Combining efforts with sourcegraph/lsif-java

[chrismwendt]

Hey @jdneo 👋 Sourcegraph has been prototyping an LSIF indexer for Java https://github.com/sourcegraph/lsif-java using JavaParser, and I'm curious about the pros/cons of using Eclipse JDT compared to JavaParser. It'd be great to combine efforts here!

From my brief research:

• https://github.com/sourcegraph/lsif-java is ~10x faster
• https://github.com/microsoft/lsif-java supports jump-to-definition on fields/methods of an object (e.g. bar in foo.bar()), whereas https://github.com/sourcegraph/lsif-java doesn't (yet, although I made an unsuccessful attempt in Cross-file j2d/refs scip-code/scip-java#1)
• https://github.com/microsoft/lsif-java appears to jump-to-definition into dependencies (with jdt:// URIs), which could be combined with monikers to provide cross-repository and cross-project code intelligence
• Spoon supports cross-file j2d/refs and is bit easier to use than JavaParser Switch to Spoon scip-code/scip-java#2

What are your near/long-term plans for this indexer? We're interested in contributing 😄

[jdneo]

Hey @chrismwendt,

Thank you for the information and the great research effort! 😄

the pros/cons of using Eclipse JDT

Since the VS Code Java Language Server is based on JDT.LS, so we choose it as the AST Parser for the sake of quick implementation.

Also adding @akaroml for awareness.

[chrismwendt]

Notes from video chat with Sheng, Jimbo, and Chris:

• Microsoft chose to use Eclipse JDT.LS because they're from the VS Code Java team and they're already familiar with it, no experience with JavaParser/Spoon
• Main features of JDT.LS relevant to Microsoft: can resolve source positions of dependencies, emit diagnostics, and the parser is error tolerant
• microsoft/lsif-java is being used by the "Rich code nav" feature in VS Code (code intel on PRs viewed in VS Code), no usage stats (any chance you can find out?)
• microsoft/lsif-java is slow because it loads the indexer as a plugin into an Eclipse workspace prior to indexing (Spoon/JavaParser can be run and used directly)
• Microsoft is currently planning for the next 6 months of priorities, and microsoft/lsif-java is not high on the list and will probably be in maintenance mode (they're focusing more on the Java language server support in VS Code)
• It's not clear if combining efforts would be beneficial for both sides because:
• microsoft/lsif-java is in maintenance mode, development from the Sourcegraph side could cause problems
• sourcegraph/lsif-java almost supports all of Sourcegraph's use cases (all that's left is cross-repo, which is WIP)
• Sheng will ask his manager/PM for more context on collaboration and post back here

[chrismwendt]

Notes from video chat with Rome, Xiaokai, Sheng, Christina, and Chris:

• Microsoft will try sourcegraph/lsif-java to see if Spoon alleviates some problems with using Eclipse JDT directly (startup time, project build success rate, and needing to download all dependencies first)
• Microsoft will report back with their findings
• Chris (from a technical perspective) thinks Microsoft would be very helpful in at least these areas:
  • Finding the root cause and fixing some open issues where Eclipse JDT causes Spoon to crash/hang
  • Adding support for more build configurations
• Both sourcegraph/lsif-java and microsoft/lsif-java work on ~50% of projects

If I forgot something, please edit this comment 🙂