Use filtering queries to do batched AI querying#2670
Conversation
starcke
changed the title
starcke
force-pushed
the
starcke/apply-slice-filter
branch
4 times, most recently
from
db8d942 to
edbcae8
Compare
starcke
force-pushed
the
starcke/apply-slice-filter
branch
from
edbcae8 to
f6c492d
Compare
charisk
left a comment
charisk
left a comment
There was a problem hiding this comment.
Looks good to me and works. Just some stylistic, optional suggestions.
|
|
||
| describe("getCandidates", () => { | ||
| it("doesnt return methods that are already modelled", () => { | ||
| const externalApiUsages: ExternalApiUsage[] = []; |
There was a problem hiding this comment.
Any reason you're initialising and then doing push? Can just do it in one line.
const externalApiUsages: ExternalApiUsage[] = [
{
...
}
];There was a problem hiding this comment.
Yeah I probably just copied it from the limit case. I'll change it.
| externalApiUsages.push({ | ||
| library: "my.jar", | ||
| signature: `org.my.A#x${i}()`, | ||
|
|
| method.methodParameters, | ||
| ]); | ||
|
|
||
| const filter = { |
There was a problem hiding this comment.
Do we have a type we could use here to help out with some type safety? Same for the syntheticConfigPath
There was a problem hiding this comment.
We decided to do this in a followup.
Co-authored-by: Charis Kyriakou <charisk@users.noreply.github.com>
adityasharad
left a comment
adityasharad
left a comment
There was a problem hiding this comment.
I really like the use of data extensions to configure the batch requests. Some suggestions on the pack aspects; I haven't looked closely at the rest.
| extensions: [ | ||
| { | ||
| addsTo: { | ||
| pack: `codeql/${language}-queries`, |
There was a problem hiding this comment.
In future this will be the automodel queries pack.
| * @param candidateMethods | ||
| * @returns | ||
| */ | ||
| export async function generateCandidateFilterPack( |
There was a problem hiding this comment.
Possible performance optimisation: would it make sense to generate the pack in a temp location once up front, at the start of using the data extensions editor, but update the data extensions within the pack with a different set of candidate methods each time you need to run a filter? That might be slightly faster.
There was a problem hiding this comment.
Any chance that there can be multiple runs happening at the same time? Also, need to make sure that the temp folder is different for each open vscode window.
There was a problem hiding this comment.
I think we use this approach in a few other places, so I'll defer changes to a followup. I have created an issue to discuss this with the team.
…eml-queries.ts Co-authored-by: Aditya Sharad <6874315+adityasharad@users.noreply.github.com>
4 participants
This implements batched LLM querying with the following logic:
Checklist
ready-for-doc-reviewlabel there.