Add "View Autofixes" feature for variant analysis results by jcogs33 · Pull Request #4065 · github/vscode-codeql

jcogs33 · 2025-06-29T02:12:01Z

Description

Adds a "View Autofixes" feature for variant analysis results. This is a canary-only feature.
Generates autofixes for selected variant analysis results and opens the resulting autofix outputs in a markdown file.
Activated via the "View Autofixes" button in the variant analysis webview or via right-clicking the variant analysis query history item.

At a high-level, the implementation:

Checks for a local checkout of autofix
Sets up to run autofix by:
- Overriding the relevant query help
- Getting the relevant SARIF(s)
- Downloading source root(s)
Runs autofix locally
Outputs autofix results in a markdown file

Commit-by-commit review recommended. Let me know if you want me to split anything onto a separate PR.

Consideration

See in-line comments
Change from my original implementation: Instead of downloading databases and then extracting the source root from the database, I directly download the source root since it seemed a bit faster. Let me know if you think the database approach is better.
I will make a follow-up PR to convert this to support Go autofix instead of cocofix, and I'll update the queryHelpOverrideDirectory to the new go/pkg/autofix/prompt/qhelps directory in that PR.

Testing

I've manually tested this feature but have not added unit/view/integration tests since it seems another canary feature did not add tests. Let me know if I should add tests after all.
Note: There is a known issue with the E2E CI test failing. It is not caused by changes in this PR.

…ix limit

This function will increment file numbers when running autofix in a loop

extensions/ql-vscode/src/common/interface-types.ts

extensions/ql-vscode/src/variant-analysis/variant-analysis-view.ts

extensions/ql-vscode/src/variant-analysis/view-autofixes.ts

Reference codeQL.autofix.path setting instead of env var

jcogs33 · 2025-07-04T01:55:09Z

There's one more comment from @koesie10 that should be addressed

@aeisenberg Which comment are you referring to? There were four things I still was planning to address: three comments related to editing parts of downloadPublicCommitSource, and one comment about getting the autofix CLI logs into the extension logs.

aeisenberg · 2025-07-04T03:25:59Z

Oh...there might be more things that I missed. I'm out tomorrow. I think it's also a holiday in the US, so I can take another look on Monday.

jcogs33 · 2025-07-06T23:04:12Z

@aeisenberg @koesie10 I've addressed all review comments. Let me know if you want any other changes, especially regarding the environment variables.

koesie10

Thanks, this looks good to me. I have some optional comments, but I don't think any of those are necessary.

extensions/ql-vscode/src/variant-analysis/view-autofixes.ts

koesie10 · 2025-07-07T09:46:01Z

extensions/ql-vscode/src/variant-analysis/view-autofixes.ts

+  // Get autofix binary.
+  // Switch to Go binary in the future and have user pass full path
+  // in an environment variable instead of hardcoding part here.
+  const cocofixBin = join(process.cwd(), localAutofixPath, "bin", "cocofix.js");


If the Go binary accepts the same parameters as cocofix, would it makes sense to already add a config setting for this and use a default of bin/cocofix.js?

I need to experiment with the Go binary manually. I'm not sure yet if it accepts all of the same parameters. I'll likely edit this into a config setting when I create a follow-up PR for switching to Go.

extensions/ql-vscode/src/variant-analysis/view-autofixes.ts

koesie10 · 2025-07-07T09:50:47Z

extensions/ql-vscode/src/variant-analysis/view-autofixes.ts

+      env: {
+        CAPI_DEV_KEY: process.env.CAPI_DEV_KEY,
+        PATH: process.env.PATH,
+      },


I don't think this is really necessary if it's unlikely that this will be used. I haven't run autofix myself locally, so I'm not sure if it depends on any environment variables.

aeisenberg

Looks generally good. I see one area where I think you can simplify things.

extensions/ql-vscode/src/variant-analysis/view-autofixes.ts

jcogs33 · 2025-07-07T23:08:04Z

@koesie10 @aeisenberg I've responded to all new review comments. Thanks for the thorough reviews! 🙇

koesie10

LGTM, thanks!

aeisenberg

Great job!

I haven't tried this locally, but the code looks good. It's likely that the failing E2E job is transient. I'll kick it off again and hopefully it will work.

jcogs33 · 2025-07-08T16:43:31Z

@aeisenberg There's a known issue with the E2E tests failing since end of March. The issue tracking it is in the private code-scanning repo. So I think this PR can be merged despite that test failure?

aeisenberg · 2025-07-08T16:49:51Z

I'm fine with that.

Jami Cogswell added 22 commits June 27, 2025 19:12

Add 'View Autofixes' button to variant analysis view

8dfcf8b

Add 'View Autofixes' query history command for variant analysis items

1103437

Add 'view-autofixes.ts' file for main implementation

5f5a20f

Check for local autofix

fb39455

Override query help

27bdc31

Get full names (owner/repo) of the selected repositories

9a8bd79

Get storage paths for autofix output

66d9526

Add function to process the selected repositories

9c4d7a7

Get SARIF for variant analysis repo

2afd2ee

Read 'repo_task.json' file

2bcf42e

Download source root

b0b8b62

Add function for running autofix on a given repo

05c398f

Get storage paths for repository-specific autofix output

4d5437f

Set up to run autofix on repo results

5dfaf3f

Add function for setting variable autofix args

03585af

Add function for executing autofix

19400a9

Add function for running autofix on a given SARIF

3c7d9ca

Run autofix for case when the repo result count does not exceed the f…

74e8277

…ix limit

Add function that appends a suffix to a given file path

e7076f9

This function will increment file numbers when running autofix in a loop

Run autofix for case when the repo result count exceeds the fix limit

e04eb62

Add markdown formatting to autofix output

2c12d71

Combine and output final autofix results

6b47932

jcogs33 force-pushed the jcogs33/view-autofixes branch 3 times, most recently from 8095989 to c5f0e5e Compare June 30, 2025 21:29

Add canary

3ff5d64

jcogs33 force-pushed the jcogs33/view-autofixes branch from c5f0e5e to 3ff5d64 Compare July 2, 2025 00:44

jcogs33 commented Jul 2, 2025

View reviewed changes

extensions/ql-vscode/src/common/interface-types.ts Outdated Show resolved Hide resolved

jcogs33 commented Jul 2, 2025

View reviewed changes

extensions/ql-vscode/src/variant-analysis/variant-analysis-view.ts Outdated Show resolved Hide resolved

jcogs33 commented Jul 2, 2025

View reviewed changes

extensions/ql-vscode/src/variant-analysis/view-autofixes.ts Outdated Show resolved Hide resolved

Jami Cogswell added 4 commits July 4, 2025 00:13

Switch fs/promises imports to fs-extra

41a3612

Replace three backticks in file content

b7f84fc

Use 'extLogger' instead of 'NotificationLogger'

492b805

Update doc

21a466a

Reference codeQL.autofix.path setting instead of env var

Jami Cogswell added 2 commits July 6, 2025 20:07

downloadPublicCommitSource: use 'fetch' API and 'tmp-promise'

561cd6c

Log autofix stdout/stderr to the extension logs

8ed9e6a

jcogs33 requested review from aeisenberg and koesie10 July 6, 2025 23:02

koesie10 approved these changes Jul 7, 2025

View reviewed changes

aeisenberg reviewed Jul 7, 2025

View reviewed changes

extensions/ql-vscode/src/variant-analysis/view-autofixes.ts Outdated Show resolved Hide resolved

Jami Cogswell added 8 commits July 7, 2025 20:06

Fix path to extracted source root directory

cb4bfbc

Remove checkoutDir if source root download fails

6333c9e

Remove unneeded 'progressCallback' undefined conditional

0e6aa81

Simplify source root downloading with 'pipe'

2c04019

Call 'cliServer.resolveMetadata' instead of 'tryGetQueryMetadata'

7f9d7ae

showWarningMessage when number of repos is limited

4e76a0d

Add names of config settings to error messages

bf6d8e7

Fix timeout error caused by oversimplification of 'onData'

1f5cc90

jcogs33 requested review from aeisenberg and koesie10 July 7, 2025 23:08

koesie10 approved these changes Jul 8, 2025

View reviewed changes

aeisenberg approved these changes Jul 8, 2025

View reviewed changes

aeisenberg merged commit 5328aa2 into github:main Jul 8, 2025
18 of 20 checks passed

jcogs33 deleted the jcogs33/view-autofixes branch July 8, 2025 16:54

Conversation

jcogs33 commented Jun 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Consideration

Testing

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jcogs33 commented Jul 4, 2025

Uh oh!

aeisenberg commented Jul 4, 2025

Uh oh!

jcogs33 commented Jul 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

koesie10 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

koesie10 Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

jcogs33 Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

koesie10 Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

aeisenberg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jcogs33 commented Jul 7, 2025

Uh oh!

koesie10 left a comment

Choose a reason for hiding this comment

Uh oh!

aeisenberg left a comment

Choose a reason for hiding this comment

Uh oh!

jcogs33 commented Jul 8, 2025

Uh oh!

aeisenberg commented Jul 8, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jcogs33 commented Jun 29, 2025 •

edited

Loading

jcogs33 commented Jul 6, 2025 •

edited

Loading