Adds pre-push readiness skill#11935
Conversation
| ```bash | ||
| dart run script/tool/bin/flutter_plugin_tools.dart \ | ||
| format --run-on-changed-packages | ||
| ``` |
There was a problem hiding this comment.
Just saw that we could also use the --set-exit-if-changed option. But we could format directly since it might help the developer. @LouiseHsu what do you think?
There was a problem hiding this comment.
I think what you have is good already!
There was a problem hiding this comment.
I think it depends on what you want the scope of this skill to be. My gut says that the skill should not format but instead should say that formatting changes were found. That way one of the evals we can run is "did any files change when running this skill" and enforce that the answer is no. But that only makes sense if the skills "check pre push" skill and not a "prepare for push" skil. So it comes down to a scope question.
Second we are designing this as a system. The precommits should handle formatting(and analyze) so maybe the right answer is to do no formatting checks.
| @@ -0,0 +1,184 @@ | |||
| --- | |||
| name: "pre-push-skill" | |||
| description: "A comprehensive pre-push checklist for contributing to the flutter/packages repository." | |||
There was a problem hiding this comment.
So from what I remember, skills are invoked only based off the description and the name. So maybe you can make the desc more detailed? something like
"Executes the required pre-push steps for the flutter/packages repository. Call this tool immediately whenever the user asks to push, requests a code review before committing, or wants to validate their local changes for a pull request. Do NOT use this tool if the user is working in flutter/flutter or flutter/engine."
There was a problem hiding this comment.
I see. Good call, I like your description! Updated it.
There was a problem hiding this comment.
Code Review
This pull request adds a new markdown document, SKILL.md, defining the pre-push validation steps and checklist for the flutter/packages repository. The review feedback suggests two improvements to the instructions: first, recommending that developers commit rather than stash their changes during the initial working tree check to ensure latest changes are validated; and second, clarifying that any auto-generated changes in the final check should be committed rather than discarded.
| If this command outputs anything, the code WAS NOT ready to push. | ||
| The developer must commit or stash their changes | ||
| before you can proceed with the remaining validation steps. | ||
| Do not continue if there are uncommitted changes. |
There was a problem hiding this comment.
Instructing the developer to stash their changes is counterproductive for validation. If they stash their changes, the working tree is reverted to the HEAD commit, meaning the subsequent validation steps (formatting, analysis, tests) will run on the older code without their latest changes. Instead, they should be instructed to commit their changes (e.g., as a temporary WIP commit) so that those changes can actually be validated.
| If this command outputs anything, the code WAS NOT ready to push. | |
| The developer must commit or stash their changes | |
| before you can proceed with the remaining validation steps. | |
| Do not continue if there are uncommitted changes. | |
| If this command outputs anything, the code WAS NOT ready to push. | |
| The developer must commit their changes (e.g., as a temporary WIP commit) | |
| before you can proceed with the remaining validation steps. | |
| Do not continue if there are uncommitted changes. |
| If this command outputs anything, the code WAS NOT ready to push. | ||
| The changes must be cleaned up before the developer can push. |
There was a problem hiding this comment.
Using the phrase 'cleaned up' might be confusing, as it could be interpreted as discarding the changes (e.g., via git checkout or git clean). Since any changes at this stage are likely formatting fixes or version updates generated by the validation tools, they should be committed instead of discarded. Clarify that these changes must be committed.
| If this command outputs anything, the code WAS NOT ready to push. | |
| The changes must be cleaned up before the developer can push. | |
| If this command outputs anything, the code WAS NOT ready to push. | |
| The changes must be committed before the developer can push. |
| @@ -0,0 +1,184 @@ | |||
| --- | |||
| name: "pre-push-skill" | |||
| description: "Executes the required pre-push steps for the flutter/packages repository. Call this tool immediately whenever the user asks to push, requests a code review before committing, or wants to validate their local changes can become a pull request. Do NOT use this tool if the user is working in flutter/flutter or flutter/engine." | |||
There was a problem hiding this comment.
I am not sure this should trigger on request for "code review before committing". Consider instead
| description: "Executes the required pre-push steps for the flutter/packages repository. Call this tool immediately whenever the user asks to push, requests a code review before committing, or wants to validate their local changes can become a pull request. Do NOT use this tool if the user is working in flutter/flutter or flutter/engine." | |
| description: "Executes the required pre-push steps for the flutter/packages repository. Call this tool immediately whenever the user asks to push, asks if we/you are ready to push, or wants to validate their local changes can become a pull request. [Do NOT use this tool if the user is working in flutter/flutter or flutter/engine](https://github.com/flutter/flutter/pull/188373)." |
Why did you need "flutter/flutter#188373"? This skill should be loaded locally so it should not leak to those environments.
| This skill provides a fully automated pre-push validation script | ||
| and a mental checklist for developers attempting to push code changes | ||
| to any of the packages in the flutter/packages repository. | ||
| It directly answers the question: **"Am I ready to push?"** | ||
|
|
||
| Follow the steps below before pushing any current code changes upstream | ||
| via `git push` to a package or finishing your task. | ||
| If any step fails, continue with the following steps | ||
| to provide a comprehensive list of required fixes for the user. |
There was a problem hiding this comment.
Agent authored skills really like adding this secondary description but once this is read the skill is already loaded. If there is important information here like "Am I ready to push" Then it should be in the description in the frontmatter. Otherwise it should be removed.
| Because the tooling natively relies on `git diff` | ||
| to determine which packages have changed, | ||
| it **completely ignores uncommitted changes**. | ||
| You must ensure the working tree is clean before running any checks. |
There was a problem hiding this comment.
Which tooling?
"completely ignores uncommitted changes" I dont think this is true. More importantly I think part of the definition of "ready to push" is that git is in a clean state. I think this whole section could be
| Because the tooling natively relies on `git diff` | |
| to determine which packages have changed, | |
| it **completely ignores uncommitted changes**. | |
| You must ensure the working tree is clean before running any checks. | |
| The first step is ensuring that all changes are committed and there are no tracked files with lingering non committed state. |
| ``` | ||
|
|
||
| If this command outputs anything, the code WAS NOT ready to push. | ||
| The developer must commit or stash their changes |
There was a problem hiding this comment.
Try to avoid "who" unless you specifically need a human to take the step for safety, policy etc.
FWIW we could say that the agent should do this but I think it is better to say that the response from this skill is
"No the code is not ready to push. Git has uncommitted changes."
There was a problem hiding this comment.
This skill should go in camera_android_camerax/.agents/skills for now.
|
|
||
| ## 2. Check for Changed Packages | ||
|
|
||
| Because `--run-on-changed-packages` defaults to checking the entire repository |
There was a problem hiding this comment.
We could make this way more simple if you decided that https://github.com/flutter/packages/pull/11935/changes#r3461096989 is a good idea. Just hardcode the flag to only check camera_android_camerax
| ## 3. Format Code | ||
|
|
||
| Consistent code style is required for all pull requests. | ||
| The repository uses auto-formatters (like `dart format`, `clang-format`) |
There was a problem hiding this comment.
Not not but maybe later we should reference the check-readiness skill to ensure the formatters are on path and acceptable.
| ```bash | ||
| dart run script/tool/bin/flutter_plugin_tools.dart \ | ||
| format --run-on-changed-packages | ||
| ``` |
There was a problem hiding this comment.
I think it depends on what you want the scope of this skill to be. My gut says that the skill should not format but instead should say that formatting changes were found. That way one of the evals we can run is "did any files change when running this skill" and enforce that the answer is no. But that only makes sense if the skills "check pre push" skill and not a "prepare for push" skil. So it comes down to a scope question.
Second we are designing this as a system. The precommits should handle formatting(and analyze) so maybe the right answer is to do no formatting checks.
|
|
||
| ```bash | ||
| dart run script/tool/bin/flutter_plugin_tools.dart \ | ||
| dart-test --run-on-changed-packages |
There was a problem hiding this comment.
again this is a place where we could simplify the command if we scoped it to camera_android_camerax
There was a problem hiding this comment.
This also has the semi dangerous assumption that main was not broken and that we are merging back into main. Those assumptions are probably fine for a first version of the skill but we might want to articulate a more nuanced version.
|
|
||
| ```bash | ||
| dart run script/tool/bin/flutter_plugin_tools.dart update-release-info \ | ||
| --version=minimal --base-branch=main \ |
There was a problem hiding this comment.
probably should not hardcode --version=minimal. Consider instead articulating how to decide the scope of a version change.
| ```bash | ||
| dart run script/tool/bin/flutter_plugin_tools.dart update-release-info \ | ||
| --version=minimal --base-branch=main \ | ||
| --changelog="<description of your changes>" |
There was a problem hiding this comment.
I think we have docs on how to write a changelog. Lets include a link to those here.
|
|
||
| ## 8. Final Clean Working Tree Check | ||
|
|
||
| Before pushing, ensure that all your fixes, formatting changes, |
There was a problem hiding this comment.
Lets make a call on if this skill is only about evaluating if the changes are ready to push or if its scope it to actually make the changes.
|
|
||
| --- | ||
|
|
||
| ## Final Review |
There was a problem hiding this comment.
Instead of using temporal works like final. Lets come up with something else. Maybe Non-mechanical?
| - **Tests:** Check if any test files were added or modified | ||
| alongside the source code changes. If not, proactively ask the user | ||
| if they'd like you to generate tests (Critical for PR acceptance!). |
There was a problem hiding this comment.
Lets be a bit more specific about what types of tests we expect.
| - **Tests:** Check if any test files were added or modified | |
| alongside the source code changes. If not, proactively ask the user | |
| if they'd like you to generate tests (Critical for PR acceptance!). | |
| - **Tests:** It is expected that virtually all changes required a test see [Test Documentation](https://github.com/flutter/flutter/blob/master/docs/ecosystem/testing/Plugin-Tests.md). Evaluate the change against that testing rubric If they are not give a quote from the testing documentation on what type of test for which feature needs to be added. If you think the change does not meet the documented quality bar tell the user and only push if they approve the test coverage. |
| Then, provide the developer with a brief summary | ||
| of what you verified automatically. | ||
| If the code is ready to push, provide them with this short checklist | ||
| of items they will need to handle when opening their pull request: |
There was a problem hiding this comment.
I think this should actually output the gh pr create command that should be used.
| of items they will need to handle when opening their pull request: | ||
|
|
||
| - Ensure the PR title starts with the package name in brackets | ||
| (e.g., `[camera_android] Fix crash`). |
There was a problem hiding this comment.
Give instructions for prs that touch multiple packages.
| - Ensure the PR title starts with the package name in brackets | ||
| (e.g., `[camera_android] Fix crash`). | ||
| - Ensure the PR description links to at least one issue that is being fixed. | ||
| - Ensure they have signed the CLA. |
There was a problem hiding this comment.
We should remove this for now. We have all signed the cla any bot account will have been approved already.
| (e.g., `[camera_android] Fix crash`). | ||
| - Ensure the PR description links to at least one issue that is being fixed. | ||
| - Ensure they have signed the CLA. | ||
| - Ensure the branch is up to date with the main branch |
There was a problem hiding this comment.
This should happen as part of the early steps before formatting and especially before changelog verification. I would really stick to have an easy to update branch that becomes hard to update because you added changelogs and version bumps.
There was a problem hiding this comment.
Meta comments:
- We need to decide if file changes are part of this skill or if this skill has no file changes and instead the agents responds to failures like a test.
Adds a new
pre-push-skillskill that verifies changes made to a packages are ready to be "pushed", i.e. the committed changes could successfully be made into a pull request that follows the requirements of the flutter/packages repository (including some of the items in the checklist below).Pre-Review Checklist
[shared_preferences]///).If you need help, consider asking for advice on the #hackers-new channel on Discord.
Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the
gemini-code-assistbot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.Footnotes
Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling. ↩ ↩2