Skip to content

Fix bump-version to use YAML parsing for packages.yml updates #2148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
devin-ai-integration wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Fix bump-version to use YAML parsing for packages.yml updates #2148

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8229635
Fix bump-version to use YAML parsing for packages.yml updates
devin-ai-integration[bot] Mar 12, 2026
e69aaab
Add guard for invalid dbt-package-version input
devin-ai-integration[bot] Mar 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions .github/scripts/bump_packages_version.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
import os
import sys

import yaml

PACKAGES_FILE = "./elementary/monitor/dbt_project/packages.yml"
HELPER_COMMENTS = """
# NOTE - for unreleased CLI versions we often need to update the package version to a commit hash (please leave this
# commented, so it will be easy to access)
# - git: https://github.com/elementary-data/dbt-data-reliability.git
# revision: <COMMIT_HASH>
# When releasing a new version of the package, if the current version is using a commit hash, update the version to the new version.
# - package: elementary-data/elementary
# version: {version}
"""


def bump_packages_version(version: str) -> None:
with open(PACKAGES_FILE) as f:
data = yaml.safe_load(f)
Comment on lines +19 to +20
Copy link

@coderabbitai coderabbitai bot Mar 12, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

python -m pip install --quiet PyYAML

python - <<'PY'
from pathlib import Path
import yaml

path = Path("elementary/monitor/dbt_project/packages.yml")
src = path.read_text()

before = [line for line in src.splitlines() if line.lstrip().startswith("#")]
round_tripped = yaml.dump(
    yaml.safe_load(src),
    default_flow_style=False,
    sort_keys=False,
)
after = [line for line in round_tripped.splitlines() if line.lstrip().startswith("#")]

print(f"comments_before={len(before)}")
print(f"comments_after={len(after)}")
print("sample_comments_before:")
for line in before[:10]:
    print(line)
PY

Repository: elementary-data/elementary

Length of output: 622

PyYAML's safe_load/dump strip all comments from packages.yml.

The file contains 7 substantive comments (e.g., version management notes for unreleased CLI versions), all of which are dropped during the round-trip through yaml.safe_load() at lines 19–20 and yaml.dump() at lines 52–57. This contradicts the PR's stated goal of preserving commented content. Use a round-trip YAML editor that retains comments and formatting instead.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/scripts/bump_packages_version.py around lines 19 - 20, The script
currently uses PyYAML's yaml.safe_load() and yaml.dump(), which strip comments
from PACKAGES_FILE during the read/modify/write round-trip; replace this with a
round-trip YAML API that preserves comments (e.g., ruamel.yaml's YAML()
loader/dumper) so comments and formatting in PACKAGES_FILE are retained when
reading into the variable currently named data and when writing back (the code
paths that call yaml.dump at the end of the script). Update imports and replace
yaml.safe_load(...) / yaml.dump(...) usage with the ruamel YAML().load(...) and
YAML().dump(...) workflow (or an equivalent round-trip-preserving YAML library)
while ensuring the same data modifications still target the in-memory variable
(data) and are written back to the same file.

Copy link
Contributor Author

@devin-ai-integration devin-ai-integration bot Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is by design. The script intentionally uses yaml.safe_load() / yaml.dump() to strip comments, then re-appends the helper comments block explicitly (lines 58-59). This ensures the comments are always up-to-date with the correct version, regardless of what state the file was in before. Using ruamel.yaml for round-trip preservation would add complexity without benefit here, since the comments need to be regenerated with the new version anyway.

Copy link

@coderabbitai coderabbitai bot Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skipped: comment is from another GitHub bot.


packages = data.get("packages") or []

new_packages = []
elementary_found = False
for pkg in packages:
if "git" in pkg and "dbt-data-reliability" in pkg["git"]:
# Replace git hash reference with proper package reference
new_packages.append(
{
"package": "elementary-data/elementary",
"version": version,
}
)
elementary_found = True
elif pkg.get("package") == "elementary-data/elementary":
# Update existing package version
pkg["version"] = version
new_packages.append(pkg)
elementary_found = True
else:
new_packages.append(pkg)

if not elementary_found:
print(
"::error::Could not find elementary-data/elementary or "
"dbt-data-reliability entry in packages.yml"
)
sys.exit(1)

data["packages"] = new_packages
with open(PACKAGES_FILE, "w") as f:
yaml.dump(data, f, default_flow_style=False, sort_keys=False)

# Append the helper comments for developer convenience
with open(PACKAGES_FILE, "a") as f:
f.write(HELPER_COMMENTS.format(version=version))

print(f"Updated packages.yml to version {version}")


if __name__ == "__main__":
version = os.environ.get("PKG_VERSION", "")
if not version:
print("::error::PKG_VERSION environment variable is not set")
sys.exit(1)
bump_packages_version(version)
30 changes: 22 additions & 8 deletions .github/workflows/bump-version.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,16 +61,30 @@ jobs:
- name: Bump version
run: |
sed -i 's/^version = "[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*"/version = "${{ inputs.cli-version }}"/' ./pyproject.toml
- name: Bump version for package (using input)
if: ${{ needs.validate-version.outputs.validated-dbt-package-version != ''}}
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.10"
- name: Install PyYAML
run: pip install pyyaml
- name: Determine package version
id: pkg-version
run: |
sed -i 's/version: [0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*$/version: ${{ needs.validate-version.outputs.validated-dbt-package-version }}/' ./elementary/monitor/dbt_project/packages.yml
sed -i 's/version: [0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*$/version: ${{ needs.validate-version.outputs.validated-dbt-package-version }}/' ./docs/_snippets/quickstart-package-install.mdx
- name: Bump version for package (using default)
if: ${{ needs.validate-version.outputs.validated-dbt-package-version == ''}}
if [ -n "${{ inputs.dbt-package-version }}" ] && [ -z "${{ needs.validate-version.outputs.validated-dbt-package-version }}" ]; then
echo "::error::Invalid dbt-package-version input"
exit 1
elif [ -n "${{ needs.validate-version.outputs.validated-dbt-package-version }}" ]; then
echo "version=${{ needs.validate-version.outputs.validated-dbt-package-version }}" >> $GITHUB_OUTPUT
else
echo "version=${{ needs.validate-version.outputs.default-dbt-package-version }}" >> $GITHUB_OUTPUT
fi
- name: Bump version for packages.yml
env:
PKG_VERSION: ${{ steps.pkg-version.outputs.version }}
run: python .github/scripts/bump_packages_version.py
- name: Bump version for docs snippet
run: |
sed -i 's/version: [0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*$/version: ${{ needs.validate-version.outputs.default-dbt-package-version }}/' ./elementary/monitor/dbt_project/packages.yml
sed -i 's/version: [0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*$/version: ${{ needs.validate-version.outputs.default-dbt-package-version }}/' ./docs/_snippets/quickstart-package-install.mdx
sed -i 's/version: [0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*$/version: ${{ steps.pkg-version.outputs.version }}/' ./docs/_snippets/quickstart-package-install.mdx
- name: Commit changes
run: git commit -am "release v${{ inputs.cli-version }}"
- name: Push code
Expand Down
Loading