Skip to content

[DX-4553] Add docker build cache dance - #1608

Merged
kalverra merged 25 commits into
mainfrom
cacheDance
Jul 27, 2026
Merged

[DX-4553] Add docker build cache dance#1608
kalverra merged 25 commits into
mainfrom
cacheDance

Conversation

@kalverra

@kalverra kalverra commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Fix --mount=type=cache

Use the buildkit-cache-dance action to properly enable caching docker layers with --mount=type=cache in CI. See Docker docs on caching for more info.

Consolidate Caching Inputs

1. Unified cache-mode Enum Preset

  • Consolidated docker-restore-cache and docker-save-cache into a single cache-mode input parameter with standard presets:
    • auto (Default): Save on push/schedule, restore on pull_request and other events.
    • read-write (or true): Force save and restore.
    • read-only: Restore cache only.
    • write-only: Save cache only.
    • off (or false): Disable GHA layer caching.

2. Auto-Inferred buildkit-cache-dance via cache-map

  • Callers no longer need to specify cache-dance: "true" explicitly. Providing a non-empty cache-map automatically enables buildkit-cache-dance.
  • cache-map added as the standard input name, with cache-dance-cache-map retained as a backward-compatible fallback alias.

@kalverra
kalverra marked this pull request as ready for review July 24, 2026 15:43
@kalverra
kalverra requested a review from a team as a code owner July 24, 2026 15:43
@kalverra
kalverra requested review from erikburt and jmank88 July 24, 2026 15:43
Comment thread actions/ctf-build-image/action.yml Outdated
Comment thread actions/ctf-build-image/scripts/test-cache-event-overrides.sh
Comment thread actions/build-push-docker/action.yml Outdated
Comment thread actions/build-push-docker/scripts/test-cache-dance-config.sh
Comment thread actions/build-push-docker/action.yml Fixed
Comment thread actions/ctf-build-image/action.yml Fixed
Comment thread actions/build-push-docker/action.yml Fixed

@erikburt erikburt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would like to see the /chainlink pr tested against the new changes too.

Comment thread actions/build-push-docker/action.yml Outdated
Comment thread actions/ctf-build-image/scripts/go-get-overrides.sh

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Docker build/push composite actions to support persistent BuildKit cache mounts in CI and simplifies cache configuration via a unified cache-mode preset plus an implicit cache-map-driven enablement of buildkit-cache-dance.

Changes:

  • Add cache-mode and cache-map inputs to ctf-build-image and build-push-docker, and resolve save/restore behavior via a shared cache-mode preset script.
  • Enable BuildKit cache-mount persistence in build-push-docker using reproducible-containers/buildkit-cache-dance, with JSON validation for cache-map.
  • Refactor/expand shell scripts and add local test scripts; add actionlint as a pre-commit hook.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
lefthook.yml Adds a pre-commit actionlint command for workflow YAML linting.
actions/ctf-build-image/scripts/test-plugin-overrides.sh Updates plugin override test harness and makes “real run” conditional on yq.
actions/ctf-build-image/scripts/test-go-get-overrides.sh Expands Go override tests, including an unauthorized-module failure case.
actions/ctf-build-image/scripts/test-cache-event-overrides.sh Adds cache-mode resolution tests for ctf-build-image.
actions/ctf-build-image/scripts/resolve-cache-settings.sh Introduces cache-mode preset resolution (save/restore booleans).
actions/ctf-build-image/scripts/plugin-overrides.sh Improves dry-run behavior and malformed override handling.
actions/ctf-build-image/scripts/go-get-overrides.sh Adds a Go module override script using go mod edit -replace + go mod tidy.
actions/ctf-build-image/README.md Documents new cache-mode / cache-map inputs.
actions/ctf-build-image/action.yml Wires new caching inputs through to build-push-docker and refactors override steps.
actions/build-push-docker/scripts/validate-cache-dance-config.sh Adds cache-map JSON validation and emits cache-dance outputs.
actions/build-push-docker/scripts/test-cache-event-overrides.sh Adds cache-mode resolution tests for build-push-docker.
actions/build-push-docker/scripts/test-cache-dance-config.sh Adds tests for cache-map validation / implicit cache-dance enablement.
actions/build-push-docker/scripts/resolve-cache-settings.sh Introduces cache-mode preset resolution (save/restore booleans).
actions/build-push-docker/README.md Updates inputs table to include cache-mode/cache-map.
actions/build-push-docker/action.yml Integrates cache-mode resolution + buildkit-cache-dance into the build flow.
.changeset/tough-clouds-drive.md Announces minor releases for both actions due to caching feature additions.
Comments suppressed due to low confidence (1)

actions/ctf-build-image/action.yml:179

  • The action now runs go-get-overrides.sh without ensuring Go is installed/configured. Since the script invokes go mod edit/go mod tidy, this can fail on runners that don't already have Go (the previous actions/setup-go step was removed). Add actions/setup-go back (and keep it conditional on inputs.go-get-overrides).
    - name: Process Go dependency overrides
      shell: bash
      env:
        ACTION_PATH: ${{ github.action_path }}
        GO_OVERRIDES: ${{ inputs.go-get-overrides }}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread actions/ctf-build-image/action.yml Outdated
Comment thread actions/build-push-docker/action.yml Outdated
Comment thread actions/build-push-docker/README.md Outdated
Comment thread actions/build-push-docker/scripts/validate-cache-dance-config.sh
Comment thread actions/ctf-build-image/scripts/go-get-overrides.sh Outdated
Comment thread .changeset/tough-clouds-drive.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 5 comments.

Comment thread actions/ctf-build-image/action.yml
Comment thread actions/ctf-build-image/scripts/go-get-overrides.sh Outdated
Comment thread actions/build-push-docker/scripts/validate-cache-dance-config.sh
Comment thread actions/ctf-build-image/scripts/plugin-overrides.sh Outdated
Comment thread lefthook.yml

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 7 comments.

Comments suppressed due to low confidence (2)

lefthook.yml:24

  • actionlint is invoked directly, but there’s no evidence in this repo that the actionlint binary is installed/managed for local dev. This hook will fail for contributors who don’t already have actionlint on PATH. Consider skipping with a warning when it’s missing, or invoking it via a managed toolchain (if available).
    actionlint:
      description: "Run actionlint"
      glob: "{workflows/*/*,.github/workflows/*}.{yml,yaml}"
      run: actionlint {staged_files}

actions/ctf-build-image/scripts/test-plugin-overrides.sh:54

  • If you add an EXIT trap for cleanup, rm "$TEMP_MANIFEST" can fail (file already removed) and terminate the script due to set -e. Using rm -f avoids that failure mode.
# Clean up
rm "$TEMP_MANIFEST"
echo "All tests for plugin-overrides.sh completed."

Comment thread actions/ctf-build-image/scripts/plugin-overrides.sh
Comment thread actions/ctf-build-image/scripts/go-get-overrides.sh Outdated
Comment thread actions/build-push-docker/scripts/validate-cache-dance-config.sh
Comment thread actions/build-push-docker/action.yml
Comment thread actions/ctf-build-image/scripts/test-plugin-overrides.sh
Comment thread actions/build-push-docker/scripts/resolve-cache-settings.sh Outdated
Comment thread actions/ctf-build-image/scripts/resolve-cache-settings.sh Outdated
Comment thread actions/build-push-docker/action.yml Outdated
Comment thread actions/build-push-docker/action.yml Outdated
Comment thread actions/build-push-docker/action.yml Outdated
Comment thread actions/build-push-docker/README.md Outdated
Comment thread actions/ctf-build-image/action.yml Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

actions/build-push-docker/action.yml:133

  • The composite action references inputs.cache-dance-cache-map later (as a backward-compatible alias), but that input is not declared in the inputs: block. Referencing an undefined input can cause expression evaluation errors at runtime; add the alias input (deprecated) so the validate step can read it safely.
  cache-mode:
    description: |
      Caching mode preset: "auto" (default), "read-write" (or "true"), "read-only", "write-only", or "off" (or "false").
    required: false
    default: "auto"
  cache-map:
    description: |
      JSON string mapping cache mount paths/names for buildkit-cache-dance (e.g. '{"go-mod-cache": "/go/pkg/mod"}').
      Passing a non-empty cache-map automatically enables buildkit-cache-dance.
      See: https://github.com/reproducible-containers/buildkit-cache-dance
      See: https://docs.docker.com/build/ci/github-actions/cache/#cache-mounts
    required: false
    default: ""

actions/build-push-docker/action.yml:402

  • validate-cache-dance outputs like cache-dance and cache-map contain hyphens, which are not safely accessible via dot notation in expressions. Use bracket notation when reading these outputs (or rename the output keys).
      if: ${{ steps.validate-cache-dance.outputs.cache-dance == 'true' }}
      uses: reproducible-containers/buildkit-cache-dance@5422eac04292c961a382e0f584ea0f03ad9da723 # v3.4.0
      with:
        cache-map: ${{ steps.validate-cache-dance.outputs.cache-map }}

Comment thread actions/build-push-docker/action.yml
Comment thread actions/build-push-docker/scripts/validate-cache-dance-config.sh Outdated
@kalverra
kalverra merged commit 90a98cc into main Jul 27, 2026
18 checks passed
@kalverra
kalverra deleted the cacheDance branch July 27, 2026 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants