Skip to content

Add multi-arch (amd64+arm64) buildx support for alpine Docker image#414

Draft
ihalatci wants to merge 1 commit into
developfrom
ihalatci-arm64-buildx-engine
Draft

Add multi-arch (amd64+arm64) buildx support for alpine Docker image#414
ihalatci wants to merge 1 commit into
developfrom
ihalatci-arm64-buildx-engine

Conversation

@ihalatci

@ihalatci ihalatci commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

What

Adds linux/arm64 build capability to the Citus alpine Docker image by replacing the single-arch Python Docker SDK build/tag/push path in packaging_automation/publish_docker.py with docker buildx build subprocess calls.

Refs citusdata/citus#8612.

Why

The base images (postgres:18.4 / postgres:18.4-alpine) already publish arm64/v8 manifests. The alpine Citus image builds Citus from source (./configure && make install) with no arch-specific packages, so it is arm64-capable today. The Debian images install pinned .deb packages that don't yet exist for arm64 (handled separately), so only alpine goes multi-arch here — every other image stays amd64-only with functionally unchanged behavior.

Changes

  • get_platforms() — gates alpine multi-arch behind env var DOCKER_BUILD_MULTI_ARCH (truthy set 1/true/yes/on). Default OFFlinux/amd64 for all images, preserving today's behavior. When ON, only alpinelinux/amd64,linux/arm64.
  • run_buildx_build() — one docker buildx build call per image:
    • publishing → --push (multi-arch manifests can't be --loaded; must go straight to the registry),
    • not-publishing + single-arch → --load (parity with old local-image behavior),
    • not-publishing + multi-arch → build-only (validates arm64 compile in PRs, output discarded).
  • Rewrote the 3 build functions to make one buildx call each; the tagged flow passes the full semver cascade (e.g. 14, 14.1, 14.1.0) as multiple -t flags in a single build+push.
  • Removed dead SDK code (import docker, module-level clients, flush_logs()).

Companion changes (ride together to avoid a knowingly-red CI)

  • Test: relaxed test_publish_tagged_docker_images_alpine to drop the images.get(...) asserts and rely on run_buildx_build's check=True (gate-agnostic; with the var ON alpine is build-only so there's no local image to get).
  • Workflow: added docker/setup-qemu-action@v3 + docker/setup-buildx-action@v3 to publish-docker-image-tests.yml. DOCKER_BUILD_MULTI_ARCH is intentionally left unset so tools PR/unit CI stays amd64-fast; arm64 is exercised at publish time by the citusdata/docker workflows.

Compatibility & follow-ups

  • amd64-only images keep functionally-equivalent behavior (default-OFF gate).
  • buildx --push requires a prior docker login — the citusdata/docker publish workflows already log in; buildx reads ~/.docker/config.json, so ordering is compatible.
  • After merge: a new citusdata/tools release tag must be cut and the citusdata/docker workflows repinned (currently citusdata/tools@v0.8.36).
  • QEMU/buildx setup on the citusdata/docker publish workflows is handled separately.

Risk

Emulated arm64 alpine source-build under QEMU is slow (30–60+ min). Mitigated by the default-OFF env gate — the tools repo never pays this cost unless a workflow explicitly opts in.

Validation

Local: py_compile OK on both Python files; grep confirms zero leftover docker_client/docker_api_client/flush_logs/import docker refs; AST check confirms all test-imported symbols present and flush_logs removed. Full pytest (real builds cloning citusdata/docker) not run locally — it runs in CI.

Diff stat: 3 files changed, 53 insertions(+), 48 deletions(-).

Replace the single-arch Python Docker SDK build/tag/push path in
publish_docker.py with `docker buildx build` subprocess calls so the
alpine Citus image (built from source, no arch-specific packages) can
produce linux/amd64 + linux/arm64 manifests.

- get_platforms() gates alpine multi-arch behind DOCKER_BUILD_MULTI_ARCH
  (default OFF -> amd64 only); every other image stays amd64-only.
- run_buildx_build() makes one buildx call per image: --push when
  publishing, --load for local single-arch parity, build-only for
  multi-arch non-publish (validates arm64 compile, output discarded).
- Tagged flow passes the full semver cascade as multiple -t flags in a
  single build+push; removed dead SDK clients and flush_logs().
- Relax alpine tagged test to rely on buildx check=True.
- Add QEMU + buildx setup to publish-docker-image-tests.yml (var unset,
  keeping tools CI amd64-fast).

Refs citusdata/citus#8612

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants