Skip to content

acc: make acceptance tests work in Databricks development environments#5662

Merged
pietern merged 2 commits into
mainfrom
acc-tests-ignore-local-dev-probes
Jun 22, 2026
Merged

acc: make acceptance tests work in Databricks development environments#5662
pietern merged 2 commits into
mainfrom
acc-tests-ignore-local-dev-probes

Conversation

@pietern

@pietern pietern commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

The acceptance harness blocks external network access during tests. In some Databricks-internal development environments, local tooling makes network calls that trip the sandbox and fail the suite (CI is unaffected), and a stale local toolchain produced confusing diffs instead of clear errors.

  • Ignore local probes that aren't test traffic (e.g. requests to bare localhost) and disable the local background beacons, so behavior matches CI.
  • Assert the required toolchain up front so a missing/stale tool fails fast: jq, uv, ruff, and python >= 3.11 (provisioned via uv).

Testing: full direct acceptance suite passes locally in a Databricks dev environment; unit tests for the tool checks and python provisioning included.

This pull request and its description were written by Isaac.

@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 12:55 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 12:55 — with GitHub Actions Inactive
@pietern pietern force-pushed the acc-tests-ignore-local-dev-probes branch from 149eac3 to 5c5df59 Compare June 19, 2026 13:02
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:02 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:02 — with GitHub Actions Inactive
@pietern pietern force-pushed the acc-tests-ignore-local-dev-probes branch from 5c5df59 to dc7f93b Compare June 19, 2026 13:05
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:06 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:06 — with GitHub Actions Inactive
@pietern pietern force-pushed the acc-tests-ignore-local-dev-probes branch from dc7f93b to 8f59ba8 Compare June 19, 2026 13:09
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:10 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:10 — with GitHub Actions Inactive
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
version: "0.8.9"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now run uv find python from the Taskfile instead of assuming the right version exists.

This means we need to make sure uv is available before using the Taskfile.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 4ece006

Run: 27940299125

Env 🟨​KNOWN ✅​pass 🙈​skip Time
🟨​ aws linux 1 216 99 3:19
🟨​ aws windows 1 218 97 2:40
🟨​ aws-ucws linux 1 297 18 3:38
🟨​ aws-ucws windows 1 299 16 3:27
🟨​ azure linux 1 216 98 3:15
🟨​ azure windows 1 218 96 3:16
🟨​ azure-ucws linux 1 299 15 4:00
🟨​ azure-ucws windows 1 301 13 3:19
🟨​ gcp linux 1 215 100 3:42
🟨​ gcp windows 1 217 98 2:41
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K 🟨​K

The acceptance harness blocks external network access during tests. In some
Databricks-internal development environments, local tooling issues network
calls that trip the sandbox and fail the suite even though CI passes. Ignore
the local probes that are not test traffic and disable the background beacons
so tests behave the same as on CI.

Also assert the external toolchain (jq, uv, ruff) and provision python via uv
up front, so a missing or stale tool fails fast with a clear message.

Co-authored-by: Isaac
@pietern pietern force-pushed the acc-tests-ignore-local-dev-probes branch from 8f59ba8 to a9da026 Compare June 19, 2026 13:37
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:37 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is June 19, 2026 13:37 — with GitHub Actions Inactive
@pietern pietern enabled auto-merge June 19, 2026 14:22

@denik denik left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, failing early and with clear message is going to save a lot of time.

nit: binary version checking & localhost probe thing feel completely unrelated and both deserve each own PRs.

Comment thread libs/testserver/server.go
Comment thread acceptance/acceptance_test.go Outdated
Take the required version as a string argument and compare with
golang.org/x/mod/semver in each tool file, with the version (and the reason for
it) surfaced at the call site. Group the four checks into a single
requirePrerequisites helper to keep testAccept short. On Windows, verify the
python3 on PATH meets the floor before skipping provisioning.

Co-authored-by: Isaac
@pietern pietern temporarily deployed to test-trigger-is June 22, 2026 08:39 — with GitHub Actions Inactive
@pietern pietern temporarily deployed to test-trigger-is June 22, 2026 08:39 — with GitHub Actions Inactive
@pietern

pietern commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

@denik The two changes are related in that both are needed to get a passing test run.

@pietern pietern requested a review from denik June 22, 2026 08:45
@pietern pietern added this pull request to the merge queue Jun 22, 2026
Merged via the queue into main with commit c6ce0ec Jun 22, 2026
33 checks passed
@pietern pietern deleted the acc-tests-ignore-local-dev-probes branch June 22, 2026 12:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants