Skip to content

Flaky: AddressTypeIntegrationTests intermittently fails all 3 CI retry attempts #666

Description

@jvsena42

AddressTypeIntegrationTests fails intermittently in CI, exhausting all 3 retry attempts and turning the Run Integration Tests job red on unrelated PRs.

The most consistent offender is testPruneEmptyPreservesTypesWithBalance, but the failing set varies from attempt to attempt within a single run, and sibling tests fail with LDK NodeError.WalletOperationTimeout. That points at regtest sync timing on the runner rather than a logic bug.

Evidence

Run 31718516581 (PR #662 branch, 2026-08-13) — all 3 attempts failed, different sets each time:

Attempt Failures
1 testPruneEmptyAddressTypesAfterRestore (WalletOperationTimeout), testPruneEmptyPreservesTypesWithBalance
2 testPruneEmptyPreservesTypesWithBalance, testSetMonitoringDisableForEmptyTypeSucceeds, testSetMonitoringDisableWithBalanceFails (WalletOperationTimeout)
3 testPruneEmptyPreservesTypesWithBalance

This is not branch-specific. The identical failure — same test, same two assertions — occurred on master at e656eb5 in run 31486252063 (2026-08-11), also exhausting all 3 attempts:

testSetMonitoringEnable, XCTAssertTrue failed
testPruneEmptyPreservesTypesWithBalance, XCTAssertTrue failed
testPruneEmptyPreservesTypesWithBalance, XCTAssertEqual failed: ("2") is not equal to ("3")

The same suite passed on master at fba3c67 in run 31718249460 a couple of hours before the PR run failed, so it is genuinely intermittent rather than a persistent break.

Analysis

The two failing assertions in testPruneEmptyPreservesTypesWithBalance are BitkitTests/AddressTypeIntegrationTests.swift:286-288:

let addLegacy = await settings.setMonitoring(.legacy, enabled: true, wallet: nil)
XCTAssertTrue(addLegacy)
XCTAssertEqual(settings.addressTypesToMonitor.count, 3)

On the enabled: true path, SettingsViewModel.setMonitoring returns false only when addAddressTypeToMonitor or the following sync() throws — it rolls back addressTypesToMonitor and bails (Bitkit/ViewModels/SettingsViewModel.swift:348-355). That rollback is exactly what produces the observed count of 2 instead of 3. Combined with the explicit WalletOperationTimeout errors in sibling tests, the throw is an LDK/Electrum sync timeout under CI load.

A likely contributor is the fixed sleep the test uses to wait for the regtest deposit to confirm (AddressTypeIntegrationTests.swift:281-283):

try await blocktank.regtestMineBlocks(6)
try await Task.sleep(nanoseconds: 15_000_000_000)
try await settings.lightningService.sync()

15 seconds is enough locally but not reliably on a loaded GitHub runner, and every test in the suite goes through the same setupWalletAndNode() → node start → sync path, so a slow runner degrades the whole suite at once.

Suggested fix

  • Replace the fixed Task.sleep with polling on an actual condition (deposit visible in the address-type balance) with a generous overall timeout, instead of one fixed wait.
  • Consider retrying/extending the timeout on sync() inside the integration-test helpers, or raising LDK's wallet-operation timeout for the test configuration.
  • Surface the underlying error in the assertion message — setMonitoring currently swallows it into a Bool, so the CI log shows only XCTAssertTrue failed with no cause.

Impact

Unrelated PRs get a red Run Integration Tests check and need manual re-runs. Discovered while investigating CI on #662, whose diff touches only hardware-wallet/Trezor code and nothing in this path.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions