tests/e2e: redesign exec-wait with bracket-delimited segments and external selector#807
Draft
natalie-o-perret wants to merge 18 commits intomasterfrom
Draft
tests/e2e: redesign exec-wait with bracket-delimited segments and external selector#807natalie-o-perret wants to merge 18 commits intomasterfrom
natalie-o-perret wants to merge 18 commits intomasterfrom
Conversation
- `testscript_api_test.go` (build tag `api`) with `json-setenv` and `wait-instance-state` custom commands - Each scenario owns its full resource lifecycle (create → test → delete) - First scenario: compute instance reboot - CI job runs on every push via `test-e2e-api`
Add a ptyInput struct that carries an optional waitFor pattern alongside the byte sequence to write. When waitFor is set, the input goroutine in runInPTY polls the shared, mutex-protected PTY output buffer until the pattern appears (or a 10-second deadline expires) before writing — instead of relying on a blind 300 ms sleep. Switch the PTY output collector from bufio.Scanner line-reads to a raw []byte read loop so that partial-line prompts (e.g. "[+] API Key: ") are captured immediately, before the process receives any input. Introduce @wait:<pattern> syntax in the execpty stdin file format. A @wait: line is consumed as metadata and attached to the very next input token, making "send Ctrl+C only once the API-key prompt is visible" easy to express without any timing magic. Apply the fix to the two scenarios that were previously racy: - add_cancel_with_ctrl_c.txtar (@wait:[+] API Key: before @ctrl+c) - add_cancel_with_ctrl_d.txtar (@wait:[+] API Key: before @ctrl+d)
…wait: Now that runInPTY can gate each input write on a specific string appearing in the PTY output, every scenario can declare exactly when each keystroke should be sent, making test execution fully event-driven with zero timing magic. Remove the 300 ms fallback sleep from the input goroutine in runInPTY. Each ptyInput now either carries a @wait: pattern (blocks until the expected prompt text appears in accumulated PTY output) or is sent immediately as part of a sequential interaction with a prompt already on screen. Annotate every multi-step scenario: add_cancel_during_secret — waits for [+] API Key: then [+] Secret Key: add_cancel_during_zone — waits for each of the three text prompts then Default zone add_interactive_basic — same three text prompts + Default zone add_interactive_duplicate_name — additionally waits for 'already exist' and '[?] Set [' add_interactive_empty_validation — waits for each validation error message before re-entering add_interactive_make_new_default — text prompts + Default zone + [?] Set [ add_interactive_second_account — same add_interactive_zone_navigation — text prompts + Default zone (multiple @down with no extra wait) config_cancel_during_menu — waits for 'Configured accounts' before navigating
… synchronisation #804
The main.yml build job only fired on push events, so a CHANGELOG.md-only commit (skipped by paths-ignore) left the PR with no Build check on its current head. Adding pull_request with the same paths-ignore ensures the build always runs against the PR head commit.
…[sc-167368] Add findInstance helper in cmd/compute/instance/instance_find.go wrapping FindListInstancesResponseInstances. On a not-found error it returns a zone-aware message with a hint to use -z or list all instances: instance "foo" not found in zone ch-gva-2 Hint: use -z <zone> to specify a different zone, or run 'exo compute instance list' to see instances across all zones All 24 call sites across the instance subcommands are migrated. Adds e2e scenario tests/e2e/scenarios/with-api/compute/instance_not_found_error.txtar.
…[sc-156206] When the API rejects the update request (e.g. duplicate name), the returned operation is nil. The missing error guard before DecorateAsyncOperation caused a confusing "operation is nil" error instead of surfacing the real API message. Also adds an e2e testscript scenario covering description update, name rename, and the duplicate-name conflict path.
…ernal selector Replace the -- delimiter and until=field:value option with bracket-delimited [ cmd1 ] [ cmd2 ] [ selector ] segments. The selector is any process that reads cmd2 stdout on stdin and exits 0 when the condition is satisfied (e.g. jq, grep). This makes exec-wait composable and not tied to JSON: any tool that follows Unix exit-code conventions works as a predicate. Also replace splitByDoubleDash with splitByBrackets.
- Run cmd1 and cmd2 via exec.Command directly instead of runCLIWithEnv, so the exo binary is not double-prefixed when 'exo' appears in brackets - Prepend exoBinary's directory to PATH in buildPollEnv so that 'exo' in bracket commands resolves to the correct binary
CombinedOutput mixes spinner text (written to stderr by DecorateAsyncOperation) into stdout, breaking JSON parsing. Use cmd.Output() + separate stderr buffer so only clean JSON stdout is passed to set= extraction and the selector.
Renames the execpty custom testscript command to exec-pty for consistency with exec-wait, and updates its syntax to use bracket delimiters for the command: exec-pty --stdin=<file> [ <binary> [args...] ] This separates the --stdin option cleanly from the command being run, and makes exec-pty visually consistent with exec-wait. splitByBrackets is moved from testscript_api_test.go (build tag: api) to testscript_test.go (always compiled) so that cmdExecPTY can use it without requiring the api build tag. All 11 .txtar scenario files updated to the new syntax.
Replace --stdin=<file> flag with a second bracket group: exec-pty [ <binary> [args...] ] [ <stdin-file> ] This removes the last flag-style option from exec-pty, making the syntax fully consistent with exec-wait where everything is expressed as bracket groups.
… --predicate=[] Replace positional bracket groups with named ones: exec-wait [set=...] --action=[ cmd... ] --polling=[ cmd... ] --predicate=[ cmd... ] The "=[" is attached to the flag name as a single token, making each group's purpose self-documenting. Adds splitByNamedBrackets() alongside the existing splitByBrackets() (still used by exec-pty).
exec-pty --stdin=<file> ( <binary> [args...] ) The stdin file is an explicit named flag rather than a second positional group, keeping the command group as the sole ( ) block.
Writing credentials to a config file on disk during API tests is unnecessary — the CLI reads EXOSCALE_API_KEY/EXOSCALE_API_SECRET directly from the environment and ignores config files when they are set. Removing the file also eliminates the risk of secrets appearing in leftover test artifacts. HOME/XDG_CONFIG_HOME isolation is kept so tests cannot accidentally read the developer's real config file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the
--delimiter anduntil=field:valueoption inexec-waitwith bracket-delimited[ cmd1 ] [ cmd2 ] [ selector ]segments.Motivation
The previous design hardcoded JSON as the only predicate format (
until=field:value). The new design delegates predicate evaluation to an external process that reads cmd2's stdout on stdin and exits 0 when satisfied. This makesexec-waitcomposable with any Unix tool:Changes
splitByDoubleDashreplaced bysplitByBracketscmdExecWaitrewritten to parse 3 bracket groups and pipe cmd2 stdout into the selector processinstance_reboot.txtarupdated to use the new syntaxcmdWaitInstanceState(unused) was already removed in a prior commit