feat(project-engine-client): stateful mock core — store, resource ops, seeds (LLMO-5460)

[aliciadriani]

What

The stateful Project Engine Counterfact mock — LLMO-5460 (epic LLMO-5459), work-plan steps 5–8. A generic in-memory store + the confirmed stateful resource ops, the live Counterfact runner that wires them into per-path handlers, named seeds, a test-only reset route, and an in-package E2E that drives the real client against a booted mock.

Stacked on #1661 → #1660. Base is the wrapper branch; GitHub retargets to main as the stack merges.

Spike outcome (confirmed against both consumers)

Grepped the real consumers to set the AC floor (docs/mock-statefulness.md):

• spacecat-api-service is the only Project Engine consumer; llmo-data-retrieval-service makes zero calls.
• The write-then-read spine is projects, ai_models (per project), prompts (aio, per project). The prompt slice was reconciled against the consumer's actual call sites (rest-transport.js) and the spec: create is POST .../aio/prompts/tagged, list is POST .../aio/prompts/by_tags, delete is DELETE .../aio/prompts — not the POST /aio/prompts the spike first guessed (delete-only in the spec). publish and the GET /v1/ai_models / GET /v1/languages lookups stay on Counterfact's auto-generated response.

Contents

• src/mock/store.js — generic InMemoryStore: collection-keyed CRUD + seed/reset, deep-clone on every read/write (a loaded seed is never mutated; reset() restores a pristine copy).
• src/mock/stateful.js — the confirmed stateful set as pure ops over the store (per-workspace/per-project collection scoping). No Counterfact/HTTP coupling, so unit-tested directly.
• src/mock/seeds.js — named seed sets (empty-workspace, workspace-with-data), DEFAULT_SEED, SEED_IDS. Seed shapes match the spec response models (e.g. prompts as AIOPromptWithStatus).
• src/mock/context.js — the Counterfact per-path Context: wires the pure store + ops to a seed selected at startup and exposes reset().
• src/mock/run.js — the live mock runner. Materializes the committed handlers into a gitignored .counterfact/ tree (as .ts, so the transpiler emits loadable .cjs) and launches Counterfact with --serve (no appended spec stubs). npm run mock.
• src/mock/counterfact/routes/** — the stateful per-path handlers (projects, ai_models, aio prompts tagged/by_tags/delete) plus the test-only __reset control route.
• test/mock/*.test.js — unit tests for store, stateful ops, seeds, and context.
• test/e2e/project-engine-mock.e2e.js — boots the live mock and drives it through the real client; gated behind MOCK_E2E=1, outside the default npm test glob. Exercises the prompt write-then-read spine (tagged → by_tags) with response validation on, so the response envelopes are confirmed spec-valid.
• test/types/index.type-test.ts — compile-only guard that the public type surface stays assignable to Client<paths> and never degrades to any (npm run test:types).
• CI — a path-gated E2E (project-engine mock) job (type-surface check + live E2E), scoped via a merge-base diff so it only runs when this package changes.
• docs/mock-statefulness.md — the spike decision rule + confirmed inventory.

Validation

• npm test — 51 passing, 100% statements / branches / functions / lines
• npm run lint — clean
• npm run test:types — clean
• npm run test:e2e — 8 passing against the live mock (response validation on)

Follow-up (separate PR)

Adopting the mock into spacecat-api-service's own E2E/local-dev (replacing its Project Engine calls against this mock) lands with the adoption PR — out of scope here, which delivers the mock and its self-contained E2E.

🤖 Generated with Claude Code

[github-actions]

This PR will trigger a minor release when merged.

[aliciadriani]

Type-surface finding (surfaced by the new CI guard)

The Type surface check (npm run test:types, added in this PR) caught a real contract/implementation mismatch within minutes of landing — exactly the argument for guarding the type surface rather than deferring it.

The mismatch:

• The generated paths mark Auth-Data-Jwt as a required per-operation header — faithful to the Semrush API contract.
• But the client owns that header via middleware and sets it with headers.set(...), so any value a consumer passes is silently overwritten.

Why it matters (footgun, not cosmetics): the client's public type (Client<paths>) forces every call site in llmo + api-service to pass an Auth-Data-Jwt token that does nothing. It looks load-bearing, so someone will eventually pass a "real" token there and lose time discovering it's ignored. The E2E suite never caught this (plain JS, type-blind); the type-test did.

Options:

1. (leaning) Narrow the client's exported type — a mapped type over paths that drops the injected Auth-Data-Jwt from each operation's params.header, so the public surface reflects that the client supplies it.
2. Accept it — rejected: keeps a recurring footgun in every consumer.

Hard constraint (option 1): do the Omit at the client's exported type only. Do not touch the vendored spec, src/generated/types.ts, or the Pydantic models — those are the honest API contract and the spec-is-the-contract gate must hold.

Parked for the #4 adoption (where real call sites make the cost and the exact shape-to-omit concrete; the transform is a non-trivial recursive mapped type). The client type itself lives in #1661. The type-test fixture moves in lockstep — whatever's decided, test/types/index.type-test.ts updates with it (drop the header: { 'Auth-Data-Jwt': ... } line from the typed-call assertion when the surface is narrowed), or the guard will just re-flag the old surface.

Tracked as an LLMO-5461 sub-task linked to the #4 adoption work.

[aliciadriani]

Inline comments below carry the proposed fix for each finding from the review.

Process — Should Fix (not inline): the PR description is stale relative to the branch. The body says step 5 (Counterfact runner) and step 8 (E2E + CI wiring) are "Deferred to a follow-up PR," but this branch already contains run.js, the route handlers, __reset.js, the full E2E suite, and the new E2E (project-engine mock) CI job. The "Contents" list also omits context.js, run.js, the handlers, and the type-surface test. Please refresh the description to match the 24-file / +1579 diff so reviewers can size scope.

Fix: update the PR body — move steps 5 & 8 from "Deferred" to "Contents," and add context.js, run.js, the counterfact handlers, __reset.js, the E2E suite, and test/types/index.type-test.ts to the contents list.

[aliciadriani]

Should Fix — this create handler is mounted on a path the spec and the consumer don't use.

The spec defines only delete on /aio/prompts (aio-delete-prompt-by-ids-v2) — there is no POST here. The real create is POST /aio/prompts/tagged (aio-create-prompts-with-tags) and the real read is POST /aio/prompts/by_tags (aio-list-prompts-by-tag-ids) — exactly what docs/mock-statefulness.md lists as the consumer's stateful prompt ops.

Consequences as written:

• The E2E creates aio prompts (v2) case drives a non-spec POST, so there's no 200 response schema for Counterfact to validate the { id, name } envelope against — the "response validation stays on" guarantee doesn't cover this handler.
• When spacecat-api-service runs against the mock, prompt create (tagged) and read (by_tags) hit Counterfact's stateless stubs, so the documented prompt write-then-read spine isn't actually stateful.

Fix: keep DELETE in this file, move create into a new aio/prompts/tagged.js, and add a aio/prompts/by_tags.js read handler (the store + ops already support both via createMany / list). Repoint the E2E case at /v2/.../aio/prompts/tagged.

// aio/prompts/tagged.js — POST create (operationId aio-create-prompts-with-tags)
export function POST($) {
  const { path, body, context } = $;
  const created = context.ops.prompts.createMany(
    { workspaceId: path.id, projectId: path.project_id },
    (body?.items ?? []).map((text) => ({ name: text })),
  );
  const first = created[0] ?? {};
  return $.response[200].json({ id: first.id, name: first.name });
}

// aio/prompts/by_tags.js — POST list/read (operationId aio-list-prompts-by-tag-ids)
// shape the envelope to match the spec's 200 response for this op
export function POST($) {
  const { path, context } = $;
  const items = context.ops.prompts.list({ workspaceId: path.id, projectId: path.project_id });
  return $.response[200].json({ items, page: 1, total: items.length });
}

If this is intentionally deferred to the adoption PR, please scope it out in the description and drop the POST /aio/prompts row from the README's stateful-endpoints table, which currently advertises it as "create prompts."

[aliciadriani]

Nit — seed/created prompt field mismatch. Seed prompts use text, but prompts.createMany stores { name } and the create handler returns { id, name }. If a by_tags read handler is added (see the prompts.js comment), seeded vs. created prompts will have inconsistent shapes. Align the seed to name:

Suggested change

	{ id: 'prompt-1', text: 'What is the best running shoe?' },
	{ id: 'prompt-1', name: 'What is the best running shoe?' },

[aliciadriani]

Nit (optional, no change required). create with an explicit existing id silently overwrites via Map.set (a real API would 409). Fine for a mock; flagging only because seeds use fixed ids, so a seed-id collision would be swallowed.

If you want collisions to surface during dev, the minimal guard is:

if (entity.id && this.#collection(name).has(entity.id)) {
  throw new Error(`duplicate id ${entity.id} in collection ${name}`);
}

Otherwise leave as-is — overwrite-on-duplicate is acceptable mock behaviour.

[aliciadriani]

Nit (no change required). This POST returns 200 while the projects create returns 201. CI response-validation is on and green, so 200 is what the spec declares for add ai_model — the inconsistency is spec-driven, not a bug. Noting only for awareness.

[aliciadriani]

Nit (no change required). prompts.removeMany (and ai_models.removeMany) compute a removed count that this handler discards — it always returns 204. The spec DELETE is 204, so this is correct; the count is exercised at the ops layer in stateful.test.js. Flagging only so the unused return value reads as deliberate.

[aliciadriani]

Resolved — prompts stateful slice reconciled against the consumer and the spec (commit 4c510c1).

Follow-up to Should Fix #1 (the POST /aio/prompts handler). The prompts statefulness was reconciled against the actual consumer call sites (spacecat-api-service — src/support/serenity/rest-transport.js) and the spec, rather than the spike's guessed default, which had landed on a path the spec defines as delete-only.

What was wrong. The stateful create handler was mounted on POST /v2/.../aio/prompts, incorrect three ways:

• Path — the spec defines only delete on /aio/prompts (aio-delete-prompt-by-ids-v2); there is no POST operation there.
• Body — it read { items: [text] }; the real request is AIOTaggedPromptsCreateRequest → { prompts: { [tagName]: [text, ...] } }.
• Status — it returned 200; the spec's create returns 201.

Because the spec declares no 200 response for that path, the { id, name } envelope was never response-validated — false coverage that looked like the floor was met.

The actual consumer flow (rest-transport.js):

• create — POST /aio/prompts/tagged (aio-create-prompts-with-tags), body { prompts: { [tag]: [text] } }
• list / read — POST /aio/prompts/by_tags (aio-list-prompts-by-tag-ids), body { tag_ids, page, limit, ... }
• delete — DELETE /aio/prompts, body { ids }

handleListPrompts writes via tagged then reads via by_tags, depending on items[].{ id, name, tags } — a genuine write-then-read spine, so prompts does belong in the stateful set.

Fix (option a — wire the correct handlers, repoint the E2E):

• Removed the bogus POST from aio/prompts.js; kept DELETE → 204 (the only op the spec defines on that path).
• Added aio/prompts/tagged.js → 201 IDsWithStatsResponse { ids, existing_count }, wired to the in-memory store; created prompts carry a synthesized AIOTag so by_tags filtering is meaningful.
• Added aio/prompts/by_tags.js → 200 AIOPromptsWithStatusListResponse { items, page, total, unassigned }; empty tag_ids lists all, otherwise OR-filters by tag id.
• Realigned the workspace-with-data prompt seed to the real AIOPromptWithStatus shape (name + tags, replacing the stale text field) — also resolves the seed/created field mismatch (review nit chore: initial setup #2).
• Repointed the E2E to drive tagged → by_tags with a real write-then-read assertion, plus a delete-reflects-in-list case.
• Updated the README stateful-endpoints table.

Validation. The E2E now exercises the real spec operations with response validation on, so the 201/200 envelopes are confirmed spec-valid — the false-coverage gap is closed. All gates green: npm test (51 passing, 100% statements / branches / functions / lines), npm run lint, npm run test:types, npm run test:e2e (8 passing against the live mock).

Spec quirk worth flagging. On tagged, the workspace id is declared as a query parameter, though it is a path segment everywhere else and in the consumer's URL builder (aioPromptsPath). The handler reads $.path.id to match how the consumer actually calls it, and the E2E confirms it end-to-end. Noted because the generated types may model that parameter oddly.

[rainer-friederich]

Heads-up for the rebase onto #1661 (now that it carries the generation-time overlay spec/overlays/corrections.yaml → build/openapi3.json): src/mock/run.js needs two coupled changes, otherwise the mock will not reflect the corrected spec.

1. GET /v1/ai_models will 404 in the mock. run.js feeds Counterfact the pristine vendored swagger (SPEC = spec/projectengine_swagger_public.yaml), which by design does not contain /v1/ai_models — that path exists only in the overlaid build/openapi3.json. So "GET /v1/ai_models stays on Counterfacts auto-generated response" only holds if Counterfact is fed the overlaid artifact. → Point SPEC at build/openapi3.json, and ensure npm run spec:convert && npm run spec:overlay (i.e. npm run generate) has run before the mock boots (build/ is gitignored).

2. The route prefix breaks when you switch specs. Counterfact honors a Swagger-2.0 basePath as a serving prefix, but does not honor an OAS3 servers[0].url (verified empirically). The current raw-swagger setup gets /enterprise/projects/api for free from basePath; the converted/overlaid build/openapi3.json carries it only as servers[0].url, which Counterfact ignores → the real client (which appends /enterprise/projects/api) would 404. → Add --prefix /enterprise/projects/api to the Counterfact args. It applies to both the committed stateful handlers and the spec-derived routes, so the whole surface stays under the /enterprise/projects/api base path.

--no-validate-request already makes the Auth-Data-Jwt removal a no-op for the mock, so nothing is needed there.

Net diff to run.js:

const SPEC = join(packageRoot, 'build', 'openapi3.json'); // was spec/projectengine_swagger_public.yaml
// ...ensure build/openapi3.json exists first (npm run generate)...
[findCounterfactBin(), SPEC, BASE_PATH, '--port', String(PORT), '--serve',
 '--prefix', '/enterprise/projects/api', '--no-validate-request', '--no-update-check']

Alternative (bigger change): keep feeding Swagger 2.0 so basePath keeps prefixing (no --prefix needed), but apply the overlay to a Swagger-2.0 artifact so /v1/ai_models is present there. Since the overlay currently targets OAS3 post-conversion, feeding the overlaid build/openapi3.json + --prefix is the smaller change.