Conversation
…le ORM Replace the single-package npm project with a pnpm workspaces monorepo, migrate from Aurora Serverless v2 + Prisma to Aurora DSQL + Drizzle ORM, and replace ESLint + Prettier with oxlint + oxfmt. Architecture: - apps/webapp (Next.js), apps/async-job (Lambda), apps/cdk (CDK infra) - packages/db (Drizzle schema, DSQL client, migration runner) - packages/shared-types (job payload Zod schemas) Database: - Aurora DSQL with IAM auth (no VPC, no NAT, no Bastion) - Drizzle ORM with relations() for joins (no FK, DSQL constraint) - Custom migration runner: hash verification, 1 DDL/tx, DSQL SQL validation - dsql-migrator CDK Construct: Docker Lambda + CDK Trigger for deploy-time migration Tooling: - oxlint with type-aware linting (oxlint-tsgolint) replaces ESLint + tsc --noEmit - oxfmt with .oxfmtrc.json matching original .prettierrc (singleQuote, printWidth 120) - shamefully-hoist removed; missing implicit deps made explicit Docker: - All Dockerfiles use pnpm install (no --filter) for strict mode compatibility - esbuild bundles to .mjs (Lambda ESM requirement) - IgnoreMode.DOCKER on all Docker assets to read root .dockerignore - Verified: all 3 images build and respond to Lambda RIE invocations Refs: #98, #91, discussion #94
…gration - Fix client.ts: use Proxy for lazy db singleton to prevent crash when getPool is imported without DSQL_ENDPOINT set (e.g. migration CLI) - Fix migrate script: add --env-file=.env to load DSQL credentials - Document VPC ENI cleanup workaround in migration guide for v2→v3 update scenario where Lambda ENIs block security group deletion
Replace the flat checklist with 8 incremental stages (12a-12h): static checks → build → local Docker build → local migration → local dev server e2e → cdk deploy → deployed e2e → Lambda migration. Each stage catches a different class of bugs that later stages cannot surface efficiently. Documents known issues (VPC ENI cleanup) at the stage where they manifest.
…heck - Add drizzle-kit as devDependency with drizzle.config.ts - Add check-dsql-compat.ts validator (runs after generate) - Add 'generate' pnpm script: drizzle-kit generate && check - Initialize drizzle-kit snapshot from existing 0001_initial.sql - Document schema change workflow in AGENTS.md
check-dsql-compat.ts now auto-fixes: - statement-breakpoint → blank lines (runner splits on \n\n) - CREATE INDEX → CREATE INDEX ASYNC - Removes REFERENCES / FOREIGN KEY lines Still errors on unfixable patterns: ALTER COLUMN TYPE, DROP COLUMN, SERIAL.
The existing migrate.ts had hash-based verification that broke on formatter changes, lacked validation for most DSQL-incompatible ALTER TABLE patterns, and did not support .ts/.mjs batch migrations. - Extract pure functions (transform/validate/validateStatement) into dsql-compat.ts; check-dsql-compat.ts becomes a thin CLI wrapper - Remove hash verification from _migrations table (name-based skip) - Add validation for SET/DROP NOT NULL, SET/DROP DEFAULT, DROP CONSTRAINT, TRUNCATE - Fix REFERENCES removal to preserve column definitions (strip only the REFERENCES clause, not the entire line) - Fix statement-breakpoint replacement to produce blank lines (\n\n) for correct statement splitting - Add .ts/.mjs migration support (export default async function) - Update CDK Construct to memorySize 2048 / timeout 15min - Add unit tests (43) and DSQL integration tests (16 passed, 2 skipped due to oxlint no-restricted-syntax not yet supported) - Colocate tests with source files (foo.test.ts next to foo.ts) - Add packages/db/README.md with usage, constraints, and caveats - Update AGENTS.md with ALTER TABLE constraints, unfixable error workflow, and test colocation convention
v3 introduces four simultaneous breaking changes (DSQL, Drizzle, pnpm, oxlint) but the decision rationale was scattered across internal planning documents. This commit creates public-facing documentation that records intent — not just what changed, but why — so future maintainers and AI agents can reproduce or re-derive the same decisions. - Move DESIGN_PRINCIPLES.md from design/ to repo root, remove design/ dir - Create docs/v3.0.0/design.md: motivation, target architecture, key design decisions with intent for each technology choice - Create docs/v3.0.0/adr.md: three ADRs (DSQL+Drizzle+migrator, pnpm workspaces, oxlint+oxfmt) in Nygard format with rejected alternatives - Create docs/v3.0.0/migration-prompt.md: phased migration meta-prompt for AI agents with checkpoints and data-loss safeguards, replacing the previous v3-pnpm-workspaces-prompt.md - Add Japanese translations (*.ja.md) for all three documents - Update DESIGN_PRINCIPLES.md: replace "ADRs are not used yet" with versioned docs/ convention, add Migration guides section with BREAKING CHANGE footer convention - Update README.md path reference
Merge information from implementation plans (plan.md, drizzle-dsql-migrator-plan.md) into the permanent documentation before the plans are deleted. - Split adr.ja.md into 3 individual ADR files (adr-001, adr-002, adr-003) to match the convention that each ADR is an immutable, self-contained record - Rewrite design.ja.md as implementation specification that references ADRs for motivation/rationale, eliminating duplicated content between the two - Add DSQL DDL constraints, SQL transform rules, connection pattern, test design, and ESM Proxy rationale to design doc (from plan.md sources) - Fix v2 problem descriptions: job.Dockerfile existed but package.json was shared; NAT Instance was t4g.nano (~$3/mo) not ~$30; add Aurora Serverless v2 operational issues (cold start, scheduled job keep-alive, retry cost) - Add "Major version process" section to DESIGN_PRINCIPLES.md defining the artifact flow: research → ADR → design doc → implementation plan → code → migration prompt, with explicit committed/not-committed distinction - Move drizzle-dsql-migrator-plan.md to .kiro/specs/
Review v3 docs (ADRs, design doc, DESIGN_PRINCIPLES.md, migration-prompt) against implementation code, DSQL official docs, and v2 codebase. Fixes: - ADR-001 Context: v2 used serverlessV2MinCapacity: 0 (auto-pause), not 0.5 ACU minimum billing. Rewrite to describe auto-pause limitations (cold start, scheduled jobs preventing pause) - ADR-001 Decision: add drizzle-kit migrate SERIAL PRIMARY KEY incompatibility (dialect.ts uses SERIAL for its migrations table) - ADR-001 Consequences: inline DDL constraint summary for self-containedness (previously required reading design doc) - design.ja.md: document globalThis singleton for Next.js hot-reload connection leak prevention (was only documenting Proxy for ESM) - DESIGN_PRINCIPLES.md: add [.lang] suffix to artifact path conventions to match actual .ja.md files, fix broken migration-prompt link Cleanup: - Remove implementation plans (.kiro/specs/) — all content verified as transferred to ADRs and design doc - Remove English doc drafts (adr.md, design.md, migration-prompt.md) — superseded by .ja.md versions
Enable all default plugins (eslint, typescript, unicorn, oxc) that were previously dropped when plugins field was explicitly set, plus react, jsx-a11y, import, and vitest for broader coverage. Key changes: - oxlintrc.json: add 7 new plugins, enable import/no-cycle for circular dependency detection, disable rules incompatible with project (React 17 JSX transform, ES2022 target, side-effect imports) - Fix lint violations caught by new plugins: unnecessary template literal, missing html lang attribute, label-control association for a11y - Fix lint:ci ignore-pattern mismatch in apps/cdk - Add inline oxlint-disable for intentional test.skip and assertion-free integration tests - README: add Agentic Coding section with DSQL skill install and oxlint/oxfmt post-write hook recommendation
…ric flow The migration prompt is a meta-prompt that an AI coding agent reads to migrate a user's v2 codebase to v3. The previous version had several issues that could lead agents to make mistakes or miss critical steps. Key changes: - Remove v3 kit code details per documentation policy (agent has v3 copy) - Add "files to copy vs files to transform" section so the agent knows which files need user-specific conversion - Use schema.prisma as primary data source for schema conversion instead of pg_dump output (structured model definitions are easier to analyze) - Merge pnpm migration + monorepo restructuring + linter into one phase (pnpm-workspace.yaml references apps/*/packages/* so dirs must exist before pnpm install; linter catches DSQL-incompatible imports early) - Add dev DSQL cluster verification step using scripts/dsql.sh before touching production database - Add user-specific analysis steps: custom code inventory, VPC-dependent constructs identification, CI/CD pipeline npm/npx cleanup - Fix phase numbering (1-based, 5 phases) and CDK deploy count (2 required deploys in Phase 5, not 4) - Clarify .npmrc must not be created (strict mode is pnpm default) - Replace tsc --noEmit with oxlint typeCheck (oxlintrc.json has typeAware) - Remove redundant "breaking changes reference" section (duplicated phases) - Consolidate agent behavioral rules at the top with data loss warning
Static checks (lint, build, tsc) alone cannot catch runtime failures such as ESM module eager evaluation, .env loading issues, and Docker path resolution problems — all of which were discovered during v3 implementation (documented in .kiro/specs plan.md task 12). Add Phase 4-3 with progressive verification stages: - 4-3a: lint → build (static checks) - 4-3b: local Docker image build + docker run artifact inspection - 4-3c: local migration execution against real DSQL cluster - 4-3d: local dev server + browser verification Also fix Phase 3 checkpoint ordering to lint → build (detect DSQL incompatible patterns before attempting build).
…oding context to ADR-003 migration-prompt.ja.md was written as if the agent should execute it directly, but each derived project has unique schemas, extensions, and data volumes. Reframe the purpose section to position the document as a planning template: analyze first (Phase 1), then produce a project-specific plan, then execute. ADR-003 context was missing the primary motivation for choosing fast tooling — the kit recommends running lint + format on every file write as a post-write hook for AI coding agents (README "Agentic coding"), which requires sub-second execution.
Replace redundant root-level script aliases (dev, build, test:unit, lint, etc.) that merely wrapped `pnpm -r run <task>`. Each sub-package already defines its own scripts, so the aliases added no value. - Add simple-git-hooks + lint-staged for pre-commit quality gates (oxlint/oxfmt on staged files, then unit tests across all packages) - Keep only `prepare` script to auto-install hooks on `pnpm install` - Update CI workflow to call `pnpm -r run` directly
Lessons learned from migrating a real app (sample-ai-storybook) to v3 revealed gaps in documentation and a false-positive bug in SQL validation. - dsql-compat: strip SQL comments (block/inline) before pattern matching to prevent false positives on keywords inside comments (e.g. "-- was DROP COLUMN but recreated") - AGENTS.md: add missing TRUNCATE constraint to DSQL constraints list - db/README: add snapshot sync verification procedure after custom migrations (generate → nothing to migrate) - migration-prompt: add Json column conversion checklist (Prisma auto parse vs Drizzle manual), FK cascade deletion warning, String[] array literal conversion for pg_dump data, .env.local Docker contamination warning
The root package.json no longer has task alias scripts (dev, build, lint, test, etc.) — they were replaced with simple-git-hooks + lint-staged in f764b29, but the v3.0.0 docs were not updated. - design.ja.md: add script convention subsection documenting per-package task names with `pnpm -r run` and pre-commit hook setup - migration-prompt.ja.md: add simple-git-hooks + lint-staged setup to Phase 2-3, fix stale `pnpm run` to `pnpm -r run` in checkpoints
All Lambdas (webapp, async-job, migrator) connect to DSQL using the admin role (dsql:DbConnectAdmin), granting DDL privileges that runtime components do not need. Add ADR-004 recording the decision to keep admin for v3.0.0: custom role separation requires GRANT management per table, circular CDK-to-migration dependency for IAM mapping, and adds complexity disproportionate to the risk (IAM temp tokens already scope access to the target cluster). Add a "DB role and privilege model" section to design.ja.md summarizing the two-tier model and linking to the ADR.
The initial ADR-004 incorrectly stated that per-table GRANT statements were needed for custom roles. DSQL supports GRANT ... ON ALL TABLES IN SCHEMA for bulk grants, though ALTER DEFAULT PRIVILEGES is not supported (requiring re-grant after migrations that add tables). Revise the ADR to: - Fix GRANT management description (ON ALL TABLES, not per-table) - Add v2 comparison (also used master user, but with long-lived passwords vs v3 IAM temp tokens) - Add explicit security risk analysis for admin role (DDL + role management exposure via SQL injection, mitigated by Drizzle ORM parameterization) - Add concrete migration steps for future custom role adoption - Refocus decision rationale on CDK-to-migration bootstrap dependency as the primary reason, removing the inaccurate GRANT complexity claim Simplify the design.ja.md summary to match.
Aurora PostgreSQL Express Configuration (VPC-less, GA 2026-03-25) resolves major DSQL pain points (no FK, no JSON, no SERIAL, 1 DDL/tx, ALTER constraints). This document captures the analysis and plan for supporting both engines in the starter kit. Key decisions: - Two-branch model (main=Aurora Express, dsql=DSQL) instead of if-branching - engine/ directory convention to isolate DB-specific code from shared code - Wait for CloudFormation Express Config support before releasing Aurora Express on main; publish DSQL as v3.0.0-dsql branch in the meantime - Use drizzle-orm built-in migrate() instead of custom migration runner Includes: DSQL vs Aurora Express trade-off analysis, CloudFormation schema investigation (describe-type confirmed no Express Config properties), cost comparison, conflict analysis for two-branch maintenance, and phased rollout plan.
…Build Migrate async-job and dsql-migrator from DockerImageCode.fromImageAsset (local Docker build) to ContainerImageBuild from @cdklabs/deploy-time-build, matching the pattern already used by webapp. Also migrate the package from deploy-time-build to @cdklabs/deploy-time-build (the official cdklabs-scoped successor). This removes Docker from deployment prerequisites, eliminating Windows Docker Desktop setup friction and CI Docker-in-Docker requirements. All three images are built natively on ARM64 CodeBuild (general1.small) during cdk deploy. Trade-off: no Docker layer cache (full rebuild each deploy) and CodeBuild ARM/Small concurrent build quota defaults to 1 (builds queue sequentially; adjustable via Service Quotas).
|
|
||
| ## コンテキスト | ||
|
|
||
| キットにはコーディング時に DSQL 非互換パターンを検出するリントルールが必要 — 具体的には `drizzle-orm/pg-core` からの `serial`, `json`, `jsonb` import をブロックする `no-restricted-imports`。リンター速度は開発者体験と CI 時間に影響し、モノレポの成長に伴い重要性が増す。 |
There was a problem hiding this comment.
agent hookでの利用のため実行速度が大事だった
| RUN corepack enable && corepack prepare pnpm@10.8.1 --activate && pnpm install --frozen-lockfile | ||
| COPY packages/db/ packages/db/ | ||
| COPY packages/shared-types/ packages/shared-types/ | ||
| COPY apps/async-job/ apps/async-job/ |
apps/async-job/package.json
Outdated
| "@smithy/protocol-http": "^5", | ||
| "@smithy/signature-v4": "^5", | ||
| "drizzle-orm": "^0.44", | ||
| "zod": "^3.24.2" |
There was a problem hiding this comment.
バンドルするのでネイティブバイナリを含むもの以外dev dependenciesに
apps/cdk/jest.config.js
Outdated
| '^.+\\.tsx?$': 'ts-jest', | ||
| }, | ||
| snapshotSerializers: ['<rootDir>/test/snapshot-plugin.ts'] | ||
| snapshotSerializers: ['<rootDir>/test/snapshot-plugin.ts'], |
apps/cdk/lib/constructs/database.ts
Outdated
| }); | ||
| } | ||
|
|
||
| public grantConnect(grantee: IGrantable) { |
There was a problem hiding this comment.
せっかくgrantを作るならGrants APIにできないか検討
| COPY --from=builder /build/apps/webapp/.next/standalone ./ | ||
| COPY --from=builder /build/apps/webapp/run.sh ./run.sh | ||
|
|
||
| RUN ln -s /tmp/cache ./apps/webapp/.next/cache |
apps/webapp/next.config.ts
Outdated
|
|
||
| const nextConfig: NextConfig = { | ||
| output: 'standalone', | ||
| transpilePackages: ['@repo/db', '@repo/shared-types'], |
|
|
||
| v2 のアーキテクチャには3つの複合的な問題があった: | ||
|
|
||
| 1. **VPC のコストと Aurora Serverless v2 の運用問題**: Aurora Serverless v2 は Lambda アクセスに VPC + NAT が必要。NAT Instance(t4g.nano, ~$3/月)と Bastion Host を含む VPC の運用コストに加え、セキュリティグループ、サブネット、デプロイ変更時の Lambda Hyperplane ENI ライフサイクル問題といった運用オーバーヘッドがある。Aurora Serverless v2 自体にも問題があった: auto-pause(0 ACU)を設定してもコールドスタートに 20–30秒かかりユーザー体験が悪い、5分間隔の定期ジョブで実質常時起動になり auto-pause の恩恵を受けられない、接続エラーのリトライ・エラー処理にかなりの開発コストがかかる。 |
There was a problem hiding this comment.
「デプロイ変更」という語が不適切。
サブネット、デプロイ変更時の Lambda Hyperplane ENI ライフサイクル問題
具体的に
Aurora Serverless v2 自体にも問題
サービスが悪いのではなくユースケースに適合しないケースがあった。20-30秒のコールドスタートは24時間以上スリープした後の話。kitのコンセプトとして「インフラコストの最小化とプロトタイプ開発を素早く始められることの両立」があるのと適合しなかった。
|
|
||
| 1. **VPC のコストと Aurora Serverless v2 の運用問題**: Aurora Serverless v2 は Lambda アクセスに VPC + NAT が必要。NAT Instance(t4g.nano, ~$3/月)と Bastion Host を含む VPC の運用コストに加え、セキュリティグループ、サブネット、デプロイ変更時の Lambda Hyperplane ENI ライフサイクル問題といった運用オーバーヘッドがある。Aurora Serverless v2 自体にも問題があった: auto-pause(0 ACU)を設定してもコールドスタートに 20–30秒かかりユーザー体験が悪い、5分間隔の定期ジョブで実質常時起動になり auto-pause の恩恵を受けられない、接続エラーのリトライ・エラー処理にかなりの開発コストがかかる。 | ||
| 2. **Prisma バイナリのオーバーヘッド**: `prisma generate` がプラットフォーム固有のクエリエンジンバイナリを生成。モノレポでは Prisma クライアントを import するパッケージごとに generate が必要。バイナリは Docker イメージを肥大化させ、クロスプラットフォームビルドを複雑にする。 | ||
| 3. **package.json の非分離**: `webapp/` に Next.js・非同期ジョブ・マイグレーションランナーが同居。`job.Dockerfile` は分かれていたが `package.json` が1つのため、`npm ci` で webapp の全依存(React, Next.js, aws-amplify 等)がインストールされイメージが膨張。共有 DB コードをモノレポパッケージに抽出するには Prisma のバイナリ共有問題も同時に解決する必要がある。 |
There was a problem hiding this comment.
npm ciで webapp の全依存(React, Next.js, aws-amplify 等)がインストールされイメージが膨張
esbuildでtree-shakingされるのできちんとdependencies / devDependencies を分離できていれば回避は可能なものの、ビルド時には一時的にインストールされることと、このビルド方法についての認知負荷が問題だった。
| **データベース:** | ||
|
|
||
| - _Aurora Serverless v2(維持)_: VPC コスト、コールドスタート、定期ジョブでの常時起動問題が残る。接続エラーのリトライ処理の開発コストも解消されない。 | ||
| - _DynamoDB_: シングルテーブル設計の学習曲線が急。SQL はキットの対象読者(サーバーレス初心者の開発者)にとってよりアクセスしやすい。 |
There was a problem hiding this comment.
DynamoDBはテーブル設計のためにユースケースの網羅が必要となる点が、Aurora Serverless v2のSQLでのクエリ柔軟性と比べた時にstarter kitとして優位性がないと判断した。
Address review feedback on the v3 migration PR: - database.ts: expose DatabaseProps, default RemovalPolicy to RETAIN_ON_UPDATE_OR_DELETE, use Grant.addToPrincipal API, remove getLambdaEnvironment in favor of direct endpoint access - async-job/package.json: move all packages to devDependencies since esbuild bundles everything (no native binaries) - Dockerfiles: consolidate redundant COPY layers, replace npx with pnpm exec across all Dockerfiles - webapp.ts: restore deleted code comments explaining ContainerImageBuild usage and AMPLIFY_APP_ORIGIN circular dependency workaround - async-job.ts: restore deleted comments (EDoS concurrency limit, scheduled jobs guide) - .env.local.example: revert unnecessary region change to us-west-2 - next.config.ts: add comment explaining why transpilePackages is needed - jest.config.js: remove trailing commas introduced by formatter - ADR-001: clarify ENI lifecycle description, reframe Aurora Serverless v2 as use-case mismatch, add esbuild tree-shaking nuance to npm ci issue, revise DynamoDB rejection rationale - ADR-003: context already included agent hook speed requirement (no change needed)
Three derivative projects migrated from v2 to v3, revealing gaps in documentation, tooling, and package configuration. Documentation (migration-prompt.ja.md): - Add DSQL uuid vs text implicit cast warning to incompatibility table - Warn that Vpc.applyRemovalPolicy(RETAIN) does not propagate to child resources (subnets, route tables, IGWs); recommend two-phase deploy - Document DSQL COPY FROM STDIN support with 3,000-row tx limit Documentation (design.ja.md): - Explain why lint-staged omits typeCheck (staged-file-only args are incompatible with project-wide tsconfig resolution; CI covers it) Tooling (dsql-compat.ts): - Strip USING btree from drizzle-kit output; DSQL indexes default to B-tree and reject the USING clause Package fix (apps/cdk/package.json): - Move @types/aws-lambda from dependencies to devDependencies per AGENTS.md convention (only native binaries belong in dependencies)
…fixes Why: - sendEvent was duplicated in webapp and async-job with identical SigV4 signing logic, making maintenance error-prone. - Several minor bugs and code quality issues accumulated across the codebase. - jest v30 has compatibility issues with ts-jest v29; downgrade to stable v29. What: - Extract sendEvent into packages/event-utils shared workspace package, removing duplicate from webapp/lib/events.ts and async-job/src/events.ts. - Update Dockerfiles, package.json, and next.config.ts to reference @repo/event-utils. - Make UserNotCreatedError extend Error with proper name/message. - Add onConflictDoNothing to user insert in auth-callback to handle race conditions. - Stabilize useEventBus callback with useRef to prevent reconnections. - Add env var validation in db client (DSQL_ENDPOINT, AWS_REGION). - Fix dsql-compat REFERENCES regex to handle ON DELETE/UPDATE clauses. - Add ROLLBACK safety net for TS migration error handling. - Add JSDoc for SQL migration statement splitting convention. - Add grantConnect JSDoc with TODO for custom DB role migration. - Convert Header.tsx from client to server component (remove unused useRouter). - Fix Dockerfile ENV to use ARG passthrough pattern for build-time vars. - Fix strict equality check in next.config.ts (== to ===). - Change console.log to console.error for parse errors in async-job. - Remove hardcoded region from TranslateClient; add userId check in translate job query. - Downgrade jest from v30 to v29 for ts-jest compatibility.
…tags The bash script (scripts/dsql.sh) required jq and AWS CLI, making it platform-dependent. Rewrite as packages/db/src/cluster-cli.ts using @aws-sdk/client-dsql with idempotent create, SDK waiter, resource tagging, and a status subcommand. Rename cli.ts to migrate-cli.ts for clarity. Add Name tag to CDK-managed DSQL cluster for console identification. Document Drizzle findMany() alias pitfall in AGENTS.md and migration-prompt.ja.md — db.query.*.findMany() with exists() subqueries causes alias errors on DSQL (drizzle-orm#3068).
WIP: pnpm workspaces monorepo + Aurora DSQL + Drizzle ORM migration.
Closes #98, closes #91.
See
.kiro/specs/pnpm-workspaces/plan.mdfor full design.