feat(parser): extensible parser routing + functional parser modules (0.22.0)

.claude/skills/plotjuggler-plugin/SKILL.md

@@ -2,16 +2,14 @@
 name: plotjuggler-plugin
 description: >-
   Write or modify a PlotJuggler plugin (DataSource, MessageParser, Toolbox, or
-  Dialog) using the plotjuggler_sdk C++20 SDK. Use this whenever the task is to
-  implement, extend, debug, or build a PlotJuggler plugin — importing a file
-  format, streaming live data, parsing message payloads, adding a data-processing
-  toolbox, or building a plugin configuration dialog — even if the user does not
-  say the word "plugin" (e.g. "add a CSV loader to PlotJuggler", "make PJ read my
-  MCAP", "parse my protobuf topic", "a dialog for my source"). It gives the fast
-  path (which base class, which header, which macro), the correct CMake for both
-  in-tree and installed-SDK builds, and the load-bearing ABI/lifetime rules that
-  are easy to get wrong. Do NOT use it for changing the SDK's own ABI/protocol
-  (that is a maintainer task with different rules — see the note below).
+  Dialog) or native functional parser module using the plotjuggler_sdk. Use this
+  whenever the task is to implement, extend, debug, or build a PlotJuggler
+  extension — importing a file format, streaming live data, parsing message
+  payloads, mapping a custom type to a canonical object or scalars, adding a
+  data-processing toolbox, or building a plugin configuration dialog — even if
+  the user does not say the word "plugin". It gives the fast path, correct CMake,
+  and load-bearing ABI/lifetime rules. Do NOT use it for changing the SDK's own
+  ABI/protocol (that is a maintainer task with different rules — see below).
 ---
 
 # Writing a PlotJuggler plugin
@@ -23,6 +21,12 @@ the C-ABI plumbing (entry point, version symbol, exception trampolines, symbol
 folding) so **you never touch the raw ABI**. Your job is to override a handful of
 virtual methods and ship the library.
 
+There is also a lighter extension shape: a **functional parser module** is one
+C++17 source compiled as a native shared library, with no plugin-family
+boilerplate. Write a MessageParser plugin when you own a whole encoding. Write a
+parser module when you only need one or a few custom message types rendered as
+canonical objects or scalars. Start at `references/parser-module.md`.
+
 This skill is the author-oriented fast path. The repo's `pj_plugins/docs/` guides
 are the full reference; this steers you to the right one and front-loads the
 things that silently break a plugin.
@@ -66,6 +70,7 @@ plotjuggler_sdk::plugin_sdk)` — only the acquisition step differs:
 |---|---|---|
 | Turn a **file** or a **live source** (socket, serial, hardware) into topics | **DataSource** | `references/data-source.md` |
 | Decode a **byte payload** on a topic into named fields (JSON, protobuf, ROS, custom) | **MessageParser** | `references/message-parser.md` |
+| Map **one or a few message types** to canonical objects or scalars without owning the encoding | **Functional parser module** | `references/parser-module.md` |
 | A **tool** that reads existing data, transforms it, and writes new topics | **Toolbox** | `references/toolbox.md` |
 | A **configuration UI** (for a source/parser/toolbox, or standalone) | **Dialog** | `references/dialog.md` |
 
@@ -95,8 +100,8 @@ second argument is the **manifest JSON literal** (see Step 3).
 > ⚠ **Header-location trap.** Three base classes live under `pj_base/sdk/`, but
 > `MessageParserPluginBase` lives under **`pj_plugins/sdk/`**, and the Dialog SDK
 > lives under `pj_plugins/sdk/` too (installed there from the `dialog_protocol`
-> module). Some in-repo docs still show the parser header under `pj_base/` — that
-> is stale; use the paths in the table.
+> module). Do not substitute a `pj_base/sdk/` path for either one; use the paths
+> in the table.
 
 The quickest correct start is to copy a working example from this repo and edit it:
 `examples/sdk_consumer/` (minimal external DataSource with the full CMake),
@@ -223,6 +228,7 @@ coalescing, etc.) — read them.
 | What each family may do; capabilities; permission matrix; config contract | `pj_plugins/docs/REQUIREMENTS.md` |
 | How the C ABI / loaders / host bridge work (mostly maintainer detail) | `pj_plugins/docs/ARCHITECTURE.md` |
 | Writing each family, in depth | `pj_plugins/docs/{data-source,message-parser,toolbox,dialog-plugin}-guide.md` |
+| Authoring a native functional parser module | `references/parser-module.md`, `pj_base/include/pj_base/parser_module/README.md` |
 | Dialog `WidgetData` setters + event-handler signatures | `docs/dialog-sdk-reference.md` |
 | Builtin object types + their wire codecs | `docs/builtin_type.md`, `pj_base/include/pj_base/builtin/` |
 | Object store: publish/read objects, ownership, lazy fetch | `V4_STORE.md` |

.claude/skills/plotjuggler-plugin/references/builtin-objects.md

@@ -66,8 +66,8 @@ if (!st) return PJ::unexpected(st.error());
 Each builtin has a `*_codec.hpp` under `PJ::` (not `PJ::sdk::`) with
 `serializeXxx()` / `deserializeXxx()`. Note the asymmetry: `serializeXxx()` returns
 a plain `std::vector<uint8_t>` (serialization can't fail), while `deserializeXxx()`
-returns `PJ::Expected<...>`. The `RobotDescription` type is the exception — it
-carries its source text as-is and has no codec.
+returns `PJ::Expected<...>`. `RobotDescription` also has a canonical codec; its
+source text remains unchanged inside the small wire envelope.
 
 **MessageParsers emit objects differently — by returning, not pushing.** A parser's
 `SchemaHandler.parse_object` returns an `ObjectRecord{optional<Timestamp> ts,

.claude/skills/plotjuggler-plugin/references/message-parser.md

@@ -14,15 +14,18 @@ not a parser. A parser only decodes payloads the host hands it. A parser is righ
 when many payloads on a topic share an encoding (JSON, protobuf, ROS, a custom
 binary) and you decode each into fields.
 
+If an existing encoding already carries one custom type that you want to render
+as a canonical object or scalars, use a **functional parser module** instead of
+forking or replacing the encoding's MessageParser. See `parser-module.md`.
+
 ## Header (trap)
 
 ```cpp
 #include <pj_plugins/sdk/message_parser_plugin_base.hpp>   // pj_plugins, NOT pj_base
 ```
 
 This base class lives under `pj_plugins/sdk/`, unlike the DataSource/Toolbox bases
-which live under `pj_base/sdk/`. Some in-repo docs show it under `pj_base/` — that
-is stale.
+which live under `pj_base/sdk/`. Do not substitute a `pj_base/sdk/` include.
 
 ## The current model: register SchemaHandlers, don't override parse()
 
@@ -96,6 +99,24 @@ Key pieces:
   mapping type names → handlers scales well (the official ROS parser maps 20+
   types this way).
 
+## Route claims and functional extensions
+
+Recompile a handler-registering parser against SDK 0.22 and the base class
+automatically exposes all three host-facing tables:
+
+- `pj.parser_route_claims.v1` reports exact claims from the handler table. It
+  never reports a wildcard. The host synthesizes one universal wildcard scalar
+  claim per manifest encoding.
+- `pj.parser_functional.v1` carries complete canonical-wire objects and scalar
+  records through synchronous caller-owned sinks.
+- `pj.parser_functional.v2` keeps the scalar route unchanged and adds the object
+  splice sink. The host asks for v2 first and falls back to v1.
+
+Scalar and object routes resolve independently. Another provider may therefore
+own the object route for a type that this parser still flattens to scalars. There
+is no extra override or registration API: populate the `SchemaHandler` table and
+the base class derives these extensions.
+
 ## Optional overrides
 
 - `loadConfig(json)` / `saveConfig()` — parser options (array-size limits,

.claude/skills/plotjuggler-plugin/references/parser-module.md

@@ -0,0 +1,193 @@
+# Functional parser module
+
+A functional parser module maps one or a few message types from an existing
+encoding to canonical PlotJuggler objects or scalar fields. It is a single C++17
+source compiled as a native shared library, with no plugin-family vtable or
+MessageParser boilerplate.
+
+Use a **MessageParser plugin** when you own an encoding and must decode its broad
+type universe. Use a **parser module** when the encoding already exists and you
+only need an exact custom type rendered as an object or scalars.
+
+## Minimal native module
+
+```cpp
+#include <pj_base/parser_module/module.hpp>
+
+#include <cstdint>
+#include <limits>
+
+class RawMonoImageParser final : public pj::FunctionalParser {
+ public:
+  pj::Status bind(const pj::BindingInfo& info) override {
+    if (info.route() != pj::Route::kObject ||
+        info.expectedObjectType() != pj::ObjectWriter::kImageObjectType) {
+      return pj::Status::decline("this module only produces Image objects");
+    }
+    return pj::Status::ok();
+  }
+
+  pj::Status parseObject(pj::PayloadView payload, pj::Timestamp timestamp,
+                         pj::ObjectWriter& output) override {
+    if (payload.size > std::numeric_limits<uint32_t>::max()) {
+      return pj::Status::error("image row is too large");
+    }
+
+    auto image = output.image();
+    if (timestamp.has_value) {
+      if (auto status = image.setTimestamp(timestamp.nanoseconds);
+          !status.isOk()) {
+        return status;
+      }
+    }
+    const auto width = static_cast<uint32_t>(payload.size);
+    if (auto status = image.setWidth(width); !status.isOk()) {
+      return status;
+    }
+    if (auto status = image.setHeight(1); !status.isOk()) {
+      return status;
+    }
+    if (auto status = image.setEncoding("mono8"); !status.isOk()) {
+      return status;
+    }
+    if (auto status = image.setRowStep(width); !status.isOk()) {
+      return status;
+    }
+    return image.setData(payload);
+  }
+};
+
+PJ_FUNCTIONAL_PARSER(RawMonoImageParser)
+```
+
+`Timestamp` is per-message input. Check `has_value` before using
+`nanoseconds`. Override `parseScalars(PayloadView, Timestamp, ScalarWriter&)`
+instead, or as well, when the manifest claims the scalar route.
+
+## Claims manifest
+
+```json
+{
+  "module_abi": 1,
+  "id": "com.example.raw-mono-image",
+  "name": "Raw mono image parser",
+  "version": "1.0.0",
+  "claims": [
+    {
+      "claim_id": "raw-image-v1",
+      "encoding": "ros2msg",
+      "type_name": "example_msgs/msg/RawImage",
+      "routes": ["object"],
+      "object_type": "kImage",
+      "schema_digests": [
+        "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
+      ],
+      "priority": 0
+    }
+  ]
+}
+```
+
+- `module_abi` must equal the frozen parser-module ABI version (`1`).
+- `id` is the stable provider identity. Never change it for a display rename.
+- `name` is display-only metadata.
+- `version` is a valid SemVer string.
+- `claim_id` is stable and unique within `id`; together they form claim identity.
+- `encoding` is a case-sensitive SDK-registered encoding such as `ros2msg` or
+  `protobuf`.
+- `type_name` is encoding-normalized: `pkg/msg/Type` for `ros2msg`, or the full
+  dotted protobuf message name. Modules claim exact types, not `"*"` objects.
+- `routes` is a non-empty array containing `"scalar"`, `"object"`, or both.
+- `object_type` is required with the object route and forbidden without it. Use
+  the exact `BuiltinObjectType` spelling, such as `kImage`.
+- `schema_digests` is optional. Each entry is `sha256:` plus 64 hexadecimal
+  digits; an empty or omitted set accepts any schema digest at catalog matching.
+- `priority` is required and must be in `[-1000, 1000]`. Host-owned provenance
+  tiers outrank priority, so do not use priority as a trust signal.
+
+Manifest order fixes each claim's `claimIndex()`. The native loader copies the
+embedded bytes; catalog ingestion validates the complete JSON transactionally.
+
+## Build
+
+```cmake
+find_package(plotjuggler_sdk 0.22 REQUIRED COMPONENTS parser_module)
+
+pj_add_parser_module(raw_mono_image_parser
+  SOURCE raw_mono_image_parser.cpp
+  MANIFEST raw_mono_image_parser.module.json
+  TARGETS native
+)
+```
+
+The helper embeds the manifest, hides every non-ABI symbol, and exports the
+complete native `pj_module_*` set. SDK 0.22 accepts only `TARGETS native`;
+requesting `TARGETS wasm` stops configuration with “wasm support arrives with
+the SDK wasm loader milestone”. The shipped WASI check is structural
+conformance testing, not a wasm authoring or execution target.
+
+## Choose a schema-compatibility strategy
+
+Use the least brittle rung that fits the format:
+
+1. **Hardcode one layout** only when the wire schema is immutable and externally
+   versioned.
+2. **Digest allow-list.** Put accepted `sha256:<64-hex>` values in the claim and
+   return `Status::decline(...)` from `bind()` for an unsupported revision, so
+   the resolver may try another claim.
+3. **Inspect at bind.** Compile requested paths once with `CdrFieldLocator` from
+   a ROS 2 concatenated `.msg` bundle, or `ProtoFieldLocator` from a serialized
+   `FileDescriptorSet`, then reuse the plan for every message.
+
+`CdrFieldLocator` traverses fixed arrays, bounded/unbounded sequences, bounded
+strings, and string arrays/sequences. Cyclic schemas and traversal deeper than
+64 levels fail at bind with an error `Status`. The CDR and protobuf readers also
+bounds-check every per-message traversal.
+
+## ObjectWriter builders and splices
+
+The nine builders with frozen splice-eligible bulk fields are:
+
+- `image()`
+- `pointCloud()`
+- `depthImage()`
+- `occupancyGrid()`
+- `compressedPointCloud()`
+- `mesh3D()`
+- `videoFrame()`
+- `occupancyGridUpdate()`
+- `voxelGrid()`
+
+Every builder has `setData(PayloadView)`, which copies the bytes into the full
+canonical wire object. Every builder also has
+`setDataFromInput(InputSpanRef)`, which omits that one bulk field from the wire
+and records one splice into the exact parse payload. Obtain a safe reference
+from `CdrReader::spanRef(...)` when possible. Splice offsets are relative to the
+payload start, not the CDR encapsulation or a nested field; the writer rejects
+out-of-range, repeated, or mixed copy/splice selection before it can emit an
+invalid descriptor.
+
+## Traps
+
+- The kit is header-only and WASI-clean: no threads, filesystem, iostream, host
+  SDK linkage, or exceptions across its API. In SDK 0.22 the supported build
+  product is nevertheless native-only.
+- Return `pj::Status` / `pj::Expected<T>`; do not throw. `Blob` uses nothrow
+  allocation and protobuf matching is bounded, so allocation failure is a
+  reported data error rather than a process abort or contract strike.
+- Every view in `BindingInfo` is borrowed until `bind()` returns. Call
+  `info.owningCopy()` and retain the returned `OwnedBindingInfo` if later parses
+  need any field.
+- `Status::decline(...)` from bind means “this valid claim does not accept this
+  binding”. A module-reported negative parse result, including malformed
+  application data or allocation failure, is a data error and does not earn a
+  strike.
+- Bad/stale-token returns, a successful call that supplies a malformed output
+  descriptor, wrong route or object type, or an ineligible/out-of-bounds splice
+  are contract violations. Native hosts feed those results to
+  `ParserModuleStrikeTracker`: three strikes quarantine the claim; after one
+  successful create/bind replay, another three-strike cycle disables it for the
+  session. The tracker is host-driven state, not module-author API.
+- Instance tokens are generated index+generation values. Token `0` is reserved
+  for creation errors; stale tokens are rejected and leave a retrievable
+  diagnostic. Do not derive meaning from token bits.

.github/workflows/linux-ci.yml

@@ -146,6 +146,21 @@ jobs:
         # abigail-tools provides abidw/abidiff for the ABI drift gate (below).
         run: sudo apt-get update && sudo apt-get install -y xvfb ccache ninja-build zstd abigail-tools
 
+      - name: Install wasi-sdk 27
+        shell: bash
+        run: |
+          set -euo pipefail
+          version="27.0"
+          archive="${RUNNER_TEMP}/wasi-sdk-${version}.tar.gz"
+          install_dir="${RUNNER_TEMP}/wasi-sdk-${version}"
+          url="https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-27/wasi-sdk-${version}-x86_64-linux.tar.gz"
+
+          curl -fsSL --retry 5 --retry-all-errors "${url}" -o "${archive}"
+          mkdir -p "${install_dir}"
+          tar -xzf "${archive}" --strip-components=1 -C "${install_dir}"
+          echo "PJ_WASI_SDK_ROOT=${install_dir}" >> "${GITHUB_ENV}"
+          "${install_dir}/bin/clang++" --version
+
       - name: Configure ccache
         # Compiler-output cache for our own C++ — complementary to the Conan
         # cache (which holds prebuilt third-party packages, not our objects).
@@ -207,11 +222,17 @@ jobs:
           -DCMAKE_BUILD_TYPE=RelWithDebInfo
           -DCMAKE_C_COMPILER_LAUNCHER=ccache
           -DCMAKE_CXX_COMPILER_LAUNCHER=ccache
+          -DPJ_WASI_SDK_ROOT=${PJ_WASI_SDK_ROOT}
           -DPJ_ENABLE_ABI_CHECK=ON
 
       - name: Build
         run: cmake --build build
 
+      - name: WASI parser-module conformance
+        run: |
+          cmake --build build --target parser_module_wasm_conformance_fixture
+          ctest --test-dir build -L wasi --output-on-failure
+
       - name: ABI drift gate
         # Mechanically enforces the Release Versioning policy (CLAUDE.md): a non-MAJOR
         # change must not break ABI. Diffs the mock_data_source_plugin canary DSO against