feat: support filtering on table calculations#24
Conversation
End-to-end verification (live instance) — found & fixed a real bugRan the SDK against a live Lightdash instance with a real token. The first attempt failed with a 400, which source-reading alone hadn't caught: Root cause: a table calculation with no declared Fix (latest commit): After the fix — verified end-to-end: defined a calc, filtered So this PR is now confirmed working end-to-end, not just shape-verified — and the earlier caveat about not being able to run E2E is resolved. |
Add a TableCalculation type and TableCalculationFilter so queries can filter
on calculated values. Table-calc filters serialize under
`filters.tableCalculations` (a sibling of `filters.dimensions`), matching the
Lightdash `Filters` type; the calculation itself serializes as a
`SqlTableCalculation` (`{name, displayName, sql}`).
- `TableCalculation(name, sql, display_name=None)` with the same comparison
operators / helper methods as `Dimension`.
- `TableCalculationFilter` accepts a `TableCalculation` or a calc name string.
- `CompositeFilter.to_dict()` routes dimension vs table-calc rules into their
respective keys; the `tableCalculations` key is omitted when unused so
existing dimension-only payloads are byte-for-byte unchanged.
- `Query.table_calculations()` builder method + `model.query(table_calculations=...)`.
- Extracted a shared `_FieldFilterMixin` for the `&`/`|` operators instead of
duplicating them across filter classes.
Shapes verified against lightdash/common source types and the live API's
filter-rule parser.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
End-to-end testing against a live instance surfaced a 400 CompileError:
filtering a table calculation with a numeric operator failed because an
untyped calc is treated as a "string" field by the API
("No function ... for the filter operator 'greaterThan' on ... field of type
'string'").
- `TableCalculation` gains a `type` field (defaults to "number"); `to_dict()`
now emits it as part of the SqlTableCalculation, matching TableCalculationBase.
- Added an env-gated acceptance test (`test_query_table_calculation_filter`)
that defines a calc, filters on it, and asserts the server applied the filter
(every returned row satisfies the predicate and the count matches a
client-side computation). Verified live.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
08b5a20 to
4973701
Compare
Lightdash AND-combines the dimensions/metrics/tableCalculations filter groups at the top level, so an OR mixing a dimension and a table-calc rule cannot be expressed faithfully. Document this and soften the "combined freely" wording to AND-only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFPouw17Yc7W8kQ4tLZSrp
Closes #21
Problem
The SDK's filter system was dimension-only — there was no way to filter on table calculations, even though the API supports it via
filters.tableCalculations.Change
TableCalculation(name, sql, display_name=None, type="number")— a SQL table calculation, with the same comparison operators (==,>,between, etc.) and helpers asDimension.TableCalculationFilter— accepts aTableCalculationor a calc-name string; serializes as{target: {fieldId}, operator, values}.CompositeFilter.to_dict()now routes dimension vs table-calc rules intofilters.dimensions/filters.tableCalculationsrespectively. ThetableCalculationskey is omitted when unused, so existing dimension-only payloads are unchanged.Query.table_calculations(...)builder method +model.query(table_calculations=...)kwarg._FieldFilterMixinfor the&/|operators rather than duplicating them across filter classes.Verification
Shapes were checked against the authoritative
lightdash/commonsource types rather than guessed:Filters = { dimensions?, metrics?, tableCalculations? }, each aFilterGroup→ confirms thefilters.tableCalculationsrouting.FilterRule = { target: {fieldId}, operator, values }(server injectsid) → confirms the rule shape.SqlTableCalculation = { name, displayName, sql, type }→ confirmsTableCalculation.to_dict().End-to-end verified against a live instance (see comment): defined a calc, filtered
calc > threshold, and confirmed the server returned exactly the rows a client-side filter would. This surfaced and fixed a real bug — an untyped calc is treated asstringby the API and rejects numeric operators, hencetypenow defaults to"number".Known limitation — OR across field types
Lightdash's
Filtersshape has separate groups (dimensions/metrics/tableCalculations) that are AND-ed together at the top level. So an OR that mixes a dimension and a table-calc rule — e.g.(dim == "USA") | (calc > 0.2)— cannot be expressed faithfully; it serializes as two independent groups that the server ANDs. OR within a single field type works as expected. This is an API constraint, now documented in the SDK guide.Tests
27 new tests in
tests/test_table_calculations.py(calc serialization, operators,filters.tableCalculationsrouting, mixed dimension+calc filters, builder integration) plus an env-gated E2E acceptance test. Full unit suite green.🤖 Generated with Claude Code