Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 0 additions & 8 deletions docs-mintlify/docs/data-modeling/measures.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -361,13 +361,6 @@ measures:
format: percent
```

<Note>

The `filter` parameter requires the [Tesseract SQL planner][ref-tesseract-env]
(`CUBEJS_TESSERACT_SQL_PLANNER=true`).

</Note>

### Nested aggregates

Use the [`grain`][ref-grain] parameter with `include` to compute an aggregate
Expand Down Expand Up @@ -477,7 +470,6 @@ measures:
[ref-filter]: /reference/data-modeling/measures#filter
[ref-case]: /reference/data-modeling/measures#case
[ref-switch-dim]: /reference/data-modeling/dimensions#type
[ref-tesseract-env]: /reference/configuration/environment-variables#cubejs_tesseract_sql_planner
[ref-calendar-cubes]: /docs/data-modeling/concepts/calendar-cubes
[ref-pop-recipe]: /recipes/data-modeling/period-over-period
[link-tesseract]: https://cube.dev/blog/introducing-next-generation-data-modeling-engine
3 changes: 0 additions & 3 deletions docs-mintlify/docs/data-modeling/multi-fact-views.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -336,8 +336,6 @@ separate-subquery-then-join strategy described

This rewrite applies only when:

- The Tesseract SQL planner is enabled via
[`CUBEJS_TESSERACT_SQL_PLANNER`][ref-tesseract-env].
- Both sides of the join condition resolve to the **same underlying cube
member** (a shared dimension), and the join key is composed only of
dimensions.
Expand Down Expand Up @@ -540,5 +538,4 @@ to that fact's subquery.
[ref-view-ref]: /reference/data-modeling/view
[ref-segments]: /reference/data-modeling/segments
[ref-sql-api]: /reference/core-data-apis/sql-api
[ref-tesseract-env]: /reference/configuration/environment-variables#cubejs_tesseract_sql_planner
[link-tesseract]: https://cube.dev/blog/introducing-tesseract
Original file line number Diff line number Diff line change
Expand Up @@ -159,5 +159,4 @@ configuration option.
[ref-views]: /docs/data-modeling/views
[ref-multi-fact-views]: /docs/data-modeling/multi-fact-views
[ref-multi-stage]: /docs/data-modeling/measures#multi_stage
[ref-tesseract-env]: /reference/configuration/environment-variables#cubejs_tesseract_sql_planner
[link-tesseract]: https://cube.dev/blog/introducing-tesseract
[link-tesseract]: https://cube.dev/blog/introducing-tesseract
4 changes: 1 addition & 3 deletions docs-mintlify/reference/core-data-apis/sql-api/joins.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -227,13 +227,11 @@ GROUP BY 1
```

The `JOIN` type (`INNER`, `LEFT`, `RIGHT`, `FULL`) controls which keys are
kept. This requires the [Tesseract SQL planner][ref-tesseract-env] and only
applies to grouped queries whose `GROUP BY` is the join key. See
kept. This only applies to grouped queries whose `GROUP BY` is the join key. See
[multi-fact views][ref-multi-fact-views] for the full explanation.


[ref-views]: /docs/data-modeling/views
[ref-join-paths]: /docs/data-modeling/joins#join-paths
[ref-join-hints]: /docs/data-modeling/joins#join-hints
[ref-multi-fact-views]: /docs/data-modeling/multi-fact-views
[ref-tesseract-env]: /reference/configuration/environment-variables#cubejs_tesseract_sql_planner
8 changes: 0 additions & 8 deletions docs-mintlify/reference/data-modeling/measures.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -722,13 +722,6 @@ This is commonly used for "share of total" calculations where the denominator
must ignore a filter applied by the query — for example, computing each status's
share of the per-category total while the query is filtered to a single status.

<Note>

The `filter` parameter requires the [Tesseract SQL planner][ref-tesseract-env]
(`CUBEJS_TESSERACT_SQL_PLANNER=true`).

</Note>

<CodeGroup>

```yaml title="YAML"
Expand Down Expand Up @@ -1498,7 +1491,6 @@ cube(`orders`, {
[ref-nested-aggregate]: /docs/data-modeling/measures#nested-aggregates
[ref-calendar-cubes]: /docs/data-modeling/concepts/calendar-cubes
[ref-switch-dimensions]: /reference/data-modeling/dimensions#type
[ref-tesseract-env]: /reference/configuration/environment-variables#cubejs_tesseract_sql_planner
[ref-filters-query]: /reference/core-data-apis/rest-api/query-format#filters-format
[ref-data-masking]: /docs/data-modeling/data-access-policies#data-masking
[link-d3-format]: https://d3js.org/d3-format
Expand Down
8 changes: 7 additions & 1 deletion packages/cubejs-schema-compiler/src/adapter/BaseQuery.js
Original file line number Diff line number Diff line change
Expand Up @@ -1016,6 +1016,7 @@ export class BaseQuery {
preAggregationsMatchOnly: true,
preAggregationId: this.options.preAggregationId || null,
securityContext: this.contextSymbols.securityContext,
joinHints: this.options.joinHints,
cubestoreSupportMultistage: this.options.cubestoreSupportMultistage ?? getEnv('cubeStoreRollingWindowJoin'),
disableExternalPreAggregations: !!this.options.disableExternalPreAggregations,
subqueryJoins: this.options.subqueryJoins,
Expand Down Expand Up @@ -3366,7 +3367,12 @@ export class BaseQuery {
}
}
const primaryKeys = this.cubeEvaluator.primaryKeys[cubeName];
const orderBySql = (symbol.orderBy || []).map(o => ({ sql: this.evaluateSql(cubeName, o.sql), dir: o.dir }));
// order_by templates reference members of the measure's owning cube. When
// the measure is exposed through a view, those members may be absent or
// exposed only under an alias, so resolve them against the owning cube
// (from aliasMember, the underlying reference) instead of the view.
const orderByCubeName = symbol.aliasMember ? symbol.aliasMember.split('.')[0] : cubeName;
const orderBySql = (symbol.orderBy || []).map(o => ({ sql: this.evaluateSql(orderByCubeName, o.sql), dir: o.dir }));
let sql;
let patchedSymbol = symbol;
if (symbol.type !== 'rank') {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,17 @@ views:
- created_at
- volume_by_day_rank
# num_parcels deliberately NOT exposed

- name: orders_view_alias
cubes:
- join_path: orders
includes:
- created_at
- volume_by_day_rank
# num_parcels exposed, but only under an alias, so the order_by
# template's bare {num_parcels} still does not resolve in the view.
- name: num_parcels
alias: total_parcels
`);

if (getEnv('nativeSqlPlanner')) {
Expand Down Expand Up @@ -102,9 +113,24 @@ views:
{ orders_view__created_at_day: '2024-01-01T00:00:00.000Z', orders_view__volume_by_day_rank: '2' },
{ orders_view__created_at_day: '2024-01-02T00:00:00.000Z', orders_view__volume_by_day_rank: '1' },
], { joinGraph, cubeEvaluator, compiler }));

// Repro: the same measure through a view that exposes num_parcels only under
// an alias (the order_by template still references it by its bare name).
it('view: rank order_by references a member exposed only under an alias', async () => dbRunner.runQueryTest({
measures: ['orders_view_alias.volume_by_day_rank'],
timeDimensions: [
{ dimension: 'orders_view_alias.created_at', granularity: 'day' }
],
order: [{ id: 'orders_view_alias.created_at' }],
timezone: 'UTC',
}, [
{ orders_view_alias__created_at_day: '2024-01-01T00:00:00.000Z', orders_view_alias__volume_by_day_rank: '2' },
{ orders_view_alias__created_at_day: '2024-01-02T00:00:00.000Z', orders_view_alias__volume_by_day_rank: '1' },
], { joinGraph, cubeEvaluator, compiler }));
} else {
// Multi-stage rank is Tesseract-only.
test.skip('base cube: rank order_by references a non-selected member', () => { expect(1).toBe(1); });
test.skip('view: rank order_by references a member not exposed by the view', () => { expect(1).toBe(1); });
test.skip('view: rank order_by references a member exposed only under an alias', () => { expect(1).toBe(1); });
}
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
import { PostgresQuery } from '../../src/adapter/PostgresQuery';
import { prepareYamlCompiler } from './PrepareCompiler';

// A `rank` multi-stage measure whose `order_by` template references another
// measure by its bare name (`{num_parcels}`) must resolve that reference against
// the measure's owning cube, not against a view exposing the rank. Otherwise the
// order_by template is evaluated in the view's context during the shared JS
// member-collection pass and crashes with
// `Cannot read properties of undefined (reading '_objectWithResolvedProperties')`
// (the referenced member is absent from the view namespace, or present only under
// an alias).
describe('Multi-stage measure order_by through a view', () => {
const model = (viewIncludes: string) => `
cubes:
- name: orders
sql: "SELECT 1 AS id, 10 AS parcels, '2024-01-01'::timestamptz AS created_at"
dimensions:
- name: id
sql: id
type: number
primary_key: true
- name: created_at
sql: created_at
type: time
measures:
- name: num_parcels
type: sum
sql: parcels
- name: volume_by_day_rank
type: rank
multi_stage: true
reduce_by:
- created_at
order_by:
- sql: "{num_parcels}"
dir: desc

views:
- name: orders_view
cubes:
- join_path: orders
includes:
${viewIncludes}
`;

const buildViewRankQuery = async (viewIncludes: string) => {
const { compiler, joinGraph, cubeEvaluator } = prepareYamlCompiler(model(viewIncludes));
await compiler.compile();
const query = new PostgresQuery({ joinGraph, cubeEvaluator, compiler }, {
measures: ['orders_view.volume_by_day_rank'],
timeDimensions: [{ dimension: 'orders_view.created_at', granularity: 'day' }],
order: [{ id: 'orders_view.created_at' }],
timezone: 'UTC',
// The crash is in the shared JS member-collection pass (query construction),
// which runs under both planners; pin the planner so the asserted SQL is
// deterministic and does not require the native addon to be built.
useNativeSqlPlanner: false,
});
return query.buildSqlAndParams();
};

it('resolves order_by against the owning cube when the member is not exposed by the view', async () => {
const [sql] = await buildViewRankQuery(
' - created_at\n - volume_by_day_rank'
);
expect(sql).toContain('rank() OVER (');
expect(sql).toContain('"orders__num_parcels" desc');
});

it('resolves order_by against the owning cube when the member is exposed under an alias', async () => {
const [sql] = await buildViewRankQuery(
' - created_at\n - volume_by_day_rank\n - name: num_parcels\n alias: total_parcels'
);
expect(sql).toContain('rank() OVER (');
expect(sql).toContain('"orders__num_parcels" desc');
});
});
97 changes: 97 additions & 0 deletions packages/cubejs-schema-compiler/test/unit/pre-aggregations.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1058,4 +1058,101 @@ describe('pre-aggregations', () => {
});
});
});

// Regression: the SQL API sends query-level joinHints when the queried members alone
// don't determine the join root (e.g. dimensions from `products` and a filter on
// `customers`, both reachable only through `orders`). Native pre-aggregation matching
// re-plans the query and must receive those hints too — without them it fails with
// "Can't find join path to join 'products', 'customers'".
describe('pre-aggregation matching with query-level join hints', () => {
const { compiler, joinGraph, cubeEvaluator } = prepareJsCompiler(`
cube(\`orders\`, {
sql: \`SELECT * FROM orders\`,

joins: {
customers: {
relationship: \`many_to_one\`,
sql: \`\${CUBE.customerId} = \${customers.id}\`,
},
products: {
relationship: \`many_to_one\`,
sql: \`\${CUBE.productId} = \${products.id}\`,
},
},

dimensions: {
id: {
sql: \`id\`,
type: \`number\`,
primaryKey: true,
},
customerId: {
sql: \`customer_id\`,
type: \`number\`,
},
productId: {
sql: \`product_id\`,
type: \`number\`,
},
},
});

cube(\`customers\`, {
sql: \`SELECT * FROM customers\`,

dimensions: {
id: {
sql: \`id\`,
type: \`number\`,
primaryKey: true,
},
state: {
sql: \`state\`,
type: \`string\`,
},
},
});

cube(\`products\`, {
sql: \`SELECT * FROM products\`,

dimensions: {
id: {
sql: \`id\`,
type: \`number\`,
primaryKey: true,
},
name: {
sql: \`name\`,
type: \`string\`,
},
},
});
`);

beforeAll(async () => {
await compiler.compile();
});

it('matching does not lose query-level join hints', async () => {
const query = new PostgresQuery({ joinGraph, cubeEvaluator, compiler }, {
measures: [],
dimensions: ['products.name'],
filters: [{
member: 'customers.state',
operator: 'set',
}],
joinHints: [
['orders', 'products'],
['orders', 'customers'],
],
timezone: 'UTC',
preAggregationsSchema: '',
});

// Must not throw while determining the matching pre-aggregation.
expect(query.preAggregations?.preAggregationsDescription()).toEqual([]);
expect(query.buildSqlAndParams()[0]).toMatch(/orders/);
});
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -116,12 +116,19 @@ impl ParamsAllocator {
Ok((sql.to_string(), self.params.clone()))
} else {
let mut result_params = self.params.clone();
// Occurrences of the same native param index must map to the same
// internal param: databases like Postgres require expressions in
// GROUP BY and window PARTITION BY/ORDER BY to be strictly equal,
// and that includes param placeholders
let mut native_index_map: HashMap<usize, usize> = HashMap::new();
let sql = NATIVE_PARAMS_MATCH_RE
.replace_all(sql, |caps: &Captures| {
let ind: usize = caps[1].to_string().parse().unwrap();
let param = native_allocated_params[ind].clone();
result_params.push(param);
self.make_placeholder(result_params.len() - 1)
let internal_ind = *native_index_map.entry(ind).or_insert_with(|| {
result_params.push(native_allocated_params[ind].clone());
result_params.len() - 1
});
self.make_placeholder(internal_ind)
})
.to_string();
Ok((sql, result_params))
Expand Down Expand Up @@ -184,4 +191,35 @@ mod tests {
]
);
}

#[test]
fn repeated_native_param_maps_to_single_internal_param() {
let allocator = ParamsAllocator::new(false);

// The same native param referenced from several member expressions
// (e.g. a dimension and window function PARTITION BY inside a measure)
// must stay a single param, otherwise Postgres rejects the query with
// "must appear in the GROUP BY clause" as the expressions stop being equal.
let sql = "SELECT TO_CHAR(d, $0$) x, LAG(m) OVER (PARTITION BY TO_CHAR(d, $0$)), $1$";
let native = vec![
FilterValue::Str("month".to_string()),
FilterValue::Str("other".to_string()),
];

let (rewritten_sql, params) = allocator
.add_native_allocated_params(sql, &native)
.expect("merge should succeed");

assert_eq!(
rewritten_sql,
"SELECT TO_CHAR(d, $_0_$) x, LAG(m) OVER (PARTITION BY TO_CHAR(d, $_0_$)), $_1_$"
);
assert_eq!(
params,
vec![
FilterValue::Str("month".to_string()),
FilterValue::Str("other".to_string()),
]
);
}
}
Loading