Skip to content

v1 convention: let a join opt out of zero-filling the positions it creates (fill_value=ABSENT) #890

Description

@FabianHofmann

Note

The following content was generated by AI.

Describe the feature you'd like to see

Under the v1 convention, a reindexing join (outer, left, right) fills the
positions it creates: the linopy operand contributes the zero expression there,
and a constant operand contributes fill_value= (§10 of
doc/design/convention.rst). There is currently no way to say "create the
labels, but leave them absent"
— the zero-fill is unconditional for linopy
operands, and fill_value= applies to constant operands only.

That makes it the one remaining place where linopy picks a fill on the caller's
behalf, which is what §6/§7 otherwise rule out.

Reproducible example

Two variables on overlapping but different node sets, and a joint minimum:

import pandas as pd
import linopy as lp

lp.options["semantics"] = "v1"

m = lp.Model()
gen = m.add_variables(coords=[pd.Index(list("ab"), name="node")], name="gen")
imp = m.add_variables(coords=[pd.Index(list("bc"), name="node")], name="imp")

print(gen.add(imp, join="outer") >= 10)
Constraint (unassigned) [node: 3]:
----------------------------------
[a]: +1 gen[a]            ≥ 10.0
[b]: +1 gen[b] + 1 imp[b] ≥ 10.0
[c]: +1 imp[c]            ≥ 10.0

Rows a and c are a different constraint than the one written: the
requirement now falls entirely on whichever technology happens to exist there.
If the intent is "this applies where both are defined, and I still want the
result on the full node index so it aligns with the rest of the model", the
zero-fill is a silent modelling change.

The wanted behaviour is already reachable, just not through the join:

union = gen.indexes["node"].union(imp.indexes["node"])
print(gen.reindex(node=union) + imp.reindex(node=union))
LinearExpression [node: 3]:
---------------------------
[a]: None
[b]: +1 gen[b] + 1 imp[b]
[c]: None

Union coordinates, terms only where both sides live, and the constraint drops
at a and c under §12.

Proposal

Give that a spelling in the join API — a sentinel meaning "create the labels,
leave them absent":

gen.add(imp, join="outer", fill_value=lp.ABSENT)

fill_value=None cannot carry it, since that already means "use the operator's
default" (0). It also needs to be a distinct sentinel rather than np.nan,
because §5 makes a user-supplied NaN raise. fill_value= would have to become
meaningful for linopy operands too, where today it is accepted for constants
only.

Why this closes the gap

It fills the one empty cell in the 2×2 of where the absence came from × what
it should be worth
:

treat as zero keep absent
carried in (mask=, .where(), .shift()) a + b.fillna(0) a + b
created by a join a.add(b, join="outer") missing

With it, every combination is something the caller states explicitly and
neither mechanism carries a fill linopy chose itself. It also reconciles the
two ways of expressing "defined on a subset": masking propagates absence (§6)
while join="outer" zero-fills, and today only the masked path can express
both outcomes.

Scope note: this touches join-created positions only. Absence an operand
carries in stays untouched by the join, so §6 keeps absorbing and the algebraic
laws of §10 hold. Letting fill_value= reach carried-in absence would break
that and is explicitly not proposed here.

Related: #712, #713, #627.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions