Skip to content

v1: merge(dim=...) rejects aux coords indexed by the concat dim itself #896

Description

@FabianHofmann

Note:
Written with help of claude

Issue Description

Under v1 semantics, merge raises §11 aux-coord conflict for coordinates whose only dimension is the dimension being concatenated. Those coords aren't in conflict — they should be concatenated along with the dim.

This is the last thing blocking PyPSA from using linopy's multi-key groupby natively (see #717 / PyPSA/PyPSA#1829).

Reproducible Example

import pandas as pd, linopy as ln
from linopy import options

options["semantics"] = "v1"

m = ln.Model()
gens = pd.Index(["g0", "g1"], name="name")
links = pd.Index(["l0", "l1"], name="name")
p = m.add_variables(coords=[gens], name="Generator-p")
f = m.add_variables(coords=[links], name="Link-p")

gen_keys = pd.DataFrame({"bus": ["b0", "b1"], "carrier": ["wind", "solar"]}, index=gens)
link_keys = pd.DataFrame({"bus": ["b1", "b2"], "carrier": ["dc", "dc"]}, index=links)

per_gen = p.to_linexpr().groupby(gen_keys).sum()
per_link = f.to_linexpr().groupby(link_keys).sum()
# both: flat integer `group` dim, with `bus`/`carrier` as ("group",) aux coords

ln.merge([per_gen, per_link], dim="group")


### Actual


ValueError: Auxiliary coordinate 'carrier' has conflicting values across operands:
left=['wind', 'solar'], right=['dc', 'dc']. xarray would silently drop the
conflict; linopy raises so the caller resolves it. Use `.drop_vars('carrier')` to
remove the coord, `.assign_coords(carrier=...)` to relabel one side, or
`.isel(..., drop=True)` if the coord was introduced by a scalar isel.


(Which of the two coords gets named varies between runs`conflicting_aux_coord` iterates a `set` of coord names.)

None of the three suggested resolutions is what the caller wants: the coord must survive the merge, with each operand's values kept for its own slice.

Expected Behavior

bus and carrier are 1-D along group, the concat dim. Concatenating them is well-defined and is what legacy does today:

group:   [0, 1, 0, 1]
bus:     ['b0', 'b1', 'b1', 'b2']
carrier: ['wind', 'solar', 'dc', 'dc']

Cause

expressions.merge calls enforce_aux_conflict(data) (expressions.py:3209) without passing dim, so semantics.conflicting_aux_coord treats every non-dim coord alike. Its exclusion is name not in d.dims, which correctly skips dim coords but not coords indexed by the concat dim.

Proposed fix

Give enforce_aux_conflict the concat dim and exempt coords whose dims == (concat_dim,) — they are concatenated, not reconciled. The §11 check stays fully in force for every other aux coord, including ones on a shared non-concat dim, which is the case it exists for.

Verified by monkeypatching exactly that exemption into the reproducer above: the merge succeeds, produces the coords shown under Expected, and a downstream merged.groupby("bus").sum() gives ['b0', 'b1', 'b2'].

Why it matters

With a flat group dim, the grouping keys can only live as aux coords on that dim — that's the shape v1's own groupby emits, and the shape the legacy deprecation notice tells callers to migrate to via .reset_index("group"). So concatenating two grouped expressions is the natural next operation after grouping, and it is currently unreachable under v1 without stripping the keys, renumbering the group labels by hand, merging, and reattaching. PyPSA carries ~20 lines doing precisely that; the fix would delete them.

Installed Versions

Details Replace this line.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions