Skip to content

askrene: opt-in circular (self-rebalance) routes in getroutes#9286

Open
ksedgwic wants to merge 2 commits into
ElementsProject:masterfrom
ksedgwic:circular-askrene4
Open

askrene: opt-in circular (self-rebalance) routes in getroutes#9286
ksedgwic wants to merge 2 commits into
ElementsProject:masterfrom
ksedgwic:circular-askrene4

Conversation

@ksedgwic

@ksedgwic ksedgwic commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

This adds an opt-in allow_circular parameter to askrene-getroutes. When set, source == destination requests a self-rebalance cycle -- a route that leaves the source, traverses the network, and returns -- for moving liquidity between one's own channels. Because the cycles come from the MCF solver, a single call can move liquidity out of several channels and into several others in one pass, rather than rebalancing one channel pair per attempt. Without the flag nothing changes: source == destination is still rejected with the same error as today, so existing callers are unaffected.

The change stays out of the solver. It uses the standard node-splitting construction from network flow: at the gossmap-localmods layer, the source keeps only its outgoing edges and a synthetic sink receives mirrored copies of its incoming edges, so the MCF sees an ordinary source-to-sink problem and the solver child runs unmodified. The commit messages have the details, including how the returned route represents the closing hop.

Motivation: rebalancing tools currently have to do their own routing for circular payments; this lets them use the same solver as everything else. We have been running this patch against live rebalance traffic on mainnet (and signet).

Tests cover the flag-off (unchanged) behavior, a deterministic synthetic cycle, mirror-scid collision handling, and a self-rebalance over a real gossmap snapshot.

Changelog-Added: JSON-RPC: askrene-getroutes has a new allow_circular parameter to request self-rebalance (source == destination) routes.

@ksedgwic ksedgwic requested a review from cdecker as a code owner July 6, 2026 19:42
@ksedgwic

ksedgwic commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator Author

@Lagrang3

@Lagrang3 Lagrang3 self-requested a review July 6, 2026 20:26
ksedgwic added 2 commits July 8, 2026 12:32
source == destination is rejected as invalid input: a node cannot pay
itself a normal route.  This adds an opt-in, allow_circular: when set,
source == destination instead requests a self-rebalance cycle -- a route
that leaves source, traverses the network, and returns to source -- for
moving liquidity between one's own channels.

Without allow_circular nothing changes: source == destination still fails
with "source and destination must be different".  Only callers that
explicitly opt in are affected, so this is purely additive for every
existing caller.

The cycle is produced by splitting the source node into two distinct nodes
at the gossmap-localmods layer, before the solver child forks:

  - source itself keeps only its outgoing edges (its real channels, modulo
    any masks the caller applied via layers).
  - A synthetic us_in node receives only incoming edges: one mirror per
    still-enabled (peer -> source) direction, with the real channel's
    capacity, fees, cltv delta and htlc bounds copied over.

The MCF then solves a regular source -> us_in flow.  No direct edge
connects them, so the algorithm must traverse the network, and every route
ends at us_in with a non-empty hop sequence.  The whole change lives in the
request-setup layer in askrene.c; the solver child (MCF / BFS / Dijkstra /
get_flow_paths / find_path_or_cycle / substract_flow / substract_cycle /
routing-cost machinery and route serialisation) runs unmodified, because
us_in is a real entry in the mod-augmented gossmap, indexed in the usual
range and picked up by every loop over gossmap_max_node_idx.

The trailing (peer -> us_in) hop is KEPT in each returned route: its
amount_out_msat is the delivered amount, and its short_channel_id_dir and
node_id_out are sentinel placeholders for the synthetic us_in node.  The
caller replaces them with the real closing channel and its own node id when
building the payment, closing the cycle.

Fake mirror scids count up from 0x0x0, skipping any scid already present in
the gossmap or the request's localmods: block 0 cannot hold a real mainnet
channel, but synthetic gossmaps and layer-created channels can occupy any
scid, and a clash would silently update the occupant instead of creating
the mirror.  Allocation failure (or any other mirror-construction failure)
fails the getroutes command cleanly rather than aborting, since cln-askrene
is an important plugin and an abort would take lightningd down with it.

Changelog-Added: JSON-RPC: `getroutes` `allow_circular` returns a self-rebalance cycle when `source` equals `destination`.
Add integration coverage for the allow_circular self-rebalance path:

- test_getroutes_circular: a focused test over a synthetic gossip store
  with one deterministic cycle (us -> 1 -> 2 -> us).  Asserts that
  source == destination without allow_circular is rejected, and that with
  allow_circular the returned route keeps its trailing synthetic closing
  hop (node_id_out == the fake us_in sentinel, scid == the first mirror
  0x0x0, amount_out_msat == delivered).  Also covers mirror-scid collision
  handling: a layer-created channel squatting on 0x0x0 plus the store's own
  real block-0 channel (gossmap-compress assigns block = index + node1)
  force the allocator to skip to 0x2x0.

- test_real_data_circular: the canonical clboss-xmovefunds shape over a
  real mainnet gossmap snapshot.  Mask the source node down to a single
  drain (source-out) channel and a single distinct fill (dest-in) channel,
  then self-rebalance.  The mask makes the route deterministic at the
  endpoints: every part must leave via the drain channel and return via the
  mirrored fill channel into us_in.

Real hops in the synthetic test are pinned by node id rather than
short_channel_id_dir, since the scids the test gossip store assigns are a
harness-internal detail.

Changelog-None
@ksedgwic ksedgwic force-pushed the circular-askrene4 branch from a287403 to 65d4a82 Compare July 8, 2026 20:01
@ksedgwic

ksedgwic commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator Author

rebased, hoping for a good CI run ...

@Lagrang3

Lagrang3 commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

This is an excellent PR!

  • commits document to a great extend the changes introduced,
  • python tests included,
  • the changes are mostly nicely contained in a single function inject_circular_fake,
  • the author added useful comments in the code,
  • the documentation is excellent.

],
"default": "100"
},
"allow_circular": {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can be increasing the number of flags forever. That's why, I guess, Rusty preferred
to introduce these as auto layers. So I would prefer to adhere to that pattern.
Instead of a boolean option this would be a auto.allow_circular layer.

Comment thread plugins/askrene/askrene.c
Comment on lines +816 to +834
/* Self-rebalance handling: when source == destination, splice
* a fake "us_in" destination node into localmods and mirror
* the still-enabled (peer -> source) directions onto it.
* Algorithms see a regular s -> t flow problem and run
* unchanged. Must precede gossmap_apply_localmods so the
* augmented mods are picked up by the apply below. Only
* reached for source == destination when the caller passed
* allow_circular (json_getroutes rejects it otherwise). */
err = inject_circular_fake(info, askrene->gossmap, localmods);
if (err) {
/* localmods are not applied at this point, so fail
* directly: the fail: path would try to remove them. */
return command_fail(cmd, PAY_ROUTE_NOT_FOUND, "%s", err);
}
if (info->circular)
cmd_log(tmpctx, cmd, LOG_DBG,
"Circular routing: spliced fake us_in destination "
"node, mirrored peer -> source channels into it");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this section, isn't it better to have the following pattern: ?

if(node_id_eq(source, dest)){
    // log source==dest -> needs circular ...
    err = inject_circular_fake(...);
    if(err){
        // log needs circular but failed for reason: {err}
        return command_fail(...);
    }
    // log circular routing prepared ...
}

@Lagrang3

Lagrang3 commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

I remember @daywalker90 raised a question about circular rebalancing using askrene-getroutes.
AFAIK a channel rebalance doesn't need to be reliable, we can afford many trials, but does need to be cheap in fees.

I would like to also point out that the primitive API already present in askrene allows for a circular rebalance
just like @ksedgwic has done here. The user would need to create a fake node, fake channels and so on and at the end
translate back into real network terms in order to build the onions.

My suggestion is to do two things:

  1. Create a layer in askrene auto.optimize_fees to indicate the desire to optimize for fees. We don't necessarily need to implement the back-end immediately.
  2. Create a new json rpc at a higher level (either a new plugin or appended to an existing plugin, it can also be added to the plugins repository) that uses askrene primitives to do a circular rebalance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants