Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,8 @@ add_library(pineforge STATIC
src/engine_stream.cpp
src/engine_strategy_commands.cpp
src/engine_trade_accessors.cpp
src/reservation_expansion.cpp
src/compat/pine/reservation_expansion.cpp
src/pending_order_mirror.cpp
src/magnifier.cpp
src/math.cpp
Expand Down
1 change: 1 addition & 0 deletions docs/Doxyfile
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ INPUT = ../include/pineforge/pineforge.h \
pages/install.md \
pages/integration-cmake.md \
pages/lifecycle.md \
pages/fill-model.md \
pages/streaming.md \
pages/live-surface.md \
pages/report-schema.md \
Expand Down
87 changes: 87 additions & 0 deletions docs/frozen-market-instruction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Frozen Pine market instruction

This is a bounded representation migration of an existing Pine compatibility
policy. `compat::pine::FrozenMarketInstruction` replaces six independent
PendingOrder sidecars with one exclusive source operation. It does not make
the existing cohort selector a generic native execution contract.

## Source operation and state ownership

| Role | Live owned facts | Existing facts consumed |
| --- | --- | --- |
| Ordinary | None | Ordinary sizing and execution state |
| Transaction | Own units; frozen total transaction units | Requested side; immutable placement-cap snapshot |
| TargetedClose | Explicit target entry ID | Original QuantityRequest Units; immutable created-position side |

Only complete role construction and whole-operation revocation are supported.
Transaction and TargetedClose are mutually exclusive; there are no independent
membership, cap-retention or close-side switches. Revocation destroys the active
payload. A targeted close has no second quantity authority: later reservation
changes leave QuantityRequest's original Units amount intact.

A source transaction captures the same expression as before:

own + opposite position held net of earlier same-bar closes
+ opposite pending entries' own units

Its retained-over-cap property is true only when the Transaction role is active
and `over_pyramiding_cap_at_placement` is true. The former duplicate calculation
and the placement snapshot use the same position-side/count/cap expression;
none of those inputs changes between the two capture sites. A targeted close's
buy direction is its captured position side being SHORT.

The own amount must be positive and finite. A positive-infinite total caused
by source-sum overflow remains representable because the prior source capture
allowed it. This is not a validated native execution amount. The existing
finite-total execution guards still exclude it from frozen transaction sizing;
ordinary sizing follows the same prior path. No new overflow admission policy
is introduced by this structural change.

## Compatibility boundary retained

The selector, full-book admission and execution scopes are unchanged: historical
close calculation; FIXED sizing; default FIFO; pyramiding at most one; no costs,
slippage, magnifier, stream or risk extensions. Exact source conditions remain
in `same_bar_market_tx_scope_is_live` and its command producers.

The complete pending book must contain only accepted operations and at most
two MARKET entries with distinct IDs. An unrelated priced/raw/bracket/close-all
order, a third market entry, or leaving the scope revokes the source operations
for the whole book. Subsequent native dispatch uses the prior ordinary paths.
Same-ID replacement and named cancellation/recreation construct a fresh
operation with a new incarnation and current source snapshots.

TargetedClose is intentionally not named ReducePosition. Existing Pine behavior
can materialize an opening artifact lot if the originally targeted side is gone
and a matching entry remains pending later in the source execution order. That
path remains intact. Lowering this source operation into separate generic
native transaction/reduction instructions remains future adapter work.

This migration removes three direct PendingOrder booleans, rather than claiming
that an entire engine has reached a four-flag goal. The Pine cohort selector and
source-operation discriminator survive and must remain in compatibility audits.
No Pine selection predicate has been moved into the native contract by renaming.

## Observation and compatibility

The existing 142-field public pending-order mirror is append-only. Every old
field retains its type, size and offset, including the old full-prefix trailing
padding. The six old `sbmt_*` outputs are derived projections only; they are not
writable engine state. Ordinary/revoked operations project false flags and NaN
legacy quantity values exactly as the old inactive sidecars did.

Six fields append the source kind, own/total units and target ID (char[64],
truncation indicator and full string hash). The broker hash folds the kind and
only its live payload; QuantityRequest and placement facts are already folded
at their owning order. Metadata mutation checks refuse hidden nested fields,
changed variant/enum alternatives, removed or conditional folds, and waivers.
The aggregate representation uses internal engine/PendingOrder ABI v7 and
broker/stream fingerprint epoch7. The complete mirror has155 fields: the142
shipped fields, seven reservation facts and these six Pine instruction facts.
Public C ABI4 and stream API1 are unchanged.

`test_frozen_market_instruction` uses literal price-100 fixtures and direct
native command transitions. It pins role construction, amounts in both
orientations, quantity ownership, whole-book revocation, replacement,
cancellation, hash sensitivity, string truncation and the full public prefix.
It does not use external tapes, expected provider trades or a grader.
39 changes: 25 additions & 14 deletions docs/pages/abi-stability.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,34 +112,45 @@ notice:
- The shape of internal log lines (use them for humans, not parsers).

Rebuild generated and native C++ objects against matching engine headers and
runtime. Owner-bound leg activation changes `PendingOrder` layout again; both
`PendingOrder` and `BacktestEngine` use internal `engine_script_run_v6`. Exact
shipped149/v5 headers are compiled before mismatch links. Native and
generated-style old engine callers, plus standalone priority callers using
`vector<PendingOrder>` without any engine-method reference, must reject the
v6 archive. Matching current and historical symbol-control links must succeed.
Earlier base38/v2, f864/v3 and c45/v4 controls remain. A compilation failure
cannot masquerade as stale-layout rejection; no pairing executable runs.
runtime. Exact-owner reservation expansion and typed Pine market instructions
change `PendingOrder` layout after
shipped ff54/v6; both `PendingOrder` and `BacktestEngine` now use internal
`engine_script_run_v7`. Exact shipped ff54/v6 headers are authenticated before
compilation. Old native and generated-style engine callers and standalone
priority callers using `vector<PendingOrder>` must reject the v7 archive.
Matching current and historical symbol-control links must succeed. Earlier
base38/v2, f864/v3, c45/v4 and shipped149/v5 controls remain. Every translation
unit must compile before a mismatch link is accepted; no pairing executable runs.

The standalone `ReservationExpansionCapture`, `ReservationExpansion` and
`ReservationGrowthSource` types establish their first C++ contract in the inline
`reservation_expansion_v1` namespace, including out-of-line methods. This version
is independent of the containing engine epoch. A frozen unshipped bf312 header
closure provides a negative control for the preceding unversioned draft; these
types did not exist in shipped ff54. Standalone method and capture-argument
pairings reject both stale directions, with matching positives. Inline accessors
still require matching headers; namespace versioning is not a serialization ABI.

`PINEFORGE_HAS_SCRIPT_RUN_PREPARE_V1` remains 1: it describes the existing hook
capability, not the class layout version. Regenerate and rebuild a strategy
module to obtain complete script-state reset; replacing an archive does not
retrofit an old module. Public C function signatures, `PF_ABI_VERSION` (4),
and `strategy_stream_api_version()` (1) are unchanged. The pending-order v1
mirror keeps all 128 shipped149 field names/types/offsets and its full old prefix;
mirror keeps all 142 shipped ff54 field names/types/offsets and its full old prefix;
new typed facts append, and removed native booleans survive only as read-only
derived outputs. Size-limited reads keep old callers within their buffers.

Namespace versioning protects referenced internal C++ symbols; it does not
validate an erased `pf_strategy_t` handle. Use a handle only with functions from
its creating strategy module. A fully self-contained old module can still use
its own matching runtime; this check does not turn it into a v6 module.
its own matching runtime; this check does not turn it into a v7 module.

The integrated representation advances the broker fingerprint domain to
`pineforge-broker-state/v6` and stream fingerprint version to 6. These identify
changed serialized leg owner/bounds and Pine placement evidence, alongside
existing quantity, predecessor and birth facts. The Pine component schema remains 1; it is
independent of the aggregate fingerprint version. Prior v2/v3/v4/v5 fingerprints are
`pineforge-broker-state/v7` and stream fingerprint version to 7. These identify
changed serialized reservation capture/source ownership and Pine instruction
roles/payloads, alongside existing
leg activation, quantity, predecessor and birth facts. The Pine component schema remains 1; it is
independent of the aggregate fingerprint version. Prior v2/v3/v4/v5/v6 fingerprints are
not comparable. Fingerprints are replay checks, not serialized checkpoints or
complete hashes of private strategy state. The native runner already binds
its strategy-library SHA; its ledger format and Python provenance fingerprints
Expand Down
5 changes: 3 additions & 2 deletions docs/pages/exit-leg-activation.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,8 @@ other historical execution rules remain separate work. The effective-level
accessor still reports resolved levels rather than masking eligibility.

The internal object layout changes and requires matching C++ headers/library.
Both PendingOrder and BacktestEngine now use internal v6 with broker/stream
fingerprint v6. Exact shipped149/v5 engine and standalone PendingOrder clients
The activation slice shipped with internal v6. Subsequent reservation ownership
integration advances PendingOrder and BacktestEngine to v7 with broker/stream
fingerprint v7. Exact shipped ff54/v6 and earlier149/v5 standalone PendingOrder clients
are checked by compile/link-only mismatch controls with matching positives.
Public C ABI 4, stream API 1 and pending mirror version 1 are preserved.
122 changes: 102 additions & 20 deletions docs/pages/fill-model.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,30 @@
# Broker state and fill ownership

The broker processes a source-ordered command book against a price path. Its
physical position, logical close claims, reserved quantities and pyramiding
capacity are separate ledgers. A Pine entry ID is a reusable name; the order
incarnation and position cycle identify the owners of those ledgers.

This document describes the first extracted ownership model. Other order,
reservation and path state is still represented in `BacktestEngine` and
`PendingOrder`; their consolidation is ongoing. The current economic rules
retain their tested domains. Moving a rule into a type does not establish
that it describes every TradingView configuration.
# Order execution model {#fill_model}

PineForge is being developed as an independent C++ backtest and forward-execution
engine. Its native contracts describe orders, exposure, reservations and actual
execution events. Pine source interpretation belongs to the frontend boundary.
TradingView comparisons are compatibility evidence for a declared configuration;
they do not define every native operation.

The migration is incomplete. The current engine still contains Pine-specific
admission and scheduling rules, and the ordinary bar scan and callback-driven
scan are separate. The responsibilities below guide their consolidation without
claiming that a single unified scheduler already exists.

## Responsibilities

| Submodel | Owned facts and transitions | Boundary |
|---|---|---|
| Command admission | Original quantity request, placement observation, accepted instruction identity, rejection cause | A source frontend interprets its calls; native admission operates on explicit transaction or position intent. |
| Exposure and reservation | Position cycle, physical quantity, reservation owner, admitted growth sources, committed growth and retirement | A proposed add is not an executed add. There is one mutable reservation capacity. |
| Trigger lifecycle | Stop/limit/trail definitions, activation bounds and owner binding, stop-limit activation and trail progress | An unready leg cannot supply a price or authorize a ready sibling. |
| Execution schedule | Candidate identity, event coordinate, dependency and explicit priority | Ordering does not grant admission, create quantity or activate a leg. |
| Settlement and observation | Executed quantity, physical lots, paid fees, OCA effects, risk follow-up and callbacks | Publish observations after the corresponding state change; report projections must not change admission equity. |

These are responsibilities, not five independently switchable modes. Direction,
order kind, quantity intent and an outstanding owned claim are normal domain
state. Encoding unrelated permissions as enum values or moving them to another
object does not simplify the model.

## What qualifies as a generic flag

Expand Down Expand Up @@ -37,6 +52,32 @@ scores are regression evidence, not a justification for keeping a compensating
flag. Actual conflicting TradingView observations belong in an anomaly review
record, separate from an unknown rule or an engine defect.

## Identity and quantity

Every accepted pending object receives a fresh `incarnation`. A user-supplied ID
can be reused, and same-ID replacement can retain queue priority; neither implies
that the new object owns the old object's claims. `replaced_order_incarnation`
records the exact predecessor. A position cycle similarly distinguishes two
positions that happen to have the same direction.

`QuantityRequest` retains the original Units, Fraction or All request and its
reservation basis. The executable `PendingOrder::qty` can change through OCA
reduction or committed reserved growth. That does not rewrite the original
request, reclassify its historical partial/full meaning, or create another
quantity ledger.

An expansion capture belongs to an exact EXIT object and exposure cycle. Selected
source orders carry the receiving EXIT incarnation. After an actual primary fill,
the receiver gains only the positive same-side quantity increase belonging to its
captured cycle. A canceled or logically retired receiver cannot redirect that
growth to another exit with the same user ID.

The first later successful entry-like admission closes the capture's population.
Canceling that later admission does not erase its historical cause. Previously
captured sources can still pay their exact receiver; later unrelated adds do not
join the capture. An old-cycle capture loses live-All authority while its ordinary
finite reservation remains available for the normal settlement path.

## Opening checkpoint

An accepted opening or add can create an opening-affordability checkpoint.
Expand Down Expand Up @@ -77,6 +118,27 @@ Four historical long/short lifecycle labels had no economic consumers. They
have been removed, along with their producers; the numerical floor-zero rules
and their trade fixtures remain. Those labels are not alternate model states.

## Current dispatch sequence

The ordinary pending-order scan updates risk state, processes due opening work,
finalizes source cohorts, updates trailing/relative prices and orders the book.
It then classifies and matches an exact pending handle. A pre-exit margin slice
can change the book between matching and dispatch, so the handle is resolved
again before applying the selected order.

The primary fill updates physical exposure. Reservation growth is settled at the
existing post-primary checkpoint, before that order's OCA and risk follow-up.
Logically retired objects cannot be dispatched again while awaiting compaction.
Callbacks use committed fill events; a declined or zero-effect attempt is not
itself a fill event. Pine's optional quota interpretation separately observes
the attempt stages its own contract specifies.

There is no universal `risk > exit > entry` priority. A forced action and a user
order have event coordinates, and an earlier event must be accounted before a
later one. Forced liquidation cannot be suppressed by an unready user stop.
An earlier risk fill can also invalidate the quantity or owner of a later
candidate, requiring another resolution of its identity and eligibility.

## Distinct execution domains

An opening receipt is broker-local. It is not an identity for every output:
Expand All @@ -94,6 +156,28 @@ different information. The opening ownership extraction changes none of their
path, callback or warmup policies. Terminal-close deferral is still selected
by the existing financial policy; it has not been generalized by this model.

## Remaining consolidation

The complete scheduler should build immutable candidates, choose the earliest
eligible event coordinate, satisfy causal dependencies, and then apply explicit
priority and stable submission identity. Losing candidate scans must not commit
trigger activation. Dependency cycles must be reported rather than hidden by
another pairwise preference.

The existing two-sort arrangement does not yet provide that contract. In
particular, the sibling comparator has reproduced ordering-law failures for
unrelated interleavings and mixed trailing/non-trailing exits. Grouping by owner
alone does not repair the mixed-leg case. Replacing these sorts is separate from
the reservation-ownership change and requires its own native ordering tests and
Cloud compatibility assessment.

A native reduction must never create new exposure. Some existing Pine close
interpretations can produce a new transaction after their old target disappears;
that interpretation must remain explicit at the source boundary until lowering
to native operations is complete. The current engine also has separate forward
ingress, callback, affordability and per-leg lifecycle work. This page does not
claim those migrations or the large-function cleanup are finished.

## Verification and compatibility

Literal state tests exercise consume-before-use, exempt presence, replacement,
Expand All @@ -103,11 +187,9 @@ fixed reference population is required before reporting parity preservation.
Hashes supplement those comparisons; a matching fingerprint is not a proof
that all hidden strategy state is equal.

The new receipt replaces protected C++ members and changes the fingerprint
representation. Rebuild generated and native modules against matching headers
and runtime. The cap extraction advances the internal class namespace to
`engine_script_run_v3`, broker hash domain to `pineforge-broker-state/v3`, and
stream fingerprint prefix to 3. These pairing/serialization versions change
no financial rule; public C signatures, POD layouts and API versions remain
unchanged. See
[ABI stability](abi-stability.md).
Internal C++ layouts and broker/stream fingerprint domains are versioned separately
from the public C ABI and its append-only pending-order mirror. Rebuild generated
and native modules against matching headers and runtime. See
[ABI stability](@ref abi_stability) for the current versions and stale-object
pairing checks. A lower Boolean field count is not proof that every compatibility
policy or possible order history has been covered.
Loading
Loading