Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
226 changes: 226 additions & 0 deletions .ai/categories/polkadot-protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -757,6 +757,179 @@ For detailed error management in XCM, see Gavin Wood's blog post on [XCM Executi
</div>


---

Page Title: Asynchronous Backing

- Source (raw): https://raw.githubusercontent.com/polkadot-developers/polkadot-docs/master/.ai/pages/reference-parachains-consensus-async-backing.md
- Canonical (HTML): https://docs.polkadot.com/reference/parachains/consensus/async-backing/
- Summary: Understand how asynchronous backing pipelines parachain block production, the protocol changes it introduces on the Relay Chain, and how parachains participate safely and efficiently.

Asynchronous backing (often shortened to ***Async Backing***) is a parachain [configuration](https://github.com/paritytech/polkadot-sdk/blob/f204e3264f945c33b4cea18a49f7232c180b07c5/polkadot/primitives/src/vstaging/mod.rs#L43) set by on-chain governance. It allows collators and validators to build *some* number of blocks ahead of the relay chain during the **generation** and **backing** stages of the [Inclusion Pipeline](/reference/parachains/consensus/inclusion-pipeline).

Async Backing improves throughput of the overall Polkadot Network by using coretime more efficiently, and enables the parallel processing needed for parachains to further scale throughput via [Elastic Scaling](/reference/parachains/consensus/elastic-scaling).

## Configurations
The following configurations can be set by on-chain governance, dictating how many blocks ahead of the relay chain a given parachain's collators can run:

* [`max_candidate_depth`](https://github.com/paritytech/polkadot-sdk/blob/f204e3264f945c33b4cea18a49f7232c180b07c5/polkadot/primitives/src/vstaging/mod.rs#L49): the number of parablocks a collator can produce that are not yet included in the relay chain. A value of `2` means that there can be a maximum of 3 unincluded parablocks at any given time.
* [`allowed_ancestry_len`](https://github.com/paritytech/polkadot-sdk/blob/f204e3264f945c33b4cea18a49f7232c180b07c5/polkadot/primitives/src/vstaging/mod.rs#L54): the oldest relay parent a parablock can be built on top of. A value of `1` means collators can start building blocks 6 seconds in advance.

## Synchronous VS. Asynchronous Processing



The Polkadot-parachain protocol originally operated in synchronous mode, where both collators and validators drew context exclusively from the relay parent of the prior parablock, which lives on the relay chain. This made the Backing and Generation steps tightly coupled to the prior parablock completing the entire inclusion pipeline. As a result, one parablock could only be processed every other relay block, with just 0.5 seconds assigned for execution.

```mermaid
---
displayMode: compact
config:
themeCSS: "
#item1 { fill: #450693; stroke: #450693; } \n
#item2 { fill: #8C00FF; stroke: #8C00FF; } \n
#item3 { fill: #FFC400; stroke: #FFC400; } \n
#r { fill: #eb4172; stroke:none; font-size: 20px; } \n
svg text { font-size: 20px !important; } \n
svg .sectionTitle { font-size: 20px !important; } \n #p1padTop { display: none; } \n

/* Hide ALL task labels by default */
text.taskText,
text.taskTextOutside,
[class*='taskText'] tspan { display: none !important; } \n

/* Show labels for the 'r' group (inside or outside, incl. tspans) */
text.taskText[id^='r'],
text.taskTextOutside[id^='r'],
text[id^='r'] tspan { display: inline !important; font-size: 20px; color: #000 !important; } \n

/* Keep section titles styled */
.sectionTitle { fill: #000 !important; font-weight: 700; font-size: 18px; } \n

/* Hide the first two section titles (F1, F2). Change indexes if needed. */
.sectionTitle:nth-of-type(1),
.sectionTitle:nth-of-type(2) { display: none !important; } \n

/* Also hide SPACING row labels on the left */
text.taskTextOutside[id^='p1padTop'] { display: none !important; } \n
"
themeVariables:
sectionBkgColor: '#fff'
gantt:
numberSectionStyles: 1
barHeight: 70
gridLineStartPadding: 100
---
%%{init: {"gantt": {"barHeight": 70 }}}%%
gantt
dateFormat YYYY
axisFormat %y
tickInterval '10year'

section F1
R1 : r, 1905, 1907
R2 : r, 1911, 1913
R3 : r, 1917, 1919
R4 : r, 1923, 1925

section F2
SPACING : p1padTop, 1901, 1924

section P1
X : item1, 1900, 1901
Backing : item2, 1901, 1906
Inclusion : item3, 1906, 1912

section P2
X : item1, 1912, 1913
Backing : item2, 1913, 1918
Inclusion : item3, 1918, 1924


```



The modern protocol now uses asynchronous backing, where both collators and validators have access to [Unincluded Segments](/reference/parachains/consensus/inclusion-pipeline) as an additional context source. The Backing and Generation steps are no longer coupled to the prior block completing the full inclusion pipeline. Instead, the prior parablock only needs to complete the generation step and be added to the Unincluded Segments before the next parablock can begin the Backing and Generation steps.

This results in one parablock being processed every relay block (instead of every other relay block), and allows for more time to execute during the Generation step (0.5s β†’ 2s).

```mermaid
---
displayMode: compact
config:
themeCSS: "
#item1 { fill: #450693; stroke: #450693; } \n
#item2 { fill: #8C00FF; stroke: #8C00FF; } \n
#item3 { fill: #FFC400; stroke: #FFC400; } \n
#r { fill: #eb4172; stroke:none; font-size: 20px; } \n
svg text { font-size: 20px !important; } \n
svg .sectionTitle { font-size: 20px !important; } \n #p1padTop { display: none; } \n

/* Hide ALL task labels by default */
text.taskText,
text.taskTextOutside,
[class*='taskText'] tspan { display: none !important; } \n

/* Show labels for the 'r' group (inside or outside, incl. tspans) */
text.taskText[id^='r'],
text.taskTextOutside[id^='r'],
text[id^='r'] tspan { display: inline !important; font-size: 20px; color: #000 !important; } \n

/* Keep section titles styled */
.sectionTitle { fill: #000 !important; font-weight: 700; font-size: 18px; } \n

/* Hide the first two section titles (F1, F2). Change indexes if needed. */
.sectionTitle:nth-of-type(1),
.sectionTitle:nth-of-type(2) { display: none !important; } \n

/* Also hide SPACING row labels on the left */
text.taskTextOutside[id^='p1padTop'] { display: none !important; } \n
"
themeVariables:
sectionBkgColor: '#fff'
gantt:
numberSectionStyles: 1
barHeight: 70
gridLineStartPadding: 100
---
%%{init: {"gantt": {"barHeight": 70 }}}%%
gantt
dateFormat YYYY
axisFormat %y
tickInterval '10year'

section F1
R1 : r, 1905, 1907
R2 : r, 1911, 1913
R3 : r, 1917, 1919
R4 : r, 1923, 1925
R5 : r, 1929, 1931

section F2
SPACING : p1padTop, 1901, 1930

section P1
X : item1, 1900, 1902
Backing : item2, 1902, 1912
Inclusion : item3, 1912, 1918

section P2
X : item1, 1906, 1908
Backing : item2, 1908, 1918
Inclusion : item3, 1918, 1924

section P3
X : item1, 1912, 1914
Backing : item2, 1914, 1924
Inclusion : item3, 1924, 1930

section P4
X : item1, 1918, 1920
Backing : item2, 1920, 1930
```


---

Page Title: Blocks
Expand Down Expand Up @@ -5071,6 +5244,59 @@ By defining weights, you can trade-off the number of transactions per second and
Westend is a Parity-maintained, Polkadot SDK-based blockchain that serves as a test network for the [Polkadot](#polkadot) network.


---

Page Title: Inclusion Pipeline

- Source (raw): https://raw.githubusercontent.com/polkadot-developers/polkadot-docs/master/.ai/pages/reference-parachains-consensus-inclusion-pipeline.md
- Canonical (HTML): https://docs.polkadot.com/reference/parachains/consensus/inclusion-pipeline/
- Summary: Understand each of the core components of the Inclusion Pipeline.

The inclusion pipeline is the multi-stage process through which every parachain block (parablock) is **validated** and **secured** before being finalized in the Polkadot relay chain.

This pipeline ensures that all parablocks meet Polkadot's security guarantees through progressive verification: each parablock passes through multiple validation stages with different validator sets, ensuring that invalid parablocks cannot be finalized even if some validators *or collators* are malicious/compromised.

By configuring [Async Backing](/reference/parachains/consensus/async-backing), a parachin can run this pipeline in parallel for many blocks, allowing for high throughput.

## Pipeline Stages

The inclusion pipeline consists of three main stages:

```mermaid
%%{init: {"flowchart": {"nodeSpacing": 40, "rankSpacing": 60}}}%%
flowchart LR
%% Keep the pipeline on one row (container is hidden)
subgraph Row[" "]
direction LR
G["Generation"] --> B["Backing"] --> I["Inclusion"]
end
style Row fill:none,stroke:none

%% Context: plain text (no box) pointing to both G and B
C["Context"]:::nobox
C -.-> G
C -.-> B

classDef nobox fill:none,stroke:none,color:inherit;
```
**Context**: Context of state is provided as input in order for collators and validators to build a parablock during the generation and backing stages, respectively. This context is provided by two sources:

* **Relay Parent**: The relay chain block which a given parablock is anchored to. Note that the relay parent of a parablock and the relay block including that parablock are always different. This context source lives on the relay chain.

* **Unincluded Segments**: Chains of candidate parablocks that have yet to be included in the relay chain, i.e. they can contain blocks at any stage pre-inclusion. The core functionality that [Async Backing](/reference/parachains/consensus/async-backing) brings is the ability to build on these unincluded segments of block ancestors rather than building only on ancestors included in the relay chain state. This context source lives on the collators.

**Generation**: Collators *execute* their blockchain's core functionality to generate a new block, producing a [proof-of-validity](https://paritytech.github.io/polkadot-sdk/book/types/availability.html?#proof-of-validity) (PoV), which is passed to validators selected for backing. The PoV is composed of:

- A list of state transitions called the **block candidate**
- The values in the parachain's database that the block modifies
- The hashes of the unaffected points in the Merkle tree


**Backing**: A subset of active validators verify that the parablock follows the state transition rules of the parachain and sign a [validity statement](https://paritytech.github.io/polkadot-sdk/book/types/backing.html?#validity-attestation) about the PoV which can have a positive or negative outcome. With enough positive statements (at least 2/3 of assigned validators), the candidate is considered backable. It is then noted in a fork on the relay chain, at which point it is considered backed, ready for the next stage of the pipeline.

**Inclusion**: Validators gossip [erasure code chunks](https://paritytech.github.io/polkadot-sdk/book/types/availability.html#erasure-chunk) and put the parablock through the final [approval process](https://paritytech.github.io/polkadot-sdk/book/protocol-approval.html) before it is considered *included* in the relay chain.


---

Page Title: Install Polkadot SDK Dependencies
Expand Down
Loading