DRAFT bounty: unbrick "live" blockchain and publish procedures

[ltfschoen]

The following treasury spending proposal and bounty spending proposal were prepared using the guide and included templates here: DataHighway-DHX/documentation#139

Treasury Spending Proposal

• Treasury Proposal: Request Funding for Bounty Proposal #???

• Proponent: [DOT Address]

HINT: Use the identity module on the address before submission of proposal to allow more certainty on the voting process

• Date: DD.MM.YYYY

• Bounty Spending Proposal ID: [INPUT HERE]

• Beneficiary: [DOT Address]

HINT: List the beneficiaries, how they will each contribute to the treasury spending proposal or the associated bounty spending proposal, and the proportion of the funds that will be distributed to them.

Bounty Spending Proposal Work

• Bounty Spending Proposal Title: Prepare procedures that may be used by any Substrate-based blockchain to revive a "live" blockchain so it finalizes blocks again whilst retaining as much existing on-chain storage state as possible

• Proponent: [DOT Address]

• Date: DD.MM.YYYY

• Requested allocation value: [INPUT HERE]

• Proposed Curator reward: [INPUT HERE]

• Beneficiary:

  • The beneficiary of the treasury spending proposal funds from the Polkadot treasury is the MXC Foundation gGmbH (on behalf of the DataHighway). MXC Foundation gGmbH shall fairly distribute a proportion of the received treasury spending proposal funds to relevant recipients that contribute to the above mentioned bounty. A proportion of the received treasury spending proposal funds shall be retained by the MXC Foundation gGmbH to recoup the costs associated with the preparation of the bounty management and tender documentation, and the preparation, finalization, and publication of associated procedure(s) mentioned above.

• Short description:

  • Context of the bounty:

    • Context:
      • A bounty is proposed to formally published procedures that document how any "live" Substrate-based blockchain that has been bricked may be revived so it finalizes blocks again whilst retaining as much existing on-chain storage state as possible. The Curator is proposed to be the MXC Foundation gGmbH (on behalf of the DataHighway) who shall prepare a bounty and invite the community to review and comment on any draft procedure(s), suggest improvements, or submit new procedures and solutions. The procedures shall be finalized and publicised by the Curator following satisfactory completion of the bounty. Effective execution of the bounty shall be measured by verifying that the procedure(s) may be successfully applied to reviving the DataHighway's "live" Westlake network. This will indicate that it may be applied to benefit any other live Substrate-based blockchain.

  • Topic and categories of the bounty:

    • Topic:
      • "unbrick"
    • Categories:
      • "unbrick", "live blockchain", "hard spoon", "time warp", "fork", "spoon", "slashing", "babe", "aura"

  • Problem statement:

    • Problem Statement:
      • There are no formally published procedures that document how any "live" Substrate-based blockchain that has been bricked may be revived so it finalizes blocks again whilst retaining as much existing on-chain storage state as possible. There are reports of some Substrate-based blockchains being bricked who were using Babe and shut down their validators for more than one epoch when increasing the server storage capacity.
    • Topic and Category Details:
      • "time warp" - This is an option to revive a Substrate-based blockchain by coordinating network-wide machine time rollbacks, where you reverse the system clocks on all the validators to before the epoch where the "unexpected epoch change" occurred where it was previously able to produce and finalize blocks, and then repeat that until every epoch has a valid block and the time is back in sync. It may be necessary to write migrations using this approach. The procedures created in this bounty should explain how to sufficiently reverse the system clocks and how to resolve any slashing that may have occurred just before a blockchain was bricked, and how to write migrations. Kusama historically solved the issue of no blocks being generated for an epoch by using the infamous "time warp" approach that is published here https://medium.com/polkadot-network/kusamas-first-adventure-2cd4f439a7a4 (reference: @jam)
      • "hard spoon" (re-genesis) - This is an option that was recommended by Rob Habermeier. It has been successfully used by @h4x from Phala to upgrade a live "mainnet" runtime without purging any state. It was unsuccessfully used by @ltfschoen to hard spoon the DataHighway as whilst some of the old chain state was retained, no blocks were generated on the new blockchain. It may be necessary to write migrations using this approach to extract raw storage state from old chain so its storage values are not lost. There are two known tools that may be used to possibly achieve a hard spoon:
        • chainSpecUtils https://github.com/Phala-Network/phala-blockchain/blob/master/scripts/js/src/bin/chainSpecUtils.js is a tool used by @h4x from Phala Network that allows bootstrapping a new substrate chain with the current state of a "live" chain, but with more features and less dependency than fork-off-relaychain. To use the tool you briefly stop the "live" blockchain and use its binary to run the export-state command to dump all the storage to a chain-spec .json file that is then used as the "source" file when running the chainSpecUtils.js script, which then generates a "target" chain-spec file from it that may then be used to start a new hard spoon of the blockchain with the old state imported
        • Fork-off-relaychain https://github.com/Phala-Network/fork-off-relaychain is an alternative approach to a hard spoon

  • Proposal objective and solution(s):

    • Goals and Objectives:
      • To prepare and publish procedures that satisfy the "Problem Statement" and revive the DataHighway "live" Substrate-based blockchain using one or more of the procedures
    • Success Factors:
      • Hard spoon procedure documented and publicised on IPFS
      • Time warp procedure documented and publicised on IPFS
      • Other procedures documented and publicised on IPFS
      • DataHighway "live" Substrate-based blockchain is able to be revived using one or more of the procedures
    • Milestones & Budgets & Tools:
      • Cost breakdown:

        • Hard spoon procedure

          • ??? DOT (??? %) - Curator prepares of draft procedure (if any).
            • Draft Procedure for the Hard Spoon approaches
          • ??? DOT (??? %) - Bounty Worker(s) prepare and submit bounty brief. Curator assesses each brief and rewards each submission primarily based on how useful it is and also considering to what extent it attempts to satisfy the Bounty Work Assessment Criteria
          • ??? DOT (??? %) - Curator chooses to use the work from a variety of Bounty Worker(s) and determines how to splits the rewards and documents that
          • ??? DOT (??? %) - Curator improvements to the draft procedure based on the briefs
          • ??? DOT (??? %) - Curator publicises the procedure on IPFS

        • Time warp procedure

          • Same as above. Note: No draft procedure is available

        • Other procedures

          • Same as above. Note: No draft procedure is available

        • Total cost ??? DOT (100 %)

    • Solution Details:
      • Any Substrate-based blockchain that deems it necessary to fork or possibly time warp their blockchain would benefit