What?

This is just me trying to wrap some ideas around and getting it somewhere where I can just point and say: “this is what I’m looking for”. I may change this with the time, as I clarify my ideas w.r.t this project.

Background

Inside Floresta, the floresta-wire crate is undoubtedly the most important one. It drives our chainstate, downloading blocks and transactions from the p2p network. It is very important for the correct functioning of floresta that floresta-wire works. A misimplemented wire might cause our node to be eclipse attacked. It also is a major hot path in the node workflow, and therefore, if it performs poorly, the overall performance is reduced.

The current implementation is very simple, and is divided in the following modules:

  • UtreexoNode: The actual node, it gets all our peer’s messages and dispatches to other modules, like mempool and chainstate. It hash two flavors, that are optimized for a given part of the node’s lifetime
    • RunningNode: Meant to take less resources, handle rpc requests and watch for new blocks as they are mined
    • IBDNode: Aggressively downloads blocks from peers, takes up as much resource as we can to speed the IBD process up.
  • Mempool: Holds unconfirmed transactions until it’s confirmation. Right now, we only hold transactions that we broadcast, not from others.
  • Peer: A low-level machinery that keeps our peer alive, handles requests like ping, and passes everything else to our node to handle.

The node flavor is selected with a zero-cost struct that implements the NodeContext trait. More importantly, RunningNode spawns a IBDNode before it can start. RunningNode only starts after IBDNode returns. This is very bad for UX, since things like pulling blocks from RPC are only possible with a RunningNode. Moreover, using BIP-158 compact block filters may let users to start using their nodes almost instantly.

Another reason why this design may be problematic, we want to implement parallel-sync. The way we’re going for is to create smaller chainstates, called PartialChainState that only holds part of the whole chain. We would have smaller nodes that holds a partial chainstate, and drives it to sync downloading all blocks and validating it. But the connections are handled by the node itself, and therefore each node needs to create multiple connections, that is wasteful.

A new workflow

I want a new workflow that:

  • (i) let users start using the node as quick as possible
  • (ii) selects the best chain, using the PoW fraud proof method
  • (iii) backfills blocks using parallel sync

With that in mind, here’s the new lifetime for a Floresta wire:

When it starts up, try to find the best chain. We connect with multiple peers, ask which chain are they on. If all peers agree, we select that chain and move to the running node. If, however, we ever find a fork. We should ask for the block that causes the fork in the heavier chain and fetch state for it. Once we get the state, we should validate that block. If valid, just pick the most-work chain, pick the other one otherwise.

Once we’ve found the best chain, we need to move into the running state. This initial process should be fast, so when we jump to running we download BIP-158 block filters and start working from that point. On the PoV of our user, a few minutes after he installed a Floresta node, it’s ready to point a wallet and start using. New blocks will be validated as normal.

If the user op-in, we’ll create one or more background sync nodes, that will each download an validate a range of blocks. Once they reach their tip, they exit and notifies our main node that it finished (and the validation state). We mark that in our main chainstate and, after all sync peers complete, we have a chain synced in with full node security.

If the user don’t opt into backfilling, the past blocks will have PoW fraud proof security which, under reasonable assumptions, is already very secure.

But why?

The worse part of setting a node up (UX-wide) is waiting hours, or even days, for your node to sync-up! Before that, you can’t use anything. This new workflow will allow users to start using their nodes in almost no time, requiring a negligible amount of resources, making it a awesome fit for embedding floresta in other wallets as the ultimate backend that gives you almost full node security, but with the lightweightness of an SPV.