Shards and parallelized execution
Definition
Shards are smaller chains that are usually responsible for managing a portion of the global state of the main chain. In =nil;, there exist two types of shards.
- Execution shards process transactions that target the accounts located on these shards
- The consensus shard coordinates execution shards and submits ZKPs of the global state to Ethereum
Execution shards
All work related to processing transactions, managing cross-shard communications, storing accounts, and overseeing the local state occurs at the execution shard level. Execution shards are also tasked with managing transaction passing checks.
In =nil;, only transactions that are sent by external actors (e.g., users or dApps) go to the mempool before being executed. Transactions that are exchanged between smart contracts are not added to the mempool.
Although each shard has its own mempool, there are no restrictions for the participants on what shard they can cooperate with.
Transaction passing checks
Transactions in =nil; can either be incoming or outgoing:
- Incoming transactions trigger contract execution.
- Outgoing transactions are spawned by contracts as part of execution. These outgoing transactions are sent to other contracts located on various shards.
To support cross-shard communications, execution shards must record all necessary outgoing transactions posted in other execution shards.
Each transaction contains information about its destination account. If a shard sees that a transaction targets one of the accounts in the shard, the shard must include this transaction in its next block. Such outgoing transactions are called necessary transactions and new blocks are considered invalid unless they include all necessary transactions.
As the number of execution shards grows, this process may be changed so that shards are only tasked with tracking transactions from their neighbors. The closeness of shards can be determined by their Hamming distance.
Internal vs. external transactions
In =nil;, transactions are divided into internal and external.
Internal transactions cover all types of async communication between smart contracts on the network regardless of the shard where they are deployed. When Contract 1 (deployed on Shard 1) calls a function in Contract 2 (deployed on Shard 2), Shard 1 spawns an internal transaction and Shard 2 processes this transaction. Note that sync calls between contracts do not produce any transactions.
External transactions can originate only in one case: when any entity that exists outside of the cluster (e.g., a dApp) sends a transaction to any smart contract on the network.
For internal transactions, the transaction itself is the 'payer' entity. An internal transaction spends its value to cover processing costs. Note that the transaction only pays for the initial call to a 'top-level' smart contract. If the transaction produces a chain of sync calls, the transaction will only pay for the first call in the chain.
For external transactions, the receiving contract is the 'payer' entity. To process external transactions, the receiving contract must implement a special function that uses arbitrary logic to verify these transactions. If this function is absent, the contract will only accept internal transactions.
Regardless of the transaction type, the 'payer' is tasked with purchasing gas equal to the transaction gasLimit. The transaction fails if the balance of the 'payer' is insufficient. If execution is successful, the remaining gas is refunded to the 'payer'.
Consensus shard
In contrast to execution shards, the consensus shard is responsible for the following functions:
- Managing global consensus
- Producing 'master' ZKPs for verifying global state changes
- Setting the protocol rules and parameters
Although =nil; relies on ZKPs to verify state transitions statelessly, ZKPs can be costly to generate. As a result, a standard security protocol is used to provide security guarantees while ZKPs are generated.
The committee of an execution shard runs a Multi-Threshold BFT security protocol which is a varioation of Sync HotStuff. This is done so that the safety threshold at the local level is not tied to the safety threshold of the cluster.
After running the local consensus protocol, leaders of all committees transmit block digests and quorum certificates to the leader of the consensus shard. The consensus shard leader then proposes a new block to the consensus shard committee, which includes the entire set of validators. These validators run a HotStuff-2 consensus protocol to achieve global consensus.