Anza's New SVM API

Solana is experiencing its own “EVM” moment with the advent of the “SVM” (Solana Virtual Machine), a term currently generating significant buzz in the crypto world. Much like Ethereum in 2022, a plethora of “SVM” projects are embarking on ambitious ventures to harness Solana’s advanced transaction processing capabilities for a variety of applications beyond the Solana validator.

As the SVM ecosystem rapidly expands, developers and innovators are exploring new possibilities and pushing the boundaries of what can be achieved with this powerful technology. But what exactly is the SVM, and how can projects build with it?

Defining SVM

Engineers within the Solana ecosystem hold diverse views on the precise definition of the Solana Virtual Machine. Some insist it encompasses the entire transaction processing pipeline, from validator runtime to program execution. Others believe the true SVM is merely the lower-level eBPF virtual machine responsible for executing programs.

Most emergent SVM projects seem to favor the former, broader perspective of the Solana Virtual Machine. With this in mind, it becomes essential to break down the transaction processing pipeline and understand its components.

The Agave validator’s Bank component serves as the orchestrator of the SVM within the validator. Bank, encompassing much of the validator runtime, is responsible for managing the network’s state during a given slot. While slots can exist across different forks, this complexity extends beyond the scope of the SVM.

As the Solana network hums along, transactions are submitted to validators over network connections and are subsequently routed to the runtime for execution. During any given slot, that slot’s Bank processes transactions and attempts to pack them into a block.

The Solana protocol forwards transactions to the next leader in the leader schedule, rather than leveraging a mempool for pending transactions. If a leader can’t include a transaction in its current slot, it is forwarded to the next leader. Leaders use a Bank to pack a block, while other nodes will replay a block, attempting to rebuild a matching instance of a Bank and thus validate a block.

Regardless of whether a node is packing a block (leader) or replaying a block (validator), transactions are processed in batches. Each transaction contains one or more instructions, each of which targets a specific program.

As the above diagram depicts, instructions contain the information relevant to load accounts and determine write-locks. This concept of read-only and “writable” accounts allows parallel account data access across Bank instances.

Before transactions are processed, they undergo a “sanitization” process, which performs various checks and extracts processing information from the instructions. One useful optimization during sanitization is the extraction and deduplication of account keys from a transaction’s instructions. This allows the SVM to load the necessary keys for the entire transaction without duplication or unnecessary effort.

Once all the accounts are loaded, the SVM loads the executable program for each instruction and provisions an eBPF virtual machine to execute the program, returning the result. To avoid redundant translation from byte code to machine code, a cache mechanism is used, which stores the translated program until it absolutely needs to be recomputed (or the cache is full).

All of this end-to-end complexity occurs within the SVM component and is driven by Bank. The SVM interface used by Bank has been decoupled from Bank significantly, and this leads to many interesting opportunities surrounding this new, decoupled, well-defined SVM interface. Mainly, this means SVM can be driven by other components, outside a Solana validator.

Opportunities for SVM

With a comprehensive understanding of the SVM’s architecture, it becomes relevant to understand what this standalone software component might be used for. Many opportunities exist — both within and beyond a Solana validator node — for a standalone, composable SVM. Below are just a few of the most popular examples.

• Off-Chain Services: SVM can be used to build services that emulate Solana’s transaction processing protocol but operate entirely off-chain. This can be useful for things like simulating transactions and fuzzing/testing. In fact, this architecture could lead to major breakthroughs in the upcoming RPC v2.

• Diet Clients: SVM can enable the creation of fraud proofs to demonstrate invalid state transitions by the supermajority. This would allow for lightweight clients to operate efficiently without needing to process every transaction or maintain the full blockchain state. By relying on fraud proofs, diet clients can ensure the integrity of the network and verify the validity of transactions with minimal resource requirements, thereby enhancing scalability and security. See SIMD 0065.

• State Channels: Projects can build SVM-based state channels, which can power a wide range of exciting and creative use cases. With state channels, a protocol can restrict which types of transactions are permitted in their network, and can choose to limit connections to strictly peer-to-peer or party-based. When the channel is eventually closed, the final results of the channel’s transactions are posted to the main chain. A classic example is a token payment channel, where one or more tokens are supported, and the final balances of the channel participants are posted.

• Rollups: Networks that need to execute transactions or blocks without the full validator or consensus protocol can leverage the SVM to build rollups. Rollups can utilize the SVM as an execution layer, settling “proofs” of the state transitions within the rollup to the main chain. This architecture offers substantial scaling potential, as SVM execution can be performed in parallel.

• Avalanche Subnet: The isolated SVM execution layer could be used within an Avalanche subnet, relying on Avalanche modules for consensus and networking.

• Extended SVM: SVM could be extended with custom, protocol-compliant functionality. These “extended” SVM units could be plugged into Solana validators or any of the above listed solutions.

Anza’s primary application for the SVM is transaction processing within the Agave validator. However, as previously indicated, the SVM interface has been decoupled from the validator runtime and is now available via an an official specification.

This self-contained SVM, encompassing all the intricate details of transaction processing and program execution, is now encapsulated within the newly released solana-svm Rust crate. This library, featuring the decoupled interface, provides a versatile and composable SVM unit, capable of supporting the aforementioned use cases and beyond.

Anza’s SVM API

Anza’s new solana-svm Rust crate equips developers with the tools necessary to build SVM projects using components that operate live on Solana mainnet-beta. This ensures that the entire stack is battle-tested and guaranteed to be protocol-compliant. Additionally, it has been meticulously fine-tuned for performance and will continue to receive optimizations as the network matures.

The SVM API is broken down in detail below. The central interface to the SVM revolves around the TransactionBatchProcessor struct.

pub struct TransactionBatchProcessor<FG: ForkGraph> {
  /// Bank slot (i.e. block)
  slot: Slot,
  /// Bank epoch
  epoch: Epoch,
  /// SysvarCache is a collection of system variables that are
  /// accessible from on chain programs. It is passed to SVM from
  /// client code (e.g. Bank) and forwarded to the MessageProcessor.
  pub sysvar_cache: RwLock<SysvarCache>,
  /// Programs required for transaction batch processing
  pub program_cache: Arc<RwLock<ProgramCache<FG>>>,
  /// Builtin program ids
  pub builtin_program_ids: RwLock<HashSet<Pubkey>>,
}

Instantiating one of these batch processor objects will allow your application to process batches of sanitized Solana transactions, using all of the downstream Agave components depicted in the earlier section (BPF Loader, eBPF VM, etc.).

The main API method for processing transaction batches is load_and_execute_sanitized_transactions. It requires the following arguments:

• callbacks: A TransactionProcessingCallback trait instance which allows the transaction processor to summon information about accounts, most importantly loading them for transaction execution.

• sanitized_txs: A list of sanitized Solana transactions.

• check_results: A list of transaction check results.

• environment: The runtime environment for transaction batch processing.

• config: Configurations for customizing transaction processing behavior.

The method returns a LoadAndExecuteSanitizedTransactionsOutput, which is defined below in more detail.

Transaction Processing Callback

Downstream consumers of the SVM must implement the TransactionProcessingCallback trait in order to provide the transaction processor with the ability to load accounts and retrieve other relevant information.

pub trait TransactionProcessingCallback {
  fn get_account_shared_data(&self, pubkey: &Pubkey) -> Option<AccountSharedData>;
  fn account_matches_owners(&self, account: &Pubkey, owners: &[Pubkey]) -> Option<usize>;
  fn add_builtin_account(&self, _name: &str, _program_id: &Pubkey) {}
}

Because the API accepts a trait implementation, consumers can provide their own custom implementations of loading accounts, caching, and more. This flexibility allows projects to tailor the transaction processor to their specific needs, optimizing performance and resource management. By leveraging these customizable traits, developers can enhance the functionality and efficiency of their applications, ensuring seamless integration with the SVM while maintaining full control over critical processes.

Transaction Processing Environment

The transaction processor requires consumers to provide values describing the runtime environment to use for processing transactions, by way of the TransactionProcessingEnvironment object.

• blockhash: The blockhash to use for the transaction batch.

• epoch_total_stake: The total stake for the current epoch.

• epoch_vote_accounts: The vote accounts for the current epoch.

• feature_set: Runtime feature set to use for the transaction batch.

• lamports_per_signature: Lamports per signature to charge per transaction.

• rent_collector: Rent collector to use for the transaction batch.

These environment settings enable a precise and controlled execution context for transactions, ensuring that they adhere to specific operational standards. By configuring these parameters, developers can emulate various transaction processing environments to meet specific requirements, optimize performance, and maintain network integrity.

Transaction Processing Configuration

Consumers can provide various configurations to adjust the default behavior of the transaction processor through the TransactionProcessingConfig argument.

• account_overrides: Encapsulates overridden accounts, typically used for transaction simulation.

• compute_budget: The compute budget to use for transaction execution.

• log_messages_bytes_limit: The maximum number of bytes that log messages can consume.

• limit_to_load_programs: Whether to limit the number of programs loaded for the transaction batch.

• recording_config: Recording capabilities for transaction execution.

• transaction_account_lock_limit: The max number of accounts that a transaction may lock.

These configurations enable developers to fine-tune specific aspects of the transaction processor’s operation without altering the overall environment. By adjusting these parameters, developers can control behaviors such as logging limits, compute resource allocation, and account handling, ensuring that the transaction processor operates efficiently and meets the specific needs of their applications.

Transaction Processing Output

The transaction processor’s main API method — load_and_execute_sanitized_transactions — returns a LoadAndExecuteSanitizedTransactionsOutput object, encapsulating the result of the processed transaction batch.

• error_metrics: Error metrics for transactions that were processed.

• execute_timings: Timings for transaction batch execution.

• execution_results: List of results indicating whether or not a transaction was executed.

• loaded_transactions: List of loaded transactions that were processed.

Additional Helpers and API Methods

Developers can bypass processing transaction batches if they wish by instead using the MessageProcessor API — mainly MessageProcessor::process_message. This allows applications to process transaction messages directly, rather than a batch of transactions, but requires a bit more setup.

Additionally, a handful of public helpers are available.

• account_loader::collect_rent_from_account: Collect rent from an account if rent is still enabled.

• account_loader::validate_fee_payer: Check whether the payer account is capable of paying the transaction fee.

• account_rent_state::RentState: API for working with an account’s rent state (rent-paying, rent-exempt, etc.).

• nonce_info: Module with traits and structs for working with transaction nonces.

• program_loader::load_program_with_pubkey: Load a program by its public key.

Looking Forward

The Solana Virtual Machine will continue to evolve as the Solana network matures and communities around these exciting new SVM projects continue to grow. New use cases will arise from SVM’s use outside of the validator, likely demanding higher performance and increased composability.

Additionally, the boom of SVM projects has given rise to a new age of Solana tooling. Many teams will develop custom software solutions to tailor SVM to their project’s needs, as well as brand-new, never before seen technology including breakthroughs in areas such as bridging, settling, and proofs.

The industry as a whole stands to benefit immensely from the collaborative pursuit of open-source, consumable tooling — similar to Anza’s SVM API. This new ecosystem of SVM teams should seek to work together wherever possible to increase innovation and push the boundary of what’s possible with SVM.