AO Decoded
Welcome to AO Decoded: a book about AO, the decentralized supercomputer built on top of Arweave. This book takes an approachable look at AO and how it works from an architectural perspective.
AO is quite a unique piece of architecture, and is a powerful piece of technology. Due to its novel approach, it requires some new concepts to be learned which you might not be familiar with. This is true even if you are well-versed in the crypto space.
Who this book is for
This book is for anyone interested in understanding the architecture and purpose of AO.
The goal of this book
The goal of this book is twofold: to help you understand how AO works, and by extension, help you to realize the benefits AO offers resulting from its architecture.
This book is written in a such a way that it introduces the core technologies and familiar concepts before going into more detail about how they work, and how they differ from other decentralized networks/blockchains.
This approach is aimed at making it as easy as possible to get acquainted with AO.
Prerequisites
This book assumes no large prerequisites, but assumes a general understanding of:
- What blockchain technology is
- Some computing-related terminology
Overall, this book should generally be readable by anyone familiar with how blockchains work, and are curious about the unique and particular aspects of AO.
We hope this book helps you understand how AO works, and in turn, helps you to realize the benefits that AO Computer and Arweave can offer as a whole.
What this book is not
AO Decoded is not a guide on how to create or deploy applications on AO, or intended as developer documentation. It is also not intended as an explanation of how to run AO nodes or pieces of infrastructure.
If this is instead what you are interested in, take a look at the different cookbooks respectively.
Contributing
This book is open source, and open to contributions.
If you want to contribute content to this book, fix a typo, work on a translation, or something else entirely, you can find the source code on GitHub here.
An Introduction to Arweave
Before understanding AO, one must understand Arweave.
Arweave is what underpins the entire idea and (implementation) of AO.
You do not need to understand how it works in great detail, only the aspects of Arweave which make the creation of AO possible.
What is Arweave?
Arweave is a standard, traditional blockchain much like Bitcoin. Users can transfer value to one another with cryptocurrency tokens ($AR), and it is stored immutably on a global, decentralized ledger.
What makes Arweave different is that instead of having properties for smart contracts, like Ethereum for example, Arweave is solely focused on the storage of data.
Arweave's storage properties
Arweave's consensus mechanism is built to allow the cost-effective storage of data for a very long period of time, and data stored on Arweave is immutable. It cannot be tampered with, or individually removed.
Although Arweave has no properties to execute code, one must remember that code is just data. This code can be stored on Arweave, even if not executed.
It is still an immutable ledger which processes and logs different transactions, in a specific order.
The existence of such a ledger with these properties is important for AO.
What is AO?
AO is a decentralized compute network built on top of Arweave.
What this really means is that AO is a network of connected computers which perform computations on behalf of users. This is similar to many blockchains out there which run smart contracts, and process smart contract interactions for users.
In terms of AO, a simple way you can think of it is like this:
- Arweave stores all of the smart contract code, and interactions
- AO executes all of the actions users take
Where traditional blockchains have a single node (and network) perform both storage and computation, the separation of Arweave and AO separates these duties.
Is AO a blockchain?
The best way to think of it is that AO is very much like a blockchain. In fact, almost all operations are logged onchain, on Arweave.
However, some of AO's properties make it very different to existing blockchains.
Try to go into this book with an open mind, and don't get caught up too much on whether something fits this definition exactly.
Here are some of the properties which make AO similar to existing machines:
- There are a connected group of nodes which communicate with eachother
- Computation is paid for in cryptocurrency ($AO)
- There is some kind of consensus
- There are cryptographic signatures and attestations for user actions
The following chapters
The next section of the book will establish the very core concepts which make up an AO node, and the type of things that they can do. This will introduce concepts like processes and messages, which are similar to existing blockchain concepts.
We will then take a look at the structure of nodes in more detail, and look at how these nodes communicate with each other. This is where AO starts to deviate from traditional blockchains.
Lastly, we will look at consensus, state, and scaling.
Briefly after this section we will have a reference section which details more about AO's device mechanism, and how modularity works within a system of interconnected nodes.
AO Nodes, Processes, and Messages
AO Nodes
An AO node is a computer running HyperBEAM, AO's node client software.
These nodes operate similarly to other blockchain nodes: they communicate with each other over a network, handle transactions, compute smart contract state, and more.
For now, let's focus on the primary purpose of these nodes: performing computation.
Users of blockchains and decentralized networks want to be able to do things: transfer tokens from one account to another, swap tokens on decentralized exchanges, and interact with different types of smart contracts.
AO nodes facilitate these actions through two primary mechanisms: processes and messages.
Processes
On AO, smart contracts are colloquially referred to as processes. Processes on AO are written in the Lua programming language. AO processes contain handlers which, as you might expect, "handle" different types of transactions.
These handlers are written by developers, and contain the logic to deal with different types of smart contract actions.
For example, a decentralized exchange built on AO might have a process with the handlers "swap", "deposit", and "withdraw".
If a user sends a transaction of type "swap", the process runs the relevant code to handle a swap. If it is deposit, it will run the code in the process for deposit the user's tokens into a pool. And so on.
Messages
Transactions that users send to processes are referred to as messages on AO. Processes can also create and send messages to one another, much like how smart contracts can call other smart contracts.
Messages function identically to transactions, and in reality are signed transactions which get settled onchain (which we'll look more into later). Wehere you might think of blockchains as a chain of blocks, processes in AO have their own "chain of messages" associated with them.
Inboxes, Outboxes, and CRON Jobs
Inboxes and Outboxes
Every AO process has an Inbox and Outbox.
The Inbox accepts incoming, ordered messages for the process to handle. You can think of this much like an email inbox, or lke a list of pending transactions waiting to be processed by a smart contract.
The messages from the Inbox are read in FIFO (First In First Out) order.
Similarly, processes also have an Outbox which contains messages generated by the process itself. These could be messages to send onward to other processes, or to update its own state.
Scheduled tasks
A distnguising feature of AO is the ability for users and developers to run scheduled tasks, i.e. CRON jobs.
This functionality allows messages to be sent at specified intervals. For example, a user could set up an action to swap token x for token y every day.
Conclusion
In the previous chapter we looked at the structure of nodes, and in this one we explored further concepts like Inboxes, Outboxes, and scheduled tasks.
But how exactly are these messages ordered and sent to a process?
How does a transaction go from a user's initial message to computation?
Let's look at how these nodes communicate with each other and the unique aspect of transactions, or messages, on AO.
Transaction Lifecycle
On AO, users have a wallet with which they create and sign transactions, also known as messages on AO.
Let's take a look at a high-level overview of what happens when a message is sent on AO, and how a user gets a response from the network.
Message validation
When a user interacts with a dApp on AO, for example a decentralized exchange process, a message is created and sent to an initial node.
This initial node receives the new message, checks to see if the transaction is valid (for example it has a valid signature, has the correct structure, etc.).
If the message is valid, it forwards this message onwards to be scheduled.
Message scheduling and storage
Once a message is deemed as valid, this means it can be scheduled for settlement and storage on Arweave.
A scheduler handles all of incoming messages associated with certain processes, orders them, and then stores these messages on Arweave.
This is a crucial part of the process, as it sets in stone the transaction ordering.
Each message is stored on Arweave in an immutable, timestamped order - just like with other blockchains.
Computation
Once a user's message is stored on Arweave, the new state of a process can be computed.
Think of it like this:
- An AO node sends off a message to be stored on Arweave
- Schedulers handle the message settlement, and once done, let the AO node know
- The AO node can now read all the transactions on Arweave, and re-construct the state
- A number of AO nodes can compute this state for further verification
And this is exactly what happens with AO. An AO node reads all of the state associated with a process from Arweave, computes it, and returns the result to the user.
This is ultimately how the message lifecycle works on AO.
But, where consensus?
This approach to computation is radically different to other solutions, which also means that consensus must also be approached differently.
Let's take a look at consensus in a little bit more detail.
Consensus and Global State
What you might have realized so far in our discussion of AO's architecutre is that there has been no mention of "blocks", or adding blocks to a "blockchain".
This does happen at the Arweave level, but no blocks are being produced by the AO network itself.
Looking at the previous chapter, you may still have some unanswered questions. The largest one likely being "where is consensus?!?!?!?!?!?"
In this chapter we'll explore the security and consensus model of AO in more detail, first taking a look at the state level.
The state layer
Consensus on AO does not exist the same way as it does on Ethereum (or even Arweave, for that matter).
It provides different security mechanisms based on different needs, but ultimately, Arweave acts as the immutable state layer from which computation uses as a source of truth.
As mentioned in the previous chapter, schedulers order and store messages on Arweave, and AO nodes perform computation based on this message log on Arweave.
This means at the state layer, consensus is solved. Once a set of messages are on Arweave they are immutable and tamper-proof: in other words, anyone can read from this log of messages stored on Arweave and re-construct the state of an AO process.
Users could even run their own AO node and compute this for themselves - no need to even request computation from a node or a consortium of nodes.
From this persective, AO essentially inherits the security of Arweave for the storing and reading of message data.
Where consensus is instead important is the layer on top of this - computation, and the layer previous (scheduling).
Ensuring computational correctness
With all of the information given, this raises the question:
How do you ensure nodes return truthful results to a user?
This ultimately depends on the level of insurance for "trust" that end users or applications require.
Users, for example, could run their own AO node and compute the results from Arweave themselves. These provides a high level of verification.
Most users, however, do not end up running their own node. This means that AO nodes will typically compute results for users.
Once AO has fully converted to mainnet, these nodes will make use of economic incentives and staking in order to incentivize the returning of correct results to users.
The fact that the state is implied and re-constructable from Arweave by anyone also means that nodes are heavily disincentivized to tamper with the result.
If an AO node decided to return an unthruthful result - the "truth" is on Arweave. It only will affect the frontend, for example the displaying of an incorrect balance, which does not reflect the actual balance onchain. They cannot affect the actual state of a process.
This leaves one part of consensus to be answered:
How do you ensure that Scheduler Units order transactions in a fair, and tamper-proof way?
We will not cover that in the scope of this bok at the moment.
Scaling
As AO has no blocks or blockchain, this means it does not have a global shared state.
There is no global state for AO nodes to agree upon, as this state is stored on and read from Arweave.
This radically changes how AO can scale - as interactions with different processes do not have to wait in a queue to compute state, it allows for massively parallel execution.
Let's explore this with a more concrete example.
Sending messages at the same time
Let's assume that one user, Bob, wants to swap some tokens on a decentralized exchange.
He will send a message to the DEX process, usually through a dApp's interface, leading to a transaction lifecycle like we've already explored.
Now let's say at the same time, Alice wants to buy an NFT (Atomic Asset) from a marketplace, with a similiar transaction lifecycle.
These two actions do not modify the same state - so what's to say that they cannot happen simultaneously?
based on what we've explored in the previous chapters, we can see that not all AO nodes must compute all actions. In fact, users can provision as little or as many nodes as desired to verify the result of an action.
What this does mean though is that while Bob is sending his message to AO Node 1, Alice can, at the same time, send her message to AO Node 2. Both can get their results back from these different nodes, without waiting for them to agree agree upon consensus.
Processes are instead concerned with largely with their own state.
Horizontal scaling
With traditional blockchains, adding more nodes to the network doesn't tend to increase the speed or scalability of a blockchain.
Every node must agree on every transaction and every state change, and update their state accordingly. This is a scaling bottleneck that is solved by Layer 2 solutions such as rollups.
With AO, the more nodes on the network means the more nodes available for computation, which means the more processes can execute in parallel.
This is what's known as horizontal scaling.
Accessing the same process
To go a step further, what if Bob and Alice did want to access the same process?
In this case, the messages would get queued in the inbox of a process, and there would be a similar aspect of waiting for a result.
However, this can still be alleviated yet by sharding the process, or splitting it into multiple processes, and routing user messages to the process with the lowest amount of messages queued for processing.
This is ideal for use cases that typically become infeasible in pracitce, such as large scale onchain games or social media applications.
Conclusion
This covers most of the building blocks and core concepts which underpin the AO network.
If you'd like to learn about the architectural concepts in more detail, or explore other novel aspects of AO, please check out the references section.