© StreamingFast
EN
Subgraphs are great for accessing blockchain data, they are simple to use and they provide an out-of-the-box GraphQL API, which is pretty useful in front-end applications. However, despite the many benefits of using subgraphs, one limitation can be the time it takes to index large smart contracts.
This is where Substreams comes in. Substreams was developed for The Graph Network to parallelize the data indexing process, so that the computation power can scale to match the ever-growing dataset. Although Substreams can work independently from subgraphs, it is also one of the possible data sources for a subgraph (known as a Substreams-powered subgraph).
How do Substreams reduce the time it takes for subgraphs to index large smart contracts? Let’s take a look at a great example: Uniswap, one of the largest decentralized cryptocurrency exchanges, powers on-chain markets and DeFi — as the main source of liquidity for DeFi markets, and as the creator of essential public infrastructure.
Decentralized exchanges allow users to swap their cryptocurrencies peer to peer, without depending on a centralized entity facilitating the trade. Uniswap Labs’ smart contract expertise has transformed blockchain-based markets, allowing users to trust code instead of centralized entities. In the crypto space, a set of smart contracts working together for a specific goal is known as a protocol.
Since Uniswap’s founding in 2018, several versions of the protocol have been released: Uniswap v1, Uniswap v2 and the latest version, Uniswap v3. The Uniswap v3 protocol adds significant improvements over the prior version Uniswap v2, but it also adds more complexity within the smart contracts it uses.
Most DeFi applications, such as wallets or aggregators, and even Uniswap itself require easy access to up-to-date, accurate indexed data in order to report analytics. For example, the Uniswap info overview page requires fresh Uniswap v3 data. Imagine how problematic it would be if users of a wallet or DeFi application were making decisions or observations based on outdated data!
Using subgraphs to index the Uniswap v3 data is a great option. However, indexing the Uniswap v3 smart contract was taking Indexers more than 2 months! And this 2 month time-to-index gets longer as the protocol sees more usage — it’s the type of problem that only gets worse with time.
In addition to the problems that delays pose to users of applications, waiting months to extract data also creates challenges for developers. For instance, the software development cycle slows down significantly. Agile teams can’t iterate on several versions easily, and this leads to unsatisfied customers and product risk. Not to mention the inability for teams to proactively squash bugs in their code, as the learn-build-measure feedback loop is also made too slow.
So what is the solution? Substreams is great at speeding up indexing times, and the Uniswap v3 smart contract demonstrates this well.
Substreams is able to extract the entirety of the Uniswap v3 data and send it to a subgraph in just 20 hours. A 70x improvement over the 2 months that it used to take, when just using subgraphs. This Substreams-powered subgraphs are live on The Graph Network, and being served currently by 14 independent Indexers.
How does it work?
Substreams-powered Subgraph
A Substreams-powered subgraph provides users with a familiar GraphQL API that uses Substreams modules as its data source.
This is a great approach if your application is already using subgraphs and you want to speed up the indexing time and your development cycles without making significant changes to your codebase.
Substreams Without Subgraphs
Creating a Substreams-powered subgraph to power your front-end application is just the beginning. Substreams also allows you to consume the blockchain data in a variety of ways, sinking the data to your preferred database (PostgreSQL, MongoDB, Kafka, files…).
The great thing about Substreams is that you do not have to code several implementations to send your data to different sinks. Once your Substreams is developed, you can easily adapt it to send the data to any sink!
Processing the Substreams and writing the data to simple files took only 10 hours. That’s a massive improvement!
Substreams is even faster after the first use!
There is another benefit to using Substreams, which is the ultra fast caching system it uses.
When a Substreams package is consumed (for example, the Uniswap v3 Substreams), the data is cached on the StreamingFast servers so that it can be reused in the future.
This means that a Substrems package is even faster after the first time that you consume it. Because it’s never fast enough for Substreams!
NOTE: All the times displayed above are approximate and were collected by the StreamingFast team.
Are you ready to experience the benefits of using Substreams?
Here’s how to get started:
If you are a builder, install the Substreams CLI, get a StreamingFast authentication token and create your first Substreams. It’s super easy!
You can also explore the Substreams Registry, where other builders share their own Substreams. Maybe someone has already built a Substreams that covers your needs!