Skip to main content

​
Build an Indexer from Scratch

This guide demonstrates how SDK packages can be combined into a working indexer (called a squid). This page goes through all the technical details to make the squid architecture easier to understand. If you want to get to a working indexer ASAP, bootstrap from a template.

​
Prerequisites

Before you begin, ensure you have:

​
Project: USDT Transfers API

In this tutorial, you’ll build an indexer that tracks USDT transfers on Ethereum, saves the data to PostgreSQL, and serves it as a GraphQL API.

​
Required packages

From the requirements above, you’ll need these packages:
  • @subsquid/evm-processor - for retrieving Ethereum data
  • @subsquid/typeorm-store, @subsquid/typeorm-codegen, and @subsquid/typeorm-migration - for saving data to PostgreSQL

​
Optional packages

  • @subsquid/evm-typegen - for decoding Ethereum data and useful constants such as event topic0 values
  • @subsquid/evm-abi - as a peer dependency for the code generated by @subsquid/evm-typegen
  • @subsquid/graphql-server / OpenReader - for serving GraphQL API

​
Build the Indexer

1

Initialize the project

Create a new folder and initialize a Node.js project:
npm init
Create a .gitignore file:
.gitignore
node_modules
lib
2

Install dependencies

Install the required packages with Portal API support:
npm i dotenv typeorm @subsquid/evm-processor@portal-api @subsquid/typeorm-store @subsquid/typeorm-migration @subsquid/graphql-server @subsquid/evm-abi
The @portal-api version of @subsquid/evm-processor is required for Portal support.
Install development dependencies:
npm i typescript @subsquid/typeorm-codegen @subsquid/evm-typegen --save-dev
3

Configure TypeScript

Create a tsconfig.json file with minimal configuration:
tsconfig.json
{
  "compilerOptions": {
    "rootDir": "src",
    "outDir": "lib",
    "module": "commonjs",
    "target": "es2020",
    "esModuleInterop": true,
    "skipLibCheck": true,
    "experimentalDecorators": true,
    "emitDecoratorMetadata": true
  }
}
4

Define the data schema

Create a schema.graphql file to define your data model:
schema.graphql
type Transfer @entity {
  id: ID!
  from: String! @index
  to: String! @index
  value: BigInt!
}
Generate TypeORM classes from the schema:
npx squid-typeorm-codegen
The TypeORM classes are now available at src/model/index.ts.
5

Set up the database

Create environment variables in a .env file:
.env
DB_NAME=squid
DB_PORT=23798
RPC_ETH_HTTP=https://rpc.ankr.com/eth
Create a docker-compose.yaml file for PostgreSQL:
docker-compose.yaml
services:
  db:
    image: postgres:15
    environment:
      POSTGRES_DB: "${DB_NAME}"
      POSTGRES_PASSWORD: postgres
    ports:
      - "${DB_PORT}:5432"
Start the database:
docker compose up -d
Compile the TypeORM classes:
npx tsc
Generate and apply the database migration:
npx squid-typeorm-migration generate
npx squid-typeorm-migration apply
6

Generate ABI utilities

Create an abi folder for contract ABIs:
mkdir abi
Find the USDT contract ABI on Etherscan and save it to ./abi/usdt.json.Generate TypeScript utility classes from the ABI:
npx squid-evm-typegen src/abi ./abi/*
The utility classes are now available at src/abi/usdt.ts.
7

Create the processor

Create src/main.ts with the following code blocks:

​
Imports

main.ts
import { EvmBatchProcessor } from "@subsquid/evm-processor";
import { TypeormDatabase } from "@subsquid/typeorm-store";
import * as usdtAbi from "./abi/usdt";
import { Transfer } from "./model";

​
Processor configuration

main.ts
const processor = new EvmBatchProcessor()
  .setPortal("https://portal.sqd.dev/datasets/ethereum-mainnet")
  .setRpcEndpoint({
    url: process.env.RPC_ETH_HTTP,
    rateLimit: 10,
  })
  .setFinalityConfirmation(75) // 15 mins to finality
  .addLog({
    address: ["0xdAC17F958D2ee523a2206206994597C13D831ec7"],
    topic0: [usdtAbi.events.Transfer.topic],
  });

​
Database configuration

main.ts
const db = new TypeormDatabase();

​
Batch handler

main.ts
processor.run(db, async (ctx) => {
  const transfers: Transfer[] = [];
  for (let block of ctx.blocks) {
    for (let log of block.logs) {
      let { from, to, value } = usdtAbi.events.Transfer.decode(log);
      transfers.push(
        new Transfer({
          id: log.id,
          from,
          to,
          value,
        })
      );
    }
  }
  await ctx.store.insert(transfers);
});
Supplying a TypeormDatabase to the function causes ctx.store to be a PostgreSQL-compatible Store object.
8

Run the processor

Compile the project:
npx tsc
Start the processor:
node -r dotenv/config lib/main.js
The processor is now indexing USDT transfers from the Ethereum blockchain.
9

Start the GraphQL server

Update your .env file to include the GraphQL server port:
.env
DB_NAME=squid
DB_PORT=23798
RPC_ETH_HTTP=https://rpc.ankr.com/eth
GRAPHQL_SERVER_PORT=4350
In a separate terminal, start the GraphQL server:
npx squid-graphql-server
The GraphQL API with GraphiQL is available at localhost:4350/graphql.

​
Next Steps

Congratulations! You’ve built a complete blockchain indexer from scratch.

View Complete Code

See the final code for this tutorial

Development Flow

Learn about the general development workflow
The commands in this tutorial are often abbreviated as custom sqd commands in production squids. Use the commands.json file from the EVM template as a starter.