--- name: sqd-blockchain-data description: > Query and index blockchain data using SQD's infrastructure. Use when building blockchain data pipelines, querying on-chain data (transactions, logs, traces, token transfers), building indexers, or deploying dApp backends. Triggers on mentions of blockchain data, on-chain analytics, EVM/Solana indexing, Portal API, Pipes SDK, Squid SDK, SQD, or Subsquid. metadata: author: subsquid version: "1.1" mintlify-proj: sqd --- # SQD Blockchain Data Infrastructure Build production-grade blockchain data pipelines across 200+ networks. SQD provides four core products: **Portal API** (raw HTTP data access), **Pipes SDK** (composable TypeScript pipelines), **Squid SDK** (complete indexing framework with GraphQL), and **SQD Cloud** (managed deployment). ## Quick Reference * **Portal API Base URL**: `https://portal.sqd.dev/datasets/{dataset-slug}` * **Authentication**: Free, no API keys required * **Rate Limits**: 20 requests per 10 seconds (public), unlimited (self-hosted) * **Response Format**: Newline-delimited JSON (NDJSON), gzip-compressed — always use `--compressed` in curl * **Documentation**: [https://beta.docs.sqd.dev](https://beta.docs.sqd.dev) * **Full LLM docs**: [https://beta.docs.sqd.dev/llms-full.txt](https://beta.docs.sqd.dev/llms-full.txt) ## When to Use Each Tool | Scenario | Tool | Why | | ------------------------------------------- | --------------------- | -------------------------------- | | Non-TypeScript language (Python, Go, Rust) | **Portal API** | Language-agnostic HTTP | | One-off data export or analytics | **Portal API** | No setup, simple curl | | Custom TypeScript pipeline, non-Postgres DB | **Pipes SDK** | Lightweight, any target | | dApp backend with GraphQL API | **Squid SDK** | Auto-generated GraphQL + TypeORM | | Managed deployment with monitoring | **Squid SDK + Cloud** | One-command deploy | ## Portal API — Core Usage ### EVM Query Structure ```bash theme={"system"} curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \ -H 'Content-Type: application/json' \ -d '{ "type": "evm", "fromBlock": 18000000, "toBlock": 18001000, "fields": { "block": {"number": true, "timestamp": true}, "log": {"address": true, "topics": true, "data": true, "transactionHash": true} }, "logs": [{ "address": ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"], "topic0": ["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"] }] }' ``` **Key parameters:** * `fromBlock` (required): Starting block number * `toBlock` (optional): Ending block. Omit to stream to current head and stop * `includeAllBlocks`: If true, returns all blocks in range even without matching data * `fields`: Select which fields to return per data type — reduces payload size * `logs`, `transactions`, `traces`, `stateDiffs`: Filter arrays for each data type ### Solana Query Structure ```bash theme={"system"} curl --compressed -X POST 'https://portal.sqd.dev/datasets/solana-mainnet/stream' \ -H 'Content-Type: application/json' \ -d '{ "type": "solana", "fromBlock": 259984950, "fields": { "block": {"number": true, "timestamp": true}, "instruction": {"programId": true, "accounts": true, "data": true} }, "instructions": [{ "programId": ["TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA"], "d1": ["3"] }] }' ``` Solana discriminators `d1`/`d2`/`d4`/`d8` are 1/2/4/8-byte instruction type identifiers. Example: `"d1": ["3"]` filters for SPL Token Transfer instructions. ### Common Event Signatures (topic0) * **Transfer** (ERC-20/721): `0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef` * **Approval** (ERC-20): `0x8c5be1e5ebec7d5bd14f71427d1e84f3dd0314c0f7b2291e5b200ac8c7c3b925` * **Swap** (Uniswap V2): `0xd78ad95fa46c994b6551d0da85fc275fe613ce37657fb8d5e3d130840159d822` * **Swap** (Uniswap V3): `0xc42079f94a6350d7e6235f29174924f928cc2ac818eb64fed8004e115fbcca67` ### Endpoints | Endpoint | Method | Purpose | | ------------------- | ------ | -------------------------------------------------------------- | | `/stream` | POST | Stream blocks matching query, including real-time data | | `/finalized-stream` | POST | Stream only finalized blocks (production-safe, handles reorgs) | | `/head` | GET | Current chain head block | | `/finalized-head` | GET | Latest finalized block | | `/metadata` | GET | Dataset info: name, aliases, start block, real-time support | ### Error Codes * **400**: Invalid query format * **404**: Dataset not found (check slug at [https://portal.sqd.dev/datasets](https://portal.sqd.dev/datasets)) * **429**: Rate limit exceeded (20 req/10s on public Portal) * **500**: Internal server error ### Top Dataset Slugs | Network | Slug | Chain ID | Real-time | | ---------------- | ------------------- | -------- | --------- | | Ethereum | `ethereum-mainnet` | 1 | ✓ | | Base | `base-mainnet` | 8453 | ✓ | | Arbitrum One | `arbitrum-one` | 42161 | ✓ | | Optimism | `optimism-mainnet` | 10 | ✓ | | Polygon | `polygon-mainnet` | 137 | ✓ | | BSC | `binance-mainnet` | 56 | ✓ | | Avalanche | `avalanche-mainnet` | 43114 | ✓ | | Solana | `solana-mainnet` | — | ✓ | | Ethereum Sepolia | `ethereum-sepolia` | 11155111 | ✓ | | Base Sepolia | `base-sepolia` | 84532 | — | Full list: [https://portal.sqd.dev/datasets](https://portal.sqd.dev/datasets) ## Pipes SDK — Composable Pipelines TypeScript-only. Currently supports EVM and Solana. ```typescript theme={"system"} import { createPipeline } from "@subsquid/portal-pipes"; const pipeline = createPipeline({ portal: "https://portal.sqd.dev/datasets/ethereum-mainnet", query: { type: "evm", fromBlock: 18000000, fields: { log: { address: true, topics: true, data: true } }, logs: [{ address: ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"] }] } }); ``` Use Pipes SDK when: integrating into existing Node.js apps, storing to non-Postgres databases (ClickHouse, MongoDB, Kafka), or needing fine-grained streaming control. ## Squid SDK — Full Indexing Framework TypeScript-only. Supports EVM, Solana, Substrate, Fuel, Tron, Starknet. ```typescript theme={"system"} import { EvmBatchProcessor } from "@subsquid/evm-processor"; import { TypeormDatabase } from "@subsquid/typeorm-store"; const processor = new EvmBatchProcessor() .setPortal("https://portal.sqd.dev/datasets/ethereum-mainnet") .addLog({ address: ["0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"] }); processor.run(new TypeormDatabase(), async (ctx) => { for (const block of ctx.blocks) { for (const log of block.logs) { // decode and persist } } }); ``` Use Squid SDK when: building dApp backends with auto-generated GraphQL APIs, needing PostgreSQL with TypeORM migrations, or deploying to SQD Cloud (`sqd deploy`). ## Common Recipes **Track USDC transfers on Base:** ```bash theme={"system"} curl --compressed -X POST 'https://portal.sqd.dev/datasets/base-mainnet/stream' \ -H 'Content-Type: application/json' \ -d '{"type":"evm","fromBlock":10000000,"fields":{"block":{"number":true,"timestamp":true},"log":{"topics":true,"data":true,"transactionHash":true}},"logs":[{"address":["0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913"],"topic0":["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"]}]}' ``` **Index Uniswap V3 swaps on Ethereum:** ```bash theme={"system"} curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \ -H 'Content-Type: application/json' \ -d '{"type":"evm","fromBlock":18000000,"fields":{"block":{"number":true,"timestamp":true},"log":{"address":true,"topics":true,"data":true}},"logs":[{"topic0":["0xc42079f94a6350d7e6235f29174924f928cc2ac818eb64fed8004e115fbcca67"]}]}' ``` **Track contract deployments via traces:** ```bash theme={"system"} curl --compressed -X POST 'https://portal.sqd.dev/datasets/ethereum-mainnet/stream' \ -H 'Content-Type: application/json' \ -d '{"type":"evm","fromBlock":18000000,"toBlock":18001000,"fields":{"trace":{"type":true,"from":true,"to":true,"value":true}},"traces":[{"type":["create"]}]}' ``` ## Guidelines * Always use `--compressed` with curl — Portal returns gzip-compressed responses * Use `/finalized-stream` instead of `/stream` for production-critical applications that need reorg safety * Filter by `address` and/or `topic0` in logs to minimize response size — unfiltered queries on busy chains return massive payloads * The `fields` object controls which properties are returned per data type — request only what you need * Portal is 10-50x faster than RPC for historical data extraction * For continuous real-time streaming, reconnect to `/stream` periodically after reaching head * OpenAPI specs available at: [https://beta.docs.sqd.dev/files/evm-openapi.yaml](https://beta.docs.sqd.dev/files/evm-openapi.yaml) and [https://beta.docs.sqd.dev/files/solana-openapi.yaml](https://beta.docs.sqd.dev/files/solana-openapi.yaml) ## AI Integration * **Docs MCP Server**: `https://beta.docs.sqd.dev/mcp` — search and access SQD documentation * **Portal MCP Server**: `https://portal.sqd.workers.dev/mcp` — query blockchain data directly * **Agent Skills**: [https://github.com/subsquid-labs/agent-skills](https://github.com/subsquid-labs/agent-skills)