Migrate SDK squids to Portal - SQD Documentation

← Back to Portal Setup

Return to the Portal setup overview to explore other deployment options.

Portal is currently in closed beta. Please report any bugs or suggestions to the SQD Portal chat or to Squid Devs.

This guide walks you through migrating SDK-based squids from gateways of the open private version of SQD Network to portals of the permissionless SQD Network as your primary data source.

Prerequisites

An existing SDK-based squid using @subsquid/evm-processor with .setGateway()
Access to an SQD portal URL (see Step 2 for options)

Benefits of migrating to Portal

Reduced reliance on centralized services: The permissionless SQD Network consists of over 2500 nodes ran by independent operators, with a total capacity of roughly 2Pb. This allows for a great deal of redundancy. You can query the network without relying on any centralized services.
Improved speed: The permissionless version of SQD Network has significantly more bandwidth than the open private network. Portals use available bandwidth more effectively than gateways, resulting in 5-10 times faster data fetching in our tests. For squids not bottlenecked by write operations, this translates into better sync performance.
Simplified API: Compared to the request-response approach used by gateways, portals offer a simpler stream-based API.
Future-proof: All future development will focus on portals and the permissionless SQD Network.

Migration steps

Install portal-api version

Navigate to your squid’s folder and install the portal-api version of @subsquid/evm-processor:

npm i @subsquid/evm-processor@portal-api

Obtain a portal URL

Obtain a portal URL for your dataset. Choose the option that matches your setup:

SQD Cloud user
Self-hosted portal
Public portal (testing)

If you’re an existing user of the SQD Cloud:

Enable the preview mode
Navigate to the portals page
Click on the tile of your network and follow the instructions

Once configured, you can run your portal-powered squid both locally and in the Cloud. You can skip Step 3—it’s already configured automatically.

If you don’t have an SQD Cloud account and want to test a portal, use the following URL template:

https://portal.sqd.dev/datasets/<dataset-slug>

Replace <dataset-slug> with the last path segment of the gateway URL for your network found on this page.Example: The URL for the Ethereum dataset is:

https://portal.sqd.dev/datasets/ethereum-mainnet

Update processor configuration

Configure your squid to ingest data from the portal by replacing the .setGateway() call in your processor configuration (typically in src/processor.ts) with .setPortal().

Local or self-hosted portal
Cloud deployment

Replace .setGateway() with .setPortal() using your portal URL directly:

src/processor.ts

export const processor = new EvmBatchProcessor()
  .setPortal('<portal URL for your dataset>')

If you’re deploying to the Cloud, source the portal URL from an environment variable configured in your manifest:

src/processor.ts

import { assertNotNull } from '@subsquid/util-internal'

export const processor = new EvmBatchProcessor()
  .setPortal(
    assertNotNull(
      process.env.PORTAL_URL,
      'Required env variable PORTAL_URL is missing'
    )
  )

If your squid uses an RPC endpoint, keep that configuration for now. Portals do not currently support real-time data ingestion, but this capability is coming soon.

Your squid is now configured to source data from Portal. Test it locally to verify everything works correctly.

← Back to Portal Setup

​Prerequisites

​Benefits of migrating to Portal

​Migration steps

Prerequisites

Benefits of migrating to Portal

Migration steps