Solana DAS API Pagination: Efficient Large Dataset Querying

Overview

DAS API methods return up to 1,000 records per call. To retrieve more, you paginate — making multiple calls and crawling through pages of data. Helius supports two mechanisms: page-based and keyset pagination. Page-based pagination is the simplest way to get started. Keyset pagination is for advanced users querying across large (500k+) datasets efficiently.

When to use this

Page-based — static views, dashboards, and most everyday queries. Easy and intuitive.
Keyset (cursor or range) — large datasets (entire collections, 500k+ assets) where page-based crawling becomes slow.
Parallel keyset — the fastest option for scanning an entire collection by partitioning the address range.

Sorting options

You can sort results by different fields using the sortBy field:

Value	Sorts by	Recommended?
`id`	Asset ID in binary (default)	Yes
`created`	Date the asset was created	Yes
`recent_action`	Date the asset was last updated	No
`none`	No sorting	No

Disabling sorting yields the fastest results, but because the data is unordered you may get inconsistent results when paginating.

Page-based pagination

You specify the page number and the number of items per page. To move to the next page, increment the page number. This is easy, intuitive, and fast for most use cases.

Example

const url = `https://mainnet.helius-rpc.com/?api-key=YOUR_API_KEY`

const example = async () => {
    let page = 1;
    let items = [];
    while (true) {
        const response = await fetch(url, {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
            },
            body: JSON.stringify({
                jsonrpc: '2.0',
                id: 'my-id',
                method: 'searchAssets',
                params: {
                    grouping: ['collection', '5PA96eCFHJSFPY9SWFeRJUHrpoNF5XZL6RrE1JADXhxf'],
                    page: page,
                    limit: 1000,
                    sortBy: { sortBy: 'id', sortDirection: 'asc' },
                },
            }),
        });
        const { result } = await response.json();
        if (result.items.length == 0) {
            console.log('No items remaining');
            break;
        } else {
            console.log(`Processing results from page ${page}`);
            items.push(...result.items);
            page += 1;
        }
    }
    console.log(`Got ${items.length} total items`);
};
example();

Using pages requires the database to crawl across all items until it reaches the next page. For example, if you ask for page 100 with a page size of 1,000, the database must traverse the first 1M records before returning your data. For this reason, page-based pagination is not recommended for large datasets — keyset pagination is far better suited to those workloads.

Keyset pagination

You define pages by providing conditions that filter the dataset. For example, “get me all assets with an ID > X but an ID < Y.” You traverse the entire dataset by modifying X or Y on each call. There are two methods of keyset pagination:

Cursor-based — easier to use but less flexible.
Range-based — more complex but very flexible.

Keyset pagination is only supported when sorting by id.

Cursor-based

A DAS query without any pagination parameters returns a cursor. Pass the cursor back to the DAS API to continue from where you left off.

Example

const url = `https://mainnet.helius-rpc.com/?api-key=YOUR_API_KEY`

const example = async () => {
    let items = [];
    let cursor;
    while (true) {
        let params = {
            grouping: ['collection', '5PA96eCFHJSFPY9SWFeRJUHrpoNF5XZL6RrE1JADXhxf'],
            limit: 1000,
            sortBy: { sortBy: 'id', sortDirection: 'asc' },
        } as any;
        if (cursor != undefined) {
            params.cursor = cursor;
        }
        const response = await fetch(url, {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
            },
            body: JSON.stringify({
                jsonrpc: '2.0',
                id: 'my-id',
                method: 'searchAssets',
                params: params,
            }),
        });
        const { result } = await response.json();
        if (result.items.length == 0) {
            console.log('No items remaining');
            break;
        } else {
            console.log(`Processing results for cursor ${cursor}`);
            cursor = result.cursor;
            items.push(...result.items);
        }
    }
    console.log(`Got ${items.length} total items`);
};
example();

At the time of writing, the cursor is the last asset ID of the response; however, the cursor design is flexible and can support any string.

Range-based

To query across a range, specify before and/or after. The query is essentially “get me all assets after X but before Y.” You traverse the dataset by updating the before or after parameter on each call.

Example

const url = `https://mainnet.helius-rpc.com/?api-key=YOUR_API_KEY`

const example = async () => {
    // Two NFTs from the Tensorian collection.
    // The "start" item has a lower asset ID (in binary) than the "end" item.
    // We will traverse in ascending order.
    let start = '6CeKtAYX5USSvPCQicwFsvN4jQSHNxQuFrX2bimWrNey';
    let end = 'CzTP4fUbdfgKzwE6T94hsYV7NWf1SzuCCsmJ6RP1xsDw';
    let sortDirection = 'asc';
    let after = start;
    let before = end;
    let items = [];

    while (true) {
        const response = await fetch(url, {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
            },
            body: JSON.stringify({
                jsonrpc: '2.0',
                id: 'my-id',
                method: 'searchAssets',
                params: {
                    grouping: ['collection', '5PA96eCFHJSFPY9SWFeRJUHrpoNF5XZL6RrE1JADXhxf'],
                    limit: 1000,
                    after: after,
                    before: before,
                    sortBy: { sortBy: 'id', sortDirection: sortDirection },
                },
            }),
        });
        const { result } = await response.json();
        if (result.items.length == 0) {
            console.log('No items remaining');
            break;
        } else {
            console.log(`Processing results with (after: ${after}, before: ${before})`);
            after = result.items[result.items.length - 1].id;
            items.push(...result.items);
        }
    }
    console.log(`Got ${items.length} total items`);
};
example();

Parallel querying with keysets (advanced)

Advanced users querying large datasets (for example, entire compressed NFT collections) should use keyset-based pagination for performance. The following example shows how to query in parallel by partitioning the Solana address range and using the before/after parameters. This method is fast, efficient, and safe.

Example

In the example below, we scan the entire Tensorian collection (~10k records). It partitions the Solana address space into 8 ranges and scans those ranges concurrently. This is far faster than the other approaches.

import base58 from 'bs58';

const url = `https://mainnet.helius-rpc.com/?api-key=YOUR_API_KEY`

const main = async () => {
    let numParitions = 8;
    let partitons = partitionAddressRange(numParitions);
    let promises = [];
    for (const [i, partition] of partitons.entries()) {
        let [s, e] = partition;
        let start = bs58.encode(s);
        let end = bs58.encode(e);
        console.log(`Parition: ${i}, Start: ${start}, End: ${end}`);

        let promise: Promise<number> = new Promise(async (resolve, reject) => {
            let current = start;
            let totalForPartition = 0;
            while (true) {
                const response = await fetch(url, {
                    method: 'POST',
                    headers: {
                        'Content-Type': 'application/json',
                    },
                    body: JSON.stringify({
                        jsonrpc: '2.0',
                        id: 'my-id',
                        method: 'searchAssets',
                        params: {
                            grouping: ['collection', '5PA96eCFHJSFPY9SWFeRJUHrpoNF5XZL6RrE1JADXhxf'],
                            limit: 1000,
                            after: current,
                            before: end,
                            sortBy: { sortBy: 'id', sortDirection: 'asc' },
                        },
                    }),
                });
                const { result } = await response.json();
                totalForPartition += result.items.length;
                console.log(`Found ${totalForPartition} total items in parition ${i}`);
                if (result.items.length == 0) {
                    break;
                } else {
                    current = result.items[result.items.length - 1].id;
                }
            }
            resolve(totalForPartition);
        });
        promises.push(promise);
    }
    let results = await Promise.all(promises);
    let total = results.reduce((a, b) => a + b, 0);
    console.log(`Got ${total} total items`);
};

// Function to convert a BigInt to a byte array
function bigIntToByteArray(bigInt: bigint): Uint8Array {
    const bytes = [];
    let remainder = bigInt;
    while (remainder > 0n) {
        // use 0n for bigint literal
        bytes.unshift(Number(remainder & 0xffn));
        remainder >>= 8n;
    }
    while (bytes.length < 32) bytes.unshift(0); // pad with zeros to get 32 bytes
    return new Uint8Array(bytes);
}

function partitionAddressRange(numPartitions: number) {
    let N = BigInt(numPartitions);

    // Largest and smallest Solana addresses in integer form.
    // Solana addresses are 32 byte arrays.
    const start = 0n;
    const end = 2n ** 256n - 1n;

    // Calculate the number of partitions and partition size
    const range = end - start;
    const partitionSize = range / N;

    // Calculate partition ranges
    const partitions: Uint8Array[][] = [];
    for (let i = 0n; i < N; i++) {
        const s = start + i * partitionSize;
        const e = i === N - 1n ? end : s + partitionSize;
        partitions.push([bigIntToByteArray(s), bigIntToByteArray(e)]);
    }

    return partitions;
}

main();

Next steps

Search Assets

Filter assets by owner, collection, and tokenType.

Get Assets (NFTs)

Retrieve NFTs, collections, editions, and proofs.

DAS API reference

Full schemas for every DAS method.

​Overview

​When to use this

​Sorting options

​Page-based pagination

​Keyset pagination

​Cursor-based

​Range-based

​Parallel querying with keysets (advanced)

​Next steps

Search Assets

Get Assets (NFTs)

DAS API reference

Overview

When to use this

Sorting options

Page-based pagination

Keyset pagination

Cursor-based

Range-based

Parallel querying with keysets (advanced)

Next steps