hyperliquid.trade_history_db

Documentation for eth_defi.hyperliquid.trade_history_db Python module.

DuckDB persistence for Hyperliquid account trading data.

Stores fills, funding payments, and ledger updates for a whitelisted set of accounts (vaults or normal addresses). Incremental sync accumulates data beyond the 10K fill API limit by fetching only new records on each run.

The sync is crash-resumeable: partial batches are safely re-inserted on restart via INSERT OR IGNORE on natural primary keys.

Schema

Five tables:

  • accounts – whitelisted addresses to track

  • fills – individual trade fills from userFillsByTime

  • funding – funding payments from userFunding

  • ledger – deposit/withdrawal events from userNonFundingLedgerUpdates

  • sync_state – per-account watermarks for incremental sync

Storage location

Default: ~/.tradingstrategy/vaults/hyperliquid/trade-history.duckdb

Example:

from pathlib import Path
from eth_defi.hyperliquid.session import create_hyperliquid_session
from eth_defi.hyperliquid.trade_history_db import HyperliquidTradeHistoryDatabase

session = create_hyperliquid_session()
db = HyperliquidTradeHistoryDatabase(Path("/tmp/trade-history.duckdb"))

db.add_account("0x1e37a337ed460039d1b15bd3bc489de789768d5e", label="Growi HF")
db.sync_account(session, "0x1e37a337ed460039d1b15bd3bc489de789768d5e")

fills = db.get_fills("0x1e37a337ed460039d1b15bd3bc489de789768d5e")
print(f"Stored {len(fills)} fills")

db.close()

Module Attributes

MAX_PER_REQUEST

Maximum records per API request for fills and ledger

MAX_FUNDING_PER_REQUEST

Maximum records per API request for funding

Classes

HyperliquidTradeHistoryDatabase

DuckDB database for storing Hyperliquid account trading data.

LedgerEvent

A deposit, withdrawal, or other non-funding ledger event from DuckDB storage.
class LedgerEvent

Bases: object

A deposit, withdrawal, or other non-funding ledger event from DuckDB storage.

Represents a row from the ledger table. The event_type field contains the raw API type string (e.g. "vaultDeposit", "vaultWithdraw", "deposit", "withdraw").

timestamp: datetime.datetime

Event timestamp

timestamp_ms: int

Timestamp in milliseconds (for storage and sorting)

event_type: str

Raw API event type (e.g. “vaultDeposit”, “withdraw”, “deposit”)

usdc: float

USDC amount

vault: str | None

Associated vault address (if any)

__init__(timestamp, timestamp_ms, event_type, usdc, vault)
Parameters
Return type

None

DEFAULT_TRADE_HISTORY_DB_PATH = PosixPath('/home/runner/.tradingstrategy/vaults/hyperliquid/trade-history.duckdb')

Default DuckDB path for trade history

MAX_PER_REQUEST = 2000

Maximum records per API request for fills and ledger

MAX_FUNDING_PER_REQUEST = 500

Maximum records per API request for funding

class HyperliquidTradeHistoryDatabase

Bases: object

DuckDB database for storing Hyperliquid account trading data.

Stores fills, funding payments, and ledger updates for whitelisted accounts. Supports incremental sync that accumulates data beyond the 10K fill API limit.

The database is crash-resumeable: interrupted syncs can be safely re-run without data loss or duplicates.

Thread safety: all database operations are protected by an internal lock. Multiple threads can call sync methods concurrently – the API calls run in parallel while database writes are serialised.

Initialise the database connection.

Parameters

path – Path to the DuckDB file. Parent directories are created if needed.

__init__(path)

Initialise the database connection.

Parameters

path (pathlib.Path) – Path to the DuckDB file. Parent directories are created if needed.

close()

Close the database connection.

Uses _db_lock so any in-flight database operation completes before the connection is torn down.

save()

Force a checkpoint to ensure data is persisted to disk.

add_account(address, label=None, is_vault=True)

Add an account to the whitelist.

Idempotent — re-adding an existing account updates the label. The is_vault flag can only be upgraded from False to True, never downgraded, to prevent SCAN=top_traders runs from incorrectly clearing the vault flag on known vaults.

Parameters

  • address (eth_typing.evm.HexAddress) – Hyperliquid account address.

  • label (str | None) – Human-readable name (e.g. “Growi HF”).

  • is_vault (bool) – Whether this is a vault account.

Return type

None

VAULT_LEDGER_EVENT_TYPES: set[str] = {'vaultCreate', 'vaultDeposit', 'vaultDistribution', 'vaultLeaderCommission', 'vaultWithdraw'}

Ledger event types that only appear for vault accounts

is_vault_address(address)

Detect whether an account is a vault from its stored ledger events.

Scans for vault-specific event types (vaultCreate, vaultDeposit, vaultWithdraw, etc.) which only appear for vault accounts. This is more reliable than the is_vault flag in the accounts table, which can be incorrectly set when accounts are added via trader scanning modes.

Falls back to the is_vault flag if no ledger events exist yet.

Parameters

address (eth_typing.evm.HexAddress) – Account address.

Returns

True if vault-specific ledger events are found, or if the is_vault flag is True in the accounts table.

Return type

bool

remove_account(address, purge_data=False)

Remove an account from the whitelist.

Parameters

  • address (eth_typing.evm.HexAddress) – Account address to remove.

  • purge_data (bool) – If True, also delete all stored data for this account.

Return type

None

get_accounts(is_vault=None)

Get whitelisted accounts, optionally filtered by vault status.

Parameters

is_vault (bool | None) – If True, return only vault accounts. If False, return only trader accounts. If None (default), return all accounts.

Returns

List of account dicts with address, label, is_vault, added_at.

Return type

list[dict]

sync_account_fills(session, address, start_time=None, end_time=None, timeout=30.0, progress=None)

Fetch new fills since last sync and store them.

Incremental: only fetches fills newer than the last stored timestamp. Uses INSERT OR IGNORE to handle overlapping batches safely.

Proxy support is handled by the session’s built-in proxy rotation via post_info().

Parameters
Returns

Number of new fills inserted.

Return type

int

sync_account_funding(session, address, start_time=None, end_time=None, timeout=30.0, progress=None)

Fetch new funding payments since last sync and store them.

Proxy support is handled by the session’s built-in proxy rotation via post_info().

Parameters
Returns

Number of new funding payments inserted.

Return type

int

sync_account_ledger(session, address, start_time=None, end_time=None, timeout=30.0, progress=None)

Fetch new ledger events since last sync and store them.

Proxy support is handled by the session’s built-in proxy rotation via post_info().

Parameters
Returns

Number of new ledger events inserted.

Return type

int

sync_account(session, address, start_time=None, end_time=None, timeout=30.0, progress=None, label=None)

Sync all data types for a single account.

Each data type (fills, funding, ledger) is synced independently with its own sync_state watermark. Individual batch inserts use INSERT OR IGNORE for idempotent crash recovery.

When progress is provided, it is reused across all three data types. The bar description updates as it moves through fills, funding, and ledger.

Proxy support is handled by the session’s built-in proxy rotation. In threaded mode, each worker should receive its own session clone via clone_for_worker().

Parameters
Returns

Dict with counts: {"fills": N, "funding": N, "ledger": N}.

Return type

dict[str, int]

sync_all(session, max_workers=1, timeout=30.0, is_vault=None)

Sync whitelisted accounts, optionally filtered by vault status.

When max_workers > 1, accounts are synced in parallel using a thread pool. Each worker gets its own session clone via clone_for_worker(), which shares rate-limiter adapters and ProxyStateManager but starts on a different proxy for load distribution across IPs.

Proxy support is configured on the session itself via configure_rotator() or the rotator / proxy_urls parameter of create_hyperliquid_session().

Progress display:

  • Position 0: overall account progress

  • Positions 1..N: one bar per worker showing current data type

Parameters

  • session (eth_defi.hyperliquid.session.HyperliquidSession) – Hyperliquid API session (with optional proxy configuration).

  • max_workers (int) – Number of parallel workers for concurrent API calls.

  • timeout (float) – HTTP request timeout.

  • is_vault (bool | None) – If True, sync only vault accounts. If False, sync only trader accounts. If None (default), sync all accounts.

Returns

Dict mapping address to sync counts.

Return type

dict[str, dict[str, int]]

get_fills(address, start_time=None, end_time=None)

Get stored fills for an account as Fill objects.

Parameters
Returns

List of Fill objects sorted by timestamp ascending.

Return type

list[eth_defi.hyperliquid.position.Fill]

get_funding(address, start_time=None, end_time=None)

Get stored funding payments for an account.

Parameters
Returns

List of FundingPayment objects sorted by timestamp ascending.

Return type

list[eth_defi.hyperliquid.trade_history.FundingPayment]

get_fill_count(address)

Get the number of stored fills for an account.

Parameters

address (eth_typing.evm.HexAddress) –

Return type

int

get_funding_count(address)

Get the number of stored funding payments for an account.

Parameters

address (eth_typing.evm.HexAddress) –

Return type

int

get_ledger(address, start_time=None, end_time=None)

Get stored ledger events for an account.

Parameters
Returns

List of LedgerEvent objects sorted by timestamp ascending.

Return type

list[eth_defi.hyperliquid.trade_history_db.LedgerEvent]

get_ledger_count(address)

Get the number of stored ledger events for an account.

Parameters

address (eth_typing.evm.HexAddress) –

Return type

int

get_total_row_counts()

Get total row counts across all accounts for each table.

Returns

Dict with keys fills, funding, ledger and integer counts.

Return type

dict[str, int]

get_sync_state(address)

Get sync state for all data types for an account.

Parameters

address (eth_typing.evm.HexAddress) – Account address.

Returns

Dict mapping data_type to state dict with oldest_ts, newest_ts, row_count, last_synced.

Return type

dict[str, dict]