HyperliquidTradeHistoryDatabase

Documentation for eth_defi.hyperliquid.trade_history_db.HyperliquidTradeHistoryDatabase Python class.

class HyperliquidTradeHistoryDatabase

Bases: object

DuckDB database for storing Hyperliquid account trading data.

Stores fills, funding payments, and ledger updates for whitelisted accounts. Supports incremental sync that accumulates data beyond the 10K fill API limit.

The database is crash-resumeable: interrupted syncs can be safely re-run without data loss or duplicates.

Thread safety: all database operations are protected by an internal lock. Multiple threads can call sync methods concurrently – the API calls run in parallel while database writes are serialised.

Initialise the database connection.

Parameters

path – Path to the DuckDB file. Parent directories are created if needed.

Attributes summary

VAULT_LEDGER_EVENT_TYPES

Ledger event types that only appear for vault accounts

Methods summary

__init__(path)

Initialise the database connection.

add_account(address[, label, is_vault])

Add an account to the whitelist.

close()

Close the database connection.

get_accounts([is_vault])

Get whitelisted accounts, optionally filtered by vault status.

get_fill_count(address)

Get the number of stored fills for an account.

get_fills(address[, start_time, end_time])

Get stored fills for an account as Fill objects.

get_funding(address[, start_time, end_time])

Get stored funding payments for an account.

get_funding_count(address)

Get the number of stored funding payments for an account.

get_ledger(address[, start_time, end_time])

Get stored ledger events for an account.

get_ledger_count(address)

Get the number of stored ledger events for an account.

get_sync_state(address)

Get sync state for all data types for an account.

get_total_row_counts()

Get total row counts across all accounts for each table.

is_vault_address(address)

Detect whether an account is a vault from its stored ledger events.

remove_account(address[, purge_data])

Remove an account from the whitelist.

save()

Force a checkpoint to ensure data is persisted to disk.

sync_account(session, address[, start_time, ...])

Sync all data types for a single account.

sync_account_fills(session, address[, ...])

Fetch new fills since last sync and store them.

sync_account_funding(session, address[, ...])

Fetch new funding payments since last sync and store them.

sync_account_ledger(session, address[, ...])

Fetch new ledger events since last sync and store them.

sync_all(session[, max_workers, timeout, ...])

Sync whitelisted accounts, optionally filtered by vault status.
__init__(path)

Initialise the database connection.

Parameters

path (pathlib.Path) – Path to the DuckDB file. Parent directories are created if needed.

close()

Close the database connection.

Uses _db_lock so any in-flight database operation completes before the connection is torn down.

save()

Force a checkpoint to ensure data is persisted to disk.

add_account(address, label=None, is_vault=True)

Add an account to the whitelist.

Idempotent — re-adding an existing account updates the label. The is_vault flag can only be upgraded from False to True, never downgraded, to prevent SCAN=top_traders runs from incorrectly clearing the vault flag on known vaults.

Parameters

  • address (eth_typing.evm.HexAddress) – Hyperliquid account address.

  • label (str | None) – Human-readable name (e.g. “Growi HF”).

  • is_vault (bool) – Whether this is a vault account.

Return type

None

VAULT_LEDGER_EVENT_TYPES: set[str] = {'vaultCreate', 'vaultDeposit', 'vaultDistribution', 'vaultLeaderCommission', 'vaultWithdraw'}

Ledger event types that only appear for vault accounts

is_vault_address(address)

Detect whether an account is a vault from its stored ledger events.

Scans for vault-specific event types (vaultCreate, vaultDeposit, vaultWithdraw, etc.) which only appear for vault accounts. This is more reliable than the is_vault flag in the accounts table, which can be incorrectly set when accounts are added via trader scanning modes.

Falls back to the is_vault flag if no ledger events exist yet.

Parameters

address (eth_typing.evm.HexAddress) – Account address.

Returns

True if vault-specific ledger events are found, or if the is_vault flag is True in the accounts table.

Return type

bool

remove_account(address, purge_data=False)

Remove an account from the whitelist.

Parameters

  • address (eth_typing.evm.HexAddress) – Account address to remove.

  • purge_data (bool) – If True, also delete all stored data for this account.

Return type

None

get_accounts(is_vault=None)

Get whitelisted accounts, optionally filtered by vault status.

Parameters

is_vault (bool | None) – If True, return only vault accounts. If False, return only trader accounts. If None (default), return all accounts.

Returns

List of account dicts with address, label, is_vault, added_at.

Return type

list[dict]

sync_account_fills(session, address, start_time=None, end_time=None, timeout=30.0, progress=None)

Fetch new fills since last sync and store them.

Incremental: only fetches fills newer than the last stored timestamp. Uses INSERT OR IGNORE to handle overlapping batches safely.

Proxy support is handled by the session’s built-in proxy rotation via post_info().

Parameters
Returns

Number of new fills inserted.

Return type

int

sync_account_funding(session, address, start_time=None, end_time=None, timeout=30.0, progress=None)

Fetch new funding payments since last sync and store them.

Proxy support is handled by the session’s built-in proxy rotation via post_info().

Parameters
Returns

Number of new funding payments inserted.

Return type

int

sync_account_ledger(session, address, start_time=None, end_time=None, timeout=30.0, progress=None)

Fetch new ledger events since last sync and store them.

Proxy support is handled by the session’s built-in proxy rotation via post_info().

Parameters
Returns

Number of new ledger events inserted.

Return type

int

sync_account(session, address, start_time=None, end_time=None, timeout=30.0, progress=None, label=None)

Sync all data types for a single account.

Each data type (fills, funding, ledger) is synced independently with its own sync_state watermark. Individual batch inserts use INSERT OR IGNORE for idempotent crash recovery.

When progress is provided, it is reused across all three data types. The bar description updates as it moves through fills, funding, and ledger.

Proxy support is handled by the session’s built-in proxy rotation. In threaded mode, each worker should receive its own session clone via clone_for_worker().

Parameters
Returns

Dict with counts: {"fills": N, "funding": N, "ledger": N}.

Return type

dict[str, int]

sync_all(session, max_workers=1, timeout=30.0, is_vault=None)

Sync whitelisted accounts, optionally filtered by vault status.

When max_workers > 1, accounts are synced in parallel using a thread pool. Each worker gets its own session clone via clone_for_worker(), which shares rate-limiter adapters and ProxyStateManager but starts on a different proxy for load distribution across IPs.

Proxy support is configured on the session itself via configure_rotator() or the rotator / proxy_urls parameter of create_hyperliquid_session().

Progress display:

  • Position 0: overall account progress

  • Positions 1..N: one bar per worker showing current data type

Parameters

  • session (eth_defi.hyperliquid.session.HyperliquidSession) – Hyperliquid API session (with optional proxy configuration).

  • max_workers (int) – Number of parallel workers for concurrent API calls.

  • timeout (float) – HTTP request timeout.

  • is_vault (bool | None) – If True, sync only vault accounts. If False, sync only trader accounts. If None (default), sync all accounts.

Returns

Dict mapping address to sync counts.

Return type

dict[str, dict[str, int]]

get_fills(address, start_time=None, end_time=None)

Get stored fills for an account as Fill objects.

Parameters
Returns

List of Fill objects sorted by timestamp ascending.

Return type

list[eth_defi.hyperliquid.position.Fill]

get_funding(address, start_time=None, end_time=None)

Get stored funding payments for an account.

Parameters
Returns

List of FundingPayment objects sorted by timestamp ascending.

Return type

list[eth_defi.hyperliquid.trade_history.FundingPayment]

get_fill_count(address)

Get the number of stored fills for an account.

Parameters

address (eth_typing.evm.HexAddress) –

Return type

int

get_funding_count(address)

Get the number of stored funding payments for an account.

Parameters

address (eth_typing.evm.HexAddress) –

Return type

int

get_ledger(address, start_time=None, end_time=None)

Get stored ledger events for an account.

Parameters
Returns

List of LedgerEvent objects sorted by timestamp ascending.

Return type

list[eth_defi.hyperliquid.trade_history_db.LedgerEvent]

get_ledger_count(address)

Get the number of stored ledger events for an account.

Parameters

address (eth_typing.evm.HexAddress) –

Return type

int

get_total_row_counts()

Get total row counts across all accounts for each table.

Returns

Dict with keys fills, funding, ledger and integer counts.

Return type

dict[str, int]

get_sync_state(address)

Get sync state for all data types for an account.

Parameters

address (eth_typing.evm.HexAddress) – Account address.

Returns

Dict mapping data_type to state dict with oldest_ts, newest_ts, row_count, last_synced.

Return type

dict[str, dict]