middleware

Documentation for eth_defi.middleware Python module.

Web3 middleware.

Most are for dealing with JSON-RPC unreliability issues with retries.

• Taken from exception_retry_request.py from Web3.py

• Modified to support sleep and throttling

• Logs warnings to Python logging subsystem in the case there is need to retry

• See also eth_defi.provider.broken_provider.

Module Attributes

DEFAULT_RETRYABLE_EXCEPTIONS	List of Web3 exceptions we know we should retry after some timeout
DEFAULT_RETRYABLE_HTTP_STATUS_CODES	List of HTTP status codes we know we might want to retry after a timeout
DEFAULT_RETRYABLE_RPC_ERROR_CODES	List of ValueError status codes we know we might want to retry after a timeout
DEFAULT_RETRYABLE_RPC_ERROR_MESSAGES	Because Ethreum JSON-RPC API is horribly broken, we also need to check for error messages besides error codes.
STATIC_CALL_LIST	Ethereum JSON-RPC calls where the value never changes

Functions

configure_provider_retry(provider[, ...])	Configure provider retry settings for web3.py v7+.
construct_sign_and_send_raw_middleware_anvil(...)	Capture transactions sign and send as raw transactions - v6/v7 compatible.
exception_retry_middleware(make_request, ...)	Creates middleware that retries failed HTTP requests.
http_retry_request_with_sleep_middleware(...)	A HTTP retry middleware with sleep and backoff.
is_retryable_http_exception(exc[, ...])	Helper to check retryable errors from JSON-RPC calls.
raise_on_revert_middleware(make_request, web3)	Automatically show the transaction revert reason in Python traceback.

Classes

StaticCallCacheMiddleware	v7-style static call cache middleware.
static_call_cache_middleware	alias of eth_defi.middleware.StaticCallCacheMiddleware

Exceptions

ProbablyNodeHasNoBlock	A special exception raised when we suspect JSON-RPC node does not yet have data for a block we asked.
SomeCrappyRPCProviderException	Deal with non-standard RPC providers and whatever shitty logic they have invented for error codes

exception SomeCrappyRPCProviderException

Bases: Exception

Deal with non-standard RPC providers and whatever shitty logic they have invented for error codes

__init__(*args, **kwargs)

__new__(**kwargs)

add_note()

Exception.add_note(note) – add a note to the exception

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

DEFAULT_RETRYABLE_EXCEPTIONS: Tuple[BaseException] = (<class 'requests.exceptions.ConnectionError'>, <class 'requests.exceptions.HTTPError'>, <class 'requests.exceptions.Timeout'>, <class 'requests.exceptions.TooManyRedirects'>, <class 'web3.exceptions.BlockNotFound'>, <class 'requests.exceptions.ChunkedEncodingError'>, <class 'http.client.RemoteDisconnected'>, <class 'eth_defi.middleware.SomeCrappyRPCProviderException'>, <class 'requests.exceptions.ContentDecodingError'>)

List of Web3 exceptions we know we should retry after some timeout

For BlockNotFound see also eth_defi.rpc.broken_provider.

DEFAULT_RETRYABLE_HTTP_STATUS_CODES = (429, 500, 502, 503, 504, 525, 520, 410, 403, 400)

List of HTTP status codes we know we might want to retry after a timeout

Taken from https://stackoverflow.com/a/72302017/315168

DEFAULT_RETRYABLE_RPC_ERROR_CODES = (-32003, -32043, -32005, -32701, 42903, -32002, -32603, -32052, -32601)

List of ValueError status codes we know we might want to retry after a timeout

This is a self-managed list curated by pain.

JSON-RPC error might be mapped to ValueError if nothing else is available.

Example from Pokt Network:

ValueError: {‘message’: ‘Internal JSON-RPC error.’, ‘code’: -32603}

We assume this is a broken RPC node and Pokt will reroute the the next retried request to some other node.

See GoEthereum error codes https://github.com/ethereum/go-ethereum/blob/master/rpc/errors.go

DEFAULT_RETRYABLE_RPC_ERROR_MESSAGES = {'Internal JSON-RPC error', 'Parse error', 'Unexpected error (code=40000)', 'empty reader set', 'execution aborted (timeout = 5s)', 'header not found', 'nonce too low'}

Because Ethreum JSON-RPC API is horribly broken, we also need to check for error messages besides error codes.

See DEFAULT_RETRYABLE_RPC_ERROR_CODES.

STATIC_CALL_LIST = ('eth_chainId',)

Ethereum JSON-RPC calls where the value never changes

exception ProbablyNodeHasNoBlock

Bases: Exception

A special exception raised when we suspect JSON-RPC node does not yet have data for a block we asked.

• Calling a contract on a block before contract was deployed

• Calling a contract on a block where the node does not have the block data yet

See eth_defi.provider.fallback for details.

__init__(*args, **kwargs)

__new__(**kwargs)

add_note()

Exception.add_note(note) – add a note to the exception

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

is_retryable_http_exception(exc, retryable_exceptions=(<class 'requests.exceptions.ConnectionError'>, <class 'requests.exceptions.HTTPError'>, <class 'requests.exceptions.Timeout'>, <class 'requests.exceptions.TooManyRedirects'>, <class 'web3.exceptions.BlockNotFound'>, <class 'requests.exceptions.ChunkedEncodingError'>, <class 'http.client.RemoteDisconnected'>, <class 'eth_defi.middleware.SomeCrappyRPCProviderException'>, <class 'requests.exceptions.ContentDecodingError'>), retryable_status_codes=(429, 500, 502, 503, 504, 525, 520, 410, 403, 400), retryable_rpc_error_codes=(-32003, -32043, -32005, -32701, 42903, -32002, -32603, -32052, -32601), retryable_rpc_error_messages={'Internal JSON-RPC error', 'Parse error', 'Unexpected error (code=40000)', 'empty reader set', 'execution aborted (timeout = 5s)', 'header not found', 'nonce too low'}, method=None, params=None)

Helper to check retryable errors from JSON-RPC calls.

Retryable reasons are connection timeouts, API throttling and such.

We support various kind of exceptions and HTTP status codes we know we can try.

Parameters

• exc (Exception) – Exception raised by requests or Web3 machinery.

• retryable_exceptions (Tuple[BaseException]) – Exception raised by requests or Web3 machinery.

• retryable_status_codes (Collection[int]) – HTTP status codes we can retry. E.g. 429 Too Many requests.

• retryable_rpc_error_messages (Collection[str]) – See DEFAULT_RETRYABLE_RPC_ERROR_MESSAGES.

• method (str | None) – JSON-RPC method name we called.

• params (list | None) – Method args.

• retryable_rpc_error_codes (Collection[int]) –

exception_retry_middleware(make_request, web3, retryable_exceptions, retryable_status_codes, retryable_rpc_error_codes, retries=10, sleep=5.0, backoff=1.6)

Creates middleware that retries failed HTTP requests. Is a default middleware for HTTPProvider.

MIGRATED: Now uses compat version for v6/v7 compatibility.

See http_retry_request_with_sleep_middleware() for usage.

Parameters

• make_request (Callable[[web3.types.RPCEndpoint, Any], web3.types.RPCResponse]) –

• web3 (web3.main.Web3) –

• retryable_exceptions (Tuple[BaseException]) –

• retryable_status_codes (Collection[int]) –

• retryable_rpc_error_codes (Collection[int]) –

• retries (int) –

• sleep (float) –

• backoff (float) –

Return type

Callable[[web3.types.RPCEndpoint, Any], web3.types.RPCResponse]

http_retry_request_with_sleep_middleware(make_request, web3)

A HTTP retry middleware with sleep and backoff.

MIGRATED: In web3.py v7+, this function is deprecated in favor of ExceptionRetryConfiguration on the provider. However, for backwards compatibility, this function still works but may not be called if v7 provider retry configuration is used instead.

If you want to customise timeouts, supported exceptions and such you can directly create your own middleware using exception_retry_middleware().

Usage:

web3.middleware_onion.clear()
web3.middleware_onion.inject(http_retry_request_with_sleep_middleware, layer=0)

Parameters

• make_request (Callable[[web3.types.RPCEndpoint, Any], Any]) – Part of middleware call signature

• web3 (web3.main.Web3) – Part of middleware call signature

Returns

Web3.py middleware

Return type

Callable[[web3.types.RPCEndpoint, Any], Any]

configure_provider_retry(provider, retries=10, backoff_factor=0.5, retryable_exceptions=None)

Configure provider retry settings for web3.py v7+.

This is the recommended way to configure retries in v7+.

Parameters

• provider – HTTPProvider or AsyncHTTPProvider instance

• retries (int) – Number of retries to attempt (default 10)

• backoff_factor (float) – Initial delay multiplier (default 0.5)

• retryable_exceptions (tuple) – Tuple of exceptions to retry on

raise_on_revert_middleware(make_request, web3)

Automatically show the transaction revert reason in Python traceback.

• Designed to make writing unit tests more productive

• Transaction will already revert in eth_estimateGas call unless you have manually set the gas limit for your transaction

• If a transaction fails, this middleware display its revert reason in Python exception message

• Tested with Anvil testing backend

• May interfere with http_retry_request_with_sleep_middleware(), others, so don’t use in production

from eth_defi.middleware import revert_reason_middleware

# Fix the web3.py stock gas estimate middlware with smarted one
web3.middleware_onion.replace("gas_estimate", revert_reason_aware_buffered_gas_estimate_middleware)

# Now you check the revert reason as the following

Parameters

• make_request (Callable[[web3.types.RPCEndpoint, Any], Any]) –

• web3 (web3.main.Web3) –

Return type

Callable[[web3.types.RPCEndpoint, Any], Any]

construct_sign_and_send_raw_middleware_anvil(private_key_or_account)

Capture transactions sign and send as raw transactions - v6/v7 compatible.

Return type

Type[web3.middleware.base.Web3Middleware]

class StaticCallCacheMiddleware

Bases: web3.middleware.base.Web3Middleware

v7-style static call cache middleware.

__init__(w3)

static_call_cache_middleware

alias of eth_defi.middleware.StaticCallCacheMiddleware