cryptodatapy.extract.libraries

Submodules

Classes

`CCXT`	Retrieves data from CCXT API.
`DBnomics`	Retrieves data from DBnomics API.
`Library`	Library is an abstract base class which provides a blueprint for properties and methods for the
`PandasDataReader`	Retrieves data from Pandas Data Reader API.

Package Contents

class cryptodatapy.extract.libraries.CCXT(categories: str | List[str] = 'crypto', exchanges: List[str] | None = None, indexes: List[str] | None = None, assets: Dict[str, List[str]] | None = None, markets: Dict[str, List[str]] | None = None, market_types: List[str] = ['spot', 'future', 'perpetual_future', 'option'], fields: List[str] | None = ['open', 'high', 'low', 'close', 'volume', 'funding_rate', 'oi'], frequencies: Dict[str, str | int] | None = None, base_url: str | None = None, api_key: str | None = None, max_obs_per_call: int | None = 1000, rate_limit: Any | None = None, ip_ban_wait_time_s: float = 320.0, recovery_base_delay_s: float = 2.0, max_recovery_delay_s: float = 60.0)

Bases: cryptodatapy.extract.libraries.library.Library

Retrieves data from CCXT API.

exchange = None

exchange_async = None

data_req = None

ip_ban_wait_time_s = 320.0

recovery_base_delay_s = 2.0

max_recovery_delay_s = 60.0

data_resp = []

data

get_exchanges_info() → List[str]

Get exchanges info.

Returns:: exch – List or dataframe with info on supported exchanges.
Return type:: list or pd.DataFrame

get_indexes_info() → None: Get indexes info.

get_assets_info(exch: str, as_list: bool = False) → pandas.DataFrame | List[str]

Get assets info.

Parameters:

exch (str) – Name of exchange.
as_list (bool, default False) – Returns assets info for selected exchanges as list.

Returns:

assets – Dataframe with info on available assets or list of assets.

Return type:

list or pd.DataFrame

get_markets_info(exch: str, quote_ccy: str | None = None, mkt_type: str | None = None, as_list: bool = False) → pandas.DataFrame | List[str]

Get markets info.

Parameters:

exch (str) – Name of exchange.
quote_ccy (str, optional, default None) – Quote currency.
mkt_type (str, {'spot', 'future', 'perpetual_future', 'option'}, optional, default None) – Market type.
as_list (bool, default False) – Returns markets info as list for selected exchange.

Returns:

markets – List or dataframe with info on available markets, by exchange.

Return type:

list or pd.DataFrame

get_fields_info() → List[str]

Get fields info.

Returns:: fields – List of available fields.
Return type:: list

get_frequencies_info(exch: str) → Dict[str, str | int]

Get frequencies info.

Parameters:: exch (str) – Name of exchange for which to get available assets.
Returns:: freq – Dictionary with info on available frequencies.
Return type:: dictionary

get_rate_limit_info(exch: str) → Dict[str, str | int]

Get rate limit info.

Parameters:: exch (str) – Name of exchange.
Returns:: rate_limit – Dictionary with exchange and required minimal delay between HTTP requests that exchange in milliseconds.
Return type:: dictionary

get_metadata(exch: str) → None

Get CCXT metadata.

Parameters:: exch (str) – Name of exchange.

exponential_backoff_with_jitter(attempts: int, status_code: int | None = None) → float

Calculates and applies exponential backoff with full jitter, honoring specific error codes (403/429) using configurable instance properties.

Parameters:

attempts – The current retry number (starting at 1 after the first failure).
status_code – The HTTP status code received (e.g., 403, 429).

Returns:

The actual time slept in seconds.

async exponential_backoff_with_jitter_async(attempts: int, status_code: int | None = None) → float: Async version of exponential_backoff_with_jitter, using configurable instance properties.

_handle_exception_and_backoff(e: Exception, attempts: int) → bool

Analyzes a synchronous exception, applies backoff if recoverable, and logs the result.

Returns:: True if the error was recoverable (retries should continue). False if the error is terminal (retries should stop).

async _handle_exception_and_backoff_async(e: Exception, attempts: int) → bool

Analyzes an asynchronous exception, applies backoff if recoverable, and logs the result.

Returns:: True if the error was recoverable (retries should continue). False if the error is terminal (retries should stop).

async _fetch_ohlcv_async(ticker: str, freq: str, start_date: int, end_date: int, exch: str, trials: int = 3) → List | None

Fetches OHLCV data for a specific ticker.

Parameters:

ticker (str) – Ticker symbol.
freq (str) – Frequency of data, e.g. ‘1m’, ‘5m’, ‘1h’, ‘1d’.
start_date (str) – Start date in integers in milliseconds since Unix epoch.
end_date (str) – End date in integers in milliseconds since Unix epoch.
exch (str) – Name of exchange.
trials (int, default 3) – Number of attempts to fetch data.

Returns:

data_resp – List of timestamps with OHLCV data.

Return type:

list

_fetch_ohlcv(ticker: str, freq: str, start_date: str, end_date: str, exch: str, trials: int = 3) → List | None

Fetches OHLCV data for a specific ticker.

Parameters:

ticker (str) – Ticker symbol.
freq (str) – Frequency of data, e.g. ‘1m’, ‘5m’, ‘1h’, ‘1d’.
start_date (str) – Start date in integers in milliseconds since Unix epoch.
end_date (str) – End date in integers in milliseconds since Unix epoch.
exch (str) – Name of exchange.
trials (int, default 3) – Number of attempts to fetch data.

Returns:

data_resp – List of timestamps with OHLCV data.

Return type:

list

async _fetch_all_ohlcv_async(tickers, freq: str, start_date: int, end_date: int, exch: str, trials: int = 3, pause: int = 1) → List | None

Fetches OHLCV data for a list of tickers.

Parameters:

tickers (list) – List of ticker symbols.
freq (str) – Frequency of data, e.g. ‘1m’, ‘5m’, ‘1h’, ‘1d’.
start_date (int) – Start date in integers in milliseconds since Unix epoch.
end_date (int) – End date in integers in milliseconds since Unix epoch.
exch (str) – Name of exchange.
trials (int, default 3) – Number of attempts to fetch data.
pause (int, default 0.5) – Pause in seconds to respect the rate limit.

Returns:

data_resp – List of lists of timestamps and OHLCV data for each ticker.

Return type:

list

_fetch_all_ohlcv(tickers, freq: str, start_date: int, end_date: int, exch: str, trials: int = 3, pause: int = 1) → List | None

Fetches OHLCV data for a list of tickers.

Parameters:

tickers (list) – List of ticker symbols.
freq (str) – Frequency of data, e.g. ‘1m’, ‘5m’, ‘1h’, ‘1d’.
start_date (int) – Start date in integers in milliseconds since Unix epoch.
end_date (int) – End date in integers in milliseconds since Unix epoch.
exch (str) – Name of exchange.
trials (int, default 3) – Number of attempts to fetch data.
pause (int, default 0.5) – Pause in seconds to respect the rate limit.

Returns:

data_resp – List of lists of timestamps and OHLCV data for each ticker.

Return type:

list

async _fetch_funding_rates_async(ticker: str, start_date: int, end_date: int, exch: str, trials: int = 3) → List | None

Fetches funding rates data for a specific ticker.

Parameters:

ticker (str) – Ticker symbol.
start_date (int) – Start date in integers in milliseconds since Unix epoch.
end_date (int) – End date in integers in milliseconds since Unix epoch.
trials (int, default 3) – Number of attempts to fetch data.

Returns:

data_resp – List of dictionaries with timestamps and funding rates data.

Return type:

list

_fetch_funding_rates(ticker: str, start_date: int, end_date: int, exch: str, trials: int = 3) → List | None

Fetches funding rates data for a specific ticker.

Parameters:

ticker (str) – Ticker symbol.
start_date (int) – Start date in integers in milliseconds since Unix epoch.
end_date (int) – End date in integers in milliseconds since Unix epoch.
trials (int, default 3) – Number of attempts to fetch data.

Returns:

data_resp – List of dictionaries with timestamps and funding rates data.

Return type:

list

async _fetch_all_funding_rates_async(tickers, start_date: int, end_date: int, exch: str, trials: int = 3, pause: int = 1) → List | None

Fetches funding rates data for a list of tickers.

Parameters:

tickers (list) – List of ticker symbols.
start_date (int) – Start date in integers in milliseconds since Unix epoch.
end_date (int) – End date in integers in milliseconds since Unix epoch.
exch (str) – Name of exchange.
trials (int, default 3) – Number of attempts to fetch data.
pause (int, default 0.5) – Pause in seconds to respect the rate limit.

Returns:

data_resp – List of lists of dictionaries with timestamps and funding rates data for each ticker.

Return type:

list

_fetch_all_funding_rates(tickers, start_date: int, end_date: int, exch: str, trials: int = 3, pause: int = 1) → List | None

Fetches funding rates data for a list of tickers.

Parameters:

tickers (list) – List of ticker symbols.
start_date (int) – Start date in integers in milliseconds since Unix epoch.
end_date (int) – End date in integers in milliseconds since Unix epoch.
exch (str) – Name of exchange.
trials (int, default 3) – Number of attempts to fetch data.
pause (int, default 0.5) – Pause in seconds to respect the rate limit.

Returns:

data_resp – List of lists of dictionaries with timestamps and funding rates data for each ticker.

Return type:

list

async _fetch_open_interest_async(ticker: str, freq: str, start_date: int, end_date: int, exch: str, trials: int = 3) → List | None

Fetches open interest data for a specific ticker.

Parameters:

ticker (str) – Ticker symbol.
freq (str) – Frequency of data, e.g. ‘1m’, ‘5m’, ‘1h’, ‘1d’.
start_date (int) – Start date in integers in milliseconds since Unix epoch.
end_date (int) – End date in integers in milliseconds since Unix epoch.
exch (str) – Name of exchange.
trials (int, default 3) – Number of attempts to fetch data.

Returns:

data_resp – List of dictionaries with timestamps and open interest data.

Return type:

list

_fetch_open_interest(ticker: str, freq: str, start_date: int, end_date: int, exch: str, trials: int = 3) → List | None

Fetches open interest data for a specific ticker.

Parameters:

ticker (str) – Ticker symbol.
freq (str) – Frequency of data, e.g. ‘1m’, ‘5m’, ‘1h’, ‘1d’.
start_date (int) – Start date in integers in milliseconds since Unix epoch.
end_date (int) – End date in integers in milliseconds since Unix epoch.
exch (str) – Name of exchange.
trials (int, default 3) – Number of attempts to fetch data.

Returns:

data_resp – List of dictionaries with timestamps and open interest data.

Return type:

list

async _fetch_all_open_interest_async(tickers, freq: str, start_date: int, end_date: int, exch: str, trials: int = 3, pause: int = 1) → List | None

Fetches open interest data for a list of tickers.

Parameters:

tickers (list) – List of ticker symbols.
freq (str) – Frequency of data, e.g. ‘1m’, ‘5m’, ‘1h’, ‘1d’.
start_date (int) – Start date in integers in milliseconds since Unix epoch.
end_date (int) – End date in integers in milliseconds since Unix epoch.
exch (str) – Name of exchange.
trials (int, default 3) – Number of attempts to fetch data.
pause (int, default 0.5) – Pause in seconds to respect the rate limit.

Returns:

data_resp – List of lists of dictionaries with timestamps and open interest data for each ticker.

Return type:

list

_fetch_all_open_interest(tickers, freq: str, start_date: int, end_date: int, exch: str, trials: int = 3, pause: int = 1) → List | None

Fetches open interest data for a list of tickers.

Parameters:

tickers (list) – List of ticker symbols.
freq (str) – Frequency of data, e.g. ‘1m’, ‘5m’, ‘1h’, ‘1d’.
start_date (int) – Start date in integers in milliseconds since Unix epoch.
end_date (int) – End date in integers in milliseconds since Unix epoch.
exch (str) – Name of exchange.
trials (int, default 3) – Number of attempts to fetch data.
pause (int, default 0.5) – Pause in seconds to respect the rate limit.

Returns:

data_resp – List of lists of dictionaries with timestamps and open interest data for each ticker.

Return type:

list

convert_params(data_req: cryptodatapy.extract.datarequest.DataRequest) → cryptodatapy.extract.datarequest.DataRequest

Converts data request parameters to CCXT format.

Parameters:: data_req (DataRequest) – Parameters of data request in CryptoDataPy format.
Returns:: data_req – Parameters of data request in CCXT format.
Return type:: DataRequest

wrangle_data_resp(data_type: str) → pandas.DataFrame

Wrangle data response.

Parameters:: data_type (str) – Type of data, e.g. ‘ohlcv’, ‘funding_rate’, ‘open_interest’.
Returns:: df – Wrangled dataframe with DatetimeIndex and values in tidy format.
Return type:: pd.DataFrame

async fetch_tidy_ohlcv_async(data_req: cryptodatapy.extract.datarequest.DataRequest) → pandas.DataFrame

Gets entire OHLCV history and wrangles the data response into tidy data format.

Parameters:: data_req (DataRequest) – Parameters of data request in CryptoDataPy format.
Returns:: df – Dataframe with entire OHLCV data history retrieved and wrangled into tidy data format.
Return type:: pd.DataFrame

fetch_tidy_ohlcv(data_req: cryptodatapy.extract.datarequest.DataRequest) → pandas.DataFrame

Gets entire OHLCV history and wrangles the data response into tidy data format.

Parameters:: data_req (DataRequest) – Parameters of data request in CryptoDataPy format.
Returns:: df – Dataframe with entire OHLCV data history retrieved and wrangled into tidy data format.
Return type:: pd.DataFrame

async fetch_tidy_funding_rates_async(data_req: cryptodatapy.extract.datarequest.DataRequest) → pandas.DataFrame

Gets entire funding rates history and wrangles the data response into tidy data format.

Parameters:: data_req (DataRequest) – Parameters of data request in CryptoDataPy format.
Returns:: df – Dataframe with entire data history retrieved and wrangled into tidy data format.
Return type:: pd.DataFrame

fetch_tidy_funding_rates(data_req: cryptodatapy.extract.datarequest.DataRequest) → pandas.DataFrame

Gets entire funding rates history and wrangles the data response into tidy data format.

Parameters:: data_req (DataRequest) – Parameters of data request in CryptoDataPy format.
Returns:: df – Dataframe with entire data history retrieved and wrangled into tidy data format.
Return type:: pd.DataFrame

async fetch_tidy_open_interest_async(data_req: cryptodatapy.extract.datarequest.DataRequest) → pandas.DataFrame

Gets entire open interest history and wrangles the data response into tidy data format.

Parameters:: data_req (DataRequest) – Parameters of data request in CryptoDataPy format.
Returns:: df – Dataframe with entire data history retrieved and wrangled into tidy data format.
Return type:: pd.DataFrame

fetch_tidy_open_interest(data_req: cryptodatapy.extract.datarequest.DataRequest) → pandas.DataFrame

Gets entire open interest history and wrangles the data response into tidy data format.

Parameters:: data_req (DataRequest) – Parameters of data request in CryptoDataPy format.
Returns:: df – Dataframe with entire data history retrieved and wrangled into tidy data format.
Return type:: pd.DataFrame

async get_data_async(data_req: cryptodatapy.extract.datarequest.DataRequest) → pandas.DataFrame

Get data specified by data request.

Parameters data_req: DataRequest

Parameters of data request in CryptoDataPy format.

Returns:: df – DataFrame with DatetimeIndex (level 0), ticker (level 1), and values for selected fields (cols).
Return type:: pd.DataFrame - MultiIndex

get_data(data_req: cryptodatapy.extract.datarequest.DataRequest) → pandas.DataFrame

Get data specified by data request.

Parameters data_req: DataRequest

Parameters of data request in CryptoDataPy format.

Returns:: df – DataFrame with DatetimeIndex (level 0), ticker (level 1), and values for selected fields (cols).
Return type:: pd.DataFrame - MultiIndex

Bases: cryptodatapy.extract.libraries.library.Library

Retrieves data from DBnomics API.

static get_vendors_info(): Gets available vendors info.

get_assets_info() → None: Gets available assets info.

get_indexes_info() → None: Gets available indexes info.

get_markets_info() → None: Gets market pairs info.

get_fields_info(cat: str | None = None) → Dict[str, List[str]]

Gets fields info.

Parameters:: cat (str, {'crypto', 'eqty', 'fx', 'rates', 'cmdty', 'macro', 'alt'}, default None) – Asset class or time series category.
Returns:: fields – Dictionary with info on available fields, by category.
Return type:: dictionary

get_exchanges_info() → None: Gets exchanges info.

get_rate_limit_info() → None: Gets rate limit info.

static get_series(ticker: str) → pandas.DataFrame

Gets series from DBnomics python client.

Parameters:: ticker (str) – Ticker symbol/identifier of time series.
Returns:: df – Dataframe with DatetimeIndex and actual values (col) for requested series.
Return type:: pd.DataFrame

static wrangle_data_resp(data_req: cryptodatapy.extract.datarequest.DataRequest, data_resp: pandas.DataFrame) → pandas.DataFrame

Wrangle data response.

Parameters:

data_req (DataRequest) – Parameters of data request in CryptoDataPy format.
data_resp (pd.DataFrame) – Data response from client.

Returns:

df – Wrangled dataframe with DatetimeIndex (level 0), ticker (level 1), and values for macro time series for selected fields (cols), in tidy format.

Return type:

pd.DataFrame

get_tidy_data(data_req: cryptodatapy.extract.datarequest.DataRequest, ticker: str) → pandas.DataFrame

Submits data request to Python client and wrangles the data response into tidy data format.

Parameters:

data_req (DataRequest) – Data request parameters in CryptoDataPy format.
ticker (str) – Tickery symbol/identifier for time series.

Returns:

df – Dataframe with DatetimeIndex and field values (col) wrangled into tidy data format.

Return type:

pd.DataFrame

check_params(data_req: cryptodatapy.extract.datarequest.DataRequest) → None: Checks the parameters of the data request before requesting data to reduce API calls and improve efficiency.

get_data(data_req: cryptodatapy.extract.datarequest.DataRequest) → pandas.DataFrame

Get data macro data.

Parameters:: data_req (DataRequest) – Parameters of data request in CryptoDataPy format.
Returns:: df – DataFrame with DatetimeIndex (level 0), ticker (level 1) and values macro or off-chain fields (cols).
Return type:: pd.DataFrame - MultiIndex

class cryptodatapy.extract.libraries.Library(categories, exchanges, assets, indexes, markets, market_types, fields, frequencies, base_url, api_key, max_obs_per_call, rate_limit)

Bases: abc.ABC

Library is an abstract base class which provides a blueprint for properties and methods for the library subclass.

property categories: Returns a list of available categories for the data vendor.

property exchanges: Returns a list of available exchanges for the data vendor.

property assets: Returns a list of available assets for the data vendor.

property indexes: Returns a list of available indices for the data vendor.

property markets: Returns a list of available markets for the data vendor.

property market_types: Returns a list of available market types for the data vendor.

property fields: Returns a list of available fields for the data vendor.

property frequencies: Returns a list of available data frequencies for the data vendor.

property base_url: Returns the base url for the data vendor.

property api_key: Returns the api key for the data vendor.

property max_obs_per_call: Returns the maximum observations per API call for the data vendor.

property rate_limit: Returns the number of API calls made and remaining.

abstractmethod get_exchanges_info(): Gets info for available exchanges from the data vendor.

abstractmethod get_indexes_info(): Gets info for available indexes from the data vendor.

abstractmethod get_assets_info(): Gets info for available assets from the data vendor.

abstractmethod get_markets_info(): Gets info for available markets from the data vendor.

abstractmethod get_fields_info(data_type: str | None): Gets info for available fields from the data vendor.

abstractmethod get_rate_limit_info(): Gets the number of API calls made and remaining.

abstractmethod get_data(data_req) → pandas.DataFrame: Submits get data request to API.

static wrangle_data_resp(data_req: cryptodatapy.extract.datarequest.DataRequest, data_resp: Dict[str, Any] | pandas.DataFrame) → pandas.DataFrame

Abstractmethod:

Wrangles data response from data vendor API into tidy format.

class cryptodatapy.extract.libraries.PandasDataReader(categories: str | List[str] = ['fx', 'rates', 'eqty', 'cmdty', 'credit', 'macro'], exchanges: List[str] | None = None, indexes: Dict[str, List[str]] | None = None, assets: Dict[str, List[str]] | None = None, markets: Dict[str, List[str]] | None = None, market_types: List[str] = ['spot', 'future'], fields: Dict[str, List[str]] | None = None, frequencies: Dict[str, List[str]] | None = ['d', 'w', 'm', 'q', 'y', 'av-intraday', 'av-daily', 'av-weekly', 'av-monthly', 'av-daily-adjusted', 'av-weekly-adjusted', 'av-monthly-adjusted', 'av-forex-daily'], base_url: str | None = None, api_key: str = data_cred.alpha_vantage_api_key, max_obs_per_call: int | None = None, rate_limit: Any | None = None)

Bases: cryptodatapy.extract.libraries.library.Library

Retrieves data from Pandas Data Reader API.

data_req = None

data

static get_vendors_info(): Get vendors info.

static get_exchanges_info() → None: Get exchanges info.

static get_indexes_info() → None: Get indexes info.

static get_assets_info() → None: Get assets info.

static get_markets_info() → None: Get markets info.

get_fields_info() → Dict[str, List[str]]

Get fields info.

Returns:: fields – Dictionary with info on available fields, by category.
Return type:: dictionary

get_frequencies_info() → Dict[str, str | int]

Get frequencies info.

Returns:: freq – Dictionary with info on available frequencies.
Return type:: dictionary

static get_rate_limit_info() → None: Get rate limit info.

convert_params(data_req: cryptodatapy.extract.datarequest.DataRequest) → cryptodatapy.extract.datarequest.DataRequest

Converts data request parameters to source format.

Parameters:: data_req (DataRequest) – Parameters of data request in CryptoDataPy format.
Returns:: data_req – Parameters of data request in source format.
Return type:: DataRequest

get_series(data_req: cryptodatapy.extract.datarequest.DataRequest) → pandas.DataFrame

Gets series from python client.

Parameters:: data_req (DataRequest) – Parameters of data request in CryptoDataPy format.
Returns:: df – Dataframe with DatetimeIndex and actual values (col) for requested series.
Return type:: pd.DataFrame

wrangle_data_resp(data_req: cryptodatapy.extract.datarequest.DataRequest, data_resp: pandas.DataFrame) → pandas.DataFrame

Wrangle data response.

Parameters:

data_req (DataRequest) – Parameters of data request in CryptoDataPy format.
data_resp (pd.DataFrame) – Data response from data request.

Returns:

df – Wrangled dataframe with DatetimeIndex (level 0), ticker (level 1), and values for market or macro series for selected fields (cols), in tidy format.

Return type:

pd.DataFrame

get_tidy_data(data_req: cryptodatapy.extract.datarequest.DataRequest) → pandas.DataFrame

Gets data from FRED and wrangles the data response into tidy data format.

Parameters:: data_req (DataRequest) – Parameters of data request in CryptoDataPy format.
Returns:: df – Dataframe with DatetimeIndex (level 0), tickers (level 1) and actual values (cols), in tidy data format.
Return type:: pd.DataFrame - MultiIndex

get_data(data_req: cryptodatapy.extract.datarequest.DataRequest) → pandas.DataFrame

Get data.

Parameters data_req: DataRequest

Parameters of data request in CryptoDataPy format.

Returns:: df – DataFrame with DatetimeIndex (level 0), ticker (level 1), and values for selected fields (cols), in tidy format.
Return type:: pd.DataFrame - MultiIndex