base_wrangler ============= .. py:module:: base_wrangler Attributes ---------- .. autoapisummary:: base_wrangler.logger Classes ------- .. autoapisummary:: base_wrangler.BaseDataWrangler Functions --------- .. autoapisummary:: base_wrangler._load_field_map Module Contents --------------- .. py:data:: logger .. py:function:: _load_field_map() -> Dict[str, Dict[str, str]] Loads fields.csv from the config path and creates a nested dictionary map: {'vendor_name': {'vendor_field_lower': 'CRYPTODATAPY_FIELD'}} .. py:class:: BaseDataWrangler(data_req: cryptodatapy.extract.datarequest.DataRequest, data_resp: Union[Dict, pandas.DataFrame]) Bases: :py:obj:`abc.ABC` Base class for vendor-specific data wranglers. Handles common data cleaning, filtering, and field mapping operations. .. py:attribute:: _FIELD_MAP .. py:attribute:: _DEFAULT_AGG_MAP .. py:attribute:: data_req .. py:attribute:: data_resp .. py:attribute:: field_map .. py:method:: _convert_fields_to_lib(data_source: str) -> None Convert columns from vendor field names to CryptoDataPy standard field names using the dictionary map. Mutates self.data_resp. .. py:method:: _set_index_and_sort(index_cols: Union[str, List[str]] = 'date') -> None Sets the index and sorts the DataFrame by the index. It ensures that if 'date' is part of the index, it is converted to a date-only Timestamp (time component set to 00:00:00) while retaining the datetime64[ns] dtype for optimal index performance. .. py:method:: _filter_dates() -> None Filters data response based on start and end dates in data_req. .. py:method:: _resample(agg_func: Optional[Union[str, Dict[str, str]]] = None) -> None Resamples the DataFrame to the frequency in the data_req. Logic: 1. If agg_func is a string ('last', 'sum'), it applies to all columns. 2. If agg_func is None, it uses the DEFAULT_AGG_MAP for known columns. 3. If a column isn't in the map, it defaults to 'last'. :param agg_func: Aggregation function(s) to use during resampling. If a string is provided, it applies to all columns. If a dict is provided, it should map column names to aggregation functions. If None, the default aggregation map is used. :type agg_func: str or dict, optional .. py:method:: _reorder_columns(requested_fields: bool = False) -> None Reorders columns based on the provided column order list. :param requested_fields: If True, only requested fields are kept and ordered. If False, all columns are kept. :type requested_fields: bool .. py:method:: _clean_data() -> None Removes duplicates, NaNs (full row/col), and 0 values. .. py:method:: _convert_types() -> None Converts columns to appropriate numeric types, explicitly excluding known string/metadata columns, and uses standard pandas dtypes. .. py:method:: wrangle() -> pandas.DataFrame :abstractmethod: Abstract method for wrangling. Must be implemented by child classes.