great_expectations.datasource.new_datasource
¶
Module Contents¶
Classes¶
|
An Datasource is the glue between an ExecutionEngine and a DataConnector. |
|
An Datasource is the glue between an ExecutionEngine and a DataConnector. |
-
great_expectations.datasource.new_datasource.
logger
¶
-
class
great_expectations.datasource.new_datasource.
BaseDatasource
(name: str, execution_engine=None, data_context_root_directory: Optional[str] = None)¶ An Datasource is the glue between an ExecutionEngine and a DataConnector.
-
recognized_batch_parameters
:set¶
-
get_batch_from_batch_definition
(self, batch_definition: BatchDefinition, batch_data: Any = None)¶ Note: this method should not be used when getting a Batch from a BatchRequest, since it does not capture BatchRequest metadata.
-
get_single_batch_from_batch_request
(self, batch_request: BatchRequest)¶
-
get_batch_definition_list_from_batch_request
(self, batch_request: BatchRequest)¶ Validates batch request and utilizes the classes’ Data Connectors’ property to get a list of batch definition given
a batch request
:param : param batch_request: A BatchRequest object used to request a batch :param : return: A list of batch definitions
-
get_batch_list_from_batch_request
(self, batch_request: Union[BatchRequest, RuntimeBatchRequest])¶ Processes batch_request and returns the (possibly empty) list of batch objects.
:param : batch_request encapsulation of request parameters necessary to identify the (possibly multiple) batches :param : returns possibly empty list of batch objects; each batch object contains a dataset and associated metatada
-
_build_data_connector_from_config
(self, name: str, config: Dict[str, Any])¶ Build a DataConnector using the provided configuration and return the newly-built DataConnector.
-
get_available_data_asset_names
(self, data_connector_names: Optional[Union[list, str]] = None)¶ Returns a dictionary of data_asset_names that the specified data connector can provide. Note that some data_connectors may not be capable of describing specific named data assets, and some (such as inferred_asset_data_connector) require the user to configure data asset names.
- Parameters
data_connector_names – the DataConnector for which to get available data asset names.
- Returns
{ data_connector_name: { names: [ (data_asset_1, data_asset_1_type), (data_asset_2, data_asset_2_type) ... ] } ... }
- Return type
dictionary consisting of sets of data assets available for the specified data connectors
-
get_available_batch_definitions
(self, batch_request: BatchRequest)¶
-
self_check
(self, pretty_print=True, max_examples=3)¶
-
_validate_batch_request
(self, batch_request: BatchRequest)¶
-
property
name
(self)¶ Property for datasource name
-
property
execution_engine
(self)¶
-
property
data_connectors
(self)¶
-
property
config
(self)¶
-
-
class
great_expectations.datasource.new_datasource.
Datasource
(name: str, execution_engine=None, data_connectors=None, data_context_root_directory: Optional[str] = None)¶ Bases:
great_expectations.datasource.new_datasource.BaseDatasource
An Datasource is the glue between an ExecutionEngine and a DataConnector.
-
recognized_batch_parameters
:set¶
-
_init_data_connectors
(self, data_connector_configs: Dict[str, Dict[str, Any]])¶
-