great_expectations.datasource.new_datasource

Module Contents

Classes

BaseDatasource(name: str, execution_engine=None, data_context_root_directory: Optional[str] = None)

An Datasource is the glue between an ExecutionEngine and a DataConnector.

Datasource(name: str, execution_engine=None, data_connectors=None, data_context_root_directory: Optional[str] = None)

An Datasource is the glue between an ExecutionEngine and a DataConnector.

great_expectations.datasource.new_datasource.logger
class great_expectations.datasource.new_datasource.BaseDatasource(name: str, execution_engine=None, data_context_root_directory: Optional[str] = None)

An Datasource is the glue between an ExecutionEngine and a DataConnector.

recognized_batch_parameters :set
get_batch_from_batch_definition(self, batch_definition: BatchDefinition, batch_data: Any = None)

Note: this method should not be used when getting a Batch from a BatchRequest, since it does not capture BatchRequest metadata.

get_single_batch_from_batch_request(self, batch_request: BatchRequest)
get_batch_definition_list_from_batch_request(self, batch_request: BatchRequest)

Validates batch request and utilizes the classes’ Data Connectors’ property to get a list of batch definition given

a batch request

:param : param batch_request: A BatchRequest object used to request a batch :param : return: A list of batch definitions

get_batch_list_from_batch_request(self, batch_request: Union[BatchRequest, RuntimeBatchRequest])

Processes batch_request and returns the (possibly empty) list of batch objects.

:param : batch_request encapsulation of request parameters necessary to identify the (possibly multiple) batches :param : returns possibly empty list of batch objects; each batch object contains a dataset and associated metatada

_build_data_connector_from_config(self, name: str, config: Dict[str, Any])

Build a DataConnector using the provided configuration and return the newly-built DataConnector.

get_available_data_asset_names(self, data_connector_names: Optional[Union[list, str]] = None)

Returns a dictionary of data_asset_names that the specified data connector can provide. Note that some data_connectors may not be capable of describing specific named data assets, and some (such as inferred_asset_data_connector) require the user to configure data asset names.

Parameters

data_connector_names – the DataConnector for which to get available data asset names.

Returns

{
  data_connector_name: {
    names: [ (data_asset_1, data_asset_1_type), (data_asset_2, data_asset_2_type) ... ]
  }
  ...
}

Return type

dictionary consisting of sets of data assets available for the specified data connectors

get_available_batch_definitions(self, batch_request: BatchRequest)
self_check(self, pretty_print=True, max_examples=3)
_validate_batch_request(self, batch_request: BatchRequest)
property name(self)

Property for datasource name

property execution_engine(self)
property data_connectors(self)
property config(self)
class great_expectations.datasource.new_datasource.Datasource(name: str, execution_engine=None, data_connectors=None, data_context_root_directory: Optional[str] = None)

Bases: great_expectations.datasource.new_datasource.BaseDatasource

An Datasource is the glue between an ExecutionEngine and a DataConnector.

recognized_batch_parameters :set
_init_data_connectors(self, data_connector_configs: Dict[str, Dict[str, Any]])