great_expectations.core.batch

Module Contents

Classes

BatchDefinition(datasource_name: str, data_connector_name: str, data_asset_name: str, batch_identifiers: IDDict, batch_spec_passthrough: Optional[dict] = None)

BatchRequestBase(datasource_name: str, data_connector_name: str, data_asset_name: str, data_connector_query: Optional[dict] = None, limit: Optional[int] = None, batch_spec_passthrough: Optional[dict] = None, runtime_parameters: Optional[dict] = None, batch_identifiers: Optional[dict] = None)

This class is for internal inter-object protocol purposes only.

BatchRequest(datasource_name: str, data_connector_name: str, data_asset_name: str, data_connector_query: Optional[dict] = None, limit: Optional[int] = None, batch_spec_passthrough: Optional[dict] = None)

This class contains all attributes of a batch_request. See the comments in BatchRequestBase for design specifics.

RuntimeBatchRequest(datasource_name: str, data_connector_name: str, data_asset_name: str, runtime_parameters: dict, batch_identifiers: dict, batch_spec_passthrough: Optional[dict] = None)

This class contains all attributes of a batch_request. See the comments in BatchRequestBase for design specifics.

BatchMarkers(*args, **kwargs)

A BatchMarkers is a special type of BatchKwargs (so that it has a batch_fingerprint) but it generally does

Batch(data, batch_request: BatchRequest = None, batch_definition: BatchDefinition = None, batch_spec: BatchSpec = None, batch_markers: BatchMarkers = None, data_context=None, datasource_name=None, batch_parameters=None, batch_kwargs=None)

class great_expectations.core.batch.BatchDefinition(datasource_name: str, data_connector_name: str, data_asset_name: str, batch_identifiers: IDDict, batch_spec_passthrough: Optional[dict] = None)

Bases: great_expectations.types.SerializableDictDot

to_json_dict(self)
__repr__(self)

Return repr(self).

static _validate_batch_definition(datasource_name: str, data_connector_name: str, data_asset_name: str, batch_identifiers: IDDict)
property datasource_name(self)
property data_connector_name(self)
property data_asset_name(self)
property batch_identifiers(self)
property batch_spec_passthrough(self)
property id(self)
__eq__(self, other)

Return self==value.

__str__(self)

Return str(self).

__hash__(self)

Overrides the default implementation

class great_expectations.core.batch.BatchRequestBase(datasource_name: str, data_connector_name: str, data_asset_name: str, data_connector_query: Optional[dict] = None, limit: Optional[int] = None, batch_spec_passthrough: Optional[dict] = None, runtime_parameters: Optional[dict] = None, batch_identifiers: Optional[dict] = None)

Bases: great_expectations.types.SerializableDictDot

This class is for internal inter-object protocol purposes only. As such, it contains all attributes of a batch_request, but does not validate them. See the BatchRequest class, which extends BatchRequestBase and validates the attributes.

BatchRequestBase is used for the internal protocol purposes exclusively, not part of API for the developer users.

Previously, the very same BatchRequest was used for both the internal protocol purposes and as part of the API exposed to developers. However, while convenient for internal data interchange, using the same BatchRequest class as arguments to the externally-exported DataContext.get_batch(), DataContext.get_batch_list(), and DataContext.get_validator() API calls for obtaining batches and/or validators was insufficiently expressive to fulfill the needs of both. In the user-accessible API, BatchRequest, must enforce that all members of the triple, consisting of data_source_name, data_connector_name, and data_asset_name, are not NULL. Whereas for the internal protocol, BatchRequest is used as a flexible bag of attributes, in which any fields are allowed to be NULL. Hence, now, BatchRequestBase is dedicated for the use as the bag oof attributes for the internal protocol use, whereby NULL values are allowed as per the internal needs. The BatchRequest class extends BatchRequestBase and adds to it strong validation (described above plus additional attribute validation) so as to formally validate user specified fields.

property runtime_parameters(self)
property batch_identifiers(self)
property datasource_name(self)
property data_connector_name(self)
property data_asset_name(self)
property data_connector_query(self)
property limit(self)
property batch_spec_passthrough(self)
to_json_dict(self)
__str__(self)

Return str(self).

property id(self)
__eq__(self, other)

Return self==value.

class great_expectations.core.batch.BatchRequest(datasource_name: str, data_connector_name: str, data_asset_name: str, data_connector_query: Optional[dict] = None, limit: Optional[int] = None, batch_spec_passthrough: Optional[dict] = None)

Bases: great_expectations.core.batch.BatchRequestBase

This class contains all attributes of a batch_request. See the comments in BatchRequestBase for design specifics. limit: refers to the number of batches requested (not rows per batch)

static _validate_init_parameters(datasource_name: str, data_connector_name: str, data_asset_name: str, data_connector_query: Optional[dict] = None, limit: Optional[int] = None)
static _validate_runtime_batch_request_specific_init_parameters(runtime_parameters: dict, batch_identifiers: dict, batch_spec_passthrough: Optional[dict] = None)
class great_expectations.core.batch.RuntimeBatchRequest(datasource_name: str, data_connector_name: str, data_asset_name: str, runtime_parameters: dict, batch_identifiers: dict, batch_spec_passthrough: Optional[dict] = None)

Bases: great_expectations.core.batch.BatchRequest

This class contains all attributes of a batch_request. See the comments in BatchRequestBase for design specifics. limit: refers to the number of batches requested (not rows per batch)

__deepcopy__(self, memo)
to_json_dict(self)
class great_expectations.core.batch.BatchMarkers(*args, **kwargs)

Bases: great_expectations.core.id_dict.BatchKwargs

A BatchMarkers is a special type of BatchKwargs (so that it has a batch_fingerprint) but it generally does NOT require specific keys and instead captures information about the OUTPUT of a datasource’s fetch process, such as the timestamp at which a query was executed.

property ge_load_time(self)
class great_expectations.core.batch.Batch(data, batch_request: BatchRequest = None, batch_definition: BatchDefinition = None, batch_spec: BatchSpec = None, batch_markers: BatchMarkers = None, data_context=None, datasource_name=None, batch_parameters=None, batch_kwargs=None)

Bases: great_expectations.types.SerializableDictDot

property data(self)
property batch_request(self)
property batch_definition(self)
property batch_spec(self)
property batch_markers(self)
property data_context(self)
property datasource_name(self)
property batch_parameters(self)
property batch_kwargs(self)
to_json_dict(self)
property id(self)
__str__(self)

Return str(self).

head(self, n_rows=5, fetch_all=False)