great_expectations.execution_engine.execution_engine
¶
Module Contents¶
Classes¶
|
|
|
|
Generic enumeration. |
|
Generic enumeration. |
|
|
Helper class that provides a standard way to create an ABC using |
Generic enumeration. |
-
great_expectations.execution_engine.execution_engine.
logger
¶
-
great_expectations.execution_engine.execution_engine.
yaml
¶
-
great_expectations.execution_engine.execution_engine.
default_flow_style
= False¶
-
class
great_expectations.execution_engine.execution_engine.
NoOpDict
¶ -
__getitem__
(self, item)¶
-
__setitem__
(self, key, value)¶
-
update
(self, value)¶
-
-
class
great_expectations.execution_engine.execution_engine.
BatchData
(execution_engine)¶ -
property
execution_engine
(self)¶
-
head
(self, *args, **kwargs)¶
-
property
-
class
great_expectations.execution_engine.execution_engine.
MetricFunctionTypes
¶ Bases:
enum.Enum
Generic enumeration.
Derive from this class to define new enumerations.
-
VALUE
= value¶
-
MAP_VALUES
= value¶
-
WINDOW_VALUES
= value¶
-
AGGREGATE_VALUE
= value¶
-
-
class
great_expectations.execution_engine.execution_engine.
MetricDomainTypes
¶ Bases:
enum.Enum
Generic enumeration.
Derive from this class to define new enumerations.
-
COLUMN
= column¶
-
COLUMN_PAIR
= column_pair¶
-
MULTICOLUMN
= multicolumn¶
-
TABLE
= table¶
-
-
class
great_expectations.execution_engine.execution_engine.
DataConnectorStorageDataReferenceResolver
¶ -
DATA_CONNECTOR_NAME_TO_STORAGE_NAME_MAP
:Dict[str, str]¶
-
STORAGE_NAME_EXECUTION_ENGINE_NAME_PATH_RESOLVERS
:Dict[Tuple[str, str], Callable]¶
-
static
resolve_data_reference
(data_connector_name: str, execution_engine_name: str, template_arguments: dict)¶ Resolve file path for a (data_connector_name, execution_engine_name) combination.
-
-
class
great_expectations.execution_engine.execution_engine.
ExecutionEngine
(name=None, caching=True, batch_spec_defaults=None, batch_data_dict=None, validator=None)¶ Bases:
abc.ABC
Helper class that provides a standard way to create an ABC using inheritance.
-
recognized_batch_spec_defaults
¶
-
configure_validator
(self, validator)¶ Optionally configure the validator as appropriate for the execution engine.
-
property
active_batch_data_id
(self)¶ The batch id for the default batch data.
When an execution engine is asked to process a compute domain that does not include a specific batch_id, then the data associated with the active_batch_data_id will be used as the default.
-
property
active_batch_data
(self)¶ The data from the currently-active batch.
-
property
loaded_batch_data_dict
(self)¶ The current dictionary of batches.
-
property
loaded_batch_data_ids
(self)¶
-
property
config
(self)¶
-
property
dialect
(self)¶
-
get_batch_data
(self, batch_spec: BatchSpec)¶ Interprets batch_data and returns the appropriate data.
This method is primarily useful for utility cases (e.g. testing) where data is being fetched without a DataConnector and metadata like batch_markers is unwanted
Note: this method is currently a thin wrapper for get_batch_data_and_markers. It simply suppresses the batch_markers.
-
abstract
get_batch_data_and_markers
(self, batch_spec)¶
-
load_batch_data
(self, batch_id: str, batch_data: Any)¶ Loads the specified batch_data into the execution engine
-
_load_batch_data_from_dict
(self, batch_data_dict)¶ Loads all data in batch_data_dict into load_batch_data
-
resolve_metrics
(self, metrics_to_resolve: Iterable[MetricConfiguration], metrics: Optional[Dict[Tuple[str, str, str], MetricConfiguration]] = None, runtime_configuration: Optional[dict] = None)¶ resolve_metrics is the main entrypoint for an execution engine. The execution engine will compute the value of the provided metrics.
- Parameters
metrics_to_resolve – the metrics to evaluate
metrics – already-computed metrics currently available to the engine
runtime_configuration – runtime configuration information
- Returns
a dictionary with the values for the metrics that have just been resolved.
- Return type
resolved_metrics (Dict)
-
abstract
resolve_metric_bundle
(self, metric_fn_bundle)¶ Resolve a bundle of metrics with the same compute domain as part of a single trip to the compute engine.
-
abstract
get_domain_records
(self, domain_kwargs: dict)¶ get_domain_records computes the full-access data (dataframe or selectable) for computing metrics based on the given domain_kwargs and specific engine semantics.
- Returns
data corresponding to the compute domain
-
abstract
get_compute_domain
(self, domain_kwargs: dict, domain_type: Union[str, MetricDomainTypes])¶ get_compute_domain computes the optimal domain_kwargs for computing metrics based on the given domain_kwargs and specific engine semantics.
- Returns
data corresponding to the compute domain;
a modified copy of domain_kwargs describing the domain of the data returned in (1);
- a dictionary describing the access instructions for data elements included in the compute domain
(e.g. specific column name).
In general, the union of the compute_domain_kwargs and accessor_domain_kwargs will be the same as the domain_kwargs provided to this method.
- Return type
A tuple consisting of three elements
-
add_column_row_condition
(self, domain_kwargs, column_name=None, filter_null=True, filter_nan=False)¶ EXPERIMENTAL
Add a row condition for handling null filter.
- Parameters
domain_kwargs – the domain kwargs to use as the base and to which to add the condition
column_name – if provided, use this name to add the condition; otherwise, will use “column” key from table_domain_kwargs
filter_null – if true, add a filter for null values
filter_nan – if true, add a filter for nan values
-
resolve_data_reference
(self, data_connector_name: str, template_arguments: dict)¶ Resolve file path for a (data_connector_name, execution_engine_name) combination.
-
-
class
great_expectations.execution_engine.execution_engine.
MetricPartialFunctionTypes
¶ Bases:
enum.Enum
Generic enumeration.
Derive from this class to define new enumerations.
-
MAP_FN
= map_fn¶
-
MAP_SERIES
= map_series¶
-
MAP_CONDITION_FN
= map_condition_fn¶
-
MAP_CONDITION_SERIES
= map_condition_series¶
-
WINDOW_FN
= window_fn¶
-
WINDOW_CONDITION_FN
= window_condition_fn¶
-
AGGREGATE_FN
= aggregate_fn¶
-
property
metric_suffix
(self)¶
-