great_expectations.expectations.expectation

Module Contents

Classes

MetaExpectation()

MetaExpectation registers Expectations as they are defined, adding them to the Expectation registry.

Expectation(configuration: Optional[ExpectationConfiguration] = None)

Base class for all Expectations.

TableExpectation(configuration: Optional[ExpectationConfiguration] = None)

Base class for all Expectations.

ColumnExpectation(configuration: Optional[ExpectationConfiguration] = None)

Base class for all Expectations.

ColumnMapExpectation(configuration: Optional[ExpectationConfiguration] = None)

Base class for all Expectations.

ColumnPairMapExpectation(configuration: Optional[ExpectationConfiguration] = None)

Base class for all Expectations.

MulticolumnMapExpectation(configuration: Optional[ExpectationConfiguration] = None)

Base class for all Expectations.

Functions

camel_to_snake(name)

_format_map_output(result_format, success, element_count, nonnull_count, unexpected_count, unexpected_list, unexpected_index_list)

Helper function to construct expectation result objects for map_expectations (such as column_map_expectation

great_expectations.expectations.expectation.logger
great_expectations.expectations.expectation.p1
great_expectations.expectations.expectation.p2
great_expectations.expectations.expectation.camel_to_snake(name)
class great_expectations.expectations.expectation.MetaExpectation

Bases: abc.ABCMeta

MetaExpectation registers Expectations as they are defined, adding them to the Expectation registry.

Any class inheriting from Expectation will be registered based on the value of the “expectation_type” class attribute, or, if that is not set, by snake-casing the name of the class.

class great_expectations.expectations.expectation.Expectation(configuration: Optional[ExpectationConfiguration] = None)

Base class for all Expectations.

Expectation classes must have the following attributes set:
  1. domain_keys: a tuple of the keys used to determine the domain of the expectation

  2. success_keys: a tuple of the keys used to determine the success of the expectation.

In some cases, subclasses of Expectation (such as TableExpectation) can inherit these properties from their parent class.

They may optionally override runtime_keys and default_kwarg_values, and may optionally set an explicit value for expectation_type.

  1. runtime_keys lists the keys that can be used to control output but will not affect the actual success value of the expectation (such as result_format).

  2. default_kwarg_values is a dictionary that will be used to fill unspecified kwargs from the Expectation Configuration.

Expectation classes must implement the following:
  1. _validate

  2. get_validation_dependencies

In some cases, subclasses of Expectation, such as ColumnMapExpectation will already have correct implementations that may simply be inherited.

Additionally, they may provide implementations of:
  1. validate_configuration, which should raise an error if the configuration will not be usable for the Expectation

  2. Data Docs rendering methods decorated with the @renderer decorator. See the

version
domain_keys
success_keys
runtime_keys = ['include_config', 'catch_exceptions', 'result_format']
default_kwarg_values
legacy_method_parameters
classmethod is_abstract(cls)
classmethod _register_renderer_functions(cls)
abstract _validate(self, configuration: ExpectationConfiguration, metrics: Dict, runtime_configuration: dict = None, execution_engine: ExecutionEngine = None)
classmethod _atomic_prescriptive_template(cls, configuration=None, result=None, language=None, runtime_configuration=None, **kwargs)

Template function that contains the logic that is shared by atomic.prescriptive.summary (GE Cloud) and renderer.prescriptive (OSS GE)

classmethod _atomic_prescriptive_summary(cls, configuration=None, result=None, language=None, runtime_configuration=None, **kwargs)

Rendering function that is utilized by GE Cloud Front-end

classmethod _prescriptive_renderer(cls, configuration=None, result=None, language=None, runtime_configuration=None, **kwargs)
classmethod _diagnostic_status_icon_renderer(cls, configuration=None, result=None, language=None, runtime_configuration=None, **kwargs)
classmethod _diagnostic_unexpected_statement_renderer(cls, configuration=None, result=None, language=None, runtime_configuration=None, **kwargs)
classmethod _diagnostic_unexpected_table_renderer(cls, configuration=None, result=None, language=None, runtime_configuration=None, **kwargs)
classmethod _get_observed_value_from_evr(self, result: ExpectationValidationResult)
classmethod _atomic_diagnostic_observed_value(cls, configuration=None, result=None, language=None, runtime_configuration=None, **kwargs)

Rendering function that is utilized by GE Cloud Front-end

classmethod _diagnostic_observed_value_renderer(cls, configuration=None, result=None, language=None, runtime_configuration=None, **kwargs)
classmethod get_allowed_config_keys(cls)
metrics_validate(self, metrics: Dict, configuration: Optional[ExpectationConfiguration] = None, runtime_configuration: dict = None, execution_engine: ExecutionEngine = None)
_build_evr(self, raw_response, configuration)

_build_evr is a lightweight convenience wrapper handling cases where an Expectation implementor fails to return an EVR but returns the necessary components in a dictionary.

get_validation_dependencies(self, configuration: Optional[ExpectationConfiguration] = None, execution_engine: Optional[ExecutionEngine] = None, runtime_configuration: Optional[dict] = None)

Returns the result format and metrics required to validate this Expectation using the provided result format.

get_domain_kwargs(self, configuration: Optional[ExpectationConfiguration] = None)
get_success_kwargs(self, configuration: Optional[ExpectationConfiguration] = None)
get_runtime_kwargs(self, configuration: Optional[ExpectationConfiguration] = None, runtime_configuration: dict = None)
validate_configuration(self, configuration: Optional[ExpectationConfiguration])
validate(self, validator: Validator, configuration: Optional[ExpectationConfiguration] = None, evaluation_parameters=None, interactive_evaluation=True, data_context=None, runtime_configuration=None)
property configuration(self)
classmethod build_configuration(cls, *args, **kwargs)
run_diagnostics(self, pretty_print=True)

Produce a diagnostic report about this expectation. The current uses for this method’s output are using the JSON structure to populate the Public Expectation Gallery and enabling a fast devloop for developing new expectations where the contributors can quickly check the completeness of their expectations.

The content of the report: * name and description * “library metadata”, such as the GitHub usernames of the expectation’s authors * the execution engines the expectation is implemented for * the implemented renderers * tests in “examples” member variable * the tests are executed against the execution engines for which the expectation is implemented and the output of the test runs is included in the report.

At least one test case with include_in_gallery=True must be present in the examples to produce the metrics, renderers and execution engines parts of the report. This is due to a get_validation_dependencies requiring expectation_config as an argument.

If errors are encountered in the process of running the diagnostics, they are assumed to be due to incompleteness of the Expectation’s implementation (e.g., declaring a dependency on Metrics that do not exist). These errors are added under “errors” key in the report.

Parameters

pretty_print – TODO: this argument is not currently used. The intent is to return

a well formatted and easily readable text instead of the dictionary when the argument is set to True :return: a dictionary view of the report

_add_error_to_diagnostics_report(self, report_obj: Dict, error: Exception, stack_trace: str)
_get_examples(self, return_only_gallery_examples=True)

Get a list of examples from the object’s examples member variable.

Parameters

return_only_gallery_examples – if True, include only test examples where include_in_gallery is true

Returns

list of examples or [], if no examples exist

_get_docstring_and_short_description(self)
_choose_example(self, examples)
_instantiate_example_validation_results(self, test_batch: Batch, expectation_config: ExpectationConfiguration)
_get_supported_renderers(self, snake_name: str)
_get_test_results(self, snake_name, examples, execution_engines)
_get_rendered_result_as_string(self, rendered_result)
_get_renderer_dict(self, expectation_name: str, expectation_config: ExpectationConfiguration, validation_result: ExpectationValidationResult, standard_renderers=['renderer.answer', 'renderer.diagnostic.unexpected_statement', 'renderer.diagnostic.observed_value', 'renderer.diagnostic.status_icon', 'renderer.diagnostic.unexpected_table', 'renderer.prescriptive', 'renderer.question'])
_get_execution_engine_dict(self, upstream_metrics)
_get_upstream_metrics(self, expectation_config)
_get_library_metadata(self)
class great_expectations.expectations.expectation.TableExpectation(configuration: Optional[ExpectationConfiguration] = None)

Bases: great_expectations.expectations.expectation.Expectation, abc.ABC

Base class for all Expectations.

Expectation classes must have the following attributes set:
  1. domain_keys: a tuple of the keys used to determine the domain of the expectation

  2. success_keys: a tuple of the keys used to determine the success of the expectation.

In some cases, subclasses of Expectation (such as TableExpectation) can inherit these properties from their parent class.

They may optionally override runtime_keys and default_kwarg_values, and may optionally set an explicit value for expectation_type.

  1. runtime_keys lists the keys that can be used to control output but will not affect the actual success value of the expectation (such as result_format).

  2. default_kwarg_values is a dictionary that will be used to fill unspecified kwargs from the Expectation Configuration.

Expectation classes must implement the following:
  1. _validate

  2. get_validation_dependencies

In some cases, subclasses of Expectation, such as ColumnMapExpectation will already have correct implementations that may simply be inherited.

Additionally, they may provide implementations of:
  1. validate_configuration, which should raise an error if the configuration will not be usable for the Expectation

  2. Data Docs rendering methods decorated with the @renderer decorator. See the

domain_keys = ['batch_id', 'table', 'row_condition', 'condition_parser']
metric_dependencies
domain_type
get_validation_dependencies(self, configuration: Optional[ExpectationConfiguration] = None, execution_engine: Optional[ExecutionEngine] = None, runtime_configuration: Optional[dict] = None)

Returns the result format and metrics required to validate this Expectation using the provided result format.

validate_metric_value_between_configuration(self, configuration: Optional[ExpectationConfiguration])
_validate_metric_value_between(self, metric_name, configuration: ExpectationConfiguration, metrics: Dict, runtime_configuration: dict = None, execution_engine: ExecutionEngine = None)
class great_expectations.expectations.expectation.ColumnExpectation(configuration: Optional[ExpectationConfiguration] = None)

Bases: great_expectations.expectations.expectation.TableExpectation, abc.ABC

Base class for all Expectations.

Expectation classes must have the following attributes set:
  1. domain_keys: a tuple of the keys used to determine the domain of the expectation

  2. success_keys: a tuple of the keys used to determine the success of the expectation.

In some cases, subclasses of Expectation (such as TableExpectation) can inherit these properties from their parent class.

They may optionally override runtime_keys and default_kwarg_values, and may optionally set an explicit value for expectation_type.

  1. runtime_keys lists the keys that can be used to control output but will not affect the actual success value of the expectation (such as result_format).

  2. default_kwarg_values is a dictionary that will be used to fill unspecified kwargs from the Expectation Configuration.

Expectation classes must implement the following:
  1. _validate

  2. get_validation_dependencies

In some cases, subclasses of Expectation, such as ColumnMapExpectation will already have correct implementations that may simply be inherited.

Additionally, they may provide implementations of:
  1. validate_configuration, which should raise an error if the configuration will not be usable for the Expectation

  2. Data Docs rendering methods decorated with the @renderer decorator. See the

domain_keys = ['batch_id', 'table', 'column', 'row_condition', 'condition_parser']
domain_type
validate_configuration(self, configuration: Optional[ExpectationConfiguration])
class great_expectations.expectations.expectation.ColumnMapExpectation(configuration: Optional[ExpectationConfiguration] = None)

Bases: great_expectations.expectations.expectation.TableExpectation, abc.ABC

Base class for all Expectations.

Expectation classes must have the following attributes set:
  1. domain_keys: a tuple of the keys used to determine the domain of the expectation

  2. success_keys: a tuple of the keys used to determine the success of the expectation.

In some cases, subclasses of Expectation (such as TableExpectation) can inherit these properties from their parent class.

They may optionally override runtime_keys and default_kwarg_values, and may optionally set an explicit value for expectation_type.

  1. runtime_keys lists the keys that can be used to control output but will not affect the actual success value of the expectation (such as result_format).

  2. default_kwarg_values is a dictionary that will be used to fill unspecified kwargs from the Expectation Configuration.

Expectation classes must implement the following:
  1. _validate

  2. get_validation_dependencies

In some cases, subclasses of Expectation, such as ColumnMapExpectation will already have correct implementations that may simply be inherited.

Additionally, they may provide implementations of:
  1. validate_configuration, which should raise an error if the configuration will not be usable for the Expectation

  2. Data Docs rendering methods decorated with the @renderer decorator. See the

map_metric
domain_keys = ['batch_id', 'table', 'column', 'row_condition', 'condition_parser']
domain_type
success_keys = ['mostly']
default_kwarg_values
classmethod is_abstract(cls)
validate_configuration(self, configuration: Optional[ExpectationConfiguration])
get_validation_dependencies(self, configuration: Optional[ExpectationConfiguration] = None, execution_engine: Optional[ExecutionEngine] = None, runtime_configuration: Optional[dict] = None)

Returns the result format and metrics required to validate this Expectation using the provided result format.

_validate(self, configuration: ExpectationConfiguration, metrics: Dict, runtime_configuration: dict = None, execution_engine: ExecutionEngine = None)
class great_expectations.expectations.expectation.ColumnPairMapExpectation(configuration: Optional[ExpectationConfiguration] = None)

Bases: great_expectations.expectations.expectation.TableExpectation, abc.ABC

Base class for all Expectations.

Expectation classes must have the following attributes set:
  1. domain_keys: a tuple of the keys used to determine the domain of the expectation

  2. success_keys: a tuple of the keys used to determine the success of the expectation.

In some cases, subclasses of Expectation (such as TableExpectation) can inherit these properties from their parent class.

They may optionally override runtime_keys and default_kwarg_values, and may optionally set an explicit value for expectation_type.

  1. runtime_keys lists the keys that can be used to control output but will not affect the actual success value of the expectation (such as result_format).

  2. default_kwarg_values is a dictionary that will be used to fill unspecified kwargs from the Expectation Configuration.

Expectation classes must implement the following:
  1. _validate

  2. get_validation_dependencies

In some cases, subclasses of Expectation, such as ColumnMapExpectation will already have correct implementations that may simply be inherited.

Additionally, they may provide implementations of:
  1. validate_configuration, which should raise an error if the configuration will not be usable for the Expectation

  2. Data Docs rendering methods decorated with the @renderer decorator. See the

map_metric
domain_keys = ['batch_id', 'table', 'column_A', 'column_B', 'row_condition', 'condition_parser']
domain_type
success_keys = ['mostly']
default_kwarg_values
classmethod is_abstract(cls)
validate_configuration(self, configuration: Optional[ExpectationConfiguration])
get_validation_dependencies(self, configuration: Optional[ExpectationConfiguration] = None, execution_engine: Optional[ExecutionEngine] = None, runtime_configuration: Optional[dict] = None)

Returns the result format and metrics required to validate this Expectation using the provided result format.

_validate(self, configuration: ExpectationConfiguration, metrics: Dict, runtime_configuration: dict = None, execution_engine: ExecutionEngine = None)
class great_expectations.expectations.expectation.MulticolumnMapExpectation(configuration: Optional[ExpectationConfiguration] = None)

Bases: great_expectations.expectations.expectation.TableExpectation, abc.ABC

Base class for all Expectations.

Expectation classes must have the following attributes set:
  1. domain_keys: a tuple of the keys used to determine the domain of the expectation

  2. success_keys: a tuple of the keys used to determine the success of the expectation.

In some cases, subclasses of Expectation (such as TableExpectation) can inherit these properties from their parent class.

They may optionally override runtime_keys and default_kwarg_values, and may optionally set an explicit value for expectation_type.

  1. runtime_keys lists the keys that can be used to control output but will not affect the actual success value of the expectation (such as result_format).

  2. default_kwarg_values is a dictionary that will be used to fill unspecified kwargs from the Expectation Configuration.

Expectation classes must implement the following:
  1. _validate

  2. get_validation_dependencies

In some cases, subclasses of Expectation, such as ColumnMapExpectation will already have correct implementations that may simply be inherited.

Additionally, they may provide implementations of:
  1. validate_configuration, which should raise an error if the configuration will not be usable for the Expectation

  2. Data Docs rendering methods decorated with the @renderer decorator. See the

map_metric
domain_keys = ['batch_id', 'table', 'column_list', 'row_condition', 'condition_parser', 'ignore_row_if']
domain_type
success_keys
default_kwarg_values
classmethod is_abstract(cls)
validate_configuration(self, configuration: Optional[ExpectationConfiguration])
get_validation_dependencies(self, configuration: Optional[ExpectationConfiguration] = None, execution_engine: Optional[ExecutionEngine] = None, runtime_configuration: Optional[dict] = None)

Returns the result format and metrics required to validate this Expectation using the provided result format.

_validate(self, configuration: ExpectationConfiguration, metrics: Dict, runtime_configuration: dict = None, execution_engine: ExecutionEngine = None)
great_expectations.expectations.expectation._format_map_output(result_format, success, element_count, nonnull_count, unexpected_count, unexpected_list, unexpected_index_list)

Helper function to construct expectation result objects for map_expectations (such as column_map_expectation and file_lines_map_expectation).

Expectations support four result_formats: BOOLEAN_ONLY, BASIC, SUMMARY, and COMPLETE. In each case, the object returned has a different set of populated fields. See result_format for more information.

This function handles the logic for mapping those fields for column_map_expectations.