great_expectations.expectations.metrics.map_metric_provider
¶
Module Contents¶
Classes¶
Base class for all metric providers. |
|
Base class for all metric providers. |
|
Base class for all metric providers. |
|
Base class for all metric providers. |
Functions¶
|
Provides engine-specific support for authoring a metric_fn with a simplified signature. |
|
Provides engine-specific support for authoring a metric_fn with a simplified signature. |
|
Provides engine-specific support for authoring a metric_fn with a simplified signature. |
|
Provides engine-specific support for authoring a metric_fn with a simplified signature. A |
|
Provides engine-specific support for authoring a metric_fn with a simplified signature. |
|
Provides engine-specific support for authoring a metric_fn with a simplified signature. A |
|
Returns unexpected count for MapExpectations |
|
Return values from the specified domain that match the map-style metric in the metrics dictionary. |
|
Return values from the specified domain that match the map-style metric in the metrics dictionary. |
|
Return record counts from the specified domain that match the map-style metric in the metrics dictionary. |
|
Return values from the specified domain that match the map-style metric in the metrics dictionary. |
|
Return record counts from the specified domain that match the map-style metric in the metrics dictionary. |
|
Return values from the specified domain that match the map-style metric in the metrics dictionary. |
|
|
|
Returns respective value counts for distinct column values |
|
Return values from the specified domain (ignoring the column constraint) that match the map-style metric in the metrics dictionary. |
|
Returns unexpected count for MapExpectations |
|
Returns unexpected count for MapExpectations. This is a value metric, which is useful for |
|
Particularly for the purpose of finding unexpected values, returns all the metric values which do not meet an |
|
Return values from the specified domain that match the map-style metric in the metrics dictionary. |
|
Return record counts from the specified domain that match the map-style metric in the metrics dictionary. |
|
Return values from the specified domain that match the map-style metric in the metrics dictionary. |
|
Return record counts from the specified domain that match the map-style metric in the metrics dictionary. |
|
Returns value counts for all the metric values which do not meet an expected Expectation condition for instances |
|
Returns all rows of the metric values which do not meet an expected Expectation condition for instances |
|
|
|
|
|
Return values from the specified domain that match the map-style metric in the metrics dictionary. |
|
|
|
|
|
Return values from the specified domain that match the map-style metric in the metrics dictionary. |
|
Return record counts from the specified domain that match the map-style metric in the metrics dictionary. |
|
Return values from the specified domain that match the map-style metric in the metrics dictionary. |
|
Return record counts from the specified domain that match the map-style metric in the metrics dictionary. |
-
great_expectations.expectations.metrics.map_metric_provider.
logger
¶
-
great_expectations.expectations.metrics.map_metric_provider.
column_function_partial
(engine: Type[ExecutionEngine], partial_fn_type: str = None, **kwargs)¶ Provides engine-specific support for authoring a metric_fn with a simplified signature.
A metric function that is decorated as a column_function_partial will be called with the engine-specific column type and any value_kwargs associated with the Metric for which the provider function is being declared.
- Parameters
engine –
partial_fn_type –
**kwargs –
- Returns
An annotated metric_function which will be called with a simplified signature.
-
great_expectations.expectations.metrics.map_metric_provider.
column_condition_partial
(engine: Type[ExecutionEngine], partial_fn_type: Optional[Union[str, MetricPartialFunctionTypes]] = None, **kwargs)¶ Provides engine-specific support for authoring a metric_fn with a simplified signature.
A column_condition_partial must provide a map function that evaluates to a boolean value; it will be used to provide supplemental metrics, such as the unexpected_value count, unexpected_values, and unexpected_rows.
A metric function that is decorated as a column_condition_partial will be called with the engine-specific column type and any value_kwargs associated with the Metric for which the provider function is being declared.
- Parameters
engine –
partial_fn_type –
**kwargs –
- Returns
An annotated metric_function which will be called with a simplified signature.
-
great_expectations.expectations.metrics.map_metric_provider.
column_pair_function_partial
(engine: Type[ExecutionEngine], partial_fn_type: str = None, **kwargs)¶ Provides engine-specific support for authoring a metric_fn with a simplified signature.
A metric function that is decorated as a column_pair_function_partial will be called with the engine-specific column_list type and any value_kwargs associated with the Metric for which the provider function is being declared.
- Parameters
engine –
partial_fn_type –
**kwargs –
- Returns
An annotated metric_function which will be called with a simplified signature.
-
great_expectations.expectations.metrics.map_metric_provider.
column_pair_condition_partial
(engine: Type[ExecutionEngine], partial_fn_type: Optional[Union[str, MetricPartialFunctionTypes]] = None, **kwargs)¶ Provides engine-specific support for authoring a metric_fn with a simplified signature. A column_pair_condition_partial must provide a map function that evaluates to a boolean value; it will be used to provide supplemental metrics, such as the unexpected_value count, unexpected_values, and unexpected_rows.
A metric function that is decorated as a column_pair_condition_partial will be called with the engine-specific column_list type and any value_kwargs associated with the Metric for which the provider function is being declared.
- Parameters
engine –
partial_fn_type –
**kwargs –
- Returns
An annotated metric_function which will be called with a simplified signature.
-
great_expectations.expectations.metrics.map_metric_provider.
multicolumn_function_partial
(engine: Type[ExecutionEngine], partial_fn_type: str = None, **kwargs)¶ Provides engine-specific support for authoring a metric_fn with a simplified signature.
A metric function that is decorated as a multicolumn_function_partial will be called with the engine-specific column_list type and any value_kwargs associated with the Metric for which the provider function is being declared.
- Parameters
engine –
partial_fn_type –
**kwargs –
- Returns
An annotated metric_function which will be called with a simplified signature.
-
great_expectations.expectations.metrics.map_metric_provider.
multicolumn_condition_partial
(engine: Type[ExecutionEngine], partial_fn_type: Optional[Union[str, MetricPartialFunctionTypes]] = None, **kwargs)¶ Provides engine-specific support for authoring a metric_fn with a simplified signature. A multicolumn_condition_partial must provide a map function that evaluates to a boolean value; it will be used to provide supplemental metrics, such as the unexpected_value count, unexpected_values, and unexpected_rows.
A metric function that is decorated as a multicolumn_condition_partial will be called with the engine-specific column_list type and any value_kwargs associated with the Metric for which the provider function is being declared.
- Parameters
engine –
partial_fn_type –
**kwargs –
- Returns
An annotated metric_function which will be called with a simplified signature.
-
great_expectations.expectations.metrics.map_metric_provider.
_pandas_map_condition_unexpected_count
(cls, execution_engine: PandasExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Returns unexpected count for MapExpectations
-
great_expectations.expectations.metrics.map_metric_provider.
_pandas_column_map_condition_values
(cls, execution_engine: PandasExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return values from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_pandas_column_pair_map_condition_values
(cls, execution_engine: PandasExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return values from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_pandas_column_pair_map_condition_filtered_row_count
(cls, execution_engine: PandasExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return record counts from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_pandas_multicolumn_map_condition_values
(cls, execution_engine: PandasExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return values from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_pandas_multicolumn_map_condition_filtered_row_count
(cls, execution_engine: PandasExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return record counts from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_pandas_column_map_series_and_domain_values
(cls, execution_engine: PandasExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return values from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_pandas_map_condition_index
(cls, execution_engine: PandasExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶
-
great_expectations.expectations.metrics.map_metric_provider.
_pandas_column_map_condition_value_counts
(cls, execution_engine: PandasExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Returns respective value counts for distinct column values
-
great_expectations.expectations.metrics.map_metric_provider.
_pandas_map_condition_rows
(cls, execution_engine: PandasExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return values from the specified domain (ignoring the column constraint) that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_sqlalchemy_map_condition_unexpected_count_aggregate_fn
(cls, execution_engine: SqlAlchemyExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Returns unexpected count for MapExpectations
-
great_expectations.expectations.metrics.map_metric_provider.
_sqlalchemy_map_condition_unexpected_count_value
(cls, execution_engine: SqlAlchemyExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Returns unexpected count for MapExpectations. This is a value metric, which is useful for when the unexpected_condition is a window function.
-
great_expectations.expectations.metrics.map_metric_provider.
_sqlalchemy_column_map_condition_values
(cls, execution_engine: SqlAlchemyExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Particularly for the purpose of finding unexpected values, returns all the metric values which do not meet an expected Expectation condition for ColumnMapExpectation Expectations.
-
great_expectations.expectations.metrics.map_metric_provider.
_sqlalchemy_column_pair_map_condition_values
(cls, execution_engine: SqlAlchemyExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return values from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_sqlalchemy_column_pair_map_condition_filtered_row_count
(cls, execution_engine: SqlAlchemyExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return record counts from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_sqlalchemy_multicolumn_map_condition_values
(cls, execution_engine: SqlAlchemyExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return values from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_sqlalchemy_multicolumn_map_condition_filtered_row_count
(cls, execution_engine: SqlAlchemyExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return record counts from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_sqlalchemy_column_map_condition_value_counts
(cls, execution_engine: SqlAlchemyExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Returns value counts for all the metric values which do not meet an expected Expectation condition for instances of ColumnMapExpectation.
-
great_expectations.expectations.metrics.map_metric_provider.
_sqlalchemy_map_condition_rows
(cls, execution_engine: SqlAlchemyExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Returns all rows of the metric values which do not meet an expected Expectation condition for instances of ColumnMapExpectation.
-
great_expectations.expectations.metrics.map_metric_provider.
_spark_map_condition_unexpected_count_aggregate_fn
(cls, execution_engine: SparkDFExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶
-
great_expectations.expectations.metrics.map_metric_provider.
_spark_map_condition_unexpected_count_value
(cls, execution_engine: SparkDFExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶
-
great_expectations.expectations.metrics.map_metric_provider.
_spark_column_map_condition_values
(cls, execution_engine: SparkDFExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return values from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_spark_column_map_condition_value_counts
(cls, execution_engine: SparkDFExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶
-
great_expectations.expectations.metrics.map_metric_provider.
_spark_map_condition_rows
(cls, execution_engine: SparkDFExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶
-
great_expectations.expectations.metrics.map_metric_provider.
_spark_column_pair_map_condition_values
(cls, execution_engine: SparkDFExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return values from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_spark_column_pair_map_condition_filtered_row_count
(cls, execution_engine: SparkDFExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return record counts from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_spark_multicolumn_map_condition_values
(cls, execution_engine: SqlAlchemyExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return values from the specified domain that match the map-style metric in the metrics dictionary.
-
great_expectations.expectations.metrics.map_metric_provider.
_spark_multicolumn_map_condition_filtered_row_count
(cls, execution_engine: SparkDFExecutionEngine, metric_domain_kwargs: Dict, metric_value_kwargs: Dict, metrics: Dict[str, Any], **kwargs)¶ Return record counts from the specified domain that match the map-style metric in the metrics dictionary.
-
class
great_expectations.expectations.metrics.map_metric_provider.
MapMetricProvider
¶ Bases:
great_expectations.expectations.metrics.metric_provider.MetricProvider
Base class for all metric providers.
- MetricProvider classes must have the following attributes set:
metric_name: the name to use. Metric Name must be globally unique in a great_expectations installation.
domain_keys: a tuple of the keys used to determine the domain of the metric
value_keys: a tuple of the keys used to determine the value of the metric.
In some cases, subclasses of Expectation, such as TableMetricProvider will already have correct values that may simply be inherited.
They may optionally override the default_kwarg_values attribute.
- MetricProvider classes must implement the following:
1. _get_evaluation_dependencies. Note that often, _get_evaluation_dependencies should augment dependencies provided by a parent class; consider calling super()._get_evaluation_dependencies
In some cases, subclasses of Expectation, such as MapMetricProvider will already have correct implementations that may simply be inherited.
- Additionally, they may provide implementations of:
1. Data Docs rendering methods decorated with the @renderer decorator. See the guide “How to create renderers for custom expectations” for more information.
-
condition_domain_keys
= ['batch_id', 'table', 'row_condition', 'condition_parser']¶
-
function_domain_keys
= ['batch_id', 'table', 'row_condition', 'condition_parser']¶
-
condition_value_keys
¶
-
function_value_keys
¶
-
filter_column_isnull
= True¶
-
SQLALCHEMY_SELECTABLE_METRICS
¶
-
classmethod
_register_metric_functions
(cls)¶
-
classmethod
_get_evaluation_dependencies
(cls, metric: MetricConfiguration, configuration: Optional[ExpectationConfiguration] = None, execution_engine: Optional[ExecutionEngine] = None, runtime_configuration: Optional[dict] = None)¶
-
static
is_sqlalchemy_metric_selectable
(map_metric_provider: MetaMetricProvider)¶ - Parameters
map_metric_provider – object of type “MapMetricProvider”, whose SQLAlchemy implementation is inspected
- Returns
boolean indicating whether or not the returned value of a method implementing the metric resolves all
columns – hence the caller must not use “select_from” clause as part of its own SQLAlchemy query; otherwise an unwanted selectable (e.g., table) will be added to “FROM”, leading to duplicated and/or erroneous results.
-
class
great_expectations.expectations.metrics.map_metric_provider.
ColumnMapMetricProvider
¶ Bases:
great_expectations.expectations.metrics.map_metric_provider.MapMetricProvider
Base class for all metric providers.
- MetricProvider classes must have the following attributes set:
metric_name: the name to use. Metric Name must be globally unique in a great_expectations installation.
domain_keys: a tuple of the keys used to determine the domain of the metric
value_keys: a tuple of the keys used to determine the value of the metric.
In some cases, subclasses of Expectation, such as TableMetricProvider will already have correct values that may simply be inherited.
They may optionally override the default_kwarg_values attribute.
- MetricProvider classes must implement the following:
1. _get_evaluation_dependencies. Note that often, _get_evaluation_dependencies should augment dependencies provided by a parent class; consider calling super()._get_evaluation_dependencies
In some cases, subclasses of Expectation, such as MapMetricProvider will already have correct implementations that may simply be inherited.
- Additionally, they may provide implementations of:
1. Data Docs rendering methods decorated with the @renderer decorator. See the guide “How to create renderers for custom expectations” for more information.
-
condition_domain_keys
= ['batch_id', 'table', 'column', 'row_condition', 'condition_parser']¶
-
function_domain_keys
= ['batch_id', 'table', 'column', 'row_condition', 'condition_parser']¶
-
condition_value_keys
¶
-
function_value_keys
¶
-
classmethod
_get_evaluation_dependencies
(cls, metric: MetricConfiguration, configuration: Optional[ExpectationConfiguration] = None, execution_engine: Optional[ExecutionEngine] = None, runtime_configuration: Optional[dict] = None)¶
-
class
great_expectations.expectations.metrics.map_metric_provider.
ColumnPairMapMetricProvider
¶ Bases:
great_expectations.expectations.metrics.map_metric_provider.MapMetricProvider
Base class for all metric providers.
- MetricProvider classes must have the following attributes set:
metric_name: the name to use. Metric Name must be globally unique in a great_expectations installation.
domain_keys: a tuple of the keys used to determine the domain of the metric
value_keys: a tuple of the keys used to determine the value of the metric.
In some cases, subclasses of Expectation, such as TableMetricProvider will already have correct values that may simply be inherited.
They may optionally override the default_kwarg_values attribute.
- MetricProvider classes must implement the following:
1. _get_evaluation_dependencies. Note that often, _get_evaluation_dependencies should augment dependencies provided by a parent class; consider calling super()._get_evaluation_dependencies
In some cases, subclasses of Expectation, such as MapMetricProvider will already have correct implementations that may simply be inherited.
- Additionally, they may provide implementations of:
1. Data Docs rendering methods decorated with the @renderer decorator. See the guide “How to create renderers for custom expectations” for more information.
-
condition_domain_keys
= ['batch_id', 'table', 'column_A', 'column_B', 'row_condition', 'condition_parser', 'ignore_row_if']¶
-
function_domain_keys
= ['batch_id', 'table', 'column_A', 'column_B', 'row_condition', 'condition_parser', 'ignore_row_if']¶
-
condition_value_keys
¶
-
function_value_keys
¶
-
classmethod
_get_evaluation_dependencies
(cls, metric: MetricConfiguration, configuration: Optional[ExpectationConfiguration] = None, execution_engine: Optional[ExecutionEngine] = None, runtime_configuration: Optional[dict] = None)¶
-
class
great_expectations.expectations.metrics.map_metric_provider.
MulticolumnMapMetricProvider
¶ Bases:
great_expectations.expectations.metrics.map_metric_provider.MapMetricProvider
Base class for all metric providers.
- MetricProvider classes must have the following attributes set:
metric_name: the name to use. Metric Name must be globally unique in a great_expectations installation.
domain_keys: a tuple of the keys used to determine the domain of the metric
value_keys: a tuple of the keys used to determine the value of the metric.
In some cases, subclasses of Expectation, such as TableMetricProvider will already have correct values that may simply be inherited.
They may optionally override the default_kwarg_values attribute.
- MetricProvider classes must implement the following:
1. _get_evaluation_dependencies. Note that often, _get_evaluation_dependencies should augment dependencies provided by a parent class; consider calling super()._get_evaluation_dependencies
In some cases, subclasses of Expectation, such as MapMetricProvider will already have correct implementations that may simply be inherited.
- Additionally, they may provide implementations of:
1. Data Docs rendering methods decorated with the @renderer decorator. See the guide “How to create renderers for custom expectations” for more information.
-
condition_domain_keys
= ['batch_id', 'table', 'column_list', 'row_condition', 'condition_parser', 'ignore_row_if']¶
-
function_domain_keys
= ['batch_id', 'table', 'column_list', 'row_condition', 'condition_parser', 'ignore_row_if']¶
-
condition_value_keys
¶
-
function_value_keys
¶
-
classmethod
_get_evaluation_dependencies
(cls, metric: MetricConfiguration, configuration: Optional[ExpectationConfiguration] = None, execution_engine: Optional[ExecutionEngine] = None, runtime_configuration: Optional[dict] = None)¶