great_expectations.rule_based_profiler.parameter_builder.value_set_multi_batch_parameter_builder

Module Contents

Classes

ValueSetMultiBatchParameterBuilder(name: str, metric_domain_kwargs: Optional[Union[str, dict]] = None, metric_value_kwargs: Optional[Union[str, dict]] = None, evaluation_parameter_builder_configs: Optional[List[ParameterBuilderConfig]] = None, data_context: Optional[AbstractDataContext] = None)

Build a set of unique values across all specified batches.

Functions

_get_unique_values_from_nested_collection_of_sets(collection: Collection[Collection[Set[Any]]])

Get unique values from a collection of sets e.g. a list of sets.
class great_expectations.rule_based_profiler.parameter_builder.value_set_multi_batch_parameter_builder.ValueSetMultiBatchParameterBuilder(name: str, metric_domain_kwargs: Optional[Union[str, dict]] = None, metric_value_kwargs: Optional[Union[str, dict]] = None, evaluation_parameter_builder_configs: Optional[List[ParameterBuilderConfig]] = None, data_context: Optional[AbstractDataContext] = None)

Bases: great_expectations.rule_based_profiler.parameter_builder.MetricMultiBatchParameterBuilder

Build a set of unique values across all specified batches.

This parameter builder can be used to build a unique value_set for each of the domains specified by the DomainBuilder from all of the batches specified. This value_set can be used to create Expectations.

This unique value_set is the unique values from ALL batches accessible to the parameter builder. For example, if batch 1 has the unique values {1, 4, 8} and batch 2 {2, 8, 10} the unique values returned by this parameter builder are the set union, or {1, 2, 4, 8, 10}

Notes

  1. The computation of the unique values across batches is done within this ParameterBuilder so please be aware that testing large columns with high cardinality could require a large amount of memory.

  2. This ParameterBuilder filters null values out from the unique value_set.

exclude_field_names :Set[str]
_build_parameters(self, domain: Domain, variables: Optional[ParameterContainer] = None, parameters: Optional[Dict[str, ParameterContainer]] = None, recompute_existing_parameter_values: bool = False)

Builds ParameterContainer object that holds ParameterNode objects with attribute name-value pairs and details.

Returns

Attributes object, containing computed parameter values and parameter computation details metadata.

great_expectations.rule_based_profiler.parameter_builder.value_set_multi_batch_parameter_builder._get_unique_values_from_nested_collection_of_sets(collection: Collection[Collection[Set[Any]]]) → Set[Any]

Get unique values from a collection of sets e.g. a list of sets.

Parameters

collection – Collection of Sets containing collections of values. can be nested Collections.

Returns

Single flattened set containing unique values.