great_expectations.rule_based_profiler.parameter_builder.value_set_multi_batch_parameter_builder

Module Contents

Classes

ValueSetMultiBatchParameterBuilder(name: str, metric_domain_kwargs: Optional[Union[str, dict]] = None, metric_value_kwargs: Optional[Union[str, dict]] = None, batch_list: Optional[List[Batch]] = None, batch_request: Optional[Union[BatchRequest, RuntimeBatchRequest, dict]] = None, data_context: Optional[‘DataContext’] = None)

Build a set of unique values across all specified batches.

Functions

_get_unique_values_from_nested_collection_of_sets(collection: Collection[Collection[Set[Any]]])

Get unique values from a collection of sets e.g. a list of sets.

class great_expectations.rule_based_profiler.parameter_builder.value_set_multi_batch_parameter_builder.ValueSetMultiBatchParameterBuilder(name: str, metric_domain_kwargs: Optional[Union[str, dict]] = None, metric_value_kwargs: Optional[Union[str, dict]] = None, batch_list: Optional[List[Batch]] = None, batch_request: Optional[Union[BatchRequest, RuntimeBatchRequest, dict]] = None, data_context: Optional['DataContext'] = None)

Bases: great_expectations.rule_based_profiler.parameter_builder.MetricMultiBatchParameterBuilder

Build a set of unique values across all specified batches.

This parameter builder can be used to build a unique value_set for each of the domains specified by the DomainBuilder from all of the batches specified. This value_set can be used to create Expectations.

This unique value_set is the unique values from ALL batches accessible to the parameter builder. For example, if batch 1 has the unique values {1, 4, 8} and batch 2 {2, 8, 10} the unique values returned by this parameter builder are the set union, or {1, 2, 4, 8, 10}

Note: The computation of the unique values across batches is done within this ParameterBuilder so please be aware that testing large columns with high cardinality could require a large amount of memory.

property metric_domain_kwargs(self)
property metric_value_kwargs(self)
_build_parameters(self, parameter_container: ParameterContainer, domain: Domain, variables: Optional[ParameterContainer] = None, parameters: Optional[Dict[str, ParameterContainer]] = None)

Builds ParameterContainer object that holds ParameterNode objects with attribute name-value pairs and optional details.

return: Tuple containing computed_parameter_value and parameter_computation_details metadata.

great_expectations.rule_based_profiler.parameter_builder.value_set_multi_batch_parameter_builder._get_unique_values_from_nested_collection_of_sets(collection: Collection[Collection[Set[Any]]]) → Set[Any]

Get unique values from a collection of sets e.g. a list of sets.

Parameters

collection – Collection of Sets containing collections of values. can be nested Collections.

Returns

Single flattened set containing unique values.