`great_expectations.rule_based_profiler.types.parameter_container`¶

Module Contents¶

Classes¶

`ParameterNode`()	ParameterNode is a node of a tree structure.
`ParameterContainer`()	ParameterContainer holds root nodes of tree structures, corresponding to fully-qualified parameter names.

Functions¶

`_parse_attribute_name`(name: str)	Using the grammer defined by “attribute_name”, provides the parsing of collection (list, dictionary) access syntax:
`validate_fully_qualified_parameter_name`(fully_qualified_parameter_name: str)
`build_parameter_container_for_variables`(variables_configs: Dict[str, Any])	Build a ParameterContainer for all of the profiler config variables passed as key value pairs
`build_parameter_container`(parameter_container: ParameterContainer, parameter_values: Dict[str, Any])	Builds the ParameterNode trees, corresponding to the fully_qualified_parameter_name first-level keys.
`_build_parameter_node_tree_for_one_parameter`(parameter_node: ParameterNode, parameter_name_as_list: List[str], parameter_value: Any)	Recursively builds a tree of ParameterNode objects, creating new ParameterNode objects parsimoniously (i.e., only if
`get_parameter_value_by_fully_qualified_parameter_name`(fully_qualified_parameter_name: str, domain: Optional[Domain] = None, variables: Optional[ParameterContainer] = None, parameters: Optional[Dict[str, ParameterContainer]] = None)	Get the parameter value from the current “rule state” using the fully-qualified parameter name.
`_get_parameter_value_from_parameter_container`(fully_qualified_parameter_name: str, fully_qualified_parameter_name_as_list: List[str], parameter_container: ParameterContainer)

great_expectations.rule_based_profiler.types.parameter_container.FULLY_QUALIFIED_PARAMETER_NAME_SEPARATOR_CHARACTER :str = .¶

great_expectations.rule_based_profiler.types.parameter_container.DOMAIN_KWARGS_PARAMETER_NAME :str = domain_kwargs¶

great_expectations.rule_based_profiler.types.parameter_container.DOMAIN_KWARGS_PARAMETER_FULLY_QUALIFIED_NAME :str¶

great_expectations.rule_based_profiler.types.parameter_container.VARIABLES_KEY :str¶

great_expectations.rule_based_profiler.types.parameter_container.FULLY_QUALIFIED_PARAMETER_NAME_VALUE_KEY :str = value¶

great_expectations.rule_based_profiler.types.parameter_container.FULLY_QUALIFIED_PARAMETER_NAME_METADATA_KEY :str = details¶

great_expectations.rule_based_profiler.types.parameter_container.attribute_name¶

exception great_expectations.rule_based_profiler.types.parameter_container.ParameterAttributeNameParserError(message)¶

Bases: great_expectations.exceptions.GreatExpectationsError

Common base class for all non-exit exceptions.

great_expectations.rule_based_profiler.types.parameter_container._parse_attribute_name(name: str) → ParseResults¶

Using the grammer defined by “attribute_name”, provides the parsing of collection (list, dictionary) access syntax: List: variable[index: int] Dictionary: variable[key: str] Nested List/Dictionary: variable[index_0: int][key_0: str][index_1: int][key_1: str][key_2: str][index_2: int]…

Applicability: To be used as part of configuration (e.g., YAML-based files or text strings). Extendability: Readily extensible to include “slice” and other standard accessors (as long as no dynamic elements).

great_expectations.rule_based_profiler.types.parameter_container.validate_fully_qualified_parameter_name(fully_qualified_parameter_name: str)¶

class great_expectations.rule_based_profiler.types.parameter_container.ParameterNode¶

Bases: great_expectations.types.SerializableDotDict

ParameterNode is a node of a tree structure.

The tree is implemented as a nested dictionary that also supports the “dot” notation at every level of hierarchy. Together, these design aspects allow the entire tree to be converted into a JSON object for external compatibility.

Since the descendant nodes (i.e., sub-dictionaries) are of the same type as their parent node, then each descendant node is also a tree (or a sub-tree). Each node can support the combination of attribute name-value pairs representing values and details containing helpful information regarding how these values were obtained (tolerances, explanations, etc.). By convention, the “value” key corresponds the parameter value, while the “details” key corresponds the auxiliary details. These details can be used to set the “meta” key of the ExpectationConfiguration.

See the ParameterContainer documentation for examples of different parameter naming structures supported.

Even though, typically, only the leaf nodes (characterized by having no keys of “ParameterNode” type) store parameter values and details, intermediate nodes may also have these properties. This is important for supporting the situations where multiple long fully-qualified parameter names have overlapping intermediate parts (see below).

to_dict(self)¶

to_json_dict(self)¶

class great_expectations.rule_based_profiler.types.parameter_container.ParameterContainer¶

Bases: great_expectations.types.SerializableDictDot

ParameterContainer holds root nodes of tree structures, corresponding to fully-qualified parameter names.

While all parameter names begin with the dollar sign character (“$”), a fully-qualified parameter name is a string, whose parts are delimited by the period character (“.”).

As an example, suppose that the value of the attribute “max_num_conversion_attempts” is needed for certain processing operations. However, there could be several attributes with this name, albeit in different contexts: $parameter.date_strings.tolerances.max_num_conversion_attempts $parameter.character.encodings.tolerances.max_num_conversion_attempts The fully-qualified parameter names disambiguate the same “leaf” attribute names occurring in multiple contexts. In the present example, the use of fully-qualified parameter names makes it clear that in one context, “max_num_conversion_attempts” refers to the operations on date/time, while in the other – it applies to characters.

$variables.false_positive_threshold $parameter.date_strings.yyyy_mm_dd_hh_mm_ss_tz_date_format.value $parameter.date_strings.yyyy_mm_dd_hh_mm_ss_tz_date_format.details $parameter.date_strings.yyyy_mm_dd_date_format.value $parameter.date_strings.yyyy_mm_dd_date_format.details $parameter.date_strings.mm_yyyy_dd_hh_mm_ss_tz_date_format.value $parameter.date_strings.mm_yyyy_dd_hh_mm_ss_tz_date_format.details $parameter.date_strings.mm_yyyy_dd_date_format.value $parameter.date_strings.mm_yyyy_dd_date_format.details $parameter.date_strings.tolerances.max_abs_error_time_milliseconds $parameter.date_strings.tolerances.max_num_conversion_attempts $parameter.tolerances.mostly $parameter.tolerances.financial.usd $mean $parameter.monthly_taxi_fairs.mean_values.value[0] $parameter.monthly_taxi_fairs.mean_values.value[1] $parameter.monthly_taxi_fairs.mean_values.value[2] $parameter.monthly_taxi_fairs.mean_values.value[3] $parameter.monthly_taxi_fairs.mean_values.details $parameter.daily_taxi_fairs.mean_values.value[“friday”] $parameter.daily_taxi_fairs.mean_values.value[“saturday”] $parameter.daily_taxi_fairs.mean_values.value[“sunday”] $parameter.daily_taxi_fairs.mean_values.value[“monday”] $parameter.daily_taxi_fairs.mean_values.details $parameter.weekly_taxi_fairs.mean_values.value[1][‘friday’] $parameter.weekly_taxi_fairs.mean_values.value[18][‘saturday’] $parameter.weekly_taxi_fairs.mean_values.value[20][‘sunday’] $parameter.weekly_taxi_fairs.mean_values.value[21][‘monday’] $parameter.weekly_taxi_fairs.mean_values.details $custom.lang.character_encodings

The reason that ParameterContainer is needed is that each ParameterNode can point only to one tree structure, characterized by having a specific root-level ParameterNode object. A root-level ParameterNode object corresponds to a set of fully-qualified parameter names that have the same first part (e.g., “parameter”). However, a Domain may utilize fully-qualified parameter names that have multiple first parts (i.e., from different “name spaces”). The ParameterContainer maintains a dictionary that holds references to root-level ParameterNode objects for all parameter “name spaces” applicable to the given Domain (where the first part of all fully-qualified parameter names within the same “name space” serves as the dictionary key, and the root-level ParameterNode objects are the values).

parameter_nodes :Optional[Dict[str, ParameterNode]]¶

set_parameter_node(self, parameter_name_root: str, parameter_node: ParameterNode)¶

get_parameter_node(self, parameter_name_root: str)¶

_convert_dictionaries_to_parameter_nodes(self, source: Optional[Any] = None)¶

to_dict(self)¶

to_json_dict(self)¶: # TODO: <Alex>2/4/2022</Alex> A reference implementation can be provided, once circular import dependencies, caused by relative locations of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules are resolved.

great_expectations.rule_based_profiler.types.parameter_container.build_parameter_container_for_variables(variables_configs: Dict[str, Any]) → ParameterContainer¶

Build a ParameterContainer for all of the profiler config variables passed as key value pairs :param variables_configs: Variable key: value pairs e.g. {“variable_name”: variable_value, …}

Returns: ParameterContainer containing all variables

great_expectations.rule_based_profiler.types.parameter_container.build_parameter_container(parameter_container: ParameterContainer, parameter_values: Dict[str, Any])¶

Builds the ParameterNode trees, corresponding to the fully_qualified_parameter_name first-level keys.

:param parameter_container initialized ParameterContainer for all ParameterNode trees :param parameter_values Example of the name-value structure for building parameters (matching the type hint in the method signature): {

“$parameter.date_strings.tolerances.max_abs_error_time_milliseconds.value”: 100, # Actual value can of Any type. # The “details” dictionary is Optional. “$parameter.date_strings.tolerances.max_abs_error_time_milliseconds.details”: {

“max_abs_error_time_milliseconds”: {

“confidence”: { # Arbitrary dictionary key
“success_ratio”: 1.0, # Arbitrary entries “comment”: “matched template”, # Arbitrary entries

}

},

}, # While highly recommended, the use of “.value” and “.details” keys is conventional (it is not enforced). “$parameter.tolerances.mostly”: 9.0e-1, # The key here does not end on “.value” and no “.details” is provided. …

} :return parameter_container holds the dictionary of ParameterNode objects corresponding to roots of parameter names

This function loops through the supplied pairs of fully-qualified parameter names and their corresponding values (and any “details”) and builds the tree under a single root-level ParameterNode object for a “name space”. In particular, if any ParameterNode object in the tree (starting with the root-level ParameterNode object) already exists, it is reused; in other words, ParameterNode objects are unique per part of fully-qualified parameter names.

great_expectations.rule_based_profiler.types.parameter_container._build_parameter_node_tree_for_one_parameter(parameter_node: ParameterNode, parameter_name_as_list: List[str], parameter_value: Any)¶: Recursively builds a tree of ParameterNode objects, creating new ParameterNode objects parsimoniously (i.e., only if ParameterNode object, corresponding to a part of fully-qualified parameter names in a “name space” does not exist). :param parameter_node: root-level ParameterNode for the sub-tree, characterized by the first parameter name in list :param parameter_name_as_list: list of parts of a fully-qualified parameter name of sub-tree (or sub “name space”) :param parameter_value: value pertaining to the last part of the fully-qualified parameter name (“leaf node”)

great_expectations.rule_based_profiler.types.parameter_container.get_parameter_value_by_fully_qualified_parameter_name(fully_qualified_parameter_name: str, domain: Optional[Domain] = None, variables: Optional[ParameterContainer] = None, parameters: Optional[Dict[str, ParameterContainer]] = None) → Optional[Union[Any, ParameterNode]]¶

Get the parameter value from the current “rule state” using the fully-qualified parameter name. A fully-qualified parameter name must be a dot-delimited string, or the name of a parameter (without the dots). Args

param fully_qualified_parameter_name

str – A dot-separated string key starting with $ for fetching parameters

param domain

Domain – current Domain of interest

:param variables :param parameters

Returns: Optional[Union[Any, ParameterNode]] object corresponding to the last part of the fully-qualified parameter

name supplied as argument – a value (of type “Any”) or a ParameterNode object (containing the sub-tree structure).

great_expectations.rule_based_profiler.types.parameter_container._get_parameter_value_from_parameter_container(fully_qualified_parameter_name: str, fully_qualified_parameter_name_as_list: List[str], parameter_container: ParameterContainer) → Optional[Union[Any, ParameterNode]]¶

great_expectations.rule_based_profiler.types.parameter_container¶

Module Contents¶

Classes¶

Functions¶

`great_expectations.rule_based_profiler.types.parameter_container`¶