great_expectations.rule_based_profiler.config.base
¶
Module Contents¶
Classes¶
Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization. |
|
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization. |
|
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization. |
|
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization. |
|
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization. |
|
|
Abstract base class for Config objects. Sets the fields that must be included on a Config. |
Schema classes for configurations which extend from BaseYamlConfig must extend top-level Marshmallow Schema class. |
-
great_expectations.rule_based_profiler.config.base.
logger
¶
-
class
great_expectations.rule_based_profiler.config.base.
NotNullSchema
¶ Bases:
marshmallow.Schema
Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.
The __config_class__ attribute is utilized to point a Schema to a configuration. It is the responsibility of the child class to define its own __config_class__ to ensure proper serialization/deserialization.
Reference: https://marshmallow.readthedocs.io/en/stable/extending.html
-
make_config
(self, data: dict, **kwargs)¶ Hook to convert the schema object into its respective config type.
- Parameters
data – The dictionary representation of the configuration object
kwargs – Marshmallow-specific kwargs required to maintain hook signature (unused herein)
- Returns
An instance of configuration class, which subclasses the DictDot serialization class
- Raises
NotImplementedError – If the subclass inheriting NotNullSchema fails to define a __config_class__
-
remove_nulls_and_keep_unknowns
(self, output: dict, original: Type[DictDot], **kwargs)¶ Hook to clear the config object of any null values before being written as a dictionary. Additionally, it bypasses strict schema validation before writing to dict to ensure that dynamic attributes set through setattr are captured in the resulting object. It is important to note that only public attributes are captured through this process. Chetan - 20220126 - Note that if we tighten up the schema (remove the dynamic setattr behavior), the functionality to keep unknowns should also be removed.
- Parameters
output – Processed dictionary representation of the configuration object (leaving original intact)
original – The dictionary representation of the configuration object
kwargs – Marshmallow-specific kwargs required to maintain hook signature (unused herein)
- Returns
A cleaned dictionary that has no null values
-
-
class
great_expectations.rule_based_profiler.config.base.
DomainBuilderConfig
(class_name: str, module_name: Optional[str] = None, **kwargs)¶ Bases:
great_expectations.types.SerializableDictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
to_json_dict
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of “SerializableDictDot.to_json_dict() occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class itself. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__repr__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__repr__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__str__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__str__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
class
great_expectations.rule_based_profiler.config.base.
DomainBuilderConfigSchema
¶ Bases:
great_expectations.rule_based_profiler.config.base.NotNullSchema
Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.
The __config_class__ attribute is utilized to point a Schema to a configuration. It is the responsibility of the child class to define its own __config_class__ to ensure proper serialization/deserialization.
Reference: https://marshmallow.readthedocs.io/en/stable/extending.html
-
__config_class__
¶
-
module_name
¶
-
class_name
¶
-
-
class
great_expectations.rule_based_profiler.config.base.
ParameterBuilderConfig
(name: str, class_name: str, module_name: Optional[str] = None, evaluation_parameter_builder_configs: Optional[list] = None, **kwargs)¶ Bases:
great_expectations.types.SerializableDictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
to_json_dict
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of “SerializableDictDot.to_json_dict() occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class itself. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__repr__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__repr__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__str__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__str__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
class
great_expectations.rule_based_profiler.config.base.
ParameterBuilderConfigSchema
¶ Bases:
great_expectations.rule_based_profiler.config.base.NotNullSchema
Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.
The __config_class__ attribute is utilized to point a Schema to a configuration. It is the responsibility of the child class to define its own __config_class__ to ensure proper serialization/deserialization.
Reference: https://marshmallow.readthedocs.io/en/stable/extending.html
-
__config_class__
¶
-
name
¶
-
module_name
¶
-
class_name
¶
-
evaluation_parameter_builder_configs
¶
-
-
class
great_expectations.rule_based_profiler.config.base.
ExpectationConfigurationBuilderConfig
(expectation_type: str, class_name: str, module_name: Optional[str] = None, meta: Optional[dict] = None, validation_parameter_builder_configs: Optional[list] = None, **kwargs)¶ Bases:
great_expectations.types.SerializableDictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
to_json_dict
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of “SerializableDictDot.to_json_dict() occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class itself. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__repr__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__repr__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__str__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__str__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
class
great_expectations.rule_based_profiler.config.base.
ExpectationConfigurationBuilderConfigSchema
¶ Bases:
great_expectations.rule_based_profiler.config.base.NotNullSchema
Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.
The __config_class__ attribute is utilized to point a Schema to a configuration. It is the responsibility of the child class to define its own __config_class__ to ensure proper serialization/deserialization.
Reference: https://marshmallow.readthedocs.io/en/stable/extending.html
-
__config_class__
¶
-
module_name
¶
-
class_name
¶
-
expectation_type
¶
-
meta
¶
-
validation_parameter_builder_configs
¶
-
-
class
great_expectations.rule_based_profiler.config.base.
RuleConfig
(variables: Optional[Dict[str, Any]] = None, domain_builder: Optional[dict] = None, parameter_builders: Optional[List[dict]] = None, expectation_configuration_builders: Optional[List[dict]] = None)¶ Bases:
great_expectations.types.SerializableDictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
to_json_dict
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of “SerializableDictDot.to_json_dict() occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class itself. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__repr__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__repr__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__str__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__str__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
class
great_expectations.rule_based_profiler.config.base.
RuleConfigSchema
¶ Bases:
great_expectations.rule_based_profiler.config.base.NotNullSchema
Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.
The __config_class__ attribute is utilized to point a Schema to a configuration. It is the responsibility of the child class to define its own __config_class__ to ensure proper serialization/deserialization.
Reference: https://marshmallow.readthedocs.io/en/stable/extending.html
-
__config_class__
¶
-
variables
¶
-
domain_builder
¶
-
parameter_builders
¶
-
expectation_configuration_builders
¶
-
-
class
great_expectations.rule_based_profiler.config.base.
RuleBasedProfilerConfig
(name: str, config_version: float, rules: Dict[str, dict], id: Optional[str] = None, variables: Optional[Dict[str, Any]] = None, commented_map: Optional[CommentedMap] = None)¶ Bases:
great_expectations.core.configuration.AbstractConfig
,great_expectations.data_context.types.base.BaseYamlConfig
Abstract base class for Config objects. Sets the fields that must be included on a Config.
-
classmethod
from_commented_map
(cls, commented_map: CommentedMap)¶ Override parent implementation to pop unnecessary attrs from config.
Please see parent BaseYamlConfig for more details.
-
classmethod
get_config_class
(cls)¶
-
classmethod
get_schema_class
(cls)¶
-
to_json_dict
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of “SerializableDictDot.to_json_dict() occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class itself. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__repr__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__repr__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__str__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__str__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
classmethod
resolve_config_using_acceptable_arguments
(cls, profiler: RuleBasedProfiler, variables: Optional[Dict[str, Any]] = None, rules: Optional[Dict[str, Dict[str, Any]]] = None)¶ Reconciles variables/rules by taking into account runtime overrides and variable substitution.
Utilized in usage statistics to interact with the args provided in RuleBasedProfiler.run(). NOTE: This is a lightweight version of the RBP’s true reconiliation logic - see reconcile_profiler_variables and reconcile_profiler_rules.
- Parameters
profiler – The profiler used to invoke run().
variables – Any runtime override variables.
rules – Any runtime override rules.
- Returns
An instance of RuleBasedProfilerConfig that represents the reconciled profiler.
-
static
_substitute_variables_in_config
(rule: Rule, variables_container: ParameterContainer)¶ Recursively updates a given rule to substitute $variable references.
- Parameters
rule – The Rule object to update.
variables_container – Keeps track of $variable values to be substituted.
- Returns
The dictionary representation of the rule with all $variable references substituted.
-
classmethod
-
class
great_expectations.rule_based_profiler.config.base.
RuleBasedProfilerConfigSchema
¶ Bases:
great_expectations.core.configuration.AbstractConfigSchema
Schema classes for configurations which extend from BaseYamlConfig must extend top-level Marshmallow Schema class. Schema classes for their constituent configurations which extend DictDot leve must extend NotNullSchema class.
-
class
Meta
¶ -
unknown
¶
-
fields
= ['name', 'id', 'config_version', 'module_name', 'class_name', 'variables', 'rules']¶
-
ordered
= True¶
-
-
name
¶
-
id
¶
-
config_version
¶
-
module_name
¶
-
class_name
¶
-
variables
¶
-
rules
¶
-
class
-
great_expectations.rule_based_profiler.config.base.
expectationConfigurationBuilderConfigSchema
¶
-
great_expectations.rule_based_profiler.config.base.
parameterBuilderConfigSchema
¶
-
great_expectations.rule_based_profiler.config.base.
domainBuilderConfigSchema
¶
-
great_expectations.rule_based_profiler.config.base.
ruleConfigSchema
¶
-
great_expectations.rule_based_profiler.config.base.
ruleBasedProfilerConfigSchema
¶