great_expectations.rule_based_profiler.config.base

Module Contents

Classes

NotNullSchema()

Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.

DomainBuilderConfig(class_name: str, module_name: Optional[str] = None, **kwargs)

A convenience class for migrating away from untyped dictionaries to stronger typed objects.

DomainBuilderConfigSchema()

Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.

ParameterBuilderConfig(name: str, class_name: str, module_name: Optional[str] = None, evaluation_parameter_builder_configs: Optional[list] = None, **kwargs)

A convenience class for migrating away from untyped dictionaries to stronger typed objects.

ParameterBuilderConfigSchema()

Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.

ExpectationConfigurationBuilderConfig(expectation_type: str, class_name: str, module_name: Optional[str] = None, meta: Optional[dict] = None, validation_parameter_builder_configs: Optional[list] = None, **kwargs)

A convenience class for migrating away from untyped dictionaries to stronger typed objects.

ExpectationConfigurationBuilderConfigSchema()

Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.

RuleConfig(variables: Optional[Dict[str, Any]] = None, domain_builder: Optional[dict] = None, parameter_builders: Optional[List[dict]] = None, expectation_configuration_builders: Optional[List[dict]] = None)

A convenience class for migrating away from untyped dictionaries to stronger typed objects.

RuleConfigSchema()

Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.

RuleBasedProfilerConfig(name: str, config_version: float, rules: Dict[str, dict], id: Optional[str] = None, variables: Optional[Dict[str, Any]] = None, commented_map: Optional[CommentedMap] = None)

Abstract base class for Config objects. Sets the fields that must be included on a Config.

RuleBasedProfilerConfigSchema()

Schema classes for configurations which extend from BaseYamlConfig must extend top-level Marshmallow Schema class.

great_expectations.rule_based_profiler.config.base.logger
class great_expectations.rule_based_profiler.config.base.NotNullSchema

Bases: marshmallow.Schema

Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.

The __config_class__ attribute is utilized to point a Schema to a configuration. It is the responsibility of the child class to define its own __config_class__ to ensure proper serialization/deserialization.

Reference: https://marshmallow.readthedocs.io/en/stable/extending.html

make_config(self, data: dict, **kwargs)

Hook to convert the schema object into its respective config type.

Parameters
  • data – The dictionary representation of the configuration object

  • kwargs – Marshmallow-specific kwargs required to maintain hook signature (unused herein)

Returns

An instance of configuration class, which subclasses the DictDot serialization class

Raises

NotImplementedError – If the subclass inheriting NotNullSchema fails to define a __config_class__

remove_nulls_and_keep_unknowns(self, output: dict, original: Type[DictDot], **kwargs)

Hook to clear the config object of any null values before being written as a dictionary. Additionally, it bypasses strict schema validation before writing to dict to ensure that dynamic attributes set through setattr are captured in the resulting object. It is important to note that only public attributes are captured through this process. Chetan - 20220126 - Note that if we tighten up the schema (remove the dynamic setattr behavior), the functionality to keep unknowns should also be removed.

Parameters
  • output – Processed dictionary representation of the configuration object (leaving original intact)

  • original – The dictionary representation of the configuration object

  • kwargs – Marshmallow-specific kwargs required to maintain hook signature (unused herein)

Returns

A cleaned dictionary that has no null values

class great_expectations.rule_based_profiler.config.base.DomainBuilderConfig(class_name: str, module_name: Optional[str] = None, **kwargs)

Bases: great_expectations.types.SerializableDictDot

A convenience class for migrating away from untyped dictionaries to stronger typed objects.

Can be instantiated with arguments:

my_A = MyClassA(

foo=”a string”, bar=1,

)

Can be instantiated from a dictionary:

my_A = MyClassA(
**{

“foo”: “a string”, “bar”: 1,

}

)

Can be accessed using both dictionary and dot notation

my_A.foo == “a string” my_A.bar == 1

my_A[“foo”] == “a string” my_A[“bar”] == 1

Pairs nicely with @dataclass:

@dataclass() class MyClassA(DictDot):

foo: str bar: int

Can be made immutable:

@dataclass(frozen=True) class MyClassA(DictDot):

foo: str bar: int

For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.

to_json_dict(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of “SerializableDictDot.to_json_dict() occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class itself. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

__repr__(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__repr__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

__str__(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__str__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

class great_expectations.rule_based_profiler.config.base.DomainBuilderConfigSchema

Bases: great_expectations.rule_based_profiler.config.base.NotNullSchema

Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.

The __config_class__ attribute is utilized to point a Schema to a configuration. It is the responsibility of the child class to define its own __config_class__ to ensure proper serialization/deserialization.

Reference: https://marshmallow.readthedocs.io/en/stable/extending.html

class Meta
unknown
__config_class__
module_name
class_name
class great_expectations.rule_based_profiler.config.base.ParameterBuilderConfig(name: str, class_name: str, module_name: Optional[str] = None, evaluation_parameter_builder_configs: Optional[list] = None, **kwargs)

Bases: great_expectations.types.SerializableDictDot

A convenience class for migrating away from untyped dictionaries to stronger typed objects.

Can be instantiated with arguments:

my_A = MyClassA(

foo=”a string”, bar=1,

)

Can be instantiated from a dictionary:

my_A = MyClassA(
**{

“foo”: “a string”, “bar”: 1,

}

)

Can be accessed using both dictionary and dot notation

my_A.foo == “a string” my_A.bar == 1

my_A[“foo”] == “a string” my_A[“bar”] == 1

Pairs nicely with @dataclass:

@dataclass() class MyClassA(DictDot):

foo: str bar: int

Can be made immutable:

@dataclass(frozen=True) class MyClassA(DictDot):

foo: str bar: int

For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.

to_json_dict(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of “SerializableDictDot.to_json_dict() occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class itself. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

__repr__(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__repr__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

__str__(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__str__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

class great_expectations.rule_based_profiler.config.base.ParameterBuilderConfigSchema

Bases: great_expectations.rule_based_profiler.config.base.NotNullSchema

Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.

The __config_class__ attribute is utilized to point a Schema to a configuration. It is the responsibility of the child class to define its own __config_class__ to ensure proper serialization/deserialization.

Reference: https://marshmallow.readthedocs.io/en/stable/extending.html

class Meta
unknown
__config_class__
name
module_name
class_name
evaluation_parameter_builder_configs
class great_expectations.rule_based_profiler.config.base.ExpectationConfigurationBuilderConfig(expectation_type: str, class_name: str, module_name: Optional[str] = None, meta: Optional[dict] = None, validation_parameter_builder_configs: Optional[list] = None, **kwargs)

Bases: great_expectations.types.SerializableDictDot

A convenience class for migrating away from untyped dictionaries to stronger typed objects.

Can be instantiated with arguments:

my_A = MyClassA(

foo=”a string”, bar=1,

)

Can be instantiated from a dictionary:

my_A = MyClassA(
**{

“foo”: “a string”, “bar”: 1,

}

)

Can be accessed using both dictionary and dot notation

my_A.foo == “a string” my_A.bar == 1

my_A[“foo”] == “a string” my_A[“bar”] == 1

Pairs nicely with @dataclass:

@dataclass() class MyClassA(DictDot):

foo: str bar: int

Can be made immutable:

@dataclass(frozen=True) class MyClassA(DictDot):

foo: str bar: int

For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.

to_json_dict(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of “SerializableDictDot.to_json_dict() occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class itself. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

__repr__(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__repr__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

__str__(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__str__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

class great_expectations.rule_based_profiler.config.base.ExpectationConfigurationBuilderConfigSchema

Bases: great_expectations.rule_based_profiler.config.base.NotNullSchema

Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.

The __config_class__ attribute is utilized to point a Schema to a configuration. It is the responsibility of the child class to define its own __config_class__ to ensure proper serialization/deserialization.

Reference: https://marshmallow.readthedocs.io/en/stable/extending.html

class Meta
unknown
__config_class__
module_name
class_name
expectation_type
meta
validation_parameter_builder_configs
class great_expectations.rule_based_profiler.config.base.RuleConfig(variables: Optional[Dict[str, Any]] = None, domain_builder: Optional[dict] = None, parameter_builders: Optional[List[dict]] = None, expectation_configuration_builders: Optional[List[dict]] = None)

Bases: great_expectations.types.SerializableDictDot

A convenience class for migrating away from untyped dictionaries to stronger typed objects.

Can be instantiated with arguments:

my_A = MyClassA(

foo=”a string”, bar=1,

)

Can be instantiated from a dictionary:

my_A = MyClassA(
**{

“foo”: “a string”, “bar”: 1,

}

)

Can be accessed using both dictionary and dot notation

my_A.foo == “a string” my_A.bar == 1

my_A[“foo”] == “a string” my_A[“bar”] == 1

Pairs nicely with @dataclass:

@dataclass() class MyClassA(DictDot):

foo: str bar: int

Can be made immutable:

@dataclass(frozen=True) class MyClassA(DictDot):

foo: str bar: int

For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.

to_json_dict(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of “SerializableDictDot.to_json_dict() occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class itself. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

__repr__(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__repr__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

__str__(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__str__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

class great_expectations.rule_based_profiler.config.base.RuleConfigSchema

Bases: great_expectations.rule_based_profiler.config.base.NotNullSchema

Extension of Marshmallow Schema to facilitate implicit removal of null values before serialization.

The __config_class__ attribute is utilized to point a Schema to a configuration. It is the responsibility of the child class to define its own __config_class__ to ensure proper serialization/deserialization.

Reference: https://marshmallow.readthedocs.io/en/stable/extending.html

class Meta
unknown
__config_class__
variables
domain_builder
parameter_builders
expectation_configuration_builders
class great_expectations.rule_based_profiler.config.base.RuleBasedProfilerConfig(name: str, config_version: float, rules: Dict[str, dict], id: Optional[str] = None, variables: Optional[Dict[str, Any]] = None, commented_map: Optional[CommentedMap] = None)

Bases: great_expectations.core.configuration.AbstractConfig, great_expectations.data_context.types.base.BaseYamlConfig

Abstract base class for Config objects. Sets the fields that must be included on a Config.

classmethod from_commented_map(cls, commented_map: CommentedMap)

Override parent implementation to pop unnecessary attrs from config.

Please see parent BaseYamlConfig for more details.

classmethod get_config_class(cls)
classmethod get_schema_class(cls)
to_json_dict(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of “SerializableDictDot.to_json_dict() occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class itself. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

__repr__(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__repr__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

__str__(self)

# TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__str__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.

classmethod resolve_config_using_acceptable_arguments(cls, profiler: RuleBasedProfiler, variables: Optional[Dict[str, Any]] = None, rules: Optional[Dict[str, Dict[str, Any]]] = None)

Reconciles variables/rules by taking into account runtime overrides and variable substitution.

Utilized in usage statistics to interact with the args provided in RuleBasedProfiler.run(). NOTE: This is a lightweight version of the RBP’s true reconiliation logic - see reconcile_profiler_variables and reconcile_profiler_rules.

Parameters
  • profiler – The profiler used to invoke run().

  • variables – Any runtime override variables.

  • rules – Any runtime override rules.

Returns

An instance of RuleBasedProfilerConfig that represents the reconciled profiler.

static _substitute_variables_in_config(rule: Rule, variables_container: ParameterContainer)

Recursively updates a given rule to substitute $variable references.

Parameters
  • rule – The Rule object to update.

  • variables_container – Keeps track of $variable values to be substituted.

Returns

The dictionary representation of the rule with all $variable references substituted.

class great_expectations.rule_based_profiler.config.base.RuleBasedProfilerConfigSchema

Bases: great_expectations.core.configuration.AbstractConfigSchema

Schema classes for configurations which extend from BaseYamlConfig must extend top-level Marshmallow Schema class. Schema classes for their constituent configurations which extend DictDot leve must extend NotNullSchema class.

class Meta
unknown
fields = ['name', 'id', 'config_version', 'module_name', 'class_name', 'variables', 'rules']
ordered = True
name
id
config_version
module_name
class_name
variables
rules
great_expectations.rule_based_profiler.config.base.expectationConfigurationBuilderConfigSchema
great_expectations.rule_based_profiler.config.base.parameterBuilderConfigSchema
great_expectations.rule_based_profiler.config.base.domainBuilderConfigSchema
great_expectations.rule_based_profiler.config.base.ruleConfigSchema
great_expectations.rule_based_profiler.config.base.ruleBasedProfilerConfigSchema