great_expectations.data_context.types.base
¶
Module Contents¶
Classes¶
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
Base schema class with which to define custom schemas. |
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
Base schema class with which to define custom schemas. |
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
Base schema class with which to define custom schemas. |
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
Base schema class with which to define custom schemas. |
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
Base schema class with which to define custom schemas. |
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
Base schema class with which to define custom schemas. |
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
Base schema class with which to define custom schemas. |
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
Base schema class with which to define custom schemas. |
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
Base schema class with which to define custom schemas. |
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
Base schema class with which to define custom schemas. |
|
WARNING: This class is experimental. |
|
WARNING: This class is experimental. |
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
Base schema class with which to define custom schemas. |
Generic enumeration. |
|
Generic enumeration. |
|
|
Define base defaults for platform specific StoreBackendDefaults. |
|
Default store configs for s3 backends, with some accessible parameters |
|
Default store configs for filesystem backends, with some accessible parameters |
Default store configs for in memory backends. |
|
|
Default store configs for Google Cloud Storage (GCS) backends, with some accessible parameters |
|
Default store configs for database backends, with some accessible parameters |
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
Base schema class with which to define custom schemas. |
|
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
A convenience class for migrating away from untyped dictionaries to stronger typed objects. |
|
|
Base schema class with which to define custom schemas. |
Functions¶
|
-
great_expectations.data_context.types.base.
yaml
¶
-
great_expectations.data_context.types.base.
logger
¶
-
great_expectations.data_context.types.base.
CURRENT_GE_CONFIG_VERSION
= 3¶
-
great_expectations.data_context.types.base.
FIRST_GE_CONFIG_VERSION_WITH_CHECKPOINT_STORE
= 3¶
-
great_expectations.data_context.types.base.
CURRENT_CHECKPOINT_CONFIG_VERSION
= 1¶
-
great_expectations.data_context.types.base.
MINIMUM_SUPPORTED_CONFIG_VERSION
= 2¶
-
great_expectations.data_context.types.base.
DEFAULT_USAGE_STATISTICS_URL
= https://stats.greatexpectations.io/great_expectations/v1/usage_statistics¶
-
great_expectations.data_context.types.base.
object_to_yaml_str
(obj)¶
-
class
great_expectations.data_context.types.base.
BaseYamlConfig
(commented_map: Optional[CommentedMap] = None)¶ Bases:
great_expectations.types.SerializableDictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
_config_schema_class
¶
-
exclude_field_names
:Set[str]¶
-
classmethod
_get_schema_instance
(cls)¶
-
classmethod
from_commented_map
(cls, commented_map: CommentedMap)¶
-
_get_schema_validated_updated_commented_map
(self)¶
-
to_yaml
(self, outfile)¶ :returns None (but writes a YAML file containing the project configuration)
-
to_yaml_str
(self)¶ :returns a YAML string containing the project configuration
-
to_json_dict
(self)¶ :returns a JSON-serialiable dict containing the project configuration
-
property
commented_map
(self)¶
-
abstract classmethod
get_config_class
(cls)¶
-
abstract classmethod
get_schema_class
(cls)¶
-
class
great_expectations.data_context.types.base.
AssetConfig
(name=None, class_name=None, module_name=None, bucket=None, prefix=None, delimiter=None, max_keys=None, schema_name=None, batch_spec_passthrough=None, **kwargs)¶ Bases:
great_expectations.types.DictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
property
class_name
(self)¶
-
property
module_name
(self)¶
-
class
great_expectations.data_context.types.base.
AssetConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
name
¶
-
class_name
¶
-
module_name
¶
-
base_directory
¶
-
glob_directive
¶
-
pattern
¶
-
group_names
¶
-
bucket
¶
-
prefix
¶
-
delimiter
¶
-
max_keys
¶
-
schema_name
¶
-
batch_spec_passthrough
¶
-
table_name
¶
-
type
¶
-
validate_schema
(self, data, **kwargs)¶
-
make_asset_config
(self, data, **kwargs)¶
-
class
great_expectations.data_context.types.base.
SorterConfig
(name, class_name=None, module_name=None, orderby='asc', reference_list=None, datetime_format=None, **kwargs)¶ Bases:
great_expectations.types.DictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
property
name
(self)¶
-
property
module_name
(self)¶
-
property
class_name
(self)¶
-
property
orderby
(self)¶
-
property
reference_list
(self)¶
-
property
datetime_format
(self)¶
-
class
great_expectations.data_context.types.base.
SorterConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
name
¶
-
class_name
¶
-
module_name
¶
-
orderby
¶
-
reference_list
¶
-
datetime_format
¶
-
validate_schema
(self, data, **kwargs)¶
-
make_sorter_config
(self, data, **kwargs)¶
-
class
great_expectations.data_context.types.base.
DataConnectorConfig
(class_name, module_name=None, credentials=None, assets=None, base_directory=None, glob_directive=None, default_regex=None, batch_identifiers=None, sorters=None, batch_spec_passthrough=None, boto3_options=None, bucket=None, max_keys=None, azure_options=None, container=None, name_starts_with=None, bucket_or_name=None, max_results=None, prefix=None, delimiter=None, **kwargs)¶ Bases:
great_expectations.types.DictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
property
class_name
(self)¶
-
property
module_name
(self)¶
-
class
great_expectations.data_context.types.base.
DataConnectorConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
class_name
¶
-
module_name
¶
-
assets
¶
-
base_directory
¶
-
glob_directive
¶
-
sorters
¶
-
default_regex
¶
-
credentials
¶
-
batch_identifiers
¶
-
boto3_options
¶
-
bucket
¶
-
max_keys
¶
-
azure_options
¶
-
container
¶
-
name_starts_with
¶
-
gcs_options
¶
-
bucket_or_name
¶
-
max_results
¶
-
prefix
¶
-
delimiter
¶
-
data_asset_name_prefix
¶
-
data_asset_name_suffix
¶
-
include_schema_name
¶
-
splitter_method
¶
-
splitter_kwargs
¶
-
sampling_method
¶
-
sampling_kwargs
¶
-
excluded_tables
¶
-
included_tables
¶
-
skip_inapplicable_tables
¶
-
batch_spec_passthrough
¶
-
validate_schema
(self, data, **kwargs)¶
-
make_data_connector_config
(self, data, **kwargs)¶
-
class
great_expectations.data_context.types.base.
ExecutionEngineConfig
(class_name, module_name=None, caching=None, batch_spec_defaults=None, connection_string=None, credentials=None, spark_config=None, boto3_options=None, azure_options=None, gcs_options=None, credentials_info=None, **kwargs)¶ Bases:
great_expectations.types.DictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
property
module_name
(self)¶
-
property
class_name
(self)¶
-
property
batch_spec_defaults
(self)¶
-
class
great_expectations.data_context.types.base.
ExecutionEngineConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
class_name
¶
-
module_name
¶
-
connection_string
¶
-
credentials
¶
-
spark_config
¶
-
boto3_options
¶
-
azure_options
¶
-
gcs_options
¶
-
caching
¶
-
batch_spec_defaults
¶
-
force_reuse_spark_context
¶
-
credentials_info
¶
-
validate_schema
(self, data, **kwargs)¶
-
make_execution_engine_config
(self, data, **kwargs)¶
-
class
great_expectations.data_context.types.base.
DatasourceConfig
(class_name=None, module_name: str = 'great_expectations.datasource', execution_engine=None, data_connectors=None, data_asset_type=None, batch_kwargs_generators=None, connection_string=None, credentials=None, introspection=None, tables=None, boto3_options=None, azure_options=None, gcs_options=None, credentials_info=None, reader_method=None, reader_options=None, limit=None, **kwargs)¶ Bases:
great_expectations.types.DictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
property
class_name
(self)¶
-
property
module_name
(self)¶
-
class
great_expectations.data_context.types.base.
DatasourceConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
class_name
¶
-
module_name
¶
-
force_reuse_spark_context
¶
-
spark_config
¶
-
execution_engine
¶
-
data_connectors
¶
-
data_asset_type
¶
-
batch_kwargs_generators
¶
-
connection_string
¶
-
credentials
¶
-
introspection
¶
-
tables
¶
-
boto3_options
¶
-
azure_options
¶
-
gcs_options
¶
-
credentials_info
¶
-
reader_method
¶
-
reader_options
¶
-
limit
¶
-
validate_schema
(self, data, **kwargs)¶
-
make_datasource_config
(self, data, **kwargs)¶
-
class
great_expectations.data_context.types.base.
AnonymizedUsageStatisticsConfig
(enabled=True, data_context_id=None, usage_statistics_url=None)¶ Bases:
great_expectations.types.DictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
property
enabled
(self)¶
-
property
data_context_id
(self)¶
-
property
explicit_id
(self)¶
-
property
explicit_url
(self)¶
-
property
usage_statistics_url
(self)¶
-
class
great_expectations.data_context.types.base.
AnonymizedUsageStatisticsConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
data_context_id
¶
-
enabled
¶
-
usage_statistics_url
¶
-
_explicit_url
¶
-
make_usage_statistics_config
(self, data, **kwargs)¶
-
filter_implicit
(self, data, **kwargs)¶
-
class
great_expectations.data_context.types.base.
NotebookTemplateConfig
(file_name, template_kwargs=None)¶ Bases:
great_expectations.types.DictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
class
great_expectations.data_context.types.base.
NotebookTemplateConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
file_name
¶
-
template_kwargs
¶
-
make_notebook_template_config
(self, data, **kwargs)¶
-
class
great_expectations.data_context.types.base.
NotebookConfig
(class_name, module_name, custom_templates_module=None, header_markdown=None, footer_markdown=None, table_expectations_header_markdown=None, column_expectations_header_markdown=None, table_expectations_not_found_markdown=None, column_expectations_not_found_markdown=None, authoring_intro_markdown=None, column_expectations_markdown=None, header_code=None, footer_code=None, table_expectation_code=None, column_expectation_code=None)¶ Bases:
great_expectations.types.DictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
class
great_expectations.data_context.types.base.
NotebookConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
class_name
¶
-
module_name
¶
-
custom_templates_module
¶
-
header_markdown
¶
-
table_expectations_header_markdown
¶
-
column_expectations_header_markdown
¶
-
table_expectations_not_found_markdown
¶
-
column_expectations_not_found_markdown
¶
-
column_expectations_markdown
¶
-
header_code
¶
-
table_expectation_code
¶
-
column_expectation_code
¶
-
make_notebook_config
(self, data, **kwargs)¶
-
class
great_expectations.data_context.types.base.
NotebooksConfig
(suite_edit)¶ Bases:
great_expectations.types.DictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
class
great_expectations.data_context.types.base.
NotebooksConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
suite_edit
¶
-
make_notebooks_config
(self, data, **kwargs)¶
-
class
great_expectations.data_context.types.base.
ProgressBarsConfig
(globally: bool = True, profilers: bool = True, metric_calculations: bool = True)¶ Bases:
great_expectations.types.DictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
class
great_expectations.data_context.types.base.
ProgressBarsConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
globally
¶
-
profilers
¶
-
metric_calculations
¶
-
class
great_expectations.data_context.types.base.
ConcurrencyConfig
(enabled: bool = False)¶ Bases:
great_expectations.types.DictDot
WARNING: This class is experimental.
-
property
enabled
(self)¶ Whether or not multithreading is enabled.
-
property
max_database_query_concurrency
(self)¶ Max number of concurrent database queries to execute with mulithreading.
-
add_sqlalchemy_create_engine_parameters
(self, parameters: MutableMapping[str, Any])¶ Update SqlAlchemy parameters to prevent concurrency errors (e.g. http://sqlalche.me/e/14/3o7r) and bottlenecks.
- Parameters
parameters – SqlAlchemy create_engine parameters to which we add concurrency appropriate parameters. If the concurrency parameters are already set, those parameters are left unchanged.
-
property
-
class
great_expectations.data_context.types.base.
ConcurrencyConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
WARNING: This class is experimental.
-
enabled
¶
-
-
class
great_expectations.data_context.types.base.
GeCloudConfig
(base_url: str, account_id: str = None, access_token: str = None, organization_id: str = None)¶ Bases:
great_expectations.types.DictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
property
account_id
(self)¶
-
to_json_dict
(self)¶
-
class
great_expectations.data_context.types.base.
DataContextConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
config_version
¶
-
datasources
¶
-
expectations_store_name
¶
-
validations_store_name
¶
-
evaluation_parameter_store_name
¶
-
checkpoint_store_name
¶
-
profiler_store_name
¶
-
plugins_directory
¶
-
validation_operators
¶
-
stores
¶
-
notebooks
¶
-
data_docs_sites
¶
-
config_variables_file_path
¶
-
anonymous_usage_statistics
¶
-
progress_bars
¶
-
concurrency
¶
-
REMOVE_KEYS_IF_NONE
= ['concurrency', 'progress_bars']¶
-
remove_keys_if_none
(self, data: dict, **kwargs)¶
-
handle_error
(self, exc, data, **kwargs)¶ Log and raise our custom exception when (de)serialization fails.
-
validate_schema
(self, data, **kwargs)¶
-
class
great_expectations.data_context.types.base.
DataContextConfigDefaults
¶ Bases:
enum.Enum
Generic enumeration.
Derive from this class to define new enumerations.
-
DEFAULT_CONFIG_VERSION
¶
-
DEFAULT_EXPECTATIONS_STORE_NAME
= expectations_store¶
-
EXPECTATIONS_BASE_DIRECTORY
= expectations¶
-
DEFAULT_EXPECTATIONS_STORE_BASE_DIRECTORY_RELATIVE_NAME
¶
-
DEFAULT_VALIDATIONS_STORE_NAME
= validations_store¶
-
VALIDATIONS_BASE_DIRECTORY
= validations¶
-
DEFAULT_VALIDATIONS_STORE_BASE_DIRECTORY_RELATIVE_NAME
¶
-
DEFAULT_EVALUATION_PARAMETER_STORE_NAME
= evaluation_parameter_store¶
-
DEFAULT_EVALUATION_PARAMETER_STORE_BASE_DIRECTORY_RELATIVE_NAME
= evaluation_parameters/¶
-
DEFAULT_CHECKPOINT_STORE_NAME
= checkpoint_store¶
-
CHECKPOINTS_BASE_DIRECTORY
= checkpoints¶
-
DEFAULT_CHECKPOINT_STORE_BASE_DIRECTORY_RELATIVE_NAME
¶
-
DEFAULT_PROFILER_STORE_NAME
= profiler_store¶
-
PROFILERS_BASE_DIRECTORY
= profilers¶
-
DEFAULT_PROFILER_STORE_BASE_DIRECTORY_RELATIVE_NAME
¶
-
DEFAULT_DATA_DOCS_SITE_NAME
= local_site¶
-
DEFAULT_CONFIG_VARIABLES_FILEPATH
= uncommitted/config_variables.yml¶
-
PLUGINS_BASE_DIRECTORY
= plugins¶
-
DEFAULT_PLUGINS_DIRECTORY
¶
-
DEFAULT_VALIDATION_OPERATORS
¶
-
DEFAULT_STORES
¶
-
DEFAULT_DATA_DOCS_SITES
¶
-
-
class
great_expectations.data_context.types.base.
CheckpointConfigDefaults
¶ Bases:
enum.Enum
Generic enumeration.
Derive from this class to define new enumerations.
-
DEFAULT_CONFIG_VERSION
¶
-
-
class
great_expectations.data_context.types.base.
BaseStoreBackendDefaults
(expectations_store_name: str = DataContextConfigDefaults.DEFAULT_EXPECTATIONS_STORE_NAME.value, validations_store_name: str = DataContextConfigDefaults.DEFAULT_VALIDATIONS_STORE_NAME.value, evaluation_parameter_store_name: str = DataContextConfigDefaults.DEFAULT_EVALUATION_PARAMETER_STORE_NAME.value, checkpoint_store_name: str = DataContextConfigDefaults.DEFAULT_CHECKPOINT_STORE_NAME.value, profiler_store_name: str = DataContextConfigDefaults.DEFAULT_PROFILER_STORE_NAME.value, data_docs_site_name: str = DataContextConfigDefaults.DEFAULT_DATA_DOCS_SITE_NAME.value, validation_operators: dict = None, stores: dict = None, data_docs_sites: dict = None)¶ Bases:
great_expectations.types.DictDot
Define base defaults for platform specific StoreBackendDefaults. StoreBackendDefaults define defaults for specific cases of often used configurations. For example, if you plan to store expectations, validations, and data_docs in s3 use the S3StoreBackendDefaults and you may be able to specify less parameters.
-
class
great_expectations.data_context.types.base.
S3StoreBackendDefaults
(default_bucket_name: Optional[str] = None, expectations_store_bucket_name: Optional[str] = None, validations_store_bucket_name: Optional[str] = None, data_docs_bucket_name: Optional[str] = None, checkpoint_store_bucket_name: Optional[str] = None, profiler_store_bucket_name: Optional[str] = None, expectations_store_prefix: str = 'expectations', validations_store_prefix: str = 'validations', data_docs_prefix: str = 'data_docs', checkpoint_store_prefix: str = 'checkpoints', profiler_store_prefix: str = 'profilers', expectations_store_name: str = 'expectations_S3_store', validations_store_name: str = 'validations_S3_store', evaluation_parameter_store_name: str = 'evaluation_parameter_store', checkpoint_store_name: str = 'checkpoint_S3_store', profiler_store_name: str = 'profiler_S3_store')¶ Bases:
great_expectations.data_context.types.base.BaseStoreBackendDefaults
Default store configs for s3 backends, with some accessible parameters :param default_bucket_name: Use this bucket name for stores that do not have a bucket name provided :param expectations_store_bucket_name: Overrides default_bucket_name if supplied :param validations_store_bucket_name: Overrides default_bucket_name if supplied :param data_docs_bucket_name: Overrides default_bucket_name if supplied :param checkpoint_store_bucket_name: Overrides default_bucket_name if supplied :param profiler_store_bucket_name: Overrides default_bucket_name if supplied :param expectations_store_prefix: Overrides default if supplied :param validations_store_prefix: Overrides default if supplied :param data_docs_prefix: Overrides default if supplied :param checkpoint_store_prefix: Overrides default if supplied :param profiler_store_prefix: Overrides default if supplied :param expectations_store_name: Overrides default if supplied :param validations_store_name: Overrides default if supplied :param evaluation_parameter_store_name: Overrides default if supplied :param checkpoint_store_name: Overrides default if supplied :param profiler_store_name: Overrides default if supplied
-
class
great_expectations.data_context.types.base.
FilesystemStoreBackendDefaults
(root_directory: Optional[str] = None, plugins_directory: Optional[str] = None)¶ Bases:
great_expectations.data_context.types.base.BaseStoreBackendDefaults
Default store configs for filesystem backends, with some accessible parameters :param root_directory: Absolute directory prepended to the base_directory for each store :param plugins_directory: Overrides default if supplied
-
class
great_expectations.data_context.types.base.
InMemoryStoreBackendDefaults
¶ Bases:
great_expectations.data_context.types.base.BaseStoreBackendDefaults
Default store configs for in memory backends.
This is useful for testing without persistence.
-
class
great_expectations.data_context.types.base.
GCSStoreBackendDefaults
(default_bucket_name: Optional[str] = None, default_project_name: Optional[str] = None, expectations_store_bucket_name: Optional[str] = None, validations_store_bucket_name: Optional[str] = None, data_docs_bucket_name: Optional[str] = None, checkpoint_store_bucket_name: Optional[str] = None, profiler_store_bucket_name: Optional[str] = None, expectations_store_project_name: Optional[str] = None, validations_store_project_name: Optional[str] = None, data_docs_project_name: Optional[str] = None, checkpoint_store_project_name: Optional[str] = None, profiler_store_project_name: Optional[str] = None, expectations_store_prefix: str = 'expectations', validations_store_prefix: str = 'validations', data_docs_prefix: str = 'data_docs', checkpoint_store_prefix: str = 'checkpoints', profiler_store_prefix: str = 'profilers', expectations_store_name: str = 'expectations_GCS_store', validations_store_name: str = 'validations_GCS_store', evaluation_parameter_store_name: str = 'evaluation_parameter_store', checkpoint_store_name: str = 'checkpoint_GCS_store', profiler_store_name: str = 'profiler_GCS_store')¶ Bases:
great_expectations.data_context.types.base.BaseStoreBackendDefaults
Default store configs for Google Cloud Storage (GCS) backends, with some accessible parameters :param default_bucket_name: Use this bucket name for stores that do not have a bucket name provided :param default_project_name: Use this project name for stores that do not have a project name provided :param expectations_store_bucket_name: Overrides default_bucket_name if supplied :param validations_store_bucket_name: Overrides default_bucket_name if supplied :param data_docs_bucket_name: Overrides default_bucket_name if supplied :param checkpoint_store_bucket_name: Overrides default_bucket_name if supplied :param profiler_store_bucket_name: Overrides default_bucket_name if supplied :param expectations_store_project_name: Overrides default_project_name if supplied :param validations_store_project_name: Overrides default_project_name if supplied :param data_docs_project_name: Overrides default_project_name if supplied :param checkpoint_store_project_name: Overrides default_project_name if supplied :param profiler_store_project_name: Overrides default_project_name if supplied :param expectations_store_prefix: Overrides default if supplied :param validations_store_prefix: Overrides default if supplied :param data_docs_prefix: Overrides default if supplied :param checkpoint_store_prefix: Overrides default if supplied :param profiler_store_prefix: Overrides default if supplied :param expectations_store_name: Overrides default if supplied :param validations_store_name: Overrides default if supplied :param evaluation_parameter_store_name: Overrides default if supplied :param checkpoint_store_name: Overrides default if supplied :param profiler_store_name: Overrides default if supplied
-
class
great_expectations.data_context.types.base.
DatabaseStoreBackendDefaults
(default_credentials: Optional[Dict] = None, expectations_store_credentials: Optional[Dict] = None, validations_store_credentials: Optional[Dict] = None, checkpoint_store_credentials: Optional[Dict] = None, profiler_store_credentials: Optional[Dict] = None, expectations_store_name: str = 'expectations_database_store', validations_store_name: str = 'validations_database_store', evaluation_parameter_store_name: str = 'evaluation_parameter_store', checkpoint_store_name: str = 'checkpoint_database_store', profiler_store_name: str = 'profiler_database_store')¶ Bases:
great_expectations.data_context.types.base.BaseStoreBackendDefaults
Default store configs for database backends, with some accessible parameters :param default_credentials: Use these credentials for all stores that do not have credentials provided :param expectations_store_credentials: Overrides default_credentials if supplied :param validations_store_credentials: Overrides default_credentials if supplied :param checkpoint_store_credentials: Overrides default_credentials if supplied :param profiler_store_credentials: Overrides default_credentials if supplied :param expectations_store_name: Overrides default if supplied :param validations_store_name: Overrides default if supplied :param evaluation_parameter_store_name: Overrides default if supplied :param checkpoint_store_name: Overrides default if supplied :param profiler_store_name: Overrides default if supplied
-
class
great_expectations.data_context.types.base.
DataContextConfig
(config_version: Optional[float] = None, datasources: Optional[Union[Dict[str, DatasourceConfig], Dict[str, Dict[str, Union[Dict[str, str], str, dict]]]]] = None, expectations_store_name: Optional[str] = None, validations_store_name: Optional[str] = None, evaluation_parameter_store_name: Optional[str] = None, checkpoint_store_name: Optional[str] = None, profiler_store_name: Optional[str] = None, plugins_directory: Optional[str] = None, validation_operators=None, stores: Optional[Dict] = None, data_docs_sites: Optional[Dict] = None, notebooks=None, config_variables_file_path: Optional[str] = None, anonymous_usage_statistics=None, store_backend_defaults: Optional[BaseStoreBackendDefaults] = None, commented_map: Optional[CommentedMap] = None, concurrency: Optional[Union[ConcurrencyConfig, Dict]] = None, progress_bars: Optional[ProgressBarsConfig] = None)¶ Bases:
great_expectations.data_context.types.base.BaseYamlConfig
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
classmethod
get_config_class
(cls)¶
-
classmethod
get_schema_class
(cls)¶
-
property
config_version
(self)¶
-
to_json_dict
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of “SerializableDictDot.to_json_dict() occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class itself. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__repr__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__repr__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__str__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__str__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
class
great_expectations.data_context.types.base.
CheckpointConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
class
Meta
¶ -
unknown
¶
-
fields
= ['name', 'config_version', 'template_name', 'module_name', 'class_name', 'run_name_template', 'expectation_suite_name', 'batch_request', 'action_list', 'evaluation_parameters', 'runtime_configuration', 'validations', 'profilers', 'validation_operator_name', 'batches', 'site_names', 'slack_webhook', 'notify_on', 'notify_with', 'ge_cloud_id', 'expectation_suite_ge_cloud_id']¶
-
ordered
= True¶
-
-
REMOVE_KEYS_IF_NONE
= ['site_names', 'slack_webhook', 'notify_on', 'notify_with', 'validation_operator_name', 'batches']¶
-
ge_cloud_id
¶
-
name
¶
-
config_version
¶
-
template_name
¶
-
module_name
¶
-
class_name
¶
-
run_name_template
¶
-
expectation_suite_name
¶
-
expectation_suite_ge_cloud_id
¶
-
batch_request
¶
-
action_list
¶
-
evaluation_parameters
¶
-
runtime_configuration
¶
-
validations
¶
-
profilers
¶
-
validation_operator_name
¶
-
batches
¶
-
site_names
¶
-
slack_webhook
¶
-
notify_on
¶
-
notify_with
¶
-
validate_schema
(self, data, **kwargs)¶
-
prepare_dump
(self, data, **kwargs)¶
-
remove_keys_if_none
(self, data, **kwargs)¶
-
class
great_expectations.data_context.types.base.
CheckpointConfig
(name: Optional[str] = None, config_version: Optional[Union[int, float]] = None, template_name: Optional[str] = None, module_name: Optional[str] = None, class_name: Optional[str] = None, run_name_template: Optional[str] = None, expectation_suite_name: Optional[str] = None, batch_request: Optional[dict] = None, action_list: Optional[List[dict]] = None, evaluation_parameters: Optional[dict] = None, runtime_configuration: Optional[dict] = None, validations: Optional[List[dict]] = None, profilers: Optional[List[dict]] = None, validation_operator_name: Optional[str] = None, batches: Optional[List[dict]] = None, commented_map: Optional[CommentedMap] = None, ge_cloud_id: Optional[Union[UUID, str]] = None, site_names: Optional[Union[list, str]] = None, slack_webhook: Optional[str] = None, notify_on: Optional[str] = None, notify_with: Optional[str] = None, expectation_suite_ge_cloud_id: Optional[Union[UUID, str]] = None)¶ Bases:
great_expectations.data_context.types.base.BaseYamlConfig
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
classmethod
get_config_class
(cls)¶
-
classmethod
get_schema_class
(cls)¶
-
property
validation_operator_name
(self)¶
-
property
batches
(self)¶
-
property
ge_cloud_id
(self)¶
-
property
expectation_suite_ge_cloud_id
(self)¶
-
property
name
(self)¶
-
property
template_name
(self)¶
-
property
config_version
(self)¶
-
property
validations
(self)¶
-
property
profilers
(self)¶
-
property
module_name
(self)¶
-
property
class_name
(self)¶
-
property
run_name_template
(self)¶
-
property
batch_request
(self)¶
-
property
expectation_suite_name
(self)¶
-
property
action_list
(self)¶
-
property
site_names
(self)¶
-
property
slack_webhook
(self)¶
-
property
notify_on
(self)¶
-
property
notify_with
(self)¶
-
property
evaluation_parameters
(self)¶
-
property
runtime_configuration
(self)¶
-
__deepcopy__
(self, memo)¶
-
to_json_dict
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of “SerializableDictDot.to_json_dict() occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class itself. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__repr__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__repr__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
__str__
(self)¶ # TODO: <Alex>2/4/2022</Alex> This implementation of a custom “__str__()” occurs frequently and should ideally serve as the reference implementation in the “SerializableDictDot” class. However, the circular import dependencies, due to the location of the “great_expectations/types/__init__.py” and “great_expectations/core/util.py” modules make this refactoring infeasible at the present time.
-
class
great_expectations.data_context.types.base.
CheckpointValidationConfig
¶ Bases:
great_expectations.types.DictDot
A convenience class for migrating away from untyped dictionaries to stronger typed objects.
Can be instantiated with arguments:
- my_A = MyClassA(
foo=”a string”, bar=1,
)
Can be instantiated from a dictionary:
- my_A = MyClassA(
- **{
“foo”: “a string”, “bar”: 1,
}
)
Can be accessed using both dictionary and dot notation
my_A.foo == “a string” my_A.bar == 1
my_A[“foo”] == “a string” my_A[“bar”] == 1
Pairs nicely with @dataclass:
@dataclass() class MyClassA(DictDot):
foo: str bar: int
Can be made immutable:
@dataclass(frozen=True) class MyClassA(DictDot):
foo: str bar: int
For more examples of usage, please see test_dataclass_serializable_dot_dict_pattern.py in the tests folder.
-
class
great_expectations.data_context.types.base.
CheckpointValidationConfigSchema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.Schema
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
great_expectations.data_context.types.base.
dataContextConfigSchema
¶
-
great_expectations.data_context.types.base.
datasourceConfigSchema
¶
-
great_expectations.data_context.types.base.
dataConnectorConfigSchema
¶
-
great_expectations.data_context.types.base.
executionEngineConfigSchema
¶
-
great_expectations.data_context.types.base.
assetConfigSchema
¶
-
great_expectations.data_context.types.base.
sorterConfigSchema
¶
-
great_expectations.data_context.types.base.
anonymizedUsageStatisticsSchema
¶
-
great_expectations.data_context.types.base.
notebookConfigSchema
¶
-
great_expectations.data_context.types.base.
checkpointConfigSchema
¶
-
great_expectations.data_context.types.base.
concurrencyConfigSchema
¶
-
great_expectations.data_context.types.base.
progressBarsConfigSchema
¶