great_expectations.marshmallow__shade
¶
Submodules¶
great_expectations.marshmallow__shade.base
great_expectations.marshmallow__shade.class_registry
great_expectations.marshmallow__shade.decorators
great_expectations.marshmallow__shade.error_store
great_expectations.marshmallow__shade.exceptions
great_expectations.marshmallow__shade.fields
great_expectations.marshmallow__shade.orderedset
great_expectations.marshmallow__shade.schema
great_expectations.marshmallow__shade.types
great_expectations.marshmallow__shade.utils
great_expectations.marshmallow__shade.validate
great_expectations.marshmallow__shade.warnings
Package Contents¶
Classes¶
|
Base schema class with which to define custom schemas. |
|
class Meta options for the |
Functions¶
|
Register a method to invoke after serializing an object. The method |
|
Register a method to invoke after deserializing an object. The method |
|
Register a method to invoke before serializing an object. The method |
|
Register a method to invoke before deserializing an object. The method |
|
Register a field validator. |
|
Register a schema-level validator. |
|
Pretty-printing function that can pretty-print OrderedDicts |
-
great_expectations.marshmallow__shade.
post_dump
(fn=None, pass_many=False, pass_original=False)¶ Register a method to invoke after serializing an object. The method receives the serialized object and returns the processed object.
By default it receives a single object at a time, transparently handling the
many
argument passed to the Schema’sdump()
call. Ifpass_many=True
, the raw data (which may be a collection) is passed.If
pass_original=True
, the original data (before serializing) will be passed as an additional argument to the method.Changed in version 3.0.0:
many
is always passed as a keyword arguments to the decorated method.
-
great_expectations.marshmallow__shade.
post_load
(fn=None, pass_many=False, pass_original=False)¶ Register a method to invoke after deserializing an object. The method receives the deserialized data and returns the processed data.
By default it receives a single object at a time, transparently handling the
many
argument passed to the Schema’sload()
call. Ifpass_many=True
, the raw data (which may be a collection) is passed.If
pass_original=True
, the original data (before deserializing) will be passed as an additional argument to the method.Changed in version 3.0.0:
partial
andmany
are always passed as keyword arguments to the decorated method.
-
great_expectations.marshmallow__shade.
pre_dump
(fn=None, pass_many=False)¶ Register a method to invoke before serializing an object. The method receives the object to be serialized and returns the processed object.
By default it receives a single object at a time, transparently handling the
many
argument passed to the Schema’sdump()
call. Ifpass_many=True
, the raw data (which may be a collection) is passed.Changed in version 3.0.0:
many
is always passed as a keyword arguments to the decorated method.
-
great_expectations.marshmallow__shade.
pre_load
(fn=None, pass_many=False)¶ Register a method to invoke before deserializing an object. The method receives the data to be deserialized and returns the processed data.
By default it receives a single object at a time, transparently handling the
many
argument passed to the Schema’sload()
call. Ifpass_many=True
, the raw data (which may be a collection) is passed.Changed in version 3.0.0:
partial
andmany
are always passed as keyword arguments to the decorated method.
-
great_expectations.marshmallow__shade.
validates
(field_name: str)¶ Register a field validator.
- Parameters
field_name (str) – Name of the field that the method validates.
-
great_expectations.marshmallow__shade.
validates_schema
(fn=None, pass_many=False, pass_original=False, skip_on_field_errors=True)¶ Register a schema-level validator.
By default it receives a single object at a time, transparently handling the
many
argument passed to the Schema’svalidate()
call. Ifpass_many=True
, the raw data (which may be a collection) is passed.If
pass_original=True
, the original data (before unmarshalling) will be passed as an additional argument to the method.If
skip_on_field_errors=True
, this validation method will be skipped whenever validation errors have been detected when validating fields.Changed in version 3.0.0b1:
skip_on_field_errors
defaults to True.Changed in version 3.0.0:
partial
andmany
are always passed as keyword arguments to the decorated method.
-
exception
great_expectations.marshmallow__shade.
ValidationError
(message: typing.Union[str, typing.List, typing.Dict], field_name: str = SCHEMA, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]] = None, valid_data: typing.Union[typing.List[typing.Dict[str, typing.Any]], typing.Dict[str, typing.Any]] = None, **kwargs)¶ Bases:
great_expectations.marshmallow__shade.exceptions.MarshmallowError
Raised when validation fails on a field or schema.
Validators and custom fields should raise this exception.
- Parameters
message – An error message, list of error messages, or dict of error messages. If a dict, the keys are subitems and the values are error messages.
field_name – Field name to store the error on. If None, the error is stored as schema-level error.
data – Raw input data.
valid_data – Valid (de)serialized data.
-
normalized_messages
(self)¶
-
class
great_expectations.marshmallow__shade.
Schema
(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)¶ Bases:
great_expectations.marshmallow__shade.base.SchemaABC
Base schema class with which to define custom schemas.
Example usage:
import datetime as dt from dataclasses import dataclass from great_expectations.marshmallow__shade import Schema, fields @dataclass class Album: title: str release_date: dt.date class AlbumSchema(Schema): title = fields.Str() release_date = fields.Date() album = Album("Beggars Banquet", dt.date(1968, 12, 6)) schema = AlbumSchema() data = schema.dump(album) data # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
- Parameters
only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.
exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.
many – Should be set to True if
obj
is a collection so that the object will be serialized to a list.context – Optional context passed to
fields.Method
andfields.Function
fields.load_only – Fields to skip during serialization (write-only fields)
dump_only – Fields to skip during deserialization (read-only fields)
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
Changed in version 3.0.0: prefix parameter removed.
Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.
-
class
Meta
¶ Options object for a Schema.
Example usage:
class Meta: fields = ("id", "email", "date_created") exclude = ("password", "secret_attribute")
Available options:
fields
: Tuple or list of fields to include in the serialized result.additional
: Tuple or list of fields to include in addition to theexplicitly declared fields.
additional
andfields
are mutually-exclusive options.
include
: Dictionary of additional fields to include in the schema. It isusually better to define fields as class variables, but you may need to use this option, e.g., if your fields are Python keywords. May be an OrderedDict.
exclude
: Tuple or list of fields to exclude in the serialized result.Nested fields can be represented with dot delimiters.
dateformat
: Default format for Date <fields.Date> fields.datetimeformat
: Default format for DateTime <fields.DateTime> fields.render_module
: Module to use for loads <Schema.loads> and dumps <Schema.dumps>.Defaults to json from the standard library.
ordered
: If True, order serialization output according to theorder in which fields were declared. Output of Schema.dump will be a collections.OrderedDict.
index_errors
: If True, errors dictionaries will include the indexof invalid items in a collection.
load_only
: Tuple or list of fields to exclude from serialized results.dump_only
: Tuple or list of fields to exclude from deserializationunknown
: Whether to exclude, include, or raise an error for unknownfields in the data. Use EXCLUDE, INCLUDE or RAISE.
register
: Whether to register the Schema with marshmallow’s internalclass registry. Must be True if you intend to refer to this Schema by class name in Nested fields. Only set this to False when memory usage is critical. Defaults to True.
-
TYPE_MAPPING
:typing.Dict[type, typing.Type[ma_fields.Field]]¶
-
error_messages
:typing.Dict[str, str]¶
-
_default_error_messages
:typing.Dict[str, str]¶
-
OPTIONS_CLASS
:type¶
-
opts
:SchemaOpts¶
-
_declared_fields
:typing.Dict[str, ma_fields.Field]¶
-
_hooks
:typing.Dict[types.Tag, typing.List[str]]¶
-
__repr__
(self)¶ Return repr(self).
-
property
dict_class
(self)¶
-
property
set_class
(self)¶
-
classmethod
from_dict
(cls, fields: typing.Dict[str, typing.Union[ma_fields.Field, type]], *, name: str = 'GeneratedSchema')¶ Generate a Schema class given a dictionary of fields.
from great_expectations.marshmallow__shade import Schema, fields PersonSchema = Schema.from_dict({"name": fields.Str()}) print(PersonSchema().load({"name": "David"})) # => {'name': 'David'}
Generated schemas are not added to the class registry and therefore cannot be referred to by name in Nested fields.
- Parameters
fields (dict) – Dictionary mapping field names to field instances.
name (str) – Optional name for the class, which will appear in the
repr
for the class.
New in version 3.0.0.
-
handle_error
(self, error: ValidationError, data: typing.Any, *, many: bool, **kwargs)¶ Custom error handler function for the schema.
- Parameters
error – The ValidationError raised during (de)serialization.
data – The original input data.
many – Value of
many
on dump or load.partial – Value of
partial
on load.
New in version 2.0.0.
Changed in version 3.0.0rc9: Receives many and partial (on deserialization) as keyword arguments.
-
get_attribute
(self, obj: typing.Any, attr: str, default: typing.Any)¶ Defines how to pull values from an object to serialize.
New in version 2.0.0.
Changed in version 3.0.0a1: Changed position of
obj
andattr
.
-
static
_call_and_store
(getter_func, data, *, field_name, error_store, index=None)¶ Call
getter_func
withdata
as its argument, and store any ValidationErrors.- Parameters
getter_func (callable) – Function for getting the serialized/deserialized value from
data
.data – The data passed to
getter_func
.field_name (str) – Field name.
index (int) – Index of the item being validated, if validating a collection, otherwise None.
-
_serialize
(self, obj: typing.Union[_T, typing.Iterable[_T]], *, many: bool = False)¶ Serialize
obj
.- Parameters
obj – The object(s) to serialize.
many (bool) – True if
data
should be serialized as a collection.
- Returns
A dictionary of the serialized data
Changed in version 1.0.0: Renamed from
marshal
.
-
dump
(self, obj: typing.Any, *, many: bool = None)¶ Serialize an object to native Python data types according to this Schema’s fields.
- Parameters
obj – The object to serialize.
many – Whether to serialize obj as a collection. If None, the value for self.many is used.
- Returns
A dict of serialized data
- Return type
dict
New in version 1.0.0.
Changed in version 3.0.0b7: This method returns the serialized data rather than a
(data, errors)
duple. AValidationError
is raised ifobj
is invalid.Changed in version 3.0.0rc9: Validation no longer occurs upon serialization.
-
dumps
(self, obj: typing.Any, *args, many: bool = None, **kwargs)¶ Same as
dump()
, except return a JSON-encoded string.- Parameters
obj – The object to serialize.
many – Whether to serialize obj as a collection. If None, the value for self.many is used.
- Returns
A
json
string- Return type
str
New in version 1.0.0.
Changed in version 3.0.0b7: This method returns the serialized data rather than a
(data, errors)
duple. AValidationError
is raised ifobj
is invalid.
-
_deserialize
(self, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]], *, error_store: ErrorStore, many: bool = False, partial=False, unknown=RAISE, index=None)¶ Deserialize
data
.- Parameters
data (dict) – The data to deserialize.
error_store (ErrorStore) – Structure to store errors.
many (bool) – True if
data
should be deserialized as a collection.partial (bool|tuple) – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.
index (int) – Index of the item being serialized (for storing errors) if serializing a collection, otherwise None.
- Returns
A dictionary of the deserialized data.
-
load
(self, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]], *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None, unknown: str = None)¶ Deserialize a data structure to an object defined by this Schema’s fields.
- Parameters
data – The data to deserialize.
many – Whether to deserialize data as a collection. If None, the value for self.many is used.
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE. If None, the value for self.unknown is used.
- Returns
Deserialized data
New in version 1.0.0.
Changed in version 3.0.0b7: This method returns the deserialized data rather than a
(data, errors)
duple. AValidationError
is raised if invalid data are passed.
-
loads
(self, json_data: str, *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None, unknown: str = None, **kwargs)¶ Same as
load()
, except it takes a JSON string as input.- Parameters
json_data – A JSON string of the data to deserialize.
many – Whether to deserialize obj as a collection. If None, the value for self.many is used.
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE. If None, the value for self.unknown is used.
- Returns
Deserialized data
New in version 1.0.0.
Changed in version 3.0.0b7: This method returns the deserialized data rather than a
(data, errors)
duple. AValidationError
is raised if invalid data are passed.
-
_run_validator
(self, validator_func, output, *, original_data, error_store, many, partial, pass_original, index=None)¶
-
validate
(self, data: typing.Mapping, *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None)¶ Validate data against the schema, returning a dictionary of validation errors.
- Parameters
data – The data to validate.
many – Whether to validate data as a collection. If None, the value for self.many is used.
partial – Whether to ignore missing fields and not require any fields declared. Propagates down to
Nested
fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.
- Returns
A dictionary of validation errors.
New in version 1.1.0.
-
_do_load
(self, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]], *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None, unknown: str = None, postprocess: bool = True)¶ Deserialize data, returning the deserialized result. This method is private API.
- Parameters
data – The data to deserialize.
many – Whether to deserialize data as a collection. If None, the value for self.many is used.
partial – Whether to validate required fields. If its value is an iterable, only fields listed in that iterable will be ignored will be allowed missing. If True, all fields will be allowed missing. If None, the value for self.partial is used.
unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE. If None, the value for self.unknown is used.
postprocess – Whether to run post_load methods..
- Returns
Deserialized data
-
_normalize_nested_options
(self)¶ Apply then flatten nested schema options. This method is private API.
-
__apply_nested_option
(self, option_name, field_names, set_operation)¶ Apply nested options to nested fields
-
_init_fields
(self)¶ Update self.fields, self.load_fields, and self.dump_fields based on schema options. This method is private API.
-
on_bind_field
(self, field_name: str, field_obj: ma_fields.Field)¶ Hook to modify a field when it is bound to the Schema.
No-op by default.
-
_bind_field
(self, field_name: str, field_obj: ma_fields.Field)¶ Bind field to the schema, setting any necessary attributes on the field (e.g. parent and name).
Also set field load_only and dump_only values if field_name was specified in
class Meta
.
-
_has_processors
(self, tag)¶
-
_invoke_dump_processors
(self, tag: str, data, *, many: bool, original_data=None)¶
-
_invoke_load_processors
(self, tag: str, data, *, many: bool, original_data, partial: typing.Union[bool, types.StrSequenceOrSet])¶
-
_invoke_field_validators
(self, *, error_store: ErrorStore, data, many: bool)¶
-
_invoke_schema_validators
(self, *, error_store: ErrorStore, pass_many: bool, data, original_data, many: bool, partial: typing.Union[bool, types.StrSequenceOrSet], field_errors: bool = False)¶
-
_invoke_processors
(self, tag: str, *, pass_many: bool, data, many: bool, original_data=None, **kwargs)¶
-
class
great_expectations.marshmallow__shade.
SchemaOpts
(meta, ordered: bool = False)¶ class Meta options for the
Schema
. Defines defaults.
-
great_expectations.marshmallow__shade.
EXCLUDE
= exclude¶
-
great_expectations.marshmallow__shade.
INCLUDE
= include¶
-
great_expectations.marshmallow__shade.
RAISE
= raise¶
-
great_expectations.marshmallow__shade.
missing
¶
-
great_expectations.marshmallow__shade.
pprint
(obj, *args, **kwargs) → None¶ Pretty-printing function that can pretty-print OrderedDicts like regular dictionaries. Useful for printing the output of
marshmallow.Schema.dump()
.