great_expectations.marshmallow__shade

Package Contents

Classes

Schema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Base schema class with which to define custom schemas.

SchemaOpts(meta, ordered: bool = False)

class Meta options for the Schema. Defines defaults.

Functions

post_dump(fn=None, pass_many=False, pass_original=False)

Register a method to invoke after serializing an object. The method

post_load(fn=None, pass_many=False, pass_original=False)

Register a method to invoke after deserializing an object. The method

pre_dump(fn=None, pass_many=False)

Register a method to invoke before serializing an object. The method

pre_load(fn=None, pass_many=False)

Register a method to invoke before deserializing an object. The method

validates(field_name: str)

Register a field validator.

validates_schema(fn=None, pass_many=False, pass_original=False, skip_on_field_errors=True)

Register a schema-level validator.

pprint(obj, *args, **kwargs)

Pretty-printing function that can pretty-print OrderedDicts

great_expectations.marshmallow__shade.post_dump(fn=None, pass_many=False, pass_original=False)

Register a method to invoke after serializing an object. The method receives the serialized object and returns the processed object.

By default it receives a single object at a time, transparently handling the many argument passed to the Schema’s dump() call. If pass_many=True, the raw data (which may be a collection) is passed.

If pass_original=True, the original data (before serializing) will be passed as an additional argument to the method.

Changed in version 3.0.0: many is always passed as a keyword arguments to the decorated method.

great_expectations.marshmallow__shade.post_load(fn=None, pass_many=False, pass_original=False)

Register a method to invoke after deserializing an object. The method receives the deserialized data and returns the processed data.

By default it receives a single object at a time, transparently handling the many argument passed to the Schema’s load() call. If pass_many=True, the raw data (which may be a collection) is passed.

If pass_original=True, the original data (before deserializing) will be passed as an additional argument to the method.

Changed in version 3.0.0: partial and many are always passed as keyword arguments to the decorated method.

great_expectations.marshmallow__shade.pre_dump(fn=None, pass_many=False)

Register a method to invoke before serializing an object. The method receives the object to be serialized and returns the processed object.

By default it receives a single object at a time, transparently handling the many argument passed to the Schema’s dump() call. If pass_many=True, the raw data (which may be a collection) is passed.

Changed in version 3.0.0: many is always passed as a keyword arguments to the decorated method.

great_expectations.marshmallow__shade.pre_load(fn=None, pass_many=False)

Register a method to invoke before deserializing an object. The method receives the data to be deserialized and returns the processed data.

By default it receives a single object at a time, transparently handling the many argument passed to the Schema’s load() call. If pass_many=True, the raw data (which may be a collection) is passed.

Changed in version 3.0.0: partial and many are always passed as keyword arguments to the decorated method.

great_expectations.marshmallow__shade.validates(field_name: str)

Register a field validator.

Parameters

field_name (str) – Name of the field that the method validates.

great_expectations.marshmallow__shade.validates_schema(fn=None, pass_many=False, pass_original=False, skip_on_field_errors=True)

Register a schema-level validator.

By default it receives a single object at a time, transparently handling the many argument passed to the Schema’s validate() call. If pass_many=True, the raw data (which may be a collection) is passed.

If pass_original=True, the original data (before unmarshalling) will be passed as an additional argument to the method.

If skip_on_field_errors=True, this validation method will be skipped whenever validation errors have been detected when validating fields.

Changed in version 3.0.0b1: skip_on_field_errors defaults to True.

Changed in version 3.0.0: partial and many are always passed as keyword arguments to the decorated method.

exception great_expectations.marshmallow__shade.ValidationError(message: typing.Union[str, typing.List, typing.Dict], field_name: str = SCHEMA, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]] = None, valid_data: typing.Union[typing.List[typing.Dict[str, typing.Any]], typing.Dict[str, typing.Any]] = None, **kwargs)

Bases: great_expectations.marshmallow__shade.exceptions.MarshmallowError

Raised when validation fails on a field or schema.

Validators and custom fields should raise this exception.

Parameters
  • message – An error message, list of error messages, or dict of error messages. If a dict, the keys are subitems and the values are error messages.

  • field_name – Field name to store the error on. If None, the error is stored as schema-level error.

  • data – Raw input data.

  • valid_data – Valid (de)serialized data.

normalized_messages(self)
class great_expectations.marshmallow__shade.Schema(*, only: types.StrSequenceOrSet = None, exclude: types.StrSequenceOrSet = (), many: bool = False, context: typing.Dict = None, load_only: types.StrSequenceOrSet = (), dump_only: types.StrSequenceOrSet = (), partial: typing.Union[bool, types.StrSequenceOrSet] = False, unknown: str = None)

Bases: great_expectations.marshmallow__shade.base.SchemaABC

Base schema class with which to define custom schemas.

Example usage:

import datetime as dt
from dataclasses import dataclass

from great_expectations.marshmallow__shade import Schema, fields


@dataclass
class Album:
    title: str
    release_date: dt.date


class AlbumSchema(Schema):
    title = fields.Str()
    release_date = fields.Date()


album = Album("Beggars Banquet", dt.date(1968, 12, 6))
schema = AlbumSchema()
data = schema.dump(album)
data  # {'release_date': '1968-12-06', 'title': 'Beggars Banquet'}
Parameters
  • only – Whitelist of the declared fields to select when instantiating the Schema. If None, all fields are used. Nested fields can be represented with dot delimiters.

  • exclude – Blacklist of the declared fields to exclude when instantiating the Schema. If a field appears in both only and exclude, it is not used. Nested fields can be represented with dot delimiters.

  • many – Should be set to True if obj is a collection so that the object will be serialized to a list.

  • context – Optional context passed to fields.Method and fields.Function fields.

  • load_only – Fields to skip during serialization (write-only fields)

  • dump_only – Fields to skip during deserialization (read-only fields)

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.

Changed in version 3.0.0: prefix parameter removed.

Changed in version 2.0.0: __validators__, __preprocessors__, and __data_handlers__ are removed in favor of marshmallow.decorators.validates_schema, marshmallow.decorators.pre_load and marshmallow.decorators.post_dump. __accessor__ and __error_handler__ are deprecated. Implement the handle_error and get_attribute methods instead.

class Meta

Options object for a Schema.

Example usage:

class Meta:
    fields = ("id", "email", "date_created")
    exclude = ("password", "secret_attribute")

Available options:

  • fields: Tuple or list of fields to include in the serialized result.

  • additional: Tuple or list of fields to include in addition to the

    explicitly declared fields. additional and fields are mutually-exclusive options.

  • include: Dictionary of additional fields to include in the schema. It is

    usually better to define fields as class variables, but you may need to use this option, e.g., if your fields are Python keywords. May be an OrderedDict.

  • exclude: Tuple or list of fields to exclude in the serialized result.

    Nested fields can be represented with dot delimiters.

  • dateformat: Default format for Date <fields.Date> fields.

  • datetimeformat: Default format for DateTime <fields.DateTime> fields.

  • render_module: Module to use for loads <Schema.loads> and dumps <Schema.dumps>.

    Defaults to json from the standard library.

  • ordered: If True, order serialization output according to the

    order in which fields were declared. Output of Schema.dump will be a collections.OrderedDict.

  • index_errors: If True, errors dictionaries will include the index

    of invalid items in a collection.

  • load_only: Tuple or list of fields to exclude from serialized results.

  • dump_only: Tuple or list of fields to exclude from deserialization

  • unknown: Whether to exclude, include, or raise an error for unknown

    fields in the data. Use EXCLUDE, INCLUDE or RAISE.

  • register: Whether to register the Schema with marshmallow’s internal

    class registry. Must be True if you intend to refer to this Schema by class name in Nested fields. Only set this to False when memory usage is critical. Defaults to True.

TYPE_MAPPING :typing.Dict[type, typing.Type[ma_fields.Field]]
error_messages :typing.Dict[str, str]
_default_error_messages :typing.Dict[str, str]
OPTIONS_CLASS :type
opts :SchemaOpts
_declared_fields :typing.Dict[str, ma_fields.Field]
_hooks :typing.Dict[types.Tag, typing.List[str]]
__repr__(self)

Return repr(self).

property dict_class(self)
property set_class(self)
classmethod from_dict(cls, fields: typing.Dict[str, typing.Union[ma_fields.Field, type]], *, name: str = 'GeneratedSchema')

Generate a Schema class given a dictionary of fields.

from great_expectations.marshmallow__shade import Schema, fields

PersonSchema = Schema.from_dict({"name": fields.Str()})
print(PersonSchema().load({"name": "David"}))  # => {'name': 'David'}

Generated schemas are not added to the class registry and therefore cannot be referred to by name in Nested fields.

Parameters
  • fields (dict) – Dictionary mapping field names to field instances.

  • name (str) – Optional name for the class, which will appear in the repr for the class.

New in version 3.0.0.

handle_error(self, error: ValidationError, data: typing.Any, *, many: bool, **kwargs)

Custom error handler function for the schema.

Parameters
  • error – The ValidationError raised during (de)serialization.

  • data – The original input data.

  • many – Value of many on dump or load.

  • partial – Value of partial on load.

New in version 2.0.0.

Changed in version 3.0.0rc9: Receives many and partial (on deserialization) as keyword arguments.

get_attribute(self, obj: typing.Any, attr: str, default: typing.Any)

Defines how to pull values from an object to serialize.

New in version 2.0.0.

Changed in version 3.0.0a1: Changed position of obj and attr.

static _call_and_store(getter_func, data, *, field_name, error_store, index=None)

Call getter_func with data as its argument, and store any ValidationErrors.

Parameters
  • getter_func (callable) – Function for getting the serialized/deserialized value from data.

  • data – The data passed to getter_func.

  • field_name (str) – Field name.

  • index (int) – Index of the item being validated, if validating a collection, otherwise None.

_serialize(self, obj: typing.Union[_T, typing.Iterable[_T]], *, many: bool = False)

Serialize obj.

Parameters
  • obj – The object(s) to serialize.

  • many (bool) – True if data should be serialized as a collection.

Returns

A dictionary of the serialized data

Changed in version 1.0.0: Renamed from marshal.

dump(self, obj: typing.Any, *, many: bool = None)

Serialize an object to native Python data types according to this Schema’s fields.

Parameters
  • obj – The object to serialize.

  • many – Whether to serialize obj as a collection. If None, the value for self.many is used.

Returns

A dict of serialized data

Return type

dict

New in version 1.0.0.

Changed in version 3.0.0b7: This method returns the serialized data rather than a (data, errors) duple. A ValidationError is raised if obj is invalid.

Changed in version 3.0.0rc9: Validation no longer occurs upon serialization.

dumps(self, obj: typing.Any, *args, many: bool = None, **kwargs)

Same as dump(), except return a JSON-encoded string.

Parameters
  • obj – The object to serialize.

  • many – Whether to serialize obj as a collection. If None, the value for self.many is used.

Returns

A json string

Return type

str

New in version 1.0.0.

Changed in version 3.0.0b7: This method returns the serialized data rather than a (data, errors) duple. A ValidationError is raised if obj is invalid.

_deserialize(self, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]], *, error_store: ErrorStore, many: bool = False, partial=False, unknown=RAISE, index=None)

Deserialize data.

Parameters
  • data (dict) – The data to deserialize.

  • error_store (ErrorStore) – Structure to store errors.

  • many (bool) – True if data should be deserialized as a collection.

  • partial (bool|tuple) – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE.

  • index (int) – Index of the item being serialized (for storing errors) if serializing a collection, otherwise None.

Returns

A dictionary of the deserialized data.

load(self, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]], *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None, unknown: str = None)

Deserialize a data structure to an object defined by this Schema’s fields.

Parameters
  • data – The data to deserialize.

  • many – Whether to deserialize data as a collection. If None, the value for self.many is used.

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE. If None, the value for self.unknown is used.

Returns

Deserialized data

New in version 1.0.0.

Changed in version 3.0.0b7: This method returns the deserialized data rather than a (data, errors) duple. A ValidationError is raised if invalid data are passed.

loads(self, json_data: str, *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None, unknown: str = None, **kwargs)

Same as load(), except it takes a JSON string as input.

Parameters
  • json_data – A JSON string of the data to deserialize.

  • many – Whether to deserialize obj as a collection. If None, the value for self.many is used.

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE. If None, the value for self.unknown is used.

Returns

Deserialized data

New in version 1.0.0.

Changed in version 3.0.0b7: This method returns the deserialized data rather than a (data, errors) duple. A ValidationError is raised if invalid data are passed.

_run_validator(self, validator_func, output, *, original_data, error_store, many, partial, pass_original, index=None)
validate(self, data: typing.Mapping, *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None)

Validate data against the schema, returning a dictionary of validation errors.

Parameters
  • data – The data to validate.

  • many – Whether to validate data as a collection. If None, the value for self.many is used.

  • partial – Whether to ignore missing fields and not require any fields declared. Propagates down to Nested fields as well. If its value is an iterable, only missing fields listed in that iterable will be ignored. Use dot delimiters to specify nested fields.

Returns

A dictionary of validation errors.

New in version 1.1.0.

_do_load(self, data: typing.Union[typing.Mapping[str, typing.Any], typing.Iterable[typing.Mapping[str, typing.Any]]], *, many: bool = None, partial: typing.Union[bool, types.StrSequenceOrSet] = None, unknown: str = None, postprocess: bool = True)

Deserialize data, returning the deserialized result. This method is private API.

Parameters
  • data – The data to deserialize.

  • many – Whether to deserialize data as a collection. If None, the value for self.many is used.

  • partial – Whether to validate required fields. If its value is an iterable, only fields listed in that iterable will be ignored will be allowed missing. If True, all fields will be allowed missing. If None, the value for self.partial is used.

  • unknown – Whether to exclude, include, or raise an error for unknown fields in the data. Use EXCLUDE, INCLUDE or RAISE. If None, the value for self.unknown is used.

  • postprocess – Whether to run post_load methods..

Returns

Deserialized data

_normalize_nested_options(self)

Apply then flatten nested schema options. This method is private API.

__apply_nested_option(self, option_name, field_names, set_operation)

Apply nested options to nested fields

_init_fields(self)

Update self.fields, self.load_fields, and self.dump_fields based on schema options. This method is private API.

on_bind_field(self, field_name: str, field_obj: ma_fields.Field)

Hook to modify a field when it is bound to the Schema.

No-op by default.

_bind_field(self, field_name: str, field_obj: ma_fields.Field)

Bind field to the schema, setting any necessary attributes on the field (e.g. parent and name).

Also set field load_only and dump_only values if field_name was specified in class Meta.

_has_processors(self, tag)
_invoke_dump_processors(self, tag: str, data, *, many: bool, original_data=None)
_invoke_load_processors(self, tag: str, data, *, many: bool, original_data, partial: typing.Union[bool, types.StrSequenceOrSet])
_invoke_field_validators(self, *, error_store: ErrorStore, data, many: bool)
_invoke_schema_validators(self, *, error_store: ErrorStore, pass_many: bool, data, original_data, many: bool, partial: typing.Union[bool, types.StrSequenceOrSet], field_errors: bool = False)
_invoke_processors(self, tag: str, *, pass_many: bool, data, many: bool, original_data=None, **kwargs)
class great_expectations.marshmallow__shade.SchemaOpts(meta, ordered: bool = False)

class Meta options for the Schema. Defines defaults.

great_expectations.marshmallow__shade.EXCLUDE = exclude
great_expectations.marshmallow__shade.INCLUDE = include
great_expectations.marshmallow__shade.RAISE = raise
great_expectations.marshmallow__shade.missing
great_expectations.marshmallow__shade.pprint(obj, *args, **kwargs) → None

Pretty-printing function that can pretty-print OrderedDicts like regular dictionaries. Useful for printing the output of marshmallow.Schema.dump().