great_expectations.checkpoint

Package Contents

Classes

CloudNotificationAction(data_context: DataContext, checkpoint_ge_cloud_id: str)

CloudNotificationAction is an action which utilizes the Cloud store backend

EmailAction(data_context, renderer, smtp_address, smtp_port, sender_login, sender_password, receiver_emails, sender_alias=None, use_tls=None, use_ssl=None, notify_on=’all’, notify_with=None)

EmailAction sends an email to a given list of email addresses.

MicrosoftTeamsNotificationAction(data_context, renderer, microsoft_teams_webhook, notify_on=’all’)

MicrosoftTeamsNotificationAction sends a Microsoft Teams notification to a given webhook.

NoOpAction(data_context)

This is the base class for all actions that act on validation results

OpsgenieAlertAction(data_context, renderer, api_key, region=None, priority=’P3’, notify_on=’failure’, tags: Optional[list] = None)

OpsgenieAlertAction creates and sends an Opsgenie alert

PagerdutyAlertAction(data_context, api_key, routing_key, notify_on=’failure’)

PagerdutyAlertAction sends a pagerduty event

SlackNotificationAction(data_context, renderer, slack_webhook=None, slack_token=None, slack_channel=None, notify_on=’all’, notify_with=None, show_failed_expectations=False)

SlackNotificationAction sends a Slack notification to a given webhook.

StoreEvaluationParametersAction(data_context, target_store_name=None)

StoreEvaluationParametersAction extracts evaluation parameters from a validation result and stores them in the store

StoreMetricsAction(data_context, requested_metrics, target_store_name=’metrics_store’)

StoreMetricsAction extracts metrics from a Validation Result and stores them

StoreValidationResultAction(data_context, target_store_name=None)

StoreValidationResultAction stores a validation result in the ValidationsStore.

UpdateDataDocsAction(data_context, site_names=None, target_site_names=None)

UpdateDataDocsAction is a validation action that

ValidationAction(data_context)

This is the base class for all actions that act on validation results

Checkpoint(name: str, data_context: AbstractDataContext, config_version: Optional[Union[int, float]] = None, template_name: Optional[str] = None, run_name_template: Optional[str] = None, expectation_suite_name: Optional[str] = None, batch_request: Optional[Union[BatchRequestBase, dict]] = None, action_list: Optional[List[dict]] = None, evaluation_parameters: Optional[dict] = None, runtime_configuration: Optional[dict] = None, validations: Optional[List[dict]] = None, profilers: Optional[List[dict]] = None, validation_operator_name: Optional[str] = None, batches: Optional[List[dict]] = None, ge_cloud_id: Optional[UUID] = None, expectation_suite_ge_cloud_id: Optional[UUID] = None, default_validation_id: Optional[str] = None)

LegacyCheckpoint(name: str, data_context, validation_operator_name: Optional[str] = None, batches: Optional[List[dict]] = None)

SimpleCheckpoint(name: str, data_context, config_version: Optional[Union[int, float]] = None, template_name: Optional[str] = None, run_name_template: Optional[str] = None, expectation_suite_name: Optional[str] = None, batch_request: Optional[Union[BatchRequestBase, dict]] = None, action_list: Optional[List[dict]] = None, evaluation_parameters: Optional[dict] = None, runtime_configuration: Optional[dict] = None, validations: Optional[List[dict]] = None, profilers: Optional[List[dict]] = None, ge_cloud_id: Optional[UUID] = None, site_names: Union[str, List[str]] = ‘all’, slack_webhook: Optional[str] = None, notify_on: str = ‘all’, notify_with: Union[str, List[str]] = ‘all’, expectation_suite_ge_cloud_id: Optional[str] = None, **kwargs)

SimpleCheckpointConfigurator(name: str, data_context, site_names: Union[None, str, List[str]] = ‘all’, slack_webhook: Optional[str] = None, notify_on: str = ‘all’, notify_with: Union[str, List[str]] = ‘all’, **kwargs)

SimpleCheckpointBuilder is a convenience class to easily configure a simple

Functions

verify_dynamic_loading_support(module_name: str, package_name: Optional[str] = None)

param module_name

a possibly-relative name of a module

great_expectations.checkpoint.verify_dynamic_loading_support(module_name: str, package_name: Optional[str] = None) → None
Parameters
  • module_name – a possibly-relative name of a module

  • package_name – the name of a package, to which the given module belongs

class great_expectations.checkpoint.CloudNotificationAction(data_context: DataContext, checkpoint_ge_cloud_id: str)

Bases: great_expectations.checkpoint.actions.ValidationAction

CloudNotificationAction is an action which utilizes the Cloud store backend to deliver user-specified Notification Actions.

_run(self, validation_result_suite: ExpectationSuiteValidationResult, validation_result_suite_identifier: GXCloudIdentifier, data_asset=None, payload: Optional[Dict] = None, expectation_suite_identifier=None, checkpoint_identifier=None)
class great_expectations.checkpoint.EmailAction(data_context, renderer, smtp_address, smtp_port, sender_login, sender_password, receiver_emails, sender_alias=None, use_tls=None, use_ssl=None, notify_on='all', notify_with=None)

Bases: great_expectations.checkpoint.actions.ValidationAction

EmailAction sends an email to a given list of email addresses. Configuration .. code-block:: yaml

  • name: send_email_on_validation_result

action:

class_name: EmailAction notify_on: all # possible values: “all”, “failure”, “success” notify_with: # optional list of DataDocs site names to display in the email message. Defaults to showing all renderer:

# the class that implements the message to be sent # this is the default implementation, but you can # implement a custom one module_name: great_expectations.render.renderer.email_renderer class_name: EmailRenderer

# put the actual following information in the uncommitted/config_variables.yml file # or pass in as environment variable smtp_address: ${smtp_address} smtp_port: ${smtp_port} sender_login: ${email_address} sender_password: ${sender_password} sender_alias: ${sender_alias} # useful to send an email as an alias (default = sender_login) receiver_emails: ${receiver_emails} # string containing email addresses separated by commas use_tls: False use_ssl: True

_run(self, validation_result_suite: ExpectationSuiteValidationResult, validation_result_suite_identifier: Union[ValidationResultIdentifier, GXCloudIdentifier], data_asset=None, payload=None, expectation_suite_identifier=None, checkpoint_identifier=None)
class great_expectations.checkpoint.MicrosoftTeamsNotificationAction(data_context, renderer, microsoft_teams_webhook, notify_on='all')

Bases: great_expectations.checkpoint.actions.ValidationAction

MicrosoftTeamsNotificationAction sends a Microsoft Teams notification to a given webhook.

Configuration

- name: send_microsoft_teams_notification_on_validation_result
action:
  class_name: MicrosoftTeamsNotificationAction
  # put the actual webhook URL in the uncommitted/config_variables.yml file
  # or pass in as environment variable
  microsoft_teams_webhook: ${validation_notification_microsoft_teams_webhook}
  notify_on: all # possible values: "all", "failure", "success"
  renderer:
    # the class that implements the message to be sent
    # this is the default implementation, but you can
    # implement a custom one
    module_name: great_expectations.render.renderer.microsoft_teams_renderer
    class_name: MicrosoftTeamsRenderer
_run(self, validation_result_suite: ExpectationSuiteValidationResult, validation_result_suite_identifier: Union[ValidationResultIdentifier, GXCloudIdentifier], data_asset=None, payload=None, expectation_suite_identifier=None, checkpoint_identifier=None)
class great_expectations.checkpoint.NoOpAction(data_context)

Bases: great_expectations.checkpoint.actions.ValidationAction

This is the base class for all actions that act on validation results and are aware of a Data Context namespace structure.

The Data Context is passed to this class in its constructor.

_run(self, validation_result_suite: ExpectationSuiteValidationResult, validation_result_suite_identifier: Union[ValidationResultIdentifier, GXCloudIdentifier], data_asset, expectation_suite_identifier=None, checkpoint_identifier=None)
class great_expectations.checkpoint.OpsgenieAlertAction(data_context, renderer, api_key, region=None, priority='P3', notify_on='failure', tags: Optional[list] = None)

Bases: great_expectations.checkpoint.actions.ValidationAction

OpsgenieAlertAction creates and sends an Opsgenie alert

Configuration

- name: send_opsgenie_alert_on_validation_result
action:
  class_name: OpsgenieAlertAction
  # put the actual webhook URL in the uncommitted/config_variables.yml file
  # or pass in as environment variable
  api_key: ${opsgenie_api_key} # Opsgenie API key
  region: specifies the Opsgenie region. Populate 'EU' for Europe otherwise leave empty
  priority: specify the priority of the alert (P1 - P5) defaults to P3
  notify_on: failure # possible values: "all", "failure", "success"
_run(self, validation_result_suite: ExpectationSuiteValidationResult, validation_result_suite_identifier: Union[ValidationResultIdentifier, GXCloudIdentifier], data_asset=None, payload=None, expectation_suite_identifier=None, checkpoint_identifier=None)
class great_expectations.checkpoint.PagerdutyAlertAction(data_context, api_key, routing_key, notify_on='failure')

Bases: great_expectations.checkpoint.actions.ValidationAction

PagerdutyAlertAction sends a pagerduty event

Configuration

- name: send_pagerduty_alert_on_validation_result
action:
  class_name: PagerdutyAlertAction
  api_key: ${pagerduty_api_key} # Events API v2 key
  routing_key: # The 32 character Integration Key for an integration on a service or on a global ruleset.
  notify_on: failure # possible values: "all", "failure", "success"
_run(self, validation_result_suite: ExpectationSuiteValidationResult, validation_result_suite_identifier: Union[ValidationResultIdentifier, GXCloudIdentifier], data_asset=None, payload=None, expectation_suite_identifier=None, checkpoint_identifier=None)
class great_expectations.checkpoint.SlackNotificationAction(data_context, renderer, slack_webhook=None, slack_token=None, slack_channel=None, notify_on='all', notify_with=None, show_failed_expectations=False)

Bases: great_expectations.checkpoint.actions.ValidationAction

SlackNotificationAction sends a Slack notification to a given webhook.

Configuration

- name: send_slack_notification_on_validation_result
action:
  class_name: SlackNotificationAction
  # put the actual webhook URL in the uncommitted/config_variables.yml file
  # or pass in as environment variable
  # use slack_webhook when not using slack bot token
  slack_webhook: ${validation_notification_slack_webhook}
  # pass slack token and slack channel when not using slack_webhook
  slack_token: # token from slack app
  slack_channel: # slack channel that messages should go to
  notify_on: all # possible values: "all", "failure", "success"
  notify_with: # optional list of DataDocs site names to display in Slack message. Defaults to showing all
  renderer:
    # the class that implements the message to be sent
    # this is the default implementation, but you can
    # implement a custom one
    module_name: great_expectations.render.renderer.slack_renderer
    class_name: SlackRenderer
  show_failed_expectations: *Optional* (boolean) shows a list of failed expectation types. default is false.
_run(self, validation_result_suite: ExpectationSuiteValidationResult, validation_result_suite_identifier: Union[ValidationResultIdentifier, GXCloudIdentifier], data_asset=None, payload=None, expectation_suite_identifier=None, checkpoint_identifier=None)
class great_expectations.checkpoint.StoreEvaluationParametersAction(data_context, target_store_name=None)

Bases: great_expectations.checkpoint.actions.ValidationAction

StoreEvaluationParametersAction extracts evaluation parameters from a validation result and stores them in the store configured for this action.

Evaluation parameters allow expectations to refer to statistics/metrics computed in the process of validating other prior expectations.

Configuration

- name: store_evaluation_params
action:
  class_name: StoreEvaluationParametersAction
  # name of the store where the action will store the parameters
  # the name must refer to a store that is configured in the great_expectations.yml file
  target_store_name: evaluation_parameter_store
_run(self, validation_result_suite: ExpectationSuiteValidationResult, validation_result_suite_identifier: Union[ValidationResultIdentifier, GXCloudIdentifier], data_asset, payload=None, expectation_suite_identifier=None, checkpoint_identifier=None)
class great_expectations.checkpoint.StoreMetricsAction(data_context, requested_metrics, target_store_name='metrics_store')

Bases: great_expectations.checkpoint.actions.ValidationAction

StoreMetricsAction extracts metrics from a Validation Result and stores them in a metrics store.

Configuration

- name: store_evaluation_params
action:
  class_name: StoreMetricsAction
  # name of the store where the action will store the metrics
  # the name must refer to a store that is configured in the great_expectations.yml file
  target_store_name: my_metrics_store
_run(self, validation_result_suite: ExpectationSuiteValidationResult, validation_result_suite_identifier: Union[ValidationResultIdentifier, GXCloudIdentifier], data_asset, payload=None, expectation_suite_identifier=None, checkpoint_identifier=None)
class great_expectations.checkpoint.StoreValidationResultAction(data_context, target_store_name=None)

Bases: great_expectations.checkpoint.actions.ValidationAction

StoreValidationResultAction stores a validation result in the ValidationsStore.

Configuration

- name: store_validation_result
action:
  class_name: StoreValidationResultAction
  # name of the store where the actions will store validation results
  # the name must refer to a store that is configured in the great_expectations.yml file
  target_store_name: validations_store
_run(self, validation_result_suite: ExpectationSuiteValidationResult, validation_result_suite_identifier: Union[ValidationResultIdentifier, GXCloudIdentifier], data_asset, payload=None, expectation_suite_identifier=None, checkpoint_identifier: Optional[GXCloudIdentifier] = None)
class great_expectations.checkpoint.UpdateDataDocsAction(data_context, site_names=None, target_site_names=None)

Bases: great_expectations.checkpoint.actions.ValidationAction

UpdateDataDocsAction is a validation action that notifies the site builders of all the data docs sites of the Data Context that a validation result should be added to the data docs.

Configuration

- name: update_data_docs
action:
  class_name: UpdateDataDocsAction

You can also instruct UpdateDataDocsAction to build only certain sites by providing a site_names key with a list of sites to update:

  • name: update_data_docs

action:

class_name: UpdateDataDocsAction site_names:

  • production_site

_run(self, validation_result_suite: ExpectationSuiteValidationResult, validation_result_suite_identifier: Union[ValidationResultIdentifier, GXCloudIdentifier], data_asset, payload=None, expectation_suite_identifier=None, checkpoint_identifier=None)
class great_expectations.checkpoint.ValidationAction(data_context)

This is the base class for all actions that act on validation results and are aware of a Data Context namespace structure.

The Data Context is passed to this class in its constructor.

run(self, validation_result_suite: ExpectationSuiteValidationResult, validation_result_suite_identifier: Union[ValidationResultIdentifier, GXCloudIdentifier], data_asset, expectation_suite_identifier: Optional[ExpectationSuiteIdentifier] = None, checkpoint_identifier=None, **kwargs)
Parameters
  • validation_result_suite

  • validation_result_suite_identifier

  • data_asset

  • expectation_suite_identifier – The ExpectationSuiteIdentifier to use

  • checkpoint_identifier – The Checkpoint to use

Param

kwargs - any additional arguments the child might use

Returns

_run(self, validation_result_suite: ExpectationSuiteValidationResult, validation_result_suite_identifier: Union[ValidationResultIdentifier, GXCloudIdentifier], data_asset, expectation_suite_identifier=None, checkpoint_identifier=None)
class great_expectations.checkpoint.Checkpoint(name: str, data_context: AbstractDataContext, config_version: Optional[Union[int, float]] = None, template_name: Optional[str] = None, run_name_template: Optional[str] = None, expectation_suite_name: Optional[str] = None, batch_request: Optional[Union[BatchRequestBase, dict]] = None, action_list: Optional[List[dict]] = None, evaluation_parameters: Optional[dict] = None, runtime_configuration: Optional[dict] = None, validations: Optional[List[dict]] = None, profilers: Optional[List[dict]] = None, validation_operator_name: Optional[str] = None, batches: Optional[List[dict]] = None, ge_cloud_id: Optional[UUID] = None, expectation_suite_ge_cloud_id: Optional[UUID] = None, default_validation_id: Optional[str] = None)

Bases: great_expectations.checkpoint.checkpoint.BaseCheckpoint

run_with_runtime_args(self, template_name: Optional[str] = None, run_name_template: Optional[str] = None, expectation_suite_name: Optional[str] = None, batch_request: Optional[Union[BatchRequestBase, dict]] = None, action_list: Optional[List[dict]] = None, evaluation_parameters: Optional[dict] = None, runtime_configuration: Optional[dict] = None, validations: Optional[List[dict]] = None, profilers: Optional[List[dict]] = None, run_id: Optional[Union[str, int, float]] = None, run_name: Optional[str] = None, run_time: Optional[datetime.datetime] = None, result_format: Optional[str] = None, expectation_suite_ge_cloud_id: Optional[str] = None, **kwargs)
static construct_from_config_args(data_context: AbstractDataContext, checkpoint_store_name: str, name: str, config_version: Optional[Union[int, float]] = None, template_name: Optional[str] = None, module_name: Optional[str] = None, class_name: Optional[str] = None, run_name_template: Optional[str] = None, expectation_suite_name: Optional[str] = None, batch_request: Optional[dict] = None, action_list: Optional[List[dict]] = None, evaluation_parameters: Optional[dict] = None, runtime_configuration: Optional[dict] = None, validations: Optional[List[dict]] = None, profilers: Optional[List[dict]] = None, validation_operator_name: Optional[str] = None, batches: Optional[List[dict]] = None, site_names: Optional[Union[str, List[str]]] = None, slack_webhook: Optional[str] = None, notify_on: Optional[str] = None, notify_with: Optional[Union[str, List[str]]] = None, ge_cloud_id: Optional[str] = None, expectation_suite_ge_cloud_id: Optional[str] = None, default_validation_id: Optional[str] = None)
static instantiate_from_config_with_runtime_args(checkpoint_config: CheckpointConfig, data_context: AbstractDataContext, **runtime_kwargs)
class great_expectations.checkpoint.LegacyCheckpoint(name: str, data_context, validation_operator_name: Optional[str] = None, batches: Optional[List[dict]] = None)

Bases: great_expectations.checkpoint.checkpoint.Checkpoint

Feature Maturity

icon-90427d727afb11eda8370242ac110002 LegacyCheckpoint - Notebook - How-to Guide
Run a configured Checkpoint from a notebook.
Maturity: Experimental (to-be-deprecated in favor of Checkpoint)
Details:
API Stability: to-be-deprecated in favor of Checkpoint
Implementation Completeness: Complete
Unit Test Coverage: Partial (“golden path”-focused tests; error checking tests need to be improved)
Integration Infrastructure/Test Coverage: N/A
Documentation Completeness: Complete
Bug Risk: Low
icon-90427f2a7afb11eda8370242ac110002 LegacyCheckpoint - Command Line - How-to Guide
Run a configured checkpoint from a command line in a Terminal shell.
Maturity: Experimental (to-be-deprecated in favor of Checkpoint)
Details:
API Stability: to-be-deprecated in favor of Checkpoint
Implementation Completeness: Complete
Unit Test Coverage: Complete
Integration Infrastructure/Test Coverage: N/A
Documentation Completeness: Complete
Bug Risk: Low
icon-9042801a7afb11eda8370242ac110002 LegacyCheckpoint - Cron - How-to Guide
Use the Unix crontab command to edit the cron file and add a line that will run Checkpoint as a scheduled task.
Maturity: Experimental (to-be-deprecated in favor of Checkpoint)
Details:
API Stability: to-be-deprecated in favor of Checkpoint
Implementation Completeness: Complete
Unit Test Coverage: Complete
Integration Infrastructure/Test Coverage: N/A
Documentation Completeness: Complete
Bug Risk: Low
icon-904280ec7afb11eda8370242ac110002 LegacyCheckpoint - Airflow DAG - How-to Guide
Running a configured Checkpoint in Apache Airflow enables the triggering of data validation using an Expectation Suite directly within an Airflow DAG.
Maturity: Beta (to-be-deprecated in favor of Checkpoint)
Details:
API Stability: to-be-deprecated in favor of Checkpoint
Implementation Completeness: Partial (no operator, but probably don’t need one)
Unit Test Coverage: N/A
Integration Infrastructure/Test Coverage: Minimal
Documentation Completeness: Complete (pending how-to)
Bug Risk: Low
icon-904281aa7afb11eda8370242ac110002 LegacyCheckpoint - Kedro - How-to Guide
TODO: LegacyCheckpoint - Kedro Description
Maturity: Experimental (to-be-deprecated in favor of Checkpoint)
Details:
API Stability: to-be-deprecated in favor of Checkpoint
Implementation Completeness: Unknown
Unit Test Coverage: Unknown
Integration Infrastructure/Test Coverage: Unknown
Documentation Completeness: Minimal (none)
Bug Risk: Unknown
icon-904282687afb11eda8370242ac110002 LegacyCheckpoint - Prefect - How-to Guide
TODO: LegacyCheckpoint - Prefect Description
Maturity: Experimental (to-be-deprecated in favor of Checkpoint)
Details:
API Stability: to-be-deprecated in favor of Checkpoint
Implementation Completeness: Unknown
Unit Test Coverage: Unknown
Integration Infrastructure/Test Coverage: Unknown
Documentation Completeness: Minimal (none)
Bug Risk: Unknown
icon-904283267afb11eda8370242ac110002 LegacyCheckpoint - DBT - How-to Guide
TODO: LegacyCheckpoint - DBT Description
Maturity: Beta (to-be-deprecated in favor of Checkpoint)
Details:
API Stability: to-be-deprecated in favor of Checkpoint
Implementation Completeness: Minimal
Unit Test Coverage: Minimal (none)
Integration Infrastructure/Test Coverage: Minimal (none)
Documentation Completeness: Minimal (none)
Bug Risk: Low
property validation_operator_name(self)
property batches(self)
_run_default_validation_operator(self, assets_to_validate: List, run_id: Optional[Union[str, RunIdentifier]] = None, evaluation_parameters: Optional[dict] = None, run_name: Optional[str] = None, run_time: Optional[Union[str, datetime.datetime]] = None, result_format: Optional[Union[str, dict]] = None)
run(self, run_id=None, evaluation_parameters=None, run_name=None, run_time=None, result_format=None, **kwargs)
_get_batches_to_validate(self, batches)
class great_expectations.checkpoint.SimpleCheckpoint(name: str, data_context, config_version: Optional[Union[int, float]] = None, template_name: Optional[str] = None, run_name_template: Optional[str] = None, expectation_suite_name: Optional[str] = None, batch_request: Optional[Union[BatchRequestBase, dict]] = None, action_list: Optional[List[dict]] = None, evaluation_parameters: Optional[dict] = None, runtime_configuration: Optional[dict] = None, validations: Optional[List[dict]] = None, profilers: Optional[List[dict]] = None, ge_cloud_id: Optional[UUID] = None, site_names: Union[str, List[str]] = 'all', slack_webhook: Optional[str] = None, notify_on: str = 'all', notify_with: Union[str, List[str]] = 'all', expectation_suite_ge_cloud_id: Optional[str] = None, **kwargs)

Bases: great_expectations.checkpoint.checkpoint.Checkpoint

_configurator_class
run(self, template_name: Optional[str] = None, run_name_template: Optional[str] = None, expectation_suite_name: Optional[str] = None, batch_request: Optional[Union[BatchRequestBase, dict]] = None, action_list: Optional[List[dict]] = None, evaluation_parameters: Optional[dict] = None, runtime_configuration: Optional[dict] = None, validations: Optional[List[dict]] = None, profilers: Optional[List[dict]] = None, run_id: Optional[Union[str, RunIdentifier]] = None, run_name: Optional[str] = None, run_time: Optional[Union[str, datetime.datetime]] = None, result_format: Optional[str] = None, site_names: Union[str, List[str]] = 'all', slack_webhook: Optional[str] = None, notify_on: str = 'all', notify_with: Union[str, List[str]] = 'all', expectation_suite_ge_cloud_id: Optional[str] = None)
class great_expectations.checkpoint.SimpleCheckpointConfigurator(name: str, data_context, site_names: Union[None, str, List[str]] = 'all', slack_webhook: Optional[str] = None, notify_on: str = 'all', notify_with: Union[str, List[str]] = 'all', **kwargs)

SimpleCheckpointBuilder is a convenience class to easily configure a simple Checkpoint.

build(self)

Build a Checkpoint.

_build_checkpoint_config(self)
static _default_action_list()
_add_update_data_docs_action(self, action_list)
_add_slack_action(self, action_list: List[Dict])

The underlying SlackNotificationAction and SlackRenderer default to including links to all sites if the key notify_with is not present. We are intentionally hiding this from users of SimpleCheckpoint by having a default of “all” that sets the configuration appropriately.

_validate_site_names(self, data_context)
_validate_notify_on(self)
_validate_notify_with(self)
_validate_slack_webhook(self)
_validate_slack_configuration(self)

Guide the user toward correct configuration.