great_expectations.checkpoint
¶
Subpackages¶
Submodules¶
Package Contents¶
Classes¶
|
EmailAction sends an email to a given list of email addresses. |
|
MicrosoftTeamsNotificationAction sends a Microsoft Teams notification to a given webhook. |
|
This is the base class for all actions that act on validation results |
|
OpsgenieAlertAction creates and sends an Opsgenie alert |
|
PagerdutyAlertAction sends a pagerduty event |
|
SlackNotificationAction sends a Slack notification to a given webhook. |
|
StoreEvaluationParametersAction extracts evaluation parameters from a validation result and stores them in the store |
|
StoreMetricsAction extracts metrics from a Validation Result and stores them |
|
StoreValidationResultAction stores a validation result in the ValidationsStore. |
|
UpdateDataDocsAction is a validation action that |
|
This is the base class for all actions that act on validation results |
|
|
|
|
|
|
|
SimpleCheckpointBuilder is a convenience class to easily configure a simple |
-
class
great_expectations.checkpoint.
EmailAction
(data_context, renderer, smtp_address, smtp_port, sender_login, sender_password, receiver_emails, sender_alias=None, use_tls=None, use_ssl=None, notify_on='all', notify_with=None)¶ Bases:
great_expectations.checkpoint.actions.ValidationAction
EmailAction sends an email to a given list of email addresses. Configuration .. code-block:: yaml
name: send_email_on_validation_result
- action:
class_name: EmailAction notify_on: all # possible values: “all”, “failure”, “success” notify_with: # optional list of DataDocs site names to display in the email message. Defaults to showing all renderer:
# the class that implements the message to be sent # this is the default implementation, but you can # implement a custom one module_name: great_expectations.render.renderer.email_renderer class_name: EmailRenderer
# put the actual following information in the uncommitted/config_variables.yml file # or pass in as environment variable smtp_address: ${smtp_address} smtp_port: ${smtp_port} sender_login: ${email_address} sender_password: ${sender_password} sender_alias: ${sender_alias} # useful to send an email as an alias (default = sender_login) receiver_emails: ${receiver_emails} # string containing email addresses separated by commas use_tls: False use_ssl: True
-
_run
(self, validation_result_suite, validation_result_suite_identifier, data_asset=None, payload=None)¶
-
class
great_expectations.checkpoint.
MicrosoftTeamsNotificationAction
(data_context, renderer, microsoft_teams_webhook, notify_on='all')¶ Bases:
great_expectations.checkpoint.actions.ValidationAction
MicrosoftTeamsNotificationAction sends a Microsoft Teams notification to a given webhook.
Configuration
- name: send_microsoft_teams_notification_on_validation_result action: class_name: MicrosoftTeamsNotificationAction # put the actual webhook URL in the uncommitted/config_variables.yml file # or pass in as environment variable microsoft_teams_webhook: ${validation_notification_microsoft_teams_webhook} notify_on: all # possible values: "all", "failure", "success" renderer: # the class that implements the message to be sent # this is the default implementation, but you can # implement a custom one module_name: great_expectations.render.renderer.microsoft_teams_renderer class_name: MicrosoftTeamsRenderer
-
_run
(self, validation_result_suite, validation_result_suite_identifier, data_asset=None, payload=None)¶
-
-
class
great_expectations.checkpoint.
NoOpAction
(data_context)¶ Bases:
great_expectations.checkpoint.actions.ValidationAction
This is the base class for all actions that act on validation results and are aware of a Data Context namespace structure.
The Data Context is passed to this class in its constructor.
-
_run
(self, validation_result_suite, validation_result_suite_identifier, data_asset)¶
-
-
class
great_expectations.checkpoint.
OpsgenieAlertAction
(data_context, renderer, api_key, region=None, priority='P3', notify_on='failure')¶ Bases:
great_expectations.checkpoint.actions.ValidationAction
OpsgenieAlertAction creates and sends an Opsgenie alert
Configuration
- name: send_opsgenie_alert_on_validation_result action: class_name: OpsgenieAlertAction # put the actual webhook URL in the uncommitted/config_variables.yml file # or pass in as environment variable api_key: ${opsgenie_api_key} # Opsgenie API key region: specifies the Opsgenie region. Populate 'EU' for Europe otherwise leave empty priority: specify the priority of the alert (P1 - P5) defaults to P3 notify_on: failure # possible values: "all", "failure", "success"
-
_run
(self, validation_result_suite, validation_result_suite_identifier, data_asset=None, payload=None)¶
-
-
class
great_expectations.checkpoint.
PagerdutyAlertAction
(data_context, api_key, routing_key, notify_on='failure')¶ Bases:
great_expectations.checkpoint.actions.ValidationAction
PagerdutyAlertAction sends a pagerduty event
Configuration
- name: send_pagerduty_alert_on_validation_result action: class_name: PagerdutyAlertAction api_key: ${pagerduty_api_key} # Events API v2 key routing_key: # The 32 character Integration Key for an integration on a service or on a global ruleset. notify_on: failure # possible values: "all", "failure", "success"
-
_run
(self, validation_result_suite, validation_result_suite_identifier, data_asset=None, payload=None)¶
-
-
class
great_expectations.checkpoint.
SlackNotificationAction
(data_context, renderer, slack_webhook, notify_on='all', notify_with=None)¶ Bases:
great_expectations.checkpoint.actions.ValidationAction
SlackNotificationAction sends a Slack notification to a given webhook.
Configuration
- name: send_slack_notification_on_validation_result action: class_name: SlackNotificationAction # put the actual webhook URL in the uncommitted/config_variables.yml file # or pass in as environment variable slack_webhook: ${validation_notification_slack_webhook} notify_on: all # possible values: "all", "failure", "success" notify_with: # optional list of DataDocs site names to display in Slack message. Defaults to showing all renderer: # the class that implements the message to be sent # this is the default implementation, but you can # implement a custom one module_name: great_expectations.render.renderer.slack_renderer class_name: SlackRenderer
-
_run
(self, validation_result_suite, validation_result_suite_identifier, data_asset=None, payload=None)¶
-
-
class
great_expectations.checkpoint.
StoreEvaluationParametersAction
(data_context, target_store_name=None)¶ Bases:
great_expectations.checkpoint.actions.ValidationAction
StoreEvaluationParametersAction extracts evaluation parameters from a validation result and stores them in the store configured for this action.
Evaluation parameters allow expectations to refer to statistics/metrics computed in the process of validating other prior expectations.
Configuration
- name: store_evaluation_params action: class_name: StoreEvaluationParametersAction # name of the store where the action will store the parameters # the name must refer to a store that is configured in the great_expectations.yml file target_store_name: evaluation_parameter_store
-
_run
(self, validation_result_suite, validation_result_suite_identifier, data_asset, payload=None)¶
-
-
class
great_expectations.checkpoint.
StoreMetricsAction
(data_context, requested_metrics, target_store_name='metrics_store')¶ Bases:
great_expectations.checkpoint.actions.ValidationAction
StoreMetricsAction extracts metrics from a Validation Result and stores them in a metrics store.
Configuration
- name: store_evaluation_params action: class_name: StoreMetricsAction # name of the store where the action will store the metrics # the name must refer to a store that is configured in the great_expectations.yml file target_store_name: my_metrics_store
-
_run
(self, validation_result_suite, validation_result_suite_identifier, data_asset, payload=None)¶
-
-
class
great_expectations.checkpoint.
StoreValidationResultAction
(data_context, target_store_name=None)¶ Bases:
great_expectations.checkpoint.actions.ValidationAction
StoreValidationResultAction stores a validation result in the ValidationsStore.
Configuration
- name: store_validation_result action: class_name: StoreValidationResultAction # name of the store where the actions will store validation results # the name must refer to a store that is configured in the great_expectations.yml file target_store_name: validations_store
-
_run
(self, validation_result_suite, validation_result_suite_identifier, data_asset, payload=None)¶
-
-
class
great_expectations.checkpoint.
UpdateDataDocsAction
(data_context, site_names=None, target_site_names=None)¶ Bases:
great_expectations.checkpoint.actions.ValidationAction
UpdateDataDocsAction is a validation action that notifies the site builders of all the data docs sites of the Data Context that a validation result should be added to the data docs.
Configuration
- name: update_data_docs action: class_name: UpdateDataDocsAction
You can also instruct
UpdateDataDocsAction
to build only certain sites by providing asite_names
key with a list of sites to update:name: update_data_docs
- action:
class_name: UpdateDataDocsAction site_names:
production_site
-
_run
(self, validation_result_suite, validation_result_suite_identifier, data_asset, payload=None)¶
-
class
great_expectations.checkpoint.
ValidationAction
(data_context)¶ This is the base class for all actions that act on validation results and are aware of a Data Context namespace structure.
The Data Context is passed to this class in its constructor.
-
run
(self, validation_result_suite, validation_result_suite_identifier, data_asset, **kwargs)¶ - Parameters
validation_result_suite –
validation_result_suite_identifier –
data_asset –
- Param
kwargs - any additional arguments the child might use
- Returns
-
_run
(self, validation_result_suite, validation_result_suite_identifier, data_asset)¶
-
-
class
great_expectations.checkpoint.
Checkpoint
(name: str, data_context, config_version: Optional[Union[int, float]] = None, template_name: Optional[str] = None, module_name: Optional[str] = None, class_name: Optional[str] = None, run_name_template: Optional[str] = None, expectation_suite_name: Optional[str] = None, batch_request: Optional[Union[BatchRequest, dict]] = None, action_list: Optional[List[dict]] = None, evaluation_parameters: Optional[dict] = None, runtime_configuration: Optional[dict] = None, validations: Optional[List[dict]] = None, profilers: Optional[List[dict]] = None, validation_operator_name: Optional[str] = None, batches: Optional[List[dict]] = None, ge_cloud_id: Optional[UUID] = None)¶ -
property
name
(self)¶
-
property
data_context
(self)¶
-
property
config
(self)¶
-
property
action_list
(self)¶
-
property
ge_cloud_id
(self)¶
-
get_substituted_config
(self, config: Optional[Union[CheckpointConfig, dict]] = None, runtime_kwargs: Optional[dict] = None)¶
-
_substitute_config_variables
(self, config: CheckpointConfig)¶
-
run
(self, template_name: Optional[str] = None, run_name_template: Optional[str] = None, expectation_suite_name: Optional[str] = None, batch_request: Optional[Union[BatchRequest, dict]] = None, action_list: Optional[List[dict]] = None, evaluation_parameters: Optional[dict] = None, runtime_configuration: Optional[dict] = None, validations: Optional[List[dict]] = None, profilers: Optional[List[dict]] = None, run_id: Optional[Union[str, RunIdentifier]] = None, run_name: Optional[str] = None, run_time: Optional[Union[str, datetime.datetime]] = None, result_format: Optional[str] = None, **kwargs)¶
-
self_check
(self, pretty_print=True)¶
-
property
-
class
great_expectations.checkpoint.
LegacyCheckpoint
(name: str, data_context, validation_operator_name: Optional[str] = None, batches: Optional[List[dict]] = None)¶ Bases:
great_expectations.checkpoint.checkpoint.Checkpoint
LegacyCheckpoint - Notebook - How-to GuideRun a configured Checkpoint from a notebook.Maturity: Experimental (to-be-deprecated in favor of Checkpoint)Details:API Stability: to-be-deprecated in favor of CheckpointImplementation Completeness: CompleteUnit Test Coverage: Partial (“golden path”-focused tests; error checking tests need to be improved)Integration Infrastructure/Test Coverage: N/ADocumentation Completeness: CompleteBug Risk: LowLegacyCheckpoint - Command Line - How-to GuideRun a configured checkpoint from a command line in a Terminal shell.Maturity: Experimental (to-be-deprecated in favor of Checkpoint)Details:API Stability: to-be-deprecated in favor of CheckpointImplementation Completeness: CompleteUnit Test Coverage: CompleteIntegration Infrastructure/Test Coverage: N/ADocumentation Completeness: CompleteBug Risk: LowLegacyCheckpoint - Cron - How-to GuideUse the Unix crontab command to edit the cron file and add a line that will run Checkpoint as a scheduled task.Maturity: Experimental (to-be-deprecated in favor of Checkpoint)Details:API Stability: to-be-deprecated in favor of CheckpointImplementation Completeness: CompleteUnit Test Coverage: CompleteIntegration Infrastructure/Test Coverage: N/ADocumentation Completeness: CompleteBug Risk: LowLegacyCheckpoint - Airflow DAG - How-to GuideRunning a configured Checkpoint in Apache Airflow enables the triggering of data validation using an Expectation Suite directly within an Airflow DAG.Maturity: Beta (to-be-deprecated in favor of Checkpoint)Details:API Stability: to-be-deprecated in favor of CheckpointImplementation Completeness: Partial (no operator, but probably don’t need one)Unit Test Coverage: N/AIntegration Infrastructure/Test Coverage: MinimalDocumentation Completeness: Complete (pending how-to)Bug Risk: LowLegacyCheckpoint - Kedro - How-to GuideTODO: LegacyCheckpoint - Kedro DescriptionMaturity: Experimental (to-be-deprecated in favor of Checkpoint)Details:API Stability: to-be-deprecated in favor of CheckpointImplementation Completeness: UnknownUnit Test Coverage: UnknownIntegration Infrastructure/Test Coverage: UnknownDocumentation Completeness: Minimal (none)Bug Risk: UnknownLegacyCheckpoint - Prefect - How-to GuideTODO: LegacyCheckpoint - Prefect DescriptionMaturity: Experimental (to-be-deprecated in favor of Checkpoint)Details:API Stability: to-be-deprecated in favor of CheckpointImplementation Completeness: UnknownUnit Test Coverage: UnknownIntegration Infrastructure/Test Coverage: UnknownDocumentation Completeness: Minimal (none)Bug Risk: UnknownLegacyCheckpoint - DBT - How-to GuideTODO: LegacyCheckpoint - DBT DescriptionMaturity: Beta (to-be-deprecated in favor of Checkpoint)Details:API Stability: to-be-deprecated in favor of CheckpointImplementation Completeness: MinimalUnit Test Coverage: Minimal (none)Integration Infrastructure/Test Coverage: Minimal (none)Documentation Completeness: Minimal (none)Bug Risk: Low-
property
validation_operator_name
(self)¶
-
property
batches
(self)¶
-
_run_default_validation_operator
(self, assets_to_validate: List, run_id: Optional[Union[str, RunIdentifier]] = None, evaluation_parameters: Optional[dict] = None, run_name: Optional[str] = None, run_time: Optional[Union[str, datetime.datetime]] = None, result_format: Optional[Union[str, dict]] = {'result_format': 'SUMMARY'})¶
-
run
(self, run_id=None, evaluation_parameters=None, run_name=None, run_time=None, result_format=None, **kwargs)¶
-
get_config
(self, format='dict')¶
-
_get_batches_to_validate
(self, batches)¶
-
property
-
class
great_expectations.checkpoint.
SimpleCheckpoint
(name: str, data_context, config_version: Optional[Union[int, float]] = None, template_name: Optional[str] = None, module_name: Optional[str] = None, class_name: Optional[str] = None, run_name_template: Optional[str] = None, expectation_suite_name: Optional[str] = None, batch_request: Optional[Union[BatchRequest, dict]] = None, action_list: Optional[List[dict]] = None, evaluation_parameters: Optional[dict] = None, runtime_configuration: Optional[dict] = None, validations: Optional[List[dict]] = None, profilers: Optional[List[dict]] = None, validation_operator_name: Optional[str] = None, batches: Optional[List[dict]] = None, ge_cloud_id: Optional[UUID] = None, site_names: Optional[Union[str, List[str]]] = 'all', slack_webhook: Optional[str] = None, notify_on: Optional[str] = 'all', notify_with: Optional[Union[str, List[str]]] = 'all', **kwargs)¶ Bases:
great_expectations.checkpoint.checkpoint.Checkpoint
-
_configurator_class
¶
-
run
(self, template_name: Optional[str] = None, run_name_template: Optional[str] = None, expectation_suite_name: Optional[str] = None, batch_request: Optional[Union[BatchRequest, dict]] = None, action_list: Optional[List[dict]] = None, evaluation_parameters: Optional[dict] = None, runtime_configuration: Optional[dict] = None, validations: Optional[List[dict]] = None, profilers: Optional[List[dict]] = None, run_id: Optional[Union[str, RunIdentifier]] = None, run_name: Optional[str] = None, run_time: Optional[Union[str, datetime.datetime]] = None, result_format: Optional[str] = None, site_names: Optional[Union[str, List[str]]] = 'all', slack_webhook: Optional[str] = None, notify_on: Optional[str] = 'all', notify_with: Optional[Union[str, List[str]]] = 'all', **kwargs)¶
-
-
class
great_expectations.checkpoint.
SimpleCheckpointConfigurator
(name: str, data_context, site_names: Optional[Union[str, List[str]]] = 'all', slack_webhook: Optional[str] = None, notify_on: Optional[str] = 'all', notify_with: Optional[Union[str, List[str]]] = 'all', **kwargs)¶ SimpleCheckpointBuilder is a convenience class to easily configure a simple Checkpoint.
-
build
(self)¶ Build a Checkpoint.
-
_build_checkpoint_config
(self)¶
-
static
_default_action_list
()¶
-
_add_update_data_docs_action
(self, action_list)¶
-
_add_slack_action
(self, action_list: List[Dict])¶ The underlying SlackNotificationAction and SlackRenderer default to including links to all sites if the key notify_with is not present. We are intentionally hiding this from users of SimpleCheckpoint by having a default of “all” that sets the configuration appropriately.
-
_validate_site_names
(self, data_context)¶
-
_validate_notify_on
(self)¶
-
_validate_notify_with
(self)¶
-
_validate_slack_webhook
(self)¶
-
_validate_slack_configuration
(self)¶ Guide the user toward correct configuration.
-