Skip to main content
Version: 0.18.21

Expectation creation workflow

The following image shows the four workflows you can follow to create ExpectationsA verifiable assertion about data.:

Where do Expectations come from?

The methodology for saving and testing Expectations is the same for all workflows. GX recommends using the interactive workflow or the Data Assistant workflow to create Expectations.

Create Expectations interactively​

In this workflow, you work in a Python interpreter or Jupyter Notebook. You use a ValidatorUsed to run an Expectation Suite against data. and call Expectations as methods on it to define them in an Expectation Suite, and when you have finished you save that Expectation Suite into your Expectation StoreA connector to store and retrieve information about collections of verifiable assertions about data.. For an overview of this workflow, see How to create and edit Expectations with instant feedback from a sample batch of data.

Create Expectations with Data Assistants​

In this workflow, you use a Data AssistantA utility that asks questions about your data, gathering information to describe what is observed, and then presents Metrics and proposes Expectations based on the answers. to generate Expectations based on the input data you provide. You can preview the Metrics that these Expectations are based on, and you can save the generated Expectations as an Expectation Suite in an Expectation Store.

As with creating Expectations interactively, you start with your Data Context. However, you work in a Python environment, so you need to load or create your Data Context as an instantiated object. Next, you create a Batch Request to specify the data you would like to profile with your Data Assistant. Once you have a Batch RequestProvided to a Data Source in order to create a Batch. configured you will use it as the input for the run method of your Data Assistant, which can be accessed from your Data Context object. Once the Data Assistant has run, you will be able to review the results and save the generated Expectations to an empty Expectation Suite.

Manually define Expectations​

Advanced users can use a manual process to create Expectations. This workflow does not require Data Assets, but it does require knowledge of the configurations available for Expectations. To create Expectations manually, see how to create and edit expectations based on domain knowledge without inspecting data directly.

Create Expectations with custom methods​

Advanced users can use custom methods to generate Expectations that are based on Data Source metadata. If you want to use custom methods to create Expectations, contact a member of the support team on Slack.

Test your Expectation Suite​

After you've created and saved your Expectation Suite, GX recommends that you test it by Validating it. You can Validate an Expectation with a Checkpoint. An overview of the Validation process is provided here.

Edit a saved Expectation Suite​

See How to Edit an Expectation Suite

View Expectation Suite Expectations​

See View the Expectations in the Expectation Suite.

Next steps​