Organize Expectations into an Expectation Suite
An Expectation Suite contains a group of Expectations that describe the same set of data. Combining all the Expectations that you apply to a given set of data into an Expectation Suite allows you to evaluate them as a group, rather than individually. All of the Expectations that you use to validate your data in production workflows should be grouped into Expectation Suites.
Prerequisites
- Python version 3.9 to 3.12.
- An installation of GX Core.
- Recommended. A preconfigured Data Context.
- Recommended. A preconfigured Data Source and Data Asset connected to your data.
Procedure
- Instructions
- Sample code
-
Retrieve or create a Data Context.
In this procedure, your Data Context is stored in the variable
context
. For information on creating or connecting to a Data Context see Create a Data Context. -
Create an Expectation Suite.
To create a new Expectation Suite you will instantiate the
ExpectationSuite
class, which is available from thegreat_expectations
module.Each Expectation Suite will require a unique name. In the following code update the variable
suite_name
with a a name relevant to your data. Then execute the code to create your Expectation Suite:Python inputsuite_name = "my_expectation_suite"
suite = gx.ExpectationSuite(name=suite_name) -
Add the Expectation Suite to your Data Context
Once you have finalized the contents of your Expectation Suite you should save it to your Data Context:
Python inputsuite = context.suites.add(suite)
With a File or GX Cloud Data Context your saved Expectation Suite will be available between Python sessions. You can retrieve your Expectation Suite from your Data Context with the following code:
Python inputexisting_suite_name = (
"my_expectation_suite" # replace this with the name of your Expectation Suite
)
suite = context.suites.get(name=existing_suite_name) -
Create an Expectation.
In this procedure, your Expectation is stored in the variable
expectation
. For information on creating an Expectation see Create an Expectation. -
Add the Expectation to the Expectation Suite.
An Expectation Suite's
add_expectation(...)
method takes in an instance of an Expectation and adds it to the Expectation Suite's configuraton:Python inputsuite.add_expectation(expectation)
If you have a configured Data Source, Data Asset, and Batch Definition you can test your Expectation before adding it to your Expectation Suite. To do this see Test an Expectation.
However, if you test and modify an Expectation after you have added it to your Expectation Suite you must explicitly save those modifications before they will be pushed to the Expectation Suite's configuration:
Python inputexpectation.column = "pickup_location_id"
expectation.save()Because the
save()
method of a modified Expectation updates its Expectation Suite's configuration, thesave()
method will only function if the Expectation Suite has been added to your Data Context. -
Continue to create and add additional Expectations
Repeat the process of creating, testing, and adding Expectations to your Expectation Suite until the Expectation Suite adequately describes your data's ideal state.
import great_expectations as gx
context = gx.get_context()
# Create an Expectation Suite
suite_name = "my_expectation_suite"
suite = gx.ExpectationSuite(name=suite_name)
# Add the Expectation Suite to the Data Context
suite = context.suites.add(suite)
# Create an Expectation to put into an Expectation Suite
expectation = gx.expectations.ExpectColumnValuesToNotBeNull(column="passenger_count")
# Add the previously created Expectation to the Expectation Suite
suite.add_expectation(expectation)
# Add another Expectation to the Expectation Suite.
suite.add_expectation(
gx.expectations.ExpectColumnValuesToNotBeNull(column="pickup_datetime")
)
# Update the configuration of an Expectation, then push the changes to the Expectation Suite
expectation.column = "pickup_location_id"
expectation.save()
# Retrieve an Expectation Suite from the Data Context
existing_suite_name = (
"my_expectation_suite" # replace this with the name of your Expectation Suite
)
suite = context.suites.get(name=existing_suite_name)