Changelog
Deprecation policy
- Deprecation warnings and the supporting code are maintained for two minor versions. For example, v0.12 deprecations will only be removed as part of a v0.15 release.
- This means we have three supported minor versions in the release at any time. For example: in v0.15 we support v0.15, v0.14, and v0.13. When v0.16 ships we will support v0.16, v0.15, and v0.14 and will remove support for v0.13.
- Deprecation warnings include (in the warning message) the version that they were introduced. For example: "deprecated as of v0.13"
- Deprecation warnings are accompanied by a moniker (as a code comment) indicating when they were deprecated. For example:
# deprecated-v0.13
- Changes to methods and parameters due to deprecation are also noted in the relevant docstrings.
0.18.22
- [BUGFIX] Fix datadocs icons (v0.18) (#10561)
0.18.21
- [BUGFIX] Using
{batch}
keyword inUnexpectedRowsQuery
(#10392) (#10411) - [BUGFIX] 0.18.x Ignore unsupported INTERVAL type as part of CDM (#10414)
- [BUGFIX] 0.18.x Databricks SQL Pattern Expectation Fix (#10415)
- [MAINTENANCE] Pass
description
fromValidator
toExpectationConfiguration
(#10388)
0.18.20
- [FEATURE] Add
UnexpectedRowsExpectation
(#10342) - [BUGFIX] Remove illegible duplicate local Data Docs link from Slack renderer (#10129)
- [MAINTENANCE] Ruff 0.5.3 + PR annotations (#10128)
- [MAINTENANCE] Fix 0.18.x CI (#10199)
- [MAINTENANCE] Update column identifier tests (#8783)
0.18.19
- [FEATURE] Snowflake test for the presence of a schema in
test_connection()
(#10100) - [BUGFIX] Z-score renderer when
double_sided
(#10085) - [BUGFIX] SQLDatasource - lowercase unquoted
schema_names
for SQLAlchemy case-sensitivity compatibility (#10107) - [MAINTENANCE] Export
great_expectations.compatibility
types (#10089) - [MAINTENANCE] 0.18.x - mypy -
possibly-undefined
(#10091) - [MAINTENANCE] loosen ruamel pin (#10081)
0.18.18
- [FEATURE] Add atomic renderer for
ExpectMulticolumnSumToEqual
(#10076) (#10077) - [FEATURE] Snowflake - narrow Account Identifier regex (#10069)
- [FEATURE] Add missing atomic renderers to Expectations (#10079) (#10080)
0.18.17
- [FEATURE] Snowflake - Better Account Identifier related TestConnectionErrors (#10043)
- [BUGFIX] Fix
get_user_friendly_error_message
with non-standard formats (#10061)
0.18.16
- [FEATURE] Add new
ConfigUri
type (#10000) - [FEATURE] Snowflake - minimal support for key-pair auth (#10031)
- [FEATURE] Snowflake - Forward compatibility updates (#10036)
- [BUGFIX] fix snowflake schema introspection for
expect_column_values_to_be_in_type_list
andexpect_column_values_to_be_of_type
(#9906) - [BUGFIX] 0.18.x - Avoid writing to great_expectations.yml during init (#10040)
- [BUGFIX] Snowflake - Fix
private_key
Unicode serialization errors (#10046) - [DOCS] release_prep 0.18.15 (#9984)
- [MAINTENANCE] 0.18.x - mypy
1.7.1
->1.10
(#9988) - [MAINTENANCE] 0.18.x - CVE 2024 36039 (#10033)
- [MAINTENANCE] Revert Snowflake field requirement changes for
0.18.x
(#10012) - REVERTED - [FEATURE] SnowflakeDatasource - Require
database
andschema
fields (#9980) - REVERTED - [FEATURE] Snowflake - use
Datasource
levelschema
when creatingTableAsset
(#9992) - REVERTED - [FEATURE] Restrict substitutable sections for
Snowflake.connection_string
(#10002)
0.18.15
- [BUGFIX] fix checkpoint name in slack notif (#9964)
- [BUGFIX] 0.18.x - Prevent Error on Read Only Filesystem (#9977)
- [MAINTENANCE] 0.18.x - ruff update
0.1.11
->0.4.5
(#9975)
0.18.14
- [FEATURE] update slack notifications design (#9952)
- [BUGFIX] Set application query param for SnowflakeDatasource (#9863)
- [BUGFIX] Show FDS Asset name in DataDocs (#9950)(thanks @morphatic)
- [MAINTENANCE] Remove TIME from supported metric list (#9844)
- [MAINTENANCE] Schedule packaging pipeline to run on 0.18.x (#9921)
- [MAINTENANCE] performance, avoid HTTP round trips (#9929)
0.18.13
- [FEATURE] Remove [cloud] optional dependency (#9814)
- [BUGFIX]
scrapy
compatibility - handledir()
inconsistencies (#9830) - [MAINTENANCE] Export
great_expectations.data_context
types (#9738) - [MAINTENANCE] Retire ColumnDescriptiveMetrics - 0.18 (#9789)
0.18.12
- [FEATURE] MetricListMetricRetriever - 0.18.x (#9615)
- [MAINTENANCE] TableMetrics - BatchInspector updates (0.18.x) (#9630)
0.18.11
- [FEATURE] (0.18.x) Don't break context for invalid datasource configs (#9596)
- [MAINTENANCE] Update incorrect contract tests (#9543)
- [MAINTENANCE] Update referential integrity test so DB is only created once -
0.18
(#9545) - [MAINTENANCE] CDMs - Metrics as ENUM - 0.18 (#9572)
- [MAINTENANCE] Refactor
ColumnDescriptiveMetricsMetricRetriever
to parent class (0.18.x) (#9612)
0.18.10
- [BUGFIX] - Prevent duplicate Expectations in Validation Results (#9533)
0.18.9
- [BUGFIX] Re-enable GCS Docs Integration test (#9237)
- [BUGFIX] Fix add or update expectation suite (#9268)
- [BUGFIX] Misconfigured Expectations affecting unassociated Checkpoints (#9431)
- [BUGFIX] Should not fail loading utils if an integration is missing a type (#9428)
- [BUGFIX] Snowflake ci fix (#9458)
- [BUGFIX] Fix incorrect python version in Snowflake reqs (#9464)
- [BUGFIX] Fix clickhouse CI failure with pandas 2.2 (#9470)
- [MAINTENANCE] Add upper bound for numpy (#9257)
- [MAINTENANCE] Remove Click constraint in 0.18.x (#9255)
- [MAINTENANCE] Only do docs checks and build on develop (#9463)
- [MAINTENANCE] Ignore
Pandas
DeprecationWarning
for legacyPandasDataset
(#9471) - [MAINTENANCE] Ignore pandas
DeprecationWarning
for legacyCustomPandasDataset
(#9473) - [MAINTENANCE] Fix pandas usage due to
DeprecationWarning
(#9478) - [MAINTENANCE] Release without
docs-snippets
tests for0.18.x
(#9480)
0.18.8
- [FEATURE] Add min and max of timestamp cols to Column Descriptive Met… (#9207)
- [FEATURE]
TableAsset.test_connection()
should fail if table is not queryable. (#9198) - [BUGFIX] Disable GCS Docs Integration Test to unblock Release (#9226)
- [DOCS]
0.18.x
Updateget_context()
overload to forEphemeralDataContext
(#9183) - [MAINTENANCE] Update Contrib Pipeline to release from
0.18
branch (#9182) - [MAINTENANCE]
0.18.x
mypy
&ruff
updates (#9191) - [MAINTENANCE] Apply the contract testing fixes to 0.18 (#9225)
- [CONTRIB] ExpectColumnValuesToBePresentInOtherTable - Interface Update (#9165)
- [CONTRIB] ExpectColumnValuesToBePresentInOtherTable - Parameter Rename (#9180)
- [CONTRIB] Referential Integrity Expectation- Atomic Renderers (#9202)
- [CONTRIB] Referential Integrity - Make building of
unexpected_value
table non-case sensitive (#9224)
0.18.7
- [BUGFIX] 0.18.x - Apply
QueryAsset
splitting fix (#9160) - [CONTRIB] ExpectColumnValuesToBePresentInOtherTable - Referential Integrity (0.18 branch) (#9145)
- [CONTRIB] ExpectColumnValuesToBePresentInOtherTable - Wording update (#9151)
0.18.6
- [BUGFIX] 0.18.x: Microsoft Fabric Semantic Link API update (#9102)
- [BUGFIX]
0.18.x
cherrypickcreate_temp_tables
fixes fromdevelop
(#9124) - [BUGFIX] B/remove connection log v0.18 (#9136)
- [BUGFIX] Fix QueryAsset with
create_temp_table=False
(#9148) - [DOCS] Fix doc test to use
create_temp_table=True
with QueryAsset (#9137) - [DOCS]
how_to_connect_to_postgresql_data
usescreate_temp_table=True
(#9140) - [MAINTENANCE] Delete
test_run_multibatch_data_assistant_and_checkpoint
(#9128)
0.18.5
- [BUGFIX] validator head query limit (#9071)
- [BUGFIX] Databricks shared compute fix (#9034)
- [MAINTENANCE] Add
0.18.3
release notes to0.18.x
branch (#9074) - [MAINTENANCE] add_or_update_checkpoint with expectation_suite_ge_cloud_id (#9070)
0.18.4
- [BUGFIX] 0.18.4 - Fix Snowflake error when using alternative connection details format. (#9056)
- [BUGFIX] Fix clickhouse like operators (#9068 via #9061) (thanks @matveykortsev)
- [MAINTENANCE] Add
0.18.3
release notes to0.18.x
branch (#9074)
0.18.3
- [FEATURE] Update SnowflakeDatasource connection details (#8922)
- [DOCS] Minor Updates to Create a Data Asset (#8952)
- [DOCS] List the Available GX Cloud Expectations in Manage Expectations (#8959)
- [DOCS] DSB-484 Improve Quickstart diagram links (#8951)
- [DOCS] Update Add Validation Data or Expectation Suites to a Checkpoint (#8931)
- [MAINTENANCE] Add delete Data Source and Data Asset methods to end-to-end tests (#8956)
- [MAINTENANCE] Add Nevin and cleanup teams.yml (#8955)
- [MAINTENANCE] dry up validation status calc (#8962)
- [MAINTENANCE] Update type hints so our decorators play well with classes (#8935)
- [CONTRIB] add sqlalchemy execution engine compatability to expect_column_values… (#8942) (thanks @mcornew)
- [CONTRIB] add sqlalchemy support for expect_multicolumn_values_not_to_be_all_null (#8938) (thanks @itaise)
0.18.2
- [DOCS] Update Connect to SQL Database Data Assets (#8918)
- [DOCS] DSB-353: Add GX Cloud Architecture and Deployment Patterns page (#8904)
- [DOCS] Update descs for custom expectations (#8923)
- [DOCS] Update cloud docs for data assets (#8932)
- [DOCS] Update syntax highlighting in main README (#8811)
- [DOCS] Minor Updates to GX Cloud Documentation (#8939)
- [DOCS] add
FabricPowerBIDatasource
api docs (#8948) - [DOCS] Document Add Data Asset to Existing Data Source Functionality (#8949)
- [DOCS] Create OSS Troubleshooting Topic (#8898)
- [DOCS] Minor Updates to Create a Data Asset (#8952)
- [MAINTENANCE] Helpful Message for Expectations that require Metrics that are not available (#8906)
- [MAINTENANCE] Lint Docs tests (#8930)
- [MAINTENANCE]
PandasFilesystemDatasource
E2E tests (#8884) - [MAINTENANCE] Remove Cloud integration tests with duplicated logic (#8940)
- [MAINTENANCE] Fix error in boolean logic for local assume command (#8941)
- [MAINTENANCE] pre-commit autoupdate (#8934)
- [MAINTENANCE] Add local verify step for pact tests (#8876)
0.18.1
- [FEATURE] Expect column specified percentile value to be above a given value. (#8907) (thanks @kurt1984)
- [FEATURE] Warn users if more specific SQL Datasource is available. (#8914)
- [DOCS] Update Cloud Quickstart save expectation suite example (#8893)
- [DOCS] Update Section Title (#8905)
- [DOCS] add 'user' access token clarification to GX Cloud quickstart (#8912)
- [DOCS] Remove TOC Content Duplication (#8917)
- [DOCS] Create a New Get Support Topic (#8849)
- [MAINTENANCE] Get checkpoint endpoint contract test (#8881)
- [MAINTENANCE] Add Expectation Suite Pact tests (#8885)
- [MAINTENANCE] Do not skip spark tests in cloud stage (#8901)
- [MAINTENANCE] List checkpoints endpoint contract test (#8886)
- [MAINTENANCE] Adjust timeouts for cloud-tests services (#8910)
- [MAINTENANCE] Typing render/view (#8903)
- [MAINTENANCE] Better error msg for refrence environments (#8913)
- [MAINTENANCE] remove deprecated usage of pydantic Extra (#8896)
- [MAINTENANCE] E2E Test - Spark Filesystem with Cloud (#8889)
- [MAINTENANCE] Update ruff + black + mypy (#8902)
- [MAINTENANCE] Deal with
GxDatasourceWarning
warnings in docs-snippet tests (#8924) - [MAINTENANCE] Add JennyTee as core contributor (#8921)
- [CONTRIB] [FEATURE]-add-threshold-parameter (#8891) (thanks @tb102122)
0.18.0
- [BUGFIX] Only update self.assets if cloud deletion successful (#8894)
- [DOCS] DSB-321: Add workflow diagram to OSS Quickstart (#8804)
- [DOCS] Add @public_api to Method (#8870)
- [DOCS] Update Instances of Source Data to Data Source (#8857)
- [DOCS] Airflow tutorial (#8858) (thanks @r34ctor)
- [DOCS] Create About GX Cloud Content (#8834)
- [MAINTENANCE] Use contract test response bodies as unit test mocks (#8855)
- [MAINTENANCE] Flakey usage-stats test fixes (#8861)
- [MAINTENANCE] Run
docs-snippets
tests on every PR (#8866) - [MAINTENANCE] Increase docker compose up services timeout (#8864)
- [MAINTENANCE] Remove
docs-snippets
name
(#8867) - [MAINTENANCE] Column Descriptive Metrics: Remove compute_metrics_with_aborted_metrics (#8865)
- [MAINTENANCE] Bump @babel/traverse from 7.23.0 to 7.23.2 in /docs/docusaurus (#8838)
- [MAINTENANCE] Improve Cloud update Snowflake Data Source tests (#8868)
- [MAINTENANCE] Remove the deprecated save_changes parameters on ADC data source crud methods (#8859)
- [MAINTENANCE] Bump urllib3 from 1.26.17 to 1.26.18 in /docs_rtd (#8840)
- [MAINTENANCE] Cloud integration tests documentation (#8871)
- [MAINTENANCE] Remove deprecated code in advance of v18 release (#8863)
- [MAINTENANCE] Add contract interactions for GET and POST datasource (#8877)
- [MAINTENANCE] Add pre-commit autofix to PRs (#8873)
- [MAINTENANCE] Only run
docs-snippets
on doc changes (#8880) - [MAINTENANCE] Quieter logging (#8892)
- [MAINTENANCE] E2E Test - Pandas In Memory with Cloud (#8887)
- [MAINTENANCE] Add
SparkDatasource
End-to-End test (#8879)
0.17.23
- [FEATURE] Unexpected index list with/without values - New parameter in result_format (#8785) (thanks @MMoussabbih)
- [DOCS] Update Cloud Docs to Include Docker Methodology (#8820)
- [DOCS] Updated language & code refs for Databricks Getting Started (#8751)
- [DOCS] Remove all mentions of profilers in public docs (#8825)
- [DOCS] Create New Content for Column Descriptive Metrics in GX Cloud (#8814)
- [DOCS] Document GX Cloud Data Assistant Functionality (#8829)
- [DOCS] Update set_up_gx_cloud.md (#8832)
- [DOCS] Update Manage Expectation Suites in GX Cloud Docs (#8842)
- [DOCS] Remove Troubleshooting Entry from GX Cloud TOC (#8846)
- [MAINTENANCE] Column Descriptive Metrics: Don't add empty string for type (#8830)
- [MAINTENANCE] Add cloud api via docker compose for testing (#8803)
- [MAINTENANCE] Remove public_api decorators from RBP (#8812)
- [MAINTENANCE] mypy 1.6 (#8837)
- [MAINTENANCE] Delete legacy
BasicSuiteBuilderProfiler
(#8808) - [MAINTENANCE] Run ci-is-on-main-repo in merge queue. (#8839)
- [MAINTENANCE] Add pact contract testing with GX Cloud (#8831)
- [MAINTENANCE] Allow re-publishing Pact on same commit (#8851)
- [CONTRIB] Also run CI when a PR becomes Ready for Review (#8833)
0.17.22
- [BUGFIX] Column Descriptive Metrics: Skip finding semantic type info for UNKNOWN columns (#8815)
- [BUGFIX] Column Descriptive Metrics: Convert table name to lowercase for snowflake (#8817)
- [BUGFIX] Improve datasource crud (#8773)
- [BUGFIX] Fix Snowflake
'connection_string'
TestConnectionError
(#8816) - [BUGFIX] Fix missing
config_provider
(#8822) - [DOCS] Minor Syntax Update (#8806)
- [DOCS] Update docs to reference new
gx/
dir instead ofgreat_expectations/
(#8717) - [MAINTENANCE] Remove
ge_cloud_mode
fromStore
(#8747) - [MAINTENANCE] Bump urllib3 from 1.26.5 to 1.26.17 in /docs_rtd (#8802)
- [MAINTENANCE] Delete legacy
JsonSchemaProfiler
(#8807) - [MAINTENANCE] Column Descriptive Metrics: Default to UNKNOWN if type is not found (#8810)
- [MAINTENANCE] Column Descriptive Metrics: Empty string instead of UNKNOWN if table column type introspection fails (#8819)
- [MAINTENANCE] Delete outdated context fixtures (#8809)
- [MAINTENANCE] Bump gitpython from 3.1.35 to 3.1.37 in /docs_rtd (#8818)
- [MAINTENANCE] Fix
docs-snippets
CI step (#8827) - [CONTRIB] Updates to MulticolumnValuesSumValuesEqualToSingleColumn (#8798) (thanks @calvingdu)
- [CONTRIB] New MulticolumnValuesProductEqualToSingleColumn Expectation (#8799) (thanks @calvingdu)
0.17.21
- [FEATURE] DataDocs Renderer for
ExpectMulticolumnSumToEqual
andLikePattern
Expectations (#8739) - [FEATURE] Allow optional expectation_suite_name to be passed to RunDataAssistantEvent (#8693)
- [BUGFIX] Column Descriptive Metrics: Single connection (#8758)
- [DOCS] Updated link for Prefect integration (#8749)
- [DOCS] Fixed broken links in footer to match marketing page (#8748)
- [DOCS] Remove examples of YAML configuration for Checkpoints in docs (#8704)
- [DOCS] Remove YAML Configuration Option from Manage Checkpoints and Other Documentation (#8637)
- [DOCS] Create Snowflake Quickstart for GX Cloud (#8756)
- [DOCS] Create GX Cloud Agent Installation Topic (#8760)
- [DOCS] Update Section Link in GX Cloud TOC (#8772)
- [DOCS] Update Cloud Snowflake install instructions (#8776)
- [DOCS] Add GX Cloud Content (#8775)
- [DOCS] Update Link on Landing Page (#8794)
- [DOCS] Minor Update to Code Syntax in Quickstart (#8800)
- [DOCS] Consistent pip install instructions (#8801)
- [DOCS] Add Mermaid plugin to Docusaurus (#8793)
- [MAINTENANCE] Column Descriptive Metrics: Add metric exceptions (#8754)
- [MAINTENANCE] Update CODEOWNERS js rule to include front end community of practice (#8763)
- [MAINTENANCE] Fix CI - DictDot typing issue (#8765)
- [MAINTENANCE] Stop using
extras
to install test dependencies (#8767) - [MAINTENANCE] Skip full marker test on
merge_group
(#8766) - [MAINTENANCE] Update Fabric MeasureAsset.groupby_columns signature (#8764)
- [MAINTENANCE] Add
great_expectations_cloud
as a dependency (#8762) - [MAINTENANCE] Bump Cloud dependency to
0.0.3.dev3
(#8774) - [MAINTENANCE] Fix merge group worker usage (#8778)
- [MAINTENANCE] SQL column identifier testing - ensure expectations fail when expected (#8757)
- [MAINTENANCE] Use checkoutV3 (#8777)
- [MAINTENANCE] Expectations typings (#8784)
- [MAINTENANCE] Removing gallery parameter parse_strings_as_datetimes (#8786)
- [MAINTENANCE] Prevent block-style datasource creation with Cloud-backed contexts (#8791)
- [MAINTENANCE] Raise error if constructing FDS with individual args through base context API (#8792)
- [MAINTENANCE] Remove SimpleCheckpoint from public API docs (#8770)
- [MAINTENANCE] disallow-untyped-decorators (#8788)
- [CONTRIB] Set merge_queue in CI pipeline and turn off auto-update (#8730)
- [CONTRIB] Update notify_on_failure to not run on merge_group events (#8768)
- [CONTRIB] Adding simple tests for CloudDataStore add_expectation_suite (#8743)
- [CONTRIB] Adding tests for S3 Validation and Expectation Store prefix (#8761)
0.17.20 - YANKED
0.17.19
- [FEATURE] Microsoft FabricPowerBIDatasource (#8605)
- [BUGFIX] Ensure renderers use parent class renderers instead of using non-implemented override methods (#8745)
- [BUGFIX] Column Descriptive Metrics: Support numpy metrics (#8746)
- [BUGFIX] Column Descriptive Metrics: Add
id
to excluded list (#8750) - [DOCS] Update README.md to include newsletter and a typo fix. (#8721)
- [MAINTENANCE] Improve test coverage around
GXCloudStoreBackend.gx_cloud_response_json_to_object_dict
(#8744) - [MAINTENANCE] Improve
DatasourceDict
performance (#8711)
0.17.18
- [BUGFIX] Patch faulty
GXCloudStoreBackend.has_key
logic (#8741) - [MAINTENANCE] Additional examples of
row_condition
for Spark and SQL (#8713) - [MAINTENANCE] ruff
.0.290
(#8726)
0.17.17
- [BUGFIX] Resolve issue around listing keys with empty state in
GXCloudStoreBackend
(#8728) - [DOCS] Clarify How to Use Custom Expectations Without a Filesystem (#8700)
- [DOCS] Update Result Format Topic (#8674)
0.17.16
- [FEATURE] update data assistant run to take in validator (#8560)
- [FEATURE] Add
DataAssetStore
(#8601) - [BUGFIX] Missing DataFrame for in-memory Data Assets using GX Cloud (#8682)
- [BUGFIX] Fix FDS auto-complete for Conda installs (#8698)
- [BUGFIX] Assets with persisted data don't get added to in-memory cache (#8699)
- [BUGFIX] Use one inspector for the same engine/execution_engine (#8534)
- [BUGFIX] add try except to redshift regexp resolver (#8707) (thanks @itaise)
- [BUGFIX] Patch
docs_integration
tests failures resulting from context root dir rename (#8720) - [BUGFIX] Snowflake column name case sensitivity (#8719)
- [DOCS] Relocate Use Great Expectations with Google Cloud Platform and BigQuery (#8696)
- [MAINTENANCE] only build-docs on doc-changes (#8685)
- [MAINTENANCE] Azure Blob Stores
TupleStoreBackend
now supportscredential
string (#8683) - [MAINTENANCE] Column Descriptive Metrics: Add column types metric (#8687)
- [MAINTENANCE] Column Descriptive Metrics: Add Null Count (#8692)
- [MAINTENANCE] Bump gitpython from 3.1.33 to 3.1.35 in /docs_rtd (#8686)
- [MAINTENANCE] Comprehensive SQL column identifier tests (#8673)
- [MAINTENANCE] Add test case for trino datatypes in onboarding assitant (#8507)
- [MAINTENANCE] Add
get_all
method to Store and StoreBackend APIs (#8702) - [MAINTENANCE] Change default context root to
gx/
overgreat_expectations/
(#8423) - [MAINTENANCE] Parameterize severity in PagerdutyAction (#8714) (thanks @Sonchaegeon)
- [MAINTENANCE] Column Descriptive Metrics: Make the retriever more resilient to metric failures (#8712)
0.17.15
- [BUGFIX] Databricks tests skipped for Forks (#8662)
- [BUGFIX] Fix backslashes in url path under Windows (#8663) (thanks @MarcelBeining)
- [BUGFIX] Fix validation for
QueryAsset
withcreate_temp_table: False
(#8611) (thanks @moatazelmohtaseb) - [DOCS] Remove .py suffix from URL for module pages (#8649)
- [DOCS] Add redirects for module API docs (#8665)
- [DOCS] Fix Orphaned Pages in Prerequisites Blocks (#8666)
- [MAINTENANCE] Column Descriptive Metrics: Implement cloud data store (#8656)
- [MAINTENANCE] Update team members (#8654)
- [MAINTENANCE] Update GitPython (#8661)
- [MAINTENANCE] Column Descriptive Metrics: Return id from Data Store (#8667)
- [MAINTENANCE] Column Descriptive Metrics: ColumnDescriptiveMetricsAction tests (#8668)
- [MAINTENANCE] Column Descriptive Metrics: Add tests and cleanup (#8669)
- [MAINTENANCE] Column Descriptive Metrics: Fix cloud data store - use pydantic (#8677)
- [MAINTENANCE] Run doc-checks on every PR. (#8678)
- [MAINTENANCE] Column Descriptive Metrics - Additional Table and Column Metrics (#8679)
- [MAINTENANCE] Column Descriptive Metrics: Additional metrics mean median (#8680)
- [MAINTENANCE] Pydantic 2.0 support (#8604)
- [CONTRIB] fixing null eq safe for SQLAlchemy and rendering (#8617) (thanks @karthigaiselvanm)
0.17.14
- [BUGFIX] Use correct syntax for creating temporary views for DatabricksSQL datasources (#8655)
- [BUGFIX] Missing CloudDataContext when loading from config (#8659)
- [BUGFIX] Raise warning if data_connector cannot be built on config load (#8658)
- [DOCS] Add New Topic for Identifying Failed Table Rows (#8554)
- [DOCS] Update Request Data from a Data Asset (#8642)
- [MAINTENANCE] typing -
expect_column
-to_exist
andto_not_be_null
(#8652) - [MAINTENANCE] Column Descriptive Metrics: Synchronize domain models (#8648)
- [MAINTENANCE] Column Descriptive Metrics: Add test for MetricRepository (#8650)
0.17.13
- [FEATURE] Scripts for Identify Failed Rows (ID/PK) Doc (#8606)
- [FEATURE] Ensure FDS asset gets updated with new cloud id on add (#8610)
- [BUGFIX] Fix bug in Query template values metric when it gets int values (#8614)
- [BUGFIX] Fix bad checkpoint runtime config merge (#8626)
- [BUGFIX] Ensure that
oss_id
updates conf file rather than overwriting (#8634) - [BUGFIX] Ensure
cloud
tests don't run for public Forks (#8627) - [BUGFIX] Query Compatibility With Trino (#8600) (thanks @HadasManor)
- [BUGFIX] Always run docs-snippets action on release tag. (#8651)
- [DOCS] Remove Unused Topics and Update YAML Examples (#8581)
- [DOCS] Add Limit Validation Results in Data Docs Topic (#8571)
- [DOCS] Fix Quickstart typo (#8625)
- [DOCS] Quick Fix (#8628)
- [DOCS] Remove Instances of SimpleCheckpoint in snippets (#8622)
- [DOCS] Update Connect to SQL database source data (#8635)
- [DOCS] Remove Instances of SimpleCheckpoint (#8565)
- [DOCS] add
@public_api
toPandasDatasource
methods and assets (#8620) - [MAINTENANCE] Update Docusaurus to
2.4.1
and ensure Algolia search works (#8549) - [MAINTENANCE] add py311 min versions tests (#8621)
- [MAINTENANCE] Move
get_context
out of utils (#8623) - [MAINTENANCE] Remove reference to CLI in generated GX YAML (#8624)
- [MAINTENANCE] enable
RUF
rules (#8561) - [MAINTENANCE] Fix Azure CI packaging step (#8629)
- [MAINTENANCE] Ignore
pandas
warning emitted throughaltair
codepath (#8641) - [MAINTENANCE] Delete
ExpectationExplorer
(#8643) - [MAINTENANCE] Kill datasource cache in non-Cloud envs (#8608)
- [MAINTENANCE] databricks_sql integration tests (#8563)
- [MAINTENANCE] Fix or ignore warnings appearing in python 3.11 (#8645)
- [MAINTENANCE] Remove deprecated sqlalchemy-databricks connector (#8644)
- [MAINTENANCE] Add ruff rule to disallow use of
os.environ
outside of config files (#8588) - [MAINTENANCE] metrics typing (#8633)
- [MAINTENANCE] Delete misc batch kwarg generators (#8640)
0.17.12
- [FEATURE] Use the explicit column delimiters if passed in by user. (#8459)
- [FEATURE] ID/PK - Update Jupyter Notebooks for FDS (#8580)
- [BUGFIX] Fix incorrect type of SqlAlchemyExecutionEngine constructor's arg (#8487) (thanks @YoitoFes)
- [BUGFIX] Fix faulty logic around writing to .gitignore in context root dir (#8597)
- [BUGFIX] Update Docs test after DataContext Deprecation (#8599)
- [BUGFIX] Fix DraftDatasourceConfigAction connection string substitution (#8613)
- [DOCS] Update Data Assistant and Edit an Expectation Suite Topics (#8564)
- [MAINTENANCE] Delete deprecated
get_batch
(#8574) - [MAINTENANCE] Delete deprecated
add_batch_kwargs_generator
(#8573) - [MAINTENANCE] Add typing stubs to
requirements-types.txt
(#8501) - [MAINTENANCE] mypy 1.5 (#8586)
- [MAINTENANCE] Enable explicit-override mypy rule (#8589)
- [MAINTENANCE] Consistently use
datasources
property on context (#8596) - [MAINTENANCE] Deprecate
BaseDataContext
andDataContext
(#8584) - [MAINTENANCE] Update metrics data model to use generic value type (#8575)
- [MAINTENANCE] Update the CTA in the Data Docs footer (#8603)
- [MAINTENANCE] Clean-up: Renaming "self-initializing" to "auto-initializing" (#8609)
- [MAINTENANCE] Adding support for project_root_dir to get_context (#8388)
- [MAINTENANCE] Simplify CI YAML conditional for 0.17.12 release (#8612)
- [MAINTENANCE] Add mode param to get_context (#8593)
0.17.11
- [FEATURE] Make SQLDatasource.assets field a
tagged-union
(#8532) - [BUGFIX] DatabricksSQL TableAsset quoted identifiers (#8523)
- [BUGFIX] Fixed persist to be considered for spark fluent datasource (#8512) (thanks @jmcorreia)
- [BUGFIX] Revert Docusaurus to
2.0.0-beta21
(#8546) - [BUGFIX] add_splitter_on_column_value (#8498)
- [BUGFIX] Fix persist is not being reflected for Spark Legacy Datasource (#8544) (thanks @jmcorreia)
- [BUGFIX] Snowflake column identifiers (#8526)
- [BUGFIX] Use a randomized schema name when running snowflake tests to support concurrency (#8559)
- [BUGFIX] Skip Snowflake FDS tests for User Forks (#8567)
- [BUGFIX] Fix Agent error types (#8558)
- [BUGFIX] Query Template not always String (#8570)
- [BUGFIX] Fix link update for versioned docs when there are multiple links on the same line (#8568)
- [DOCS] Update URL to blog post for anonymous stats (#7014)
- [DOCS] Updates Quickstart guide (#8321)
- [DOCS] Relocate Compare Two Tables with the Onboarding Data Assistant (#8539)
- [DOCS] Update to exact_match parameter: expect_table_columns_to_match_set (#8521) (thanks @calvingdu)
- [DOCS] Update Spark Code Examples in Connect to In-memory Source Data (#8520)
- [DOCS] Update MetricProviders Conceptual Topic (#8538)
- [MAINTENANCE] Retire
docs-integration
tests that are already running in GitHub Actions (#8525) - [MAINTENANCE] Remove Cross-Table comparison from Docs-Integration, which is already being run (#8531)
- [MAINTENANCE] Github Actions - Better handling of tests that cannot be run in Forks (#8530)
- [MAINTENANCE] Fix marker key (#8541)
- [MAINTENANCE] Fix conditional in CI (#8542)
- [MAINTENANCE] Ignore distutils warning (#8543)
- [MAINTENANCE] Bump gitpython from 3.1.30 to 3.1.32 in /docs_rtd (#8540)
- [MAINTENANCE] Fix docs-snippets ci job (#8545)
- [MAINTENANCE] Update Docusaurus to
2.4.1
(#8508) - [MAINTENANCE] ruff
0.0.282
(#8505) - [MAINTENANCE] Add concurrency criteria to Github Actions CI/CD (#8550)
- [MAINTENANCE] Update
docs snippets
stage inci.yml
for scheduled and triggered runs (#8547) - [MAINTENANCE] Bump tornado from 6.3.2 to 6.3.3 in /docs_rtd (#8551)
- [MAINTENANCE] ruff 0.0.284 (#8552)
- [MAINTENANCE] Allow duplicate keys in checkpoint configuration with the same values (#8555)
- [MAINTENANCE] airflow220-min-versions (#8557)
- [CONTRIB] Updating Special Characters Expectation (#8382) (thanks @calvingdu)
- [CONTRIB] expect_multicolumn_values_to_be_equal (#8553) (thanks @karthigaiselvanm)
- [CONTRIB] fix query template values metric (#8562) (thanks @itaise)
0.17.9
- [BUGFIX] PR title checker's code should handle apostrophes (#8513)
- [BUGFIX] Patch
add_or_update_expectation_suite
with Cloud-backed contexts (#8522) - [DOCS] Update Code Example for Viewing a Full Checkpoint Configuration (#8492)
- [DOCS] Add New Topic for Connecting to In-Memory Source Data Using Spark (#8445)
- [DOCS] Remove Unused Documentation Style Guide Topic (#8496)
- [DOCS] Update Links and Content in the Data Context Topic (#8489)
- [DOCS] Update Connect to filesystem source data (#8483)
- [DOCS] Add step to install azure-storage-blob when using azure storage blobs… (#8156)
- [DOCS] add python version badge to README (#7040)
- [DOCS] standardizes references to supported Python versions (#8474)
- [MAINTENANCE] Remove core team attributions from changelog (#8493)
- [MAINTENANCE] Remove
azure-pipeline.yml
after migrating to Github Actions (#8494) - [MAINTENANCE] Notify on build-n-publish failure. (#8495)
- [MAINTENANCE] Send slack notification on release (#8497)
- [MAINTENANCE] Removed unused github action. (#8499)
- [MAINTENANCE] Change default testing level from WARNING to INFO (#8506)
- [MAINTENANCE] Flaky decorator added for docs-integration tests that access Cloud resources (#8510)
- [MAINTENANCE] Mob typing: update list of deprecated, never to be typed files. (#8514)
- [MAINTENANCE] Better testing for FDS quoted identifier logic (#8509)
- [MAINTENANCE] Remove Click upper bound constraint. (#8515)
- [MAINTENANCE] Docs Pipeline Clean up (#8468)
- [MAINTENANCE] Update
flaky
test (#8527)
0.17.8
- [FEATURE] Add Agent action to list table names in SQL Datasources (#8177)
- [FEATURE] Add Test Draft Config Workflow to Agent (#8410)
- [FEATURE] add run checkpoint action to agent (#8449)
- [FEATURE] DataAssistantResult should include exceptions from rules (#8429)
- [BUGFIX] Cleanup aws_postgres reference environment (#8439)
- [BUGFIX] Use consistent name between compose and cli command (#8440)
- [BUGFIX] Scaffolding .gitignore should not require writeable file system (#8362) (thanks @ivanstillfront)
- [BUGFIX] Remove unused, overwritten pytest marker. (#8441)
- [BUGFIX] get_validator throws AttributeError: 'CloudDataContext' object has no attribute 'ge_cloud_mode' (#8433)
- [BUGFIX] PP-282: fixing update_datasource method bug (#8464)
- [BUGFIX] Add
performance
marker topyproject.toml
(#8480) - [BUGFIX] Fix Postgres, Trino quoted identifier issues (#8442)
- [DOCS] Correct Typo (#8451)
- [DOCS] Admonition and Landing Page Updates (#8432)
- [DOCS] Remove Expectation Implementations by Backend Topic (#8437)
- [DOCS] Source Data and Datasource Format Update (#8435)
- [DOCS] Update the API Reference Landing Page (#8472)
- [DOCS] Fix Grid Autosizing (#8475)
- [DOCS] Update Use Great Expectations with Amazon Web Services using Athena (#8406)
- [DOCS] Fixed typos in Execution Engine glossary entry (#8086)
- [DOCS] add
public_api
decorator toTableAsset
andQueryAsset
(#8470) - [MAINTENANCE] Default snippet-check to non verbose mode (#8448)
- [MAINTENANCE] Move
test_dependency_versions
to github actions (#8427) - [MAINTENANCE] Add pytest-xdist and use by default. (#8436)
- [MAINTENANCE] Add sleep to allow services to come up before we run tests (#8454)
- [MAINTENANCE] add test services for mssql, mysql, and trino (#8447)
- [MAINTENANCE] Removed sqlalchemy_version_compatibility from required test markers. (#8456)
- [MAINTENANCE] Linting for previously excluded cli, expectations and rule_based_profiler tests (#8422)
- [MAINTENANCE] Run cloud e2e tests. (#8443)
- [MAINTENANCE] Start testing clickhouse in github actions. (#8452)
- [MAINTENANCE] Github Actions - Spark tests (#8460)
- [MAINTENANCE] Remove external_sqldialect marker as test coverage marker. (#8458)
- [MAINTENANCE] Pytest Github Action - BigQuery and Postgres (#8417)
- [MAINTENANCE] Run
marker-tests
afterunit-tests
andstatic-analysis
(#8465) - [MAINTENANCE] Run all_backend tests along with specific service tests. (#8467)
- [MAINTENANCE] AWS Glue script using Context Manager to catch
FutureWarning
(#8466) - [MAINTENANCE] Remove old required tests which have been replaced. (#8462)
- [MAINTENANCE] Run all python versions on scheduled run (#8463)
- [MAINTENANCE]
min-versions
wait forunit-tests
,static-analysis
(#8471) - [MAINTENANCE] Remove Azure Dev CI steps (#8473)
- [MAINTENANCE] Enable
B009
Bugbearget-attr-with-constant
rule (#8434) - [MAINTENANCE] Update ListTableNamesAction to use GxAgentEnvVars (#8450)
- [MAINTENANCE] Add check that each test only has one required marker. (#8477)
- [MAINTENANCE] Add test case for run_name_template using env_var (#8461)
- [MAINTENANCE] Delete
azure-pipelines-dev.yml
(#8476) - [MAINTENANCE] Test out releaser github action (#8478)
- [MAINTENANCE] Add missing marker to pyproject.toml (#8481)
- [MAINTENANCE] Slack notifications if non-pr ci github action pipeline fails (#8485)
- [MAINTENANCE] Run additional matrix steps on any non-
pull_request
event (#8486) - [MAINTENANCE] GX-Release Process update to use GitHub Actions (#8484)
0.17.7
- [FEATURE] Add Agent support for Missingness Data Assistant (#8336)
- [FEATURE] Allow a rule to fail within DataAssistant and still run other rules (#8393)
- [BUGFIX] fixing minor bug in sqlalchemy_execution_engine.py (#8374) (thanks @SaeedFarahani)
- [BUGFIX]
ensure_json_serializable
accounts forpydantic.BaseModel
(#8431) - [BUGFIX]
Table.head()
follow-up for Python versions (#8426) - [DOCS] Update docs on expectation docstrings (#8405)
- [DOCS] corrects default value for result_format in documentation (#8419)
- [DOCS] Remove Custom Expectations Overview Topic (#8391)
- [DOCS] Update contributing code readme with required markers information. (#8414)
- [DOCS] Fix markdown link in install_gx.md (#8416) (thanks @jmorakuebler)
- [DOCS] Docs TOC Reorg (#8421)
- [DOCS] Fix snippet reference in how_to_use_great_expectations_with_sql.md (#8430) (thanks @jmorakuebler)
- [MAINTENANCE] Experimental column descriptive metrics repository (#8335)
- [MAINTENANCE] Mark remaining datasource tests. (#8399)
- [MAINTENANCE] Pytest Mark - Metrics Folder (#8400)
- [MAINTENANCE] Add marks to missing expectations tests. (#8401)
- [MAINTENANCE] Add test for marker coverage and remove
integration
marker. (#8394) - [MAINTENANCE] Add ci test for test marker coverage (#8402)
- [MAINTENANCE] mypy 1.4.1 (#8226)
- [MAINTENANCE] snowflake regex (#8403)
- [MAINTENANCE] Resolve misc typing issues in metrics code (#8411)
- [MAINTENANCE] Bump certifi from 2022.12.7 to 2023.7.22 in /docs_rtd (#8415)
- [MAINTENANCE] conditional
snowflake-connector-python
version bump (#8412) - [MAINTENANCE] Refactor
Table.head()
forsqlalchemy
(#8234) - [MAINTENANCE] List required markers when verify marker test fails. (#8413)
- [MAINTENANCE] Bump version of
autoupdate
GH action (#8425) - [MAINTENANCE] Bump Ubuntu version in
autoupdate
GH action (#8428) - [MAINTENANCE] Update error message around
context.add_checkpoint
when neither name nor checkpoint are provided (#8313) - [MAINTENANCE] move Azure docs and public API steps to Github Actions (#8408)
- [MAINTENANCE] Use constants for GX directory and YAML references (#8420)
- [MAINTENANCE] Clean up Databricks SQL FDS impl and docs (#8424)
- [CONTRIB] Pyspark Implementation for expect_column_values_to_be_valid_currency_code (#8418) (thanks @calvingdu)
- [CONTRIB] Adding docs for missingness data assistant (#8379)
0.17.6
- [FEATURE] Register MissingnessDataAssistant (#8337)
- [BUGFIX] Ensure that Fluent Datasources support database table names in lowercase for Oracle, DB2, and Snowflake (#8327)
- [BUGFIX] Ensure that GX supports database column names in lowercase and mixcase for Oracle, DB2, and Snowflake (#8345)
- [BUGFIX] Handle ValueError in agent action (#8369)
- [BUGFIX] Ensure that SQLAlchemy is installed for SQL Data Source and TableAsset Fluent Data Source Module (#8361)
- [BUGFIX] DataAssistantResult should not error on get_expectation_suite without name (#8370)
- [BUGFIX] Exclude appropriate SnowflakeDatasource fields when creating an execution engine (#8371)
- [DOCS] Improvements on how_to_edit_expectationsuite guide (#8096) (thanks @Ismar11)
- [DOCS] Update Slack Link to Point to Discourse (#7840)
- [DOCS] Adds a consolidated overview of using GX as a conceptual guide (#8045)
- [DOCS] Updated the Execution Engine terms page (#8309)
- [DOCS] Add docs around Snowflake FDS (#8340)
- [DOCS] Update link in Data Asset Glossary Topic (#8349)
- [DOCS] updated docs to remove SimpleCheckpoint (#8352)
- [DOCS] Remove Outdated SQLAlchemy 2.0 Admonitions (#8357)
- [DOCS] Update Links to Get started with Great Expectations and Databricks (#8350)
- [DOCS] Add MetricProviders Conceptual Content (#8175)
- [DOCS] Add Information for Adding Custom Parameters to Custom Expectations (#8172)
- [DOCS] Remove index.md Files and Add Redirects (#8332)
- [MAINTENANCE] Remove
trace_docs_deps
script in CI (#8305) - [MAINTENANCE] GH Actions pytest marker test matrix (#8277)
- [MAINTENANCE] Pytest Mark - Render folder (#8301)
- [MAINTENANCE] Fix docs pipeline check changes (#8315)
- [MAINTENANCE] Add test markers to tests/datasource/fluent/XXX/X.py (#8316)
- [MAINTENANCE] Pytest Mark - Profiling (#8318)
- [MAINTENANCE]
invoke deps
task (#8311) - [MAINTENANCE] Fix requirements and CONTRIBUTING_CODE doc (#8312)
- [MAINTENANCE] gitignore for GCP credentials (#8323)
- [MAINTENANCE] Pytest Mark - DataContext Part 2 (#8317)
- [MAINTENANCE] Pytest Mark - ExecutionEngine (#8324)
- [MAINTENANCE] Pytest Mark - Integration and Expectations (#8319)
- [MAINTENANCE] Minimum version of
numpy
bumped to1.20.3
(#8326) - [MAINTENANCE] Disable Airflow provider tests in CI due to external test failures (#8342)
- [MAINTENANCE] Bump minimum version of
numpy==1.21.6
for Python 3.9 (#8341) - [MAINTENANCE] Update algolia index (#8273)
- [MAINTENANCE] Pytest Mark - DataConnector (#8338)
- [MAINTENANCE] Replace many repetitive slow integration tests for ParameterBuilder components with a few fast unit tests (#8302)
- [MAINTENANCE] Finish marking fluent datasource tests. (#8347)
- [MAINTENANCE] Bump word-wrap from 1.2.3 to 1.2.4 in /docs/docusaurus (#8346)
- [MAINTENANCE] Add test markers to datasource/batch_kwarg_generator tests. (#8348)
- [MAINTENANCE] Re-mark slow tests. (#8356)
- [MAINTENANCE] Update standard node package (#8339)
- [MAINTENANCE] Making tests for column names insensitivity for Oracle, DB2, and Snowflake easier to read (#8358)
- [MAINTENANCE] Add detect-private-key pre-commit hook (#8363)
- [MAINTENANCE] Pytest Mark - Top-level
test/
(#8359) - [MAINTENANCE] Unpin upper bound on Click dependency (#8360)
- [MAINTENANCE] Revert Click pin removal (#8367)
- [MAINTENANCE] Bump pygments from 2.7.4 to 2.15.0 in /docs_rtd (#8368)
- [MAINTENANCE] Fix non-AssertErrors for Expectations during build-gallery process (#8353)
- [MAINTENANCE] Mark datasource top level tests (#8365)
- [MAINTENANCE] add
invoke ci-tests
andinvoke service
(#8322) - [CONTRIB] Create custom expectation for Chi Square Test (#8314)
- [CONTRIB] Contributed
expect_column_to_have_no_months_missing
(#8307) (thanks @HadasManor) - [CONTRIB] Add KS test custom expectation (#8344)
0.17.5
- [FEATURE] Airflow reference environment (#8257)
- [FEATURE] Missingness DataAssistant initial implementation (#8268)
- [FEATURE] DatabricksSQL FDS (#8184)
- [FEATURE] single batch missingness data assistant (#8278)
- [DOCS] Replaces outdated integration guides with redirects to versioned documentation (#8173)
- [MAINTENANCE] Remove unreferenced docs integration tests (#8228)
- [MAINTENANCE] Remove duplicate custom checks in PR pipeline (#8265)
- [MAINTENANCE] Use updated search api key (#8269)
- [MAINTENANCE] DataAssistantDispatcher should not reach into DataAssistant (#8262)
- [MAINTENANCE] GH action static analysis and unittest setup (#8272)
- [MAINTENANCE] remove algolia index action (#8264)
- [MAINTENANCE] Bump semver from 5.7.1 to 5.7.2 (#8274)
- [MAINTENANCE] Improvement to contributor documentation (#8043) (thanks @christian-bromann)
- [MAINTENANCE] Implement unit tests for BatchFilter.parse_batch_slice logic (#8280)
- [MAINTENANCE] Run GH Action CI job every 3 hours (#8281)
- [MAINTENANCE] Ensure all tests in tests/checkpoint are marked (#8282)
- [MAINTENANCE] Pytest Mark -
Validator
and utils (#8275) - [MAINTENANCE] Add marker cli to all cli tests. (#8286)
- [MAINTENANCE] Reduce maximum allowed unittest duration (#8283)
- [MAINTENANCE] Add tests for Batch slicing for SparkFilePathDatasource (#8285)
- [MAINTENANCE] Pytest Mark -
RuleBasedProfiler
and others (#8288) - [MAINTENANCE] Pin jsonschema (#8290)
- [MAINTENANCE] Remove two slow DataAssistant integration tests (#8287)
- [MAINTENANCE] Add data docs container to airflow env (#8284)
- [MAINTENANCE] xfail Cloud E2E test (#8292)
- [MAINTENANCE] Remove --verbose pytest default (#8298)
- [MAINTENANCE] Update marker on slow test from unit to filesystem. (#8296)
- [MAINTENANCE] Typing Improvements - render/util (#8279)
- [MAINTENANCE] Add pytest marks to tests/actions (#8299)
- [MAINTENANCE] Mark tests in tests/core/usage_statistics (#8295)
- [MAINTENANCE] Mark tests in /tests/data_asset (#8300)
- [MAINTENANCE] Add markers to tests/core (#8297)
- [MAINTENANCE] Apply pytest markers to root dir
tests/data_context
tests (#8293) - [MAINTENANCE] Run Github
ci
action onpush
(#8303) - [MAINTENANCE] Dont use check_for_docs_deps_changes (#8304)
- [MAINTENANCE] Increase timeout for flaky tests (#8306)
0.17.4
- [MAINTENANCE] Protect
develop
withno-commit-to-branch
pre-commit hook (#8254) - [MAINTENANCE] Change Pydantic models to utilize
by_alias=True
(#8252) - [MAINTENANCE] Support individual connection args for Snowflake FDS (#8183)
- [MAINTENANCE] Replace dynamic datasource deletion with single
delete
method (#8189) - [MAINTENANCE] Disable usage statistics when in Cloud-backed environments (#8248)
0.17.3
- [FEATURE] Examples of using different store backends in reference environments (#8211)
- [FEATURE] Add checker to ensure snippets are being used (#8178)
- [FEATURE] Host data docs for the postgres reference environment (#8221)
- [FEATURE] Add checker to ensure test files have fixture definition (#8186)
- [FEATURE] make backend_dependencies required and not optional for doc integration tests (#8216)
- [FEATURE] View data docs inside snowflake and bigquery reference environments (#8231)
- [FEATURE] Reference Environment - AWS RDS (#8222)
- [BUGFIX] Fix yarn install and condition on CI pipeline (#8217)
- [BUGFIX] Ensure DataAssistantResult.plot_expectations_and_metrics does not raise exceptions when no Metrics or Expectations are available to plot (#8238)
- [DOCS] update result_format configuration documentation (#8209)
- [DOCS] Update Install GX (#8206)
- [DOCS] minor updates to the readme files (#8245)
- [MAINTENANCE] Arranging call arguments to
build_batch_request()
utility method to be in consistent order (#8224) - [MAINTENANCE] Use python static file server (#8229)
- [MAINTENANCE] Fix race condition in integration tests using context manager (#8223)
- [MAINTENANCE] Remove a line ignoring warnings about iteritems (#8227)
- [MAINTENANCE] Correcting typographical error in test method naming. (#8233)
- [MAINTENANCE] Reference environment consistency - installed version (#8237)
- [MAINTENANCE] Reference environment consistency - unpin python version (#8239)
- [MAINTENANCE] Use data docs container for cloud blob stores (#8240)
- [MAINTENANCE] Reference Environments: Match the new quickstart (#8242)
- [MAINTENANCE] Reference Environment - AWS Credentials Clean up (#8230)
- [MAINTENANCE] Filter altair/jsonschema Deprecation warning (#8244)
- [MAINTENANCE] Filter
jsonschema.RefResolver
,ErrorTree
warnings in tests (#8246) - [MAINTENANCE] Temporarily upper bound Click due to
mypy
typing issues (#8247) - [MAINTENANCE] Update
test_deprecation.py
in advance of 0.17.3 release (#8251) - [MAINTENANCE] enable typechecking in validator.py (#8204)
- [MAINTENANCE] Refactor validations in Checkpoint to use
CheckpointValidationConfig
(#8225)
0.17.2
- [FEATURE] AWS S3 reference environment (#8166)
- [FEATURE] Snowflake FDS (#8157)
- [FEATURE] Fix Bigquery/GCS integration tests (#8149)
- [FEATURE] Python 3.11 support (#8174)
- [FEATURE] Fix azure docs tests (#8171)
- [FEATURE] Reference Environment - BigQuery (#8176)
- [FEATURE] Reference Environment - Google Cloud Storage (#8197)
- [FEATURE] Enable updating active stores and CRUD for data docs sites (#8194)
- [FEATURE] Use PythonScript task type for custom docs ci checks (#8208)
- [FEATURE] Reference Environment: Azure Blob Storage (#8212)
- [FEATURE] ExpectDaySumToBeCloseToEquivalentWeekDayMean: User can give
weeks_back
as argument (#8139) (thanks @HadasManor) - [BUGFIX] Allow for
create_temp_table
to beFalse
when creating aSqlAlchemyBatchData
object (#8160) - [BUGFIX] Raise more informative error on failure of getting an expectation suite by name (#8170)
- [BUGFIX] Enable Pandas Column Aggregate Metrics To Support Decimal Numeric Types (#8195)
- [BUGFIX] Relax numpy version requirements for python 3.10 (#8199)
- [BUGFIX] Patch additional
pytest.deprecated_call
around adding datasource with Cloud (#8219) - [BUGFIX] Fix yarn install and condition on CI pipeline (#8217)
- [DOCS] Create New Landing Pages (#7993)
- [DOCS] Add table listing supported evaluation parameter expressions to the docs (#8124)
- [DOCS] Reorganize Checkpoint Section (#8182)
- [DOCS] Getting Started with GX (#8143)
- [DOCS] Sidebar updates for Getting Started (#8196)
- [DOCS] Get Started Section Revisions (#8202)
- [DOCS] Remove Connect to a source data system from TOC (#8193)
- [MAINTENANCE] Raise error on unsupported versions of python. (#8158)
- [MAINTENANCE] Reference Environment - Update Snowflake Notebook to Specify Python 3.10 (#8167)
- [MAINTENANCE] Refactor reference environment CLI for consistency (#8169)
- [MAINTENANCE] DX-565 remove allow_cross_type_comparison from column_pair_values.a_greater_than_b (#8025)
- [MAINTENANCE] Python 3.11 SQLAlchemy import time fix (#8180)
- [MAINTENANCE] Rich comments and docstrings for DataAssistant related modules. (#8185)
- [MAINTENANCE] Fix Python 3.11 Docker excessive pip backtracking (#8187)
- [MAINTENANCE] Start cleaning up outdated deprecation warnings (#8135)
- [MAINTENANCE] Replace Non-Existent Expectation Name With Correct Existing Expectation Name (#8191)
- [MAINTENANCE] Deal with Python 3.11 deprecations (#8192)
- [MAINTENANCE] Placing S3, Trino Imports Under Compatibility Pattern (#8198)
- [MAINTENANCE] Deprecate adding legacy datasources in Cloud-backed environments (#8190)
- [MAINTENANCE] Add
pytest.deprecated_call
to misc usages ofadd_datasource
in tests (#8207) - [MAINTENANCE] Placing AWS RedShift Imports Under Compatibility Pattern (#8205)
- [MAINTENANCE] Placing Snowflake Imports Under Compatibility Pattern (#8210)
- [MAINTENANCE] Placing AWS Athena Imports Under Compatibility Pattern (#8213)
- [MAINTENANCE] Update notebook link to non-legacy docs (#8215)
- [MAINTENANCE] Placing BigQuery Imports Under Compatibility Pattern (#8214)
- [MAINTENANCE] Fix Python 3.11 async expectations tests (#8203)
0.17.1
- [FEATURE] Fix spark docs tests (#8131)
- [FEATURE] Reference Environment - Snowflake (#8148)
- [FEATURE] Fluent Data Source ABS, GCS, and S3 recursive file discovery (#8118) (thanks @toivomattila)
- [BUGFIX] Fix config substitution for substrings (#8145)
- [BUGFIX] Fix 'great_expectations[cloud]' extra install (#8151)
- [BUGFIX] Persist SQLDatasource splitters on creation (#8164)
- [DOCS] Enable docs versioning for 0.16.16 (#8125)
- [DOCS] Update set_based_column_map_expectation.py (#8142)
- [DOCS] Fix broken links and reenable link checking (#8146)
- [DOCS] Add version info to markdown links for earlier versions (#8155)
- [MAINTENANCE] update location of link in data docs footer (#8130)
- [MAINTENANCE] invoke show-automerges (#8127)
- [MAINTENANCE] Lint
assets
dir (#8123) - [MAINTENANCE] test/integration linting (#8132)
- [MAINTENANCE] minimal Anonymizer type-checking (#8106)
- [MAINTENANCE] Use Available Enums Instead of Strings for Metric Name Extensions (#8137)
- [MAINTENANCE] Add env var to avoid out of memory error when building docs (#8144)
- [MAINTENANCE] Update single cloud onboarding script to use fluent datasources (#8114)
- [MAINTENANCE] Add --bash command for postgres reference env (#8154)
- [MAINTENANCE] Add default
table_name
toTableAsset
if omitted (#8152) - [CONTRIB] expect_queried_column_pair_values_to_be_both_filled_or_null (#7949) (thanks @eden-o)
0.17.0
- [FEATURE] Agent prints stack trace on error (#8092)
- [FEATURE] Clickhouse Integration (#7719) (thanks @Plozano94)
- [FEATURE] Pandas 2.0.0 and Sqlalchemy 2.0.0 compatibility (#7633)
- [FEATURE] Better pandas query (#8101)
- [FEATURE] Add Pandas FWFAsset - fixed width file (#8119)
- [BUGFIX] Fix conditional for pyspark compatibility (#8108)
- [BUGFIX] respect result format bool only for validators and checkpoints (#8111)
- [BUGFIX] Robust Handling Of Column Types And Empty DataFrames For DataBricks/Spark Environment (#8115)
- [BUGFIX] Fix GXCloudStoreBackend updates by name (#8116)
- [BUGFIX] Patch bad mock in GCS test (#8128)
- [DOCS] Update Quickstart guide to be Cloud-compatible (#8036)
- [DOCS] Temporarily hardcode quickstart snippet due to substitution error (#8091)
- [DOCS] Update prefect gx tutorial (#8009) (thanks @discdiver)
- [DOCS] Fix line breaks in quickstart (#8098)
- [DOCS] Remove Remaining CLI Admonitions (#8070)
- [DOCS] Remove examples of specifying a test_backends list of dicts (#7816)
- [MAINTENANCE] Add tests for file and Cloud-backed quickstart workflows (#8037)
- [MAINTENANCE] Update anonymous usage statistics payloads with hashed MAC address (#8078)
- [MAINTENANCE] Check if PR is a fork in some docs_integration stages (#8090)
- [MAINTENANCE] Update how_to_host_and_share_data_docs_on_gcs (#8067)
- [MAINTENANCE] Add persistent OSS user identifier to anonymized usage statistics payloads (#8089)
- [MAINTENANCE] Update MSSQL docker container and instructions for Apple Silicon Macs (#8093)
- [MAINTENANCE] Remove hardcoded
Data Source._EXCLUDED_EXEC_ENG_ARGS
(#8100) - [MAINTENANCE] Update print_diagnostic_checklist (#8018)
- [MAINTENANCE] Type hint cleanup in usage statistics (#8105)
- [MAINTENANCE] remove unused noqa comments (#8107)
- [MAINTENANCE] Remove the mysql-mac-m1 and starburst directories from assets/docker (#8104)
- [MAINTENANCE] Typing Data Assistant Result (#8110)
- [MAINTENANCE] Ensure that new usage statistics schema changes are backwards compatible (#8109)
- [MAINTENANCE] Cleanup generate_expectation_tests (#8019)
- [MAINTENANCE] Update build_in_memory_runtime_context to accept which datasources to include (#8017)
- [MAINTENANCE] Pandas and SqlAlchemy 2.0 follow-up (#8112)
- [MAINTENANCE] Misc/docs integration uncommented (#8076)
- [MAINTENANCE] Fixes for test_dependency_versions pipeline (#8122)
0.16.16
- [FEATURE] Update mySQL docker compose to specify platform (#8046)
- [FEATURE] Update GX Cloud on job status (#8047)
- [FEATURE] DX-441 put how_to_connect_to_in_memory_data_using_pandas un… (#8057)
- [BUGFIX] Set SQLALCHEMY_WARN_20 in Dockerfile (#7931)
- [BUGFIX] Ensure CloudDataContext Add Checkpoint flow returns Checkpoint with cloud-updated values (#8062)
- [BUGFIX] Erroneous Code Duplication and Lack of Type Hints in Expectation Parent Class (#8066)
- [BUGFIX] Fix broken Cloud tests around
Checkpoint
(#8083) - [BUGFIX] fix issue-7954 (#7963) (thanks @jkingdon-ms)
- [BUGFIX] Fix Update Checkpoint for Cloud (#8084)
- [BUGFIX] Mock correct method in test (#8087)
- [DOCS] Cloud - Getting started uses Fluent Datasources (#8035)
- [DOCS] Update Links in How to use auto-initializing Expectations (#8054)
- [DOCS] Update How to create Expectations Interactively in Python (#8052)
- [DOCS] Add guidance for "DataFrameAsset.build_batch_request()" in "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/connecting_to_your_data/fluent/in_memory/how_to_connect_to_in_memory_data_using_pandas/". (#8069)
- [DOCS] Update set_based_column_map_expectation_template.py (#8068)
- [DOCS] Remove mention of CLI in evaluation_parameters terminology page. (#8075)
- [MAINTENANCE] Ruff rule DTZ - prevent naive datetimes (#8050)
- [MAINTENANCE] Pandas 1.5 schema updates (#8039)
- [MAINTENANCE] Fix CI
tests_schemas
unit test (#8055) - [MAINTENANCE] Fix CI round 2 (#8056)
- [MAINTENANCE] Minor stylistic clean up of "DataAssistant.build_metric_multi_batch_parameter_builder()" method usage (#8059)
- [MAINTENANCE] Enable Ruff - pylint rules (#8058)
- [MAINTENANCE] type-checking
core/batch.py
(#8024) - [MAINTENANCE] Improve code elegance and fix informational string (#8061)
- [MAINTENANCE] Misc refactor of
Store
CRUD (#8064) - [MAINTENANCE] Fix pandas model schema differences across python versions (#8065)
- [MAINTENANCE] Updating CLI messages to point to new docs (#8072)
- [MAINTENANCE] Update GitHub templates for issues/PRs (#8073)
- [MAINTENANCE] Update default action list in
Checkpoint
based on user environment (#8074) - [MAINTENANCE] Ruff
0.0.271
update (#8077) - [MAINTENANCE] Update PR template to reference
invoke
for linting (#8079) - [MAINTENANCE] Delete
LegacyCheckpoint
(#8082) - [MAINTENANCE] Add public_api to read_parquet (#8060)
- [MAINTENANCE] Postgres starter reference environment (#8031)
0.16.15
- [FEATURE] Checker for correct Python code snippets in documentation. (#8000)
- [FEATURE] Add scaffolding for gx-agent (#7837)
- [FEATURE] Cloud - generate unique asset names for
pandas_default
assets (#8020) - [FEATURE] Connect to Cloud for agent config (#8006)
- [FEATURE] Add
AbstractDataContext.view_validation_result
(#8033) - [FEATURE] Add Onboarding Data Assistant agent action (#7882)
- [FEATURE] put 'How to connect to postgresql data' under test (#7988)
- [BUGFIX] Remove perpetually failing test -- in preparation for release. (#8040)
- [BUGFIX] meta_notes rendered inline can fail schema validation for a valid string-only configuration (#8044)
- [BUGFIX] Ensure That DataFrame for Fluent Data Source Pandas/Spark DataFrame Data Assets is specified only in one API method (#8032)
- [DOCS] Add Windows Support Admonition (#7991)
- [DOCS] Update how_to_edit_an_existing_expectationsuite.md (#8007)
- [DOCS] Adds redirects for removed UserConfigurableProfiler documentation (#8026)
- [DOCS] Create New Expectation Classes Conceptual Guide (#8004)
- [DOCS] Check simple spelling and word order error using ChatGPT-based program (#8029)
- [DOCS] Update Quickstart to Reflect User Feedback (#8027)
- [DOCS] Cloud - Getting started uses Fluent Datasources (#8035)
- [MAINTENANCE] Update
teams.yml
(#8011) - [MAINTENANCE] Add sensible default values to Checkpoint APIs (#7992)
- [MAINTENANCE] Removing xfails that we're added to get release out. (#8014)
- [MAINTENANCE] Remove reference to Superconductive from CITATION.cff (#8016)
- [MAINTENANCE] Deleting References to UserConfigurableProfiler Documentation (#7983)
- [MAINTENANCE] mypy 1.3 + azure deps in type-checking step (#8012)
- [MAINTENANCE] Bump tornado from 6.1.0 to 6.3.2 in /docs_rtd (#8003)
- [MAINTENANCE] Add support for
Validator
in Checkpoint CRUD (#7999) - [MAINTENANCE] Remove unused .dockerignore (#8030)
- [MAINTENANCE] Fluent Data Source Update: Credentials Doc update (#7968)
- [MAINTENANCE] Persist connection for backends that need a connection to keep temporary tables alive (#7607)
0.16.14
- [FEATURE] Add "batch.columns()" convenience method to Fluent DataAsset implementation. (#7926)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/connecting_to_your_data/fluent/filesystem/how_to_connect_to_one_or_more_files_using_spark/" (#7927)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/connecting_to_your_data/fluent/filesystem/how_to_quickly_connect_to_a_single_file_with_pandas/" (#7938)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/connecting_to_your_data/fluent/cloud/how_to_connect_to_data_on_s3_using_pandas/" (#7941)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/connecting_to_your_data/fluent/cloud/how_to_connect_to_data_on_s3_using_spark/" (#7943)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/connecting_to_your_data/fluent/database/how_to_connect_to_sqlite_data/" (#7947)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/connecting_to_your_data/fluent/cloud/how_to_connect_to_data_on_gcs_using_pandas/" (#7959)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/connecting_to_your_data/fluent/cloud/how_to_connect_to_data_on_gcs_using_spark/" (#7964)
- [FEATURE] add ssm parameter support for config secrets (#7940) (thanks @Isaacwhyuenac)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/connecting_to_your_data/fluent/cloud/how_to_connect_to_data_on_azure_blob_storage_using_pandas/" (#7965)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/connecting_to_your_data/fluent/cloud/how_to_connect_to_data_on_azure_blob_storage_using_spark/" (#7967)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/setup/configuring_data_contexts/instantiating_data_contexts/how_to_instantiate_a_specific_filesystem_data_context/" (#7984)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/setup/configuring_data_contexts/initializing_data_contexts/how_to_initialize_a_filesystem_data_context_in_python/" (#7985)
- [FEATURE] Put how_to_connect_to_sql_data fluent docs under test (#7956)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/setup/configuring_data_contexts/initializing_data_contexts/how_to_initialize_a_filesystem_data_context_in_python/" and "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/setup/configuring_data_contexts/how_to_convert_an_ephemeral_data_context_to_a_filesystem_data_context/". (#7986)
- [FEATURE] putting fluent doc how_to_connect_to_a_sql_table under test (#7966)
- [FEATURE] put fluent doc "How to connect to sql data using a query" under test (#7987)
- [FEATURE] Checker for correct Python code snippets in documentation. (#7996)
- [FEATURE] NotImplementedErrors for all FDS methods when accessed from BDS (#8002)
- [BUGFIX] Correcting instructions for Contributor Package installation (#7936)
- [BUGFIX] Return qualified name when calling
TableAsset.as_selectable()
(#7942) (thanks @calabozo) - [BUGFIX] fix
add_or_update_expectation_suite
update path (#7911) - [BUGFIX] Cloud - fix
add_or_update_*()
when usingname
as keyword arg (#7952) - [BUGFIX] Change GXSqlDialect.AWSATHENA to awsathena (#7950) (thanks @calabozo)
- [BUGFIX] Setting maximum on
typing-extension
package version (#7970) - [BUGFIX] Resolve Issue 7335 (#7339) (thanks @richardohara)
- [BUGFIX] Cloud - Fix
context.sources.update_*()
POST
instead ofPUT
calls (#7989) - [BUGFIX]
meta_notes
rendered inline can fail schema validation for a valid configuration (#7995) - [BUGFIX]
xfail
two cloud tests that are blocking release 0.16.14 (#8008) - [DOCS] Technical tags in Versioned Docs reference correct version (#7935)
- [DOCS] Fix docs deploy (#7958)
- [DOCS] Add small doc change (#7957)
- [DOCS] Fix issues with technical tags links in earlier versions (#7961)
- [DOCS] Temporarily pin typing-extensions for API docs (#7977)
- [DOCS] Add optional Slack step to Cloud Quickstart (#7955)
- [DOCS] Add another small fix to doc (#7960)
- [DOCS] Versioning: Convert technical tag imports starting with /docs/ to relative paths (#7981)
- [DOCS] Edit Bigquery connection path (#7982)
- [DOCS] FDS Topic Quality Review (#7944)
- [DOCS] Updates the migration guide to direct to the last version of GX to support the V2 processes and API surface (#7976)
- [DOCS] Removes Data Connector references (#7930)
- [DOCS] add in-memory add expectation suite (#7973) (thanks @tb102122)
- [DOCS] Removes CLI documentation from current docs version (#7975)
- [DOCS] Delete CODE_OF_CONDUCT.md (#7625)
- [MAINTENANCE] Fixing pytest: renderer assertion (#7928)
- [MAINTENANCE] Update
teams.yml
(#7934) - [MAINTENANCE] Remove the -rs flag from the ci pytest invocations. (#7937)
- [MAINTENANCE] Update
GXCloudIdentifier
to return nullable attrs instead of empty strings (#7895) - [MAINTENANCE] Better error message when calling "Fluent" methods from wrong Data Source type (#7929)
- [MAINTENANCE] bump python minimum version to 3.8 (#7916)
- [MAINTENANCE] Update ruff to
0.0.269
and targetpython 3.8
(#7945) - [MAINTENANCE] black
23.3
(#7946) - [MAINTENANCE] ignore black formatting and ruff auto-fix revisions (#7953)
- [MAINTENANCE] Refactor documentation integration tests into their proper categories according to their respective backend. (#7978)
- [MAINTENANCE] Bump requests from 2.25.1 to 2.31.0 in /docs_rtd (#7969)
- [MAINTENANCE] Update Cloud Tests for release this week + Revert pin on
typing-extension
(#7980) - [MAINTENANCE] Use
Session
for allapi.greatexpectations.io
calls (#7919) - [MAINTENANCE] Revert "[FEATURE] Checker for correct Python code snippets in documentation." (#7998)
- [MAINTENANCE] Fix async Cloud tests (#8005)
0.16.13
- [FEATURE] Spark file reader support for fluent datasources (#7844)
- [FEATURE] Spark directory asset types (#7873)
- [FEATURE] Add Spark DeltaAsset type (#7872)
- [FEATURE] Add DirectoryDeltaAsset (#7877)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/connecting_to_your_data/fluent/data_assets/how_to_organize_batches_in_a_file_based_data_asset/" (#7907)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/connecting_to_your_data/fluent/data_assets/how_to_organize_batches_in_a_sql_based_data_asset/" (#7909)
- [FEATURE] Implementing Python code snippets under test for "https://legacy.017.docs.greatexpectations.io/docs/0.16.16/guides/connecting_to_your_data/fluent/filesystem/how_to_connect_to_one_or_more_files_using_pandas/" (#7922)
- [FEATURE] DataProfilerStructuredDataAssistant Float Rule (#7842) (thanks @micdavis)
- [BUGFIX] Fix inability to extend SimpleCheckpoint -- and several additional enhancements and clean up (#7879)
- [BUGFIX] Delete ExpectationSuite by name in GX Cloud (#7881)
- [BUGFIX] optional dataframe on datasources (#7862)
- [BUGFIX] Fix sparkDF cannot compute mean for DecimalType (#7867)
- [BUGFIX] Fix Cloud FDS
add_or_update_*
methods (#7908) - [BUGFIX] fix
PandasAzureBlobStorageDatasource
config substitution (#7914) - [BUGFIX] Fix remaining FDS config substitution issues (#7917)
- [DOCS] removes remaining Block-config Data Source guides (#7870)
- [DOCS] Update "How to use Great Expectations with Databricks" (#7762)
- [DOCS] CLI Edits (#7865)
- [DOCS] More doc updates to remove the CLI (#7874)
- [DOCS] Create New Templates for How-To, Tutorial, Conceptual, and Reference Documentation (#7855)
- [DOCS] Add Links to Tutorial Templates in the README (#7884)
- [DOCS] removes block-config docs for source data systems in the cloud (#7871)
- [DOCS] Continuing CLI Update (#7876)
- [DOCS] Removes the SQL block config guides. (#7886)
- [DOCS] Remove jq dependency for building docs (#7893)
- [DOCS] Less verbose logging during docs build (#7894)
- [DOCS] Corrections to Document "How to request data from a Data Asset" with proper FDS implementation and integration test. (#7896)
- [DOCS] Remove or Modify References to the CLI (#7875)
- [DOCS] Updated contribution docs (#7880)
- [DOCS] Glossary Update (#7900)
- [DOCS] Removes filesystem and core block config docs (#7913)
- [DOCS] FDS Deployment Pattern Redshift (#7868)
- [DOCS] Document and simplify local docs build (#7892)
- [DOCS] How to Edit Existing
ExpectationSuite
(#7859) - [DOCS] CLI Clean-up (#7904)
- [MAINTENANCE] FDS Documentation - Creating ExpectationSuite with Domain Knowledge (#7852)
- [MAINTENANCE] Refactor directory data asset (#7878)
- [MAINTENANCE] partial
checkpoint
type checking (#6914) - [MAINTENANCE] type-checking
checkpoint/actions.py
(#7899) - [MAINTENANCE] Trigger docs_integration on develop (#7902)
- [MAINTENANCE] Finish type-checking
checkpoint
(#7901) - [MAINTENANCE] Update How to set up GX to work with data in Azure Blob Storage (#7910)
- [MAINTENANCE]
CloudDataContext
ExpectationsStore
can only request one Suite at a time (#7905) - [MAINTENANCE] Removing engine-specific tests that assert generic behavior (#7918)
- [MAINTENANCE] Add
docs/*.py
toGXChanged
for linting (#7924) - [MAINTENANCE] Fixes Glue tests on vanilla pytest (#7925)
0.16.12
- [FEATURE] Plumbing of validation_result_url from cloud response (#7809)
- [FEATURE] Splitters work with Spark Fluent Datasources (#7832)
- [FEATURE] Update
get_context
to scaffold project structure for file-backed usecases (#7693) - [BUGFIX] Azure Package Presence/Absence Tests Strengthening (#7818)
- [BUGFIX] Handle "persist" directive in "SparkDFExecutionEngine" properly. (#7830)
- [BUGFIX] Adding support for Fluent Batch Requests to context.get_validator (#7808)
- [BUGFIX] FDS - Deletes not immediately reflected in
great_expectations.yml
(#7843) - [BUGFIX]
batching_regex
tags are now correctly rendered in docs (#7845) - [BUGFIX] Fix link checker and add to mypy type checking (#7857)
- [BUGFIX] expect_day_count_to_be_close_to_equivalent_week_day_mean (#7782) (thanks @HadasManor)
- [BUGFIX] Docs-Tests:
Connection.connect()
was causing Snowflake and BigQuery Tests to Fail (#7863) - [DOCS] Prerequisites Cleanup (#7811)
- [DOCS] Update docs for how_to_initialize_a_filesystem_data_context_in_python (#7831)
- [DOCS] Updating Checkpoint terms page (#7722)
- [DOCS] Update how to create a checkpoint with Test YAML config (#7835)
- [DOCS] Removing datasource centric test_yaml_config doc (#7836)
- [DOCS] Creating a Checkpoint from an In-Memory Dataframe (#7701)
- [DOCS] Review and Revise Great Expectations Quickstart (#7727)
- [MAINTENANCE] FDS - Datasources can rebuild their own asset data_connectors (#7826)
- [MAINTENANCE] Enable Spark-S3 Integration tests on Azure CI/CD (#7819)
- [MAINTENANCE] Clean up: Remove duplicated fixture and utilize deeper filtering mechanism for configuration assertions. (#7825)
- [MAINTENANCE] Enable S3/Spark Connecting To Your Data tests (#7828)
- [MAINTENANCE] New PR template (#7710)
- [MAINTENANCE] ruff
.0.262 -> .0.265
(#7829) - [MAINTENANCE] Boto import pattern established (#7796)
- [MAINTENANCE] Prevent TCH001 warnings for pydantic model annotations (#7846)
- [MAINTENANCE] Pin altair (#7849)
- [MAINTENANCE] Adding docs link checker to invoke (#7841)
- [MAINTENANCE] Clean up version checker message formatting (#7838)
- [MAINTENANCE] Bump nbconvert version (#7847)
- [MAINTENANCE] Return empty set instead of None (#7797)
- [MAINTENANCE] Improve misconfigured sampler error message (#7858)
- [MAINTENANCE] Fixing path formatting for DataConnector of Fluent SparkAzureBlobStorageDatasource and correction of the
SQLAlchemy
compatibility usage inTableHead
metric (#7860) - [MAINTENANCE] S3 Spark Integration Guide - Rendering Fix (#7864)
0.16.11
- [FEATURE] Add tests for
Checkpoint
utilizing SQLAlchemy style Fluent Datasources. (#7759) - [FEATURE] Spark parquet reader support for fluent datasources (#7754)
- [FEATURE] Add tests for
SimpleCheckpoint
utilizing Fluent Datasources with Pandas, Spark, and SQLAlchemy test cases. (#7778) - [FEATURE] Spark read directory of files as a single batch for CSV (#7777)
- [FEATURE] Enable passing "spark_config" through to "SparkDFExecutionEngine" constructor as arguments to "add_spark*()" Fluent Datasources methods. (#7810)
- [BUGFIX] Patch faulty version checker logic (#7783)
- [BUGFIX] Fix FDS Sqlite config round tripping (#7791)
- [BUGFIX] Correct import errors in Azure Blob Storage tests and make Azure Glob Storage and Google Cloud Storage tests more elegant. (#7795)
- [BUGFIX] Add
pytest
decorator to fixture (#7803) - [BUGFIX] fix class name (#7734) (thanks @tb102122)
- [BUGFIX] Repair handling of regular expressions partitioning for cloud file storage environments utilizing prefix directive. (#7798)
- [BUGFIX] AWS Docs reference clash (#7817)
- [BUGFIX] Cloud - Fix FDS Asset has no attribute
_data_connector
(#7813) - [BUGFIX] Upper bound
pyathena
due to breaking API in V3 (#7821) - [DOCS] FDS Deployment Pattern - Google Cloud: BigQuery and GCS (#7741)
- [DOCS] Remove temporary pin for ipython (#7784)
- [DOCS] Add CLI Admonition (#7765)
- [DOCS] Link Update (#7788)
- [DOCS] Remove Redundant Introduction Headings (#7747)
- [DOCS] Remove Prerequisites from Admonitions (#7786)
- [DOCS] Link Updates (#7781)
- [DOCS] FDS Deployment Pattern - AWS: Spark and S3 (#7775)
- [MAINTENANCE] Add check to
CloudDataContext
to ensure using latest PyPI version (#7753) - [MAINTENANCE] Cache the latest great_expectations version (#7785)
- [MAINTENANCE] Enable
flake8-bugbear
rules (#7776) - [MAINTENANCE] Cleanup of Fluent
BatchRequest
type and immutability constraints (#7769) - [MAINTENANCE] CLI warnings for
suite new
command (#7787) - [MAINTENANCE] Update pip instal extras and use AWS_ env vars (#7793)
- [MAINTENANCE] Test DirectoryCSVAsset with both str and pathlib.Path (#7801)
- [MAINTENANCE] Lint
test/data_context
(#7767) - [MAINTENANCE] FDS Documentation Update - S3 Pandas reference fixes (#7789)
- [MAINTENANCE] Update all pytest calls in CI to show reason skipped (#7806)
- [MAINTENANCE] Dont run runme_script_runner_tests stage on forks (#7807)
- [MAINTENANCE] Lint
tests/checkpoint
&tests/execution_engine
(#7804) - [MAINTENANCE] docs-integration re-start (#7735)
- [MAINTENANCE] Remove runme fixtures/stages and enable docs-integration to run automatically (#7812)
- [MAINTENANCE] Fix linting error. (#7820)
- [MAINTENANCE] Fix pin count. (#7823)
0.16.10
- [FEATURE] Add tests for
Checkpoint
utilizing Pandas and Spark style Fluent Datasources. (#7740) - [FEATURE] Fluent
BatchRequest
slicing (#7706) - [BUGFIX] Patch Expectation registry issue introduced in 0.16.9 (#7771)
- [DOCS] Remove, relocate, consolidate, and edit Contributing content (#7669)
- [DOCS] Temporarily pin ipython for python 3.8 before building api docs (#7764)
- [DOCS] Update Links in Configure Topics (#7760)
- [DOCS] Link Updates (#7768)
- [MAINTENANCE] FDS Deployment Guide - Pandas S3 fix reference (#7755)
- [MAINTENANCE] IPython Python 3.8 upper bound (#7763)
- [MAINTENANCE] breakup mypy ci steps (#7761)
- [MAINTENANCE] fix async type-check step (#7772)
- [MAINTENANCE] Bump Python version in
static_type_check
stage of async CI (#7773)
0.16.9
- [FEATURE] Implementing Fluent Datasources Support for Checkpoint (#7697)
- [FEATURE] FDS persist
DataAsset
to YAML file immediately on creation (#7705) - [FEATURE] Cloud support FDS deletes (#7682)
- [FEATURE] Persist Cloud DataAssets on creation (#7748)
- [FEATURE] Add tests for
Checkpoint
utilizing Pandas and Spark style Fluent Datasources. (#7740) - [BUGFIX] Render Correct Fonts in Data Assistant Plot graphs (#7676)
- [BUGFIX] fix rendering data asset name in microsoft teams notification (#7675)
- [BUGFIX] Register core Expectations JIT in Validator workflows (#7683)
- [BUGFIX] Data Context Data Source CRUD support for Fluent Datasources (#7660)
- [BUGFIX] Replace renamed fixture (#7711)
- [BUGFIX] Add missing pyspark reference (#7684) (thanks @willfeltman)
- [DOCS] Add fluent datasources and yaml configuration warning message (#7673)
- [DOCS] D/ /fluent connect to data overview (#7671)
- [DOCS] Update fluent In Progress Cautionary Note (#7681)
- [DOCS] Remove version reference (#7644)
- [DOCS] Removing in-progress from docs confirmed as up-to-date (#7686)
- [DOCS] Updated Data Source terms page (#7692)
- [DOCS] Removing json schema profiler documentation (#7694)
- [DOCS] Removing CLI-based suite edit workflow (#7689)
- [DOCS] Updated onboarding data assistant docs test script to Fluent-style (#7695)
- [DOCS] Update for fluent datasources: Expectations that span multiple batches evaluation params (#7668)
- [DOCS] Add fluent docs and test create and edit expectations with profiler (#7696)
- [DOCS] Quick docstring update for list_datasources (#7699)
- [DOCS] Retiring the CLI (#7700)
- [DOCS] Updating the Rule-Based Profiler doc to Fluent (#7698)
- [DOCS] Update Batch Request glossary entry. (#7716)
- [DOCS] Removed guide for no YML, redirect to EphemeralDataContext (#7702)
- [DOCS] Update for fluent datasources: Dynamically load evaluation params from a database (#7717)
- [DOCS] Update batch glossary docs. (#7726)
- [DOCS] Update for fluent datasources: Create a new Checkpoint (#7729)
- [DOCS] Temporarily revert
update_expectation_suite
call in GX Cloud quickstart (#7736) - [DOCS] Light update to How to add validations data or suites to a Checkpoint (#7703)
- [DOCS] Updating cross-table comparison guide with Fluent Datasources (#7691)
- [DOCS] Better output from
invoke public-api
report (#7730) - [DOCS] Removed unneeded calls to update datasource in docs. (#7739)
- [DOCS] FDS Deployment Pattern - AWS S3 Pandas (#7718)
- [DOCS] Update pypyi page urls (#7752)
- [MAINTENANCE] Correcting minor typographical errors and type hints issues in Checkpoint and Test Checkpoint Modules (#7665)
- [MAINTENANCE] Only attempt docs-integration pipeline when manually triggered (#7674)
- [MAINTENANCE] Clean up Checkpoint test method names and usage of batch_request_dict fixture (#7670)
- [MAINTENANCE] Correct typo in Checkpoint test method fixture (#7677)
- [MAINTENANCE] Enable remaining
CloudDataContext
ExpectationSuite
CRUD (#7646) - [MAINTENANCE] list_datasources should return FDS configs as well (#7667)
- [MAINTENANCE] Exit with error when attempting to delete a fluent style datasource using the CLI (#7687)
- [MAINTENANCE] Add warning to
datasource list
command if fluent datasources are detected (#7690) - [MAINTENANCE] ruff update
0.0.262
(#7707) - [MAINTENANCE] Adding black to invoke lint (#7715)
- [MAINTENANCE] Use same version of mypy in contrib tool (#7724)
- [MAINTENANCE] Update a Fluent Data Source related fixture name to better reflect its capabilities (#7725)
- [MAINTENANCE] Add CLI warnings when adding a checkpoint with fluent datasources (#7685)
- [MAINTENANCE] Iterate over the regex_pattern characters too in (#7720)
- [MAINTENANCE] Minor stylistic cleanup (#7732)
- [MAINTENANCE] fix get available data assets names for fds (#7723)
- [MAINTENANCE] add warning messages when using CLI to edit an expectaiton suite if fluent datasources are present (#7714)
- [MAINTENANCE] add warning to
datasource new
CLI command (#7709) - [MAINTENANCE] Add split/join logic to build_gallery process (#7572)
- [MAINTENANCE] Use invoke public-api in main CI pipeline (#7746)
- [MAINTENANCE] Add remaining
public_api
decorators for core fluent datasources (#7749) - [MAINTENANCE] FDS update schemas - fixes CI (#7751)
- [MAINTENANCE] FDS Deployment Guide - Pandas S3 fix reference (#7755)
0.16.8
- [FEATURE] add Fluent Datasources to
CloudDataContext
(#7570) - [BUGFIX] fix marshmallow schema for SQLAlchemy
connect_args
passthrough (#7614) - [BUGFIX] MapCondition Memory Inefficiencies in Spark (#7626)
- [BUGFIX] Fix capitalone_dataprofiler_expectations imports (#7658)
- [BUGFIX] CloudDataContext creates
great_expectations.yml
when adding a Fluent datasource (#7657) - [BUGFIX] Correct GX configuration structure that incorporates both V3 and Fluent Datasources (#7661)
- [BUGFIX] Patch broken
include_rendered_content
test in advance of0.16.8
release (#7663) - [DOCS] Corrects Step Numbering in How to instantiate a specific Filesystem Data Context (#7612)
- [DOCS] Corrects Heading Issue in How to host and share Data Docs on Azure Blob Storage (#7620)
- [DOCS] Update overview.md (#7627)
- [DOCS] Updates the "Interactive Mode" guide for creating Expectations (#7624)
- [DOCS] Updates the language in the banner linking the legacy site to the current docs. (#7636)
- [DOCS] Improve expect_column_values_to_be_of_type docstring (#7632)
- [DOCS] Corrects a typo found in the navigation section of the legacy docs (#7643)
- [DOCS] Add lakeFS to list of data version control tools (#7642) (thanks @rmoff)
- [DOCS] Standardize language around GX Cloud access tokens (#7621)
- [DOCS] Added IAM user and IAM assume role doc (#7634) (thanks @Reactor11)
- [DOCS] update to location of cloud callout in the OSS Quickstart (#7616)
- [MAINTENANCE] Update
teams.yml
(#7623) - [MAINTENANCE] Utilize
NotImported
for SQLAlchemy, Google Cloud Services, Azure Blob Storage, and Spark import usage (#7617) - [MAINTENANCE] Remove stray cloud test marker. (#7639)
- [MAINTENANCE] Upgrade mypy to 1.2.0 (#7645)
- [MAINTENANCE] Static type checking with python 3.8 (#7637)
- [MAINTENANCE] Static type checking with python 3.8 followup (#7647)
- [MAINTENANCE] The 'sklearn' PyPI package is deprecated, use 'scikit-learn' (#7651)
- [MAINTENANCE] numpy.typing only available after v1.20 (#7654)
- [MAINTENANCE] Update NotImported mechanism to use scoped compatibility modules (#7635)
- [MAINTENANCE] Uncap
altair
version, and bump minimum version to4.2.1
. Also uncapurllib3
version, and bump minimum version to1.26
(#7650)
0.16.7
- [FEATURE] Added AssumeRole Feature (#7547) (thanks @Reactor11)
- [BUGFIX] Fix Fluent Spark
DataConnectors
on config load (#7560) - [BUGFIX]
dataset_name
made optional parameter for Expectations (#7603) - [BUGFIX] Misc gallery bugfixes (#7611)
- [BUGFIX] Remove spark from bic Expectations since it never worked for them (#7619)
- [DOCS] Use current minor version number in drop down instead of "Current" (#7581)
- [DOCS] Adds deprecation policy to changelog page (#7585)
- [DOCS] Use the actual version after release (#7583)
- [DOCS] Update some docs_rtd requirements so the venv can be created successfully (#7580)
- [DOCS] Add Cloud quickstart (#7441)
- [DOCS] Updates how the GX Cloud Beta is referenced in the Quickstart guide. (#7594)
- [DOCS] Corrects typo in code block within in-memory Pandas guide (#7600)
- [DOCS] Updates to Contributing through GitHub (#7601)
- [DOCS] Correct expectation documentation for expect_column_max_to_be_between (#7597)
- [DOCS] Add scripts under test for "How to create and edit Expectations with instant feedback from a sample Batch of data" (#7615)
- [DOCS] Corrects Step Numbering in How to instantiate a specific Filesystem Data Context (#7612)
- [DOCS] Corrects Heading Issue in How to host and share Data Docs on Azure Blob Storage (#7620)
- [MAINTENANCE] Warning non integer slice on row for SQLAlchemy 2.0 Compatibility (#7501)
- [MAINTENANCE] Warning MetaData.bind argument deprecated for SQLAlchemy 2.0 Compatibility (#7502)
- [MAINTENANCE] Capitalize "If" in rendering of conditional Expectations (#7588)
- [MAINTENANCE] Remove pip pins in CI and in contributing_setup.md (#7587)
- [MAINTENANCE] Remove ignore of warning deprecated api features detected sqlalchemy 2 (#7584)
- [MAINTENANCE] Fix sqlalchemy 2.0 incompatible warnings (#7589)
- [MAINTENANCE] Increase minimum scipy version package to 1.6.0 to take advantage of available capabilities. (#7591)
- [MAINTENANCE] Remove s3fs dependency and upper bound for boto3 (#7598)
- [MAINTENANCE] Move Fluent Datasources Sorters into
TYPE_CHECKING
block (#7602) - [MAINTENANCE] Bump terser from 5.10.0 to 5.16.8 in /docs/docusaurus (#7486) (thanks @dependabot[bot])
- [MAINTENANCE] Bump cookiecutter from 1.7.3 to 2.1.1 in /contrib/cli (#7510) (thanks @dependabot[bot])
- [MAINTENANCE] Polish and ratchet requirements pins and upper bounds (#7604)
- [MAINTENANCE] small documentation updates (#7606)
- [MAINTENANCE] SqlAlchemy 2 Compatibility -
engine.execute()
(#7469) - [MAINTENANCE] Deprecate ColumnExpectation in favor of ColumnAggregateExpectation (#7609)
- [MAINTENANCE] Deprecate TableExpectation in favor of BatchExpectation (#7610)
- [MAINTENANCE] Explicitly test relevant modules in Sqlalchemy compatibility pipeline (#7613)
- [MAINTENANCE] Fluent Datasources: Eliminate redundant Data Source name and DataAsset name from dictionary and JSON configuration (#7573)
- [CONTRIB] add check to calculate difference between 2 dates in month (#7576) (thanks @tb102122)
- [CONTRIB] Expect Column Values to be Valid UUID - Added SqlAlchemyExecutionEngine support (#7592) (thanks @asafla)
0.16.6
- [FEATURE] Fluent
DataAsset
batch_metadata
config variables (#7513) - [FEATURE] Add batch metadata to spark add_*_asset methods (#7534)
- [BUGFIX] Fluent Data Source load from config fixes for remaining Pandas Datasources (#7442)
- [BUGFIX] Address
pandas==2.0.0
test failures (#7553) - [BUGFIX] Render prescriptive
ExpectationConfiguration
s with evaluation parameters inline (#7552) - [BUGFIX] Release Pipeline Fix (#7575)
- [DOCS] Update GX version in
_data.jsx
component (#7549) - [DOCS] Adds guides on using Ephemeral Data Contexts and updates Quickstart Next Steps (#7500)
- [DOCS] Fixes broken code block and incorrectly numbered steps in "How to organize Batches in a SQL-based Data Asset" (#7533)
- [DOCS] Update nav to match gx.io site (#7557)
- [DOCS] Corrects step numbers in "How to organize Batches in a file-based Data Asset" (#7559)
- [DOCS] Delete SLACK_GUIDELINES.md (#7566)
- [DOCS] Update syntax highlighting of code blocks in GX Cloud Getting Started guide (#7563)
- [DOCS] Fix code snippets for earlier versions (#7554)
- [DOCS] Fix typo in docs (#7568)
- [DOCS] Moar typo fix (#7569)
- [DOCS] removes the original getting started tutorial pages and redirects to the quickstart guide (#7548)
- [DOCS] Fix integral typo (#7578)
- [DOCS] Prepare earlier versions using develop (#7567)
- [DOCS] Use orange in docs logs (#7579)
- [DOCS] Add GX Cloud Onboarding Script (#7517)
- [MAINTENANCE] release prep for 0.16.5 (#7545)
- [MAINTENANCE] Test Pandas 2.0 prerelease in CI/CD (#7343)
- [MAINTENANCE] Add noqa directives for existing sqlalchemy imports (#7564)
- [MAINTENANCE] Add ruff rule for sqlalchemy imports (#7562)
- [MAINTENANCE] adding a footer to data docs with a link to the cloud page (#7532)
- [MAINTENANCE] Harden tests for
CloudDataContext
alwaysinclude_rendered_content
(#7558) - [MAINTENANCE] FluentDatasources - Quickstart Snippets converted to Named Snippets (#7550)
- [MAINTENANCE] Simplify
GXCloudStoreBackend._has_key
check (#7561) - [MAINTENANCE] Temporarily Pin
pandas<2.0.0
for compatibility (#7571) - [MAINTENANCE] SqlAlchemy 2.0 Compatibility - branched connection +
bind
argument now required (#7529) - [MAINTENANCE] Add missing docstrings to fluent
sql_datasource
splitter methods. (#7577)
0.16.5
- [FEATURE] Add batch metadata to sql datasources. (#7499)
- [BUGFIX] Fix issue running quickstart (#7539)
- [DOCS] doc 508 Updates footer links on docs pages (#7521)
- [DOCS] DSB-64 removes outdated v2/v3 references from the docs (#7519)
- [DOCS] Update CODEOWNERS (#7528)
- [DOCS] Quickstart code under test (#7542)
- [MAINTENANCE] SqlAlchemy2 Compatibility -
Row.keys()
(#7520) - [MAINTENANCE] Refactoring of CapitalOne Metrics and Profiler-Based DataAssistant for Enhanced Code Elegance (#7522)
- [MAINTENANCE] SqlAlchemy 2 Compatibility - Autoload Parameter deprecation (#7526)
- [MAINTENANCE] Bump notebook from 6.4.1 to 6.4.12 in /docs_rtd (#7511)
- [MAINTENANCE] Break out unit tests to own stage. (#7530)
- [MAINTENANCE] Bump wheel from 0.37.1 to 0.38.1 in /contrib/cli (#7493)
- [MAINTENANCE] Simplifying CapitalOne DataProfilerColumnDomainBuilder Using Default "profile_path" Argument (#7535)
- [MAINTENANCE] : Clean up ununsed imports (#7537)
- [MAINTENANCE] Fix Type-Checking steps (#7536)
- [MAINTENANCE] Disable UserConfigurableProfiler tests relying on deprecated V2 functionality (#7541)
- [MAINTENANCE] : replace ColumnMetricProvider with ColumnAggregateMetricProvider (#7538)
- [MAINTENANCE] Exclude files from deprecation warning check (#7544)
0.16.4
- [FEATURE] Add package, contributors and metrics filter in Algolia script for expectation (#7000) (thanks @kod-er)
- [FEATURE]
BatchMetadata
for all fluentDataAsset
s (#7392) - [FEATURE] Introducing CapitalOne DataProfilerColumnDomainBuilder as well as multiple improvements to CapitalOne codebase and testability. (#7498)
- [BUGFIX] Repair SparkSession initialization behavior to disallow restarting, unless explicitly instructed through configuration (#7444)
- [BUGFIX] Skip dialect specific tests if no flag passed or flag not available (#7443)
- [BUGFIX] Ensure that MetricStore Records "data_asset_name" Properly (#7458)
- [BUGFIX] Fix incorrect type hint and correct typographical errors in DomainBuilder docstrings and fill in missing docstrings (#7467)
- [BUGFIX] Reset Metrics Registry in order to keep state of Metrics Registry test cases Runs mutually consistent (#7473)
- [BUGFIX] Corrected typographical errors in two docstrings (#7506)
- [BUGFIX] Typo in min versions install (#7516)
- [DOCS] New ADR proposal for patch version support (#7451)
- [DOCS] Remove outdated instructions for building documentation (#7457)
- [DOCS] Updating the docs to gx_dev (#7455)
- [DOCS] Fixes typo in code snippet for "How to organize Batches in a file-based Data Asset" guide (#7465)
- [DOCS] Postgresql drivername fix for SQLAlchemy compatibility. Closes #7464 (#7466) (thanks @Itzblend)
- [DOCS] Corrects code snippets (#7470)
- [DOCS] Add a note about installing version of pyspark that matches Spark version (#7483)
- [DOCS] adding cloud language to quickstart (#7484)
- [DOCS] doc-409 Corrects links in databricks guide (#7485)
- [DOCS] Doc-472 Corrects numbering of steps in guide (#7478)
- [DOCS] DOC-474 Updates URL for usage statistics page (#7477)
- [DOCS] Testing ADR (#7495)
- [DOCS] corrects typo in GCS setup guide (#7514)
- [MAINTENANCE] Case whens argument change for SQLAlchemy 2.0 compatibility (#7416)
- [MAINTENANCE]
mypy
1.1.1
update (#7452) - [MAINTENANCE] Remove v2 api CLI (#7440)
- [MAINTENANCE] Bump http-cache-semantics from 4.1.0 to 4.1.1 in /docs/docusaurus (#7447)
- [MAINTENANCE] Updating language per issue 3572 (#7456)
- [MAINTENANCE] Remove v2 api expectations tests (#7439)
- [MAINTENANCE] Move docs_link_checker.py and create new docs-specific pipeline (#7422)
- [MAINTENANCE] select call style change for SQLAlchemy 2 compatibility (#7378)
- [MAINTENANCE] Decruft map_metric_provider.py (#7460)
- [MAINTENANCE] Add tests for
_register_metric_functions
in theMetricProvider
class hierarchy. (#7459) - [MAINTENANCE] SqlAlchemy2 Compatibility - implicit autocommit (#7400)
- [MAINTENANCE] Bump ua-parser-js from 0.7.31 to 0.7.34 in /docs/docusaurus (#7474)
- [MAINTENANCE] Remove SQLAlchemyDataset/Data Source (#7471)
- [MAINTENANCE] Bump json5 from 1.0.1 to 1.0.2 in /docs/docusaurus (#7475)
- [MAINTENANCE] Bump @sideway/formula from 3.0.0 to 3.0.1 in /docs/docusaurus (#7487)
- [MAINTENANCE] Bump ipython from 7.31.1 to 8.10.0 in /docs_rtd (#7491)
- [MAINTENANCE] Bump cross-fetch from 3.1.4 to 3.1.5 in /docs/docusaurus (#7488)
- [MAINTENANCE] Deprecated API features detected warning for SQLAlchemy 2.0 compatibility (#7490)
- [MAINTENANCE] type-checking implementation files (#7454)
- [MAINTENANCE] Small Refactor of ColumnDomainBuilder for code elegance and computational performance improvements (#7492)
- [MAINTENANCE] Lower pydantic requirement to v1.9.2 or greater (#7482)
- [MAINTENANCE] Connection.connect warning for SQLAlchemy 2.0 compatibility (#7489)
- [MAINTENANCE] Pass
PandasDatasource
batch_metadata
askwargs
to remove possibility ofNone
onDataAsset
model (#7503) - [MAINTENANCE] Make
dataset_name
a parameter for Expectations tests or create name fromExpectation
name, which ensures only limited number of tables created. (#7476) - [MAINTENANCE] Fix sqlalchemy warnings for pandas + sql fluent datasources (#7504)
- [MAINTENANCE] Bump gitpython from 3.1.7 to 3.1.30 in /docs_rtd (#7494)
- [MAINTENANCE] Bump certifi from 2020.6.20 to 2022.12.7 in /docs_rtd (#7497)
- [MAINTENANCE] Bump jupyter-core from 4.6.3 to 4.11.2 in /docs_rtd (#7496)
- [MAINTENANCE] Bump nbconvert from 5.6.1 to 6.5.1 in /docs_rtd (#7508)
- [MAINTENANCE] Bump numpy from 1.21.0 to 1.22.0 in /docs_rtd (#7509)
- [MAINTENANCE] Revert PR 7490 (#7515)
- [MAINTENANCE] Use YAMLHandler in tests and docs (#7507)
- [MAINTENANCE] Dedicated airflow 2.2.0 async test (#7518)
- [MAINTENANCE] Remove airflow2 min depdency test. (#7524)
- [CONTRIB] - Add new column expectation not be null and empty (#7449) (thanks @tmilitino)
0.16.3
- [BUGFIX] Fix LegacyRow import. (#7446)
0.16.2
- [FEATURE] Develop PandasDBFSDatasource (as part of Fluent Datasources) (#7372)
- [FEATURE] Make SQL datasources add asset methods public. (#7387)
- [FEATURE] Develop SparkDBFSDatasource (as part of Fluent Datasources) (#7380)
- [FEATURE] add optional
id
to Fluent Datasources and DataAsset schemas (#7334) - [FEATURE] Fluent SQLDatasource accepts arbitrary
kwargs
(#7394) - [FEATURE] Fluent
SQLDatasource
create_temp_table
(#7407) - [FEATURE] F/great 1463/add updates with datasource obj (#7401)
- [FEATURE] Fluent SparkDataframeDatasource with DataframeDataAsset (#7425)
- [BUGFIX] Add FluentBatchRequest early exit to convert_to_json_serializable (#7381)
- [BUGFIX] Handle non-string fluent batch request options in convert_to_json_serializable (#7386)
- [BUGFIX] Fix bug with case sensitive execution env (#7393)
- [BUGFIX] Fixing typographical errors and argument omissions (#7398)
- [BUGFIX] Remove Query ID from exception for query.template_values metric (#7373) (thanks @itaise)
- [BUGFIX] Fluent
PandasFilesytemDatasources
data_connector fixes (#7414) - [DOCS] Add back GX logo to README (#7391)
- [DOCS] DOC-473: Adds shared components for fluent and state management updates (#7404)
- [DOCS] DOC-473 Adds guide "How to set up GX to work with SQL databases" (#7409)
- [DOCS] DOC-473 Adds guide "How to set up GX to work with data on GCS" (#7408)
- [DOCS] Pending doc updates for Data Context state management and fluent Data Source configuration (#7301)
- [DOCS] Put data_context.md code examples under test (#7417)
- [MAINTENANCE] Make sure sqlalchemy 2.0 warnings are emitted when running pipelines (#7379)
- [MAINTENANCE] Make sure all existing warnings are ignored in full CI pipeline (#7389)
- [MAINTENANCE] Add PR title checker GitHub Action (#7365)
- [MAINTENANCE] Re-enable warnings as errors (#7383)
- [MAINTENANCE] Test against minimum SQLAlchemy versions (#7396)
- [MAINTENANCE] : split up map_metric_provider.py (#7402)
- [MAINTENANCE] Consolidate Cloud tutorials (#7395)
- [MAINTENANCE] Change for
connection.execute()
for SQLAlchemy 2 compatibility (#7384) - [MAINTENANCE] Bump SQLAlchemy lower bound to 1.4.0 (#7413)
- [MAINTENANCE] Validate
ExpectationConfiguration
before adding to suite (#7366) - [MAINTENANCE] Convert to python built in warning categories (#7415)
- [MAINTENANCE] Move NotImported to optional_imports.py and start using it. (#7421)
- [MAINTENANCE] Bump minimist from 1.2.5 to 1.2.8 in /docs/docusaurus (#7419)
- [MAINTENANCE] PandasFilesystemDatasource stubs and Schema corrections (#7428)
- [MAINTENANCE] Bump loader-utils from 2.0.2 to 2.0.4 in /docs/docusaurus (#7429)
- [MAINTENANCE] Bump webpack from 5.74.0 to 5.76.3 in /docs/docusaurus (#7430)
- [MAINTENANCE] Fix some failing tests when running pytest with no args locally (#7426)
- [MAINTENANCE] only provide
py.typed
files for fully typed sub-packages (#7438) - [MAINTENANCE] Some fluent datasource methods should be private (#7437)
- [CONTRIB] Adding support for date for the row condition parser (#7359) (thanks @maayaniti)
- [CONTRIB] Limit results for two expectations (#7403) (thanks @itaise)
- [CONTRIB] [MAINTENANCE] Custom query expectation and editing query.template_values metric (#7390) (thanks @mantasmy)
0.16.1
- [FEATURE] Fluent CRUD operation stubs (#7347)
- [FEATURE] Implement DataBricks (DBFS) DataConnector for Fluent Datasources needs (#7355)
- [BUGFIX] BigQuery performance uses updated
add_or_update_expectation_suite()
method (#7325) - [BUGFIX] Ensure Correct JSON and Dictionary Dump Output of Serialized Fluent Objects (#7336)
- [BUGFIX] Fluent Datasources - empty "assets" key on serialization (#7341)
- [BUGFIX] Use Path().glob Instead of Path(). Issue #7239 (#7327) (thanks @richardohara)
- [BUGFIX] Patch misc errors in advance of v0.16.1 release (#7371)
- [BUGFIX]
#ephemeral_data_asset
broken data docs links (#7367) - [BUGFIX] Fluent
batch_request_options
ignoreNone
values (#7368) - [BUGFIX] Add FluentBatchRequest early exit to convert_to_json_serializable (#7381)
- [DOCS] Adding ADR (#7314)
- [DOCS] Add Cloud onboarding tutorial (#7333)
- [DOCS] Fix onboarding cloud tutorial (#7348)
- [DOCS] SQL Alchemy 2 Warnigns (#7292)
- [DOCS] Update Cloud Getting Started Guide (#7364)
- [MAINTENANCE] In Fluent Datasources: Configuration Serialization Methods Do Not Need to be Public (#7332)
- [MAINTENANCE] Add crud methods to context.sources (#7328)
- [MAINTENANCE] Split out mysql and multi db docs integration tests into a separate job (#7331)
- [MAINTENANCE] [MAINTENANCE ] Include all
.pyi
files in package data (#7340) - [MAINTENANCE] Separate docs_integrations tests by backend (#7342)
- [MAINTENANCE] Parallelize integration tests - move test definitions (#7345)
- [MAINTENANCE] Update test around unneeded deprecation warning threshold (#7351)
- [MAINTENANCE] Remove deprecated code from
Validator
(#7353) - [MAINTENANCE] ruff 0.0.255 (#7349)
- [MAINTENANCE] add sqla deps to type-checking CI step (#7350)
- [MAINTENANCE] Remove deprecated code around
generator_asset
(#7356) - [MAINTENANCE] Revert accidental changes pushed to
develop
(#7363) - [MAINTENANCE] D/sql alchemy 2 warnings (#7362)
- [MAINTENANCE] Warnings are errors + Suppress warnings in preparation for supporting SQLAlchemy 2.x (#7352)
- [MAINTENANCE] Clean up miscellaneous deprecated code (#7358)
- [MAINTENANCE] Rename misc internal uses of
ge_cloud_id
(#7360) - [MAINTENANCE] Patch deprecation warning in
EVR.__eq__
(#7374) - [MAINTENANCE] remove imprecise wording in the new datasource notebook (#7369)
- [MAINTENANCE] Add additional warning ignores to pyproject.toml (#7375)
- [MAINTENANCE] Enable
snowflake
V3 Expectations Tests (#7370) - [MAINTENANCE] Temporarily disable warnings as errors for v0.16.1 release (#7382)
0.16.0
- [FEATURE] Provide S3 access/operations to new PandasDatasource (#7197)
- [FEATURE] F/great 1393/add more zep column splitters (#7203)
- [FEATURE] Fluent datasources have separate asset registries (#7193)
- [FEATURE] Improve Trino types support (#6588) (thanks @ms32035)
- [FEATURE] F/great 1393/add sql multi col splitter (#7236)
- [FEATURE]
EphemeralDataContext.convert_to_file_context()
(#7175) - [FEATURE] New Datasources: Support S3 access for SparkDatasource (#7232)
- [FEATURE] Implementation of Microsoft Azure Blob Storage DataConnector (#7246)
- [FEATURE] Implementation of Google Cloud Storage DataConnector (#7249)
- [FEATURE] Develop PandasGoogleCloudStorageDatasource (as part of New Datasources) (#7253)
- [FEATURE] Develop PandasAzureBlobStorageDatasource (as part of New Datasources) (#7254)
- [FEATURE] Develop SparkGoogleCloudStorageDatasource SparkAzureBlobStorageDatasource (as part of New Datasources) (#7255)
- [FEATURE] Enable creating Checkpoint and SimpleCheckpoint with Validator (#7275)
- [FEATURE] Editor support for Fluent Datasources dynamic
.add_<DS_TYPE>()
methods (#7273) - [FEATURE] M/great 1708/move fluent datasources out of experimental (#7294)
- [FEATURE]
PandasDatasource
in-memoryDataFrameAsset
(#7280) - [FEATURE] Fluent Datasources -
ConfigStr
config substitution support (#7309) - [BUGFIX] Ensure that Marshmallow UUIDs are converted to strings before usage (#7219)
- [BUGFIX]
docs-integration
tests failing after CRUD updates (#7220) - [BUGFIX] Adding exception logging to store-related failures (#7202) (thanks @ciguaran)
- [BUGFIX] Patch outdated Azure relative paths (#7247)
- [BUGFIX]
params_with_json_schema
is undefined inExpectSelectColumnValuesToBeUniqueWithinRecord._prescriptive_renderer
(#7261) - [BUGFIX] Patch Cloud workflow for
add_or_update_datasource
(#7242) - [BUGFIX] Stop modifying input batch request when getting a batch list. (#7269)
- [BUGFIX] Single dynamic pandas model failures don't result in all models failing when used (#7277)
- [BUGFIX] Fix typographical error in type used in "isinstance()" comparison in BatchManager (was generating "must be formal [...] Batch" warning erroneously) (#7291)
- [BUGFIX] Correct a typographical error in specific path-forming template name for SparkDatasource-Microsoft Azure Blob Storage combination (#7302)
- [BUGFIX] Use timezone-aware datetimes in checkpoint run information by default. (#7244)
- [BUGFIX]
row_conditions
now allow whitespace (#7313) - [BUGFIX] Scope
PandasDatasource
reader method types and serialization (1 of 2) (#7306) - [BUGFIX] AssertError Duing Great Expectations Installation (#7285) (thanks @richardohara)
- [BUGFIX] Provide Error Checking for "expectation_suite_name" in Checkpoint Constructor and Run Methods (#7320)
- [BUGFIX] Scope
PandasDatasource
reader method types and serialization (2 of 2) (#7318) - [BUGFIX] raise error if no identifier provided to get checkpoint/expectation suite (#7321)
- [BUGFIX] Fluent Datasources: Ensure that Data Source and DataAsset Model Serialization Does Not Include
"name"
Field (#7323) - [DOCS] Versioning instructions and fix references for versioning (#7221)
- [DOCS] Fix links to images in welcome page (#7222)
- [DOCS] Add API docs build to versioning instructions (#7230)
- [DOCS] Move readme_assets under docs/ (#7231)
- [DOCS] Fix hyperlinks in docs (#7264)
- [DOCS] Scan yaml examples for class names that should be part of the public API (#7267)
- [DOCS] Update location to run yarn commands (#7276)
- [DOCS] Fix reference to M1 Mac installation guide (#7279)
- [DOCS] Remove additional warnings from API docs build. (#7282)
- [DOCS] Fix blog links in
parse_strings_as_datetimes
warnings (#7288) - [DOCS] Fix unsupported chars in API docs (#7310)
- [DOCS] Add instructions to cli for creating data assets (#7326)
- [MAINTENANCE] No line number snippets checker (#7216)
- [MAINTENANCE] Re-enable snippet checker (#7217)
- [MAINTENANCE] Refactor New SparkDatasource in order to prepare for adding support for Cloud Files Storage environments (#7223)
- [MAINTENANCE] Repo Cleanup: Move docs related files under docs dir (#7227)
- [MAINTENANCE] Time series expectations (#7182)
- [MAINTENANCE] Contrib time series generators (#7235)
- [MAINTENANCE] ci & local type-checking parity - pyspark (#7237)
- [MAINTENANCE] Fix CI dev pipeline - skip orphaned schema check on min pandas version (#7241)
- [MAINTENANCE] Make a copy of azure pipeline definition files in new locations (#7233)
- [MAINTENANCE] Complete migration of Azure CI/CD config to
ci
directory (#7245) - [MAINTENANCE] ruff v0.0.253 - enable
PYI
rules (#7243) - [MAINTENANCE] Optionally allow
None
values inRenderedAtomicContent
(#7240) - [MAINTENANCE]
PandasDatasource
read_DataAsset
methods return aValidator
(#7226) - [MAINTENANCE] Rename default pandas ephemeral data asset (#7251)
- [MAINTENANCE] Fix linting errors in
contrib/
(#7259) - [MAINTENANCE] Keep repo root consistent (#7260)
- [MAINTENANCE] Threshold for public API report (#7262)
- [MAINTENANCE] Update the signature of Fluent datasource factory methods (#7266)
- [MAINTENANCE] Move some scripts into ci folder (#7271)
- [MAINTENANCE]
_PandasDataAsset
positional argument support (#7258) - [MAINTENANCE] Refactor to utilize common DataConnector instantiation and Connection Test Error Message templates for New FilePath Datasources (#7268)
- [MAINTENANCE] Merge
PandasDatasource
read method signatures with associated_PandasDataAsset
s (#7272) - [MAINTENANCE] Use ruff linting on scripts folder (#7270)
- [MAINTENANCE] Fix for docs build warnings (#7229)
- [MAINTENANCE] Add maybe imports (#7274)
- [MAINTENANCE] Proper mocking of AbstractDataContext for testing (#7281)
- [MAINTENANCE] Warn if sqlalchemy version is >= 2.0.0 (#7283)
- [MAINTENANCE] Remove Prefix "Batch" from "BatchSorter" and "BatchSortersDefinition" (in New Datasources) (#7284)
- [MAINTENANCE] Call the correct sqlalchemy method to eliminate deprecation warning (#7293)
- [MAINTENANCE] Fluent Datasources - initial snippet tests (#7278)
- [MAINTENANCE] add
ruff
PTH
use-pathlib
# noqa
comments (#7290) - [MAINTENANCE] Enable SparkDatasource add_asset() methods to accept "header" and "infer_schema" arguments; fix typos in docstrings. (#7296)
- [MAINTENANCE] enable
ruff
use-pathlib
PTH
rules (#7297) - [MAINTENANCE] Replace fluent data connector names with a constant (#7299)
- [MAINTENANCE] Apply ruff path rules for contrib (#7300)
- [MAINTENANCE] Rename column splitter to splitter. (#7303)
- [MAINTENANCE] Run airflow operator tests in GX pipeline (#7298)
- [MAINTENANCE] Add build docs to block PR pipeline (#7307)
- [MAINTENANCE] Linting for
tests/datasource
(#7308) - [MAINTENANCE] Fix test_build_docs stage (#7311)
- [MAINTENANCE] Elide Duplication of "name" Key from Fluent Data Source and DataAsset Configuration (#7312)
- [MAINTENANCE] Remove nonexistent pypi ref in README. Add WIP warning in README. (#7304)
- [MAINTENANCE] Fix a bug in the hourly generator. Add hourly functionality to genera… (#7305)
- [MAINTENANCE] Update allowed deprecation warning threshold in advance of
0.16.0
(#7317) - [MAINTENANCE] Fluent Data Source stubs for dynamic Pandas add_asset methods (#7315)
- [MAINTENANCE] Stub files for Fluent
PandasDatasource
andpy.typed
(#7322) - [MAINTENANCE]
row-condition
also includes tabs in conditional expectation (#7324) - [CONTRIB] -add athena credential support (#7186) (thanks @tmilitino)
- [CONTRIB] expect_multicolumn_sum_values_to_be_equal_to_single_column (#7224) (thanks @swittchawa)
- [CONTRIB] [BUGFIX] Update custom multi-column expectation logic and default kwargs (#7252) (thanks @yussaaa)
- [CONTRIB] Feature/expect column values to match thai (#7238) (thanks @swittchawa)
0.15.50
- [FEATURE] Utilize DataConnector in service of new DataAsset implementations (#7094)
- [FEATURE] F/great 1393/add initial non datetime sql splitters (#7183)
- [FEATURE] Experimental
PandasDatasource
with single-batch_PandasDataAsset
s (#7173) - [FEATURE] Experimental filesystem
DataAsset
spath
in batch request options and batch metadata (#7129) - [FEATURE] Default
PandasDatasource
(#7196) - [BUGFIX] : Allow CLI to work with
RuntimeDataConnector
(#7187) (thanks @luke321321) - [BUGFIX] Patch GX Cloud
validator.save_expectation_suite()
workflow (#7189) - [BUGFIX] Dynamic pandas asset model field substitution (#7212)
- [DOCS] Use named snippets part 5 (#7181)
- [DOCS] Use named snippets part 4 (#7176)
- [DOCS] Use named snippets part 6 (#7171)
- [DOCS] Use named snippets part 7 (#7192)
- [DOCS] Use named snippets part 9 (#7195)
- [DOCS] Use named snippets part 11 (#7201)
- [DOCS] Use named snippets part 8 (#7194)
- [DOCS] Use named snippets part 10 (#7199)
- [DOCS] Fix broken link in anonymous_usage_statistics.md (#7211) (thanks @Erin-GX)
- [DOCS] Use named snippets part 12 (#7214)
- [MAINTENANCE] Update changelog when updated release 0.15.49 (#7179)
- [MAINTENANCE] Raise store config
ClassInstantiationError
from originalDataContextError
(#7174) - [MAINTENANCE] New Datasources: Delineate "SparkFilesystemDatasource" (instead of "SparkDatasource") -- to be congruent with "PandasFilesystemDatasource" (#7178)
- [MAINTENANCE] Small Refactor to Enable PandasDatsource for multiple Storage Environments (#7190)
- [MAINTENANCE] Replace
regex
withbatching_regex
for fluent filesystem-like datasources (#7207) - [MAINTENANCE] Misc
DataContext
state management & API cleanup (#7215)
0.15.49
- [FEATURE] Enable customization of candidate Regular Expression patterns when running OnboardingDataAssistant (#7104)
- [FEATURE] Enable
gx.get_context()
to work without any inputs (#7074) - [FEATURE] Add
datasource
arg toDataContext
Datasource
CRUD (#7070) - [FEATURE] Update zep to use sqlalchemy_data_splitter.py (#7151)
- [FEATURE] ZEP - Dynamically define
add_<ASSET_TYPE>_asset()
methods if needed (#7121) - [FEATURE] add expectation_column_values_to_be_continuous (#5861) (thanks @jmoskovc)
- [BUGFIX] Rename experimental
get_batch_request
tobuild_batch_request
(#7107) - [BUGFIX] Remove version from versions.json (#7109)
- [BUGFIX] Properly Enable/Disable Spark Integration Tests Depending on pyspark Installation for New Datasources (#7132)
- [BUGFIX] Copy previous versions after checking out the current commit (#7142)
- [BUGFIX]
TupleAzureBlobStoreBackend
no longer gives warning when obfuscating connection string (#7139) - [BUGFIX] Patch inconsistent ordering within GCP test asserts (#7130)
- [BUGFIX] Parse pandas version correctly for development builds (#7147) (thanks @jtilly)
- [BUGFIX] Patch broken rendered content Cloud tests (#7155)
- [BUGFIX] pydantic>=1.10.4 - ImportError: cannot import name dataclass_transform (#7163)
- [BUGFIX] ID/PK Spark and Sql fall back when
unexpected_index_column_names
have not been defined (#7150) - [BUGFIX] Patch broken Cloud test blocking 0.15.49 release (#7177)
- [DOCS] Add CRUD API matrix to
AbstractDataContext
docstring (#7079) - [DOCS] Build API docs using latest released version (#7067)
- [DOCS] Add displayHTML method to view Data Docs (#7125) (thanks @swittchawa)
- [DOCS] Use named snippets part 1 (#7131)
- [DOCS] : fix capitalization of Slack (#7136) (thanks @JoelGritter)
- [DOCS] Remove sitemap.xml (#7141)
- [DOCS] doc-464 consolidating and standardizing snippets (#7154)
- [DOCS] Use named snippets part 2 (#7143)
- [DOCS] Use named snippets part 3 (#7169) (thanks @jmoskovc)
- [MAINTENANCE] Remove Extra Character from ID/PK Example README (#7098)
- [MAINTENANCE] Rename experimental
get_batch_request
tobuild_batch_request
(#7095) - [MAINTENANCE] Fix incorrect label on "How to configure a SQL Data Source" docs page (#7106)
- [MAINTENANCE] Update dependency on pydantic (#7111)
- [MAINTENANCE] Move experimental
base_directory
from_FilesystemDataAsset
toPandasDatasource
andSparkDatasource
(#7078) - [MAINTENANCE] Use secret to store algolia api key (#7115)
- [MAINTENANCE] Fluent Datasources - don't register "private" Data Source classes (#7124)
- [MAINTENANCE] ZEP - Realign pandas asset args for
Datasource
levelbase_directory
(#7123) - [MAINTENANCE] format notebooks with
black
(#7054) - [MAINTENANCE] mypy
v1.0.0
(#7138) - [MAINTENANCE] Output Consistent Data Format from "table.head" Metric for every ExecutionEngine (#7134)
- [MAINTENANCE] ruff 0.0.246 update (#7137)
- [MAINTENANCE] Refactor sql splitter to take selectable instead of str. (#7133)
- [MAINTENANCE] Update V3 DataConnector utilities to support New Datasources (ZEP) (#7144)
- [MAINTENANCE] Change all instances of
create_expectation_suite
toadd_expectation_suite
in tests, docs, and source code (#7117) - [MAINTENANCE] Clean up pathlib.Path() usage in DataConnector utilities and restore tighter formatting in great_expectations/util.py (#7149)
- [MAINTENANCE] Clean up
mypy
violations inCardinalityChecker
(#7146) - [MAINTENANCE] Remove unused dockerfile (#7152)
- [MAINTENANCE] Delete cli v012 tests. (#7159)
- [MAINTENANCE] ZEP - update asset factories method signatures from asset models (#7096)
- [MAINTENANCE] Bump minimum version of
pytest
(#7164) - [MAINTENANCE] Clean up additional deprecation warnings from outdated CRUD API (#7156)
- [MAINTENANCE] Experimental
PandasDatasource
, single-batch_PandasDataAsset
s, related schemas (#7158) - [MAINTENANCE] Removing path for
--v2-api upgrade
and informative message (#7170) - [CONTRIB] Add experimental expectation to check column values after split (#7120) (thanks @ace-racer)
- [CONTRIB] added new Expectations - India_zip_code expectation and not_to_be_future_date expectation (#6086) (thanks @prachijain136)
- [CONTRIB] Update the rendered text for min and max values to be clearer. (#7166)
0.15.48
- [FEATURE] Place FilesystemDataAsset into separate module (its functionality is used by both PandasDatasource and SparkDatasource) (#7025)
- [FEATURE] Add SQL query data asset for new experimental datasources (#6999)
- [FEATURE] Experimental
DataAsset
test_connection
(#7019) - [FEATURE] ZEP - generate pandas assets (#7044)
- [FEATURE] Experimental Splitter connection testing (#7051)
- [FEATURE] ID/PK
ColumnPairExpectations
andMultiColumnMapExpectations
- Spark (#7001) - [FEATURE] Add
expectation_suite
arg toDataContext
ExpectationSuite
CRUD (#7059) - [FEATURE] ID/PK ColumnPairExpectations and MultiColumnMapExpectations - SQL (#7046)
- [FEATURE] Introducing General-Purpose Wrapper for Regular Expressions Parsing and incorporating it in "_FilesystemDataAsset" (#7062)
- [FEATURE] Add
checkpoint
arg toDataContext
Checkpoint
CRUD (#7066) - [FEATURE] Add
profiler
arg toDataContext
Profiler
CRUD (#7060) - [FEATURE] Add API action (#6902) (thanks @itaise)
- [BUGFIX] ID/PK - Rendering
ColumnPair
andMultiColumn
Expectations in DataDocs (#7041) - [BUGFIX]
ColumnPairExpectation
tests need to consider 2 possible GroupBy results (#7045) - [BUGFIX] zep - always serialize type field (#7056)
- [BUGFIX] ZEP - html asset generation on pandas
1.1
(#7068) - [BUGFIX] fix ZEP pandas min tests (#7084)
- [BUGFIX] Skip all ZEP pandas datasource tests for min pandas (#7091)
- [DOCS] Add new
DataContext
CRUD to public API (#7058) - [DOCS] DOC-461 remove unlinked link (#7083)
- [DOCS] Adding algolia click events (#7085)
- [DOCS] Versioning for documentation (#7033)
- [MAINTENANCE] linting for
/contrib
(#7005) - [MAINTENANCE] Deprecate old
DataContext
CRUD methods (#7031) - [MAINTENANCE] Simplify logic for
add_or_update
(#7035) - [MAINTENANCE] ZEP - test type-checking (#7028)
- [MAINTENANCE] ZEP put schemas under source control (#6988)
- [MAINTENANCE] Add
id
as a param to any CRUD methods withge_cloud_id
(#7036) - [MAINTENANCE] Minor Cleanup (#7047)
- [MAINTENANCE] replace
isort
withruff
sorting rules (#6907) - [MAINTENANCE] Standardize
Checkpoint
CRUD (#6962) - [MAINTENANCE] ruff
0.0.241
(#7048) - [MAINTENANCE] finish linting
great_expectations
(#7050) - [MAINTENANCE] fix -
FutureWarning: pandas.Float64Index is deprecated
when importinggreat_expectations
(#7055) - [MAINTENANCE] Move path mapping out of ExecutionEngine into DataConnector (#7065)
- [MAINTENANCE] Add ruff TCH001 noqa annotations (#7072)
- [MAINTENANCE] Enable ruff
TCH
rules (#7073) - [MAINTENANCE] Cache dependency installation during build of
Dockerfile.tests
(#7071) - [MAINTENANCE] Fix most typing errors in
DataAssistantResult
(#7010) - [MAINTENANCE] add
ge_dev
+gx_dev
to.gitignore
(#6860) - [MAINTENANCE] Refactor Filesystem DataAsset into FilePath DataAsset and Filesystem DataAsset (later inherits former) (#7075)
- [MAINTENANCE] Refactor
Store
andStoreBackend
to leverage new CRUD methods (#7081) - [MAINTENANCE] Delete
gx_venv
from project root (#7080) - [MAINTENANCE] correct typo in data assistants portion of the getting started tutorial (#7088)
- [MAINTENANCE] ID/PK - CompoundColumnsUnique is filtered only for SQL (#7087)
- [MAINTENANCE] Minor clean up to make
DataConnector
method names less confusing. (#7089) - [MAINTENANCE] Refactor DataConnector Path Format Utilities For Better Encapsulation (#7092)
- [MAINTENANCE] Temporarily xfail ID/PK tests due to Pandas min version conflicts (#7097)
- [MAINTENANCE] Remove Extra Character from ID/PK Example README (#7098)
- [CONTRIB] Row condition parser sqlalchemy: adding support for != operator & adding support all operators for string (#7053) (thanks @maayaniti)
0.15.47
- [FEATURE] ZEP - dynamic pandas asset schema definitions (#6780)
- [FEATURE] ID/PK
ColumnPairExpectations
andMultiColumnMapExpectations
- Pandas (#6941) - [FEATURE] Experimental
Datasource
andDataAsset
connection testing (#6844) - [FEATURE] Implement Experimental SparkDatasource with CSVDataAsset (#6981)
- [FEATURE] Place FilesystemDataAsset into separate module (its functionality is used by both PandasDatasource and SparkDatasource) (#7025)
- [BUGFIX] Snowflake/Oracle/DB2 <--> SQLAlchemy table and column names case insensitivity representation (#6951)
- [BUGFIX] try except import of pandas types (#6983)
- [BUGFIX] fix jsonschema - altair conflict (#6984)
- [BUGFIX] Temporarily disable items with issues rendering (#6997)
- [BUGFIX] Fix Renderer Configuration for expectation expect_column_values_to_not_be_in_set #6963 (#6990) (thanks @jmcorreia)
- [BUGFIX] Patch logic error in new
add_or_update
methods (#7021) - [BUGFIX] Pandas ID/PK - bugfix for column name and update tests (#7015)
- [DOCS] Regex-Based, Set-Based, Query-Based, & Actions Docstrings (#6863)
- [DOCS] Documentation for classes and methods within ExecutionEngine class hierarchy (#6936)
- [DOCS] Enable use of code blocks in Returns: section (#6946)
- [DOCS] Add missing data connectors and data contexts (#6945)
- [DOCS] DOC-280: How to use GX with AWS S3 and Spark (#6782)
- [DOCS] Adding docstrings per the list (#6931)
- [DOCS] Docstrings for
DataContext
child classes andDataAssistantResult.to_json_dict
(#6956) - [DOCS] batch docstring (#6939)
- [DOCS] BatchDefinition (#6940)
- [DOCS] Add metric provider to public api report (#6958)
- [DOCS] BatchRequest (#6943)
- [DOCS] head (#6944)
- [DOCS] Add public_api. Docstring is fine already (#6955)
- [DOCS] Add public API docstring for
RuleBasedProfiler
(#6947) - [DOCS] Adding docstrings for metric providers (#6960)
- [DOCS] Add docstrings for several data connectors (#6949)
- [DOCS] Adds docstring to class configured data connector classes (#6961)
- [DOCS] Add public API docstring for
validate_configuration
onexpect_column_value_z_scores_to_be_less_than
andexpect_column_values_to_match_json_schema
(#6873) - [DOCS] Expectations Class DocStrings (#6950)
- [DOCS] D/dx 237/tal docstrings (#6959)
- [DOCS] Add delete_checkpoint to public API (#6965)
- [DOCS] Use markdown style code blocks (#6970)
- [DOCS] DataContext and CheckpointConfig DocString (#6911)
- [DOCS] Either Documentation tag style acceptable (#6974)
- [DOCS] DocStrings for Column, Query, & Table Metric Providers & register_metric (#6971)
- [DOCS] render utils (#6975)
- [DOCS] Add public API docstring for
UserConfigurableProfiler
(#6904) - [DOCS] Add docstring for
ExpectationValidationResult
(#6968) - [DOCS] Add some json serialization docstrings. (#6880)
- [DOCS] DOC-285 new guide: how to use self initializing expectations (#5205)
- [DOCS] DOC-286 how to add support for the auto initializing framework to a custom expectation (#5300)
- [DOCS] ExpectationConfiguration, get_success_kwargs and validate api docs (#6982)
- [DOCS] rule_based_profiler_result (#6977)
- [DOCS] metric_value, metric_partial (#6978)
- [DOCS] Actions, Checkpoint, ExpectationSuiteValidationResult, RunIdentifier related docstrings (#6986)
- [DOCS] API docs support self referential links (#6998)
- [DOCS] Add rendering docstrings (#6992)
- [DOCS]
Expectations
related DocStrings (#6994) - [DOCS] MetricConfiguration DocString (#6996)
- [DOCS] Updates typo in prerequisites section (#7004) (thanks @ruankie)
- [DOCS] Update API docs landing page (#6972)
- [DOCS] Remove BaseDataContext and DataContext from the public API (#7008)
- [DOCS] Fix setup instructions for email validation (#7007) (thanks @ruankie)
- [DOCS] DOC-348 corrects typos in the aws+athena guide intro and congratulations sections (#6989)
- [DOCS] DOC-420 updates to screenshots (#7012)
- [DOCS] DOC-416 How to use GX with AWS using Redshift (#6985)
- [DOCS] Fix metric provider and reorganize sidebar (#7022)
- [DOCS] Typo - Update api_reference.md (#7024)
- [DOCS] Nest sidebar by shortest import path (#7032)
- [MAINTENANCE] Parameterized tests for ID/PK at
ColumnMapExpectation
level (#6925) - [MAINTENANCE]
ruff
->0.0.236
(#6948) - [MAINTENANCE] docstring for expect_column_values_to_not_match_regex_list's validate_configuration (#6877)
- [MAINTENANCE] Remove handrolled linters/checkers from
scripts/
and CI (#6964) - [MAINTENANCE] Remove refs to old scripts from
invoke
calls (#6967) - [MAINTENANCE] Fix some linting issues (#6973)
- [MAINTENANCE] Fix variable name error associated with adding typing and docstrings (#6980)
- [MAINTENANCE] Add test to ensure that all types in the
DataContext
hierarchy emit expected usage stats (#6915) - [MAINTENANCE] Standardize
Datasource
CRUD (#6892) - [MAINTENANCE] Typing
histogram_single_batch_parameter_builder
(#6916) - [MAINTENANCE] Add
add_expectation_suite
toDataContext
CRUD (#6926) - [MAINTENANCE] ColumnExpectation, render_evaluation_parameter_string and validate method (#6995)
- [MAINTENANCE] Add
update_expectation_suite
andadd_or_update_expectation_suite
toDataContext
CRUD (#6987) - [MAINTENANCE] Standardize
RuleBasedProfiler
CRUD (#6991) - [MAINTENANCE] Make Pandas installation with Python 3.10 less restrictive (#7013)
- [MAINTENANCE] ZEP - postgres test typing (#7023)
- [MAINTENANCE] [BUGFIX ] ZEP - pandas serde fix (#7009)
- [MAINTENANCE] add preview image for twitter and other social preview images (#7027)
- [MAINTENANCE] Update
update_
methods inDataContext
to return persisted object (#7034) - [MAINTENANCE] ZEP - use parameter that exists on min pandas version (#7037)
- [MAINTENANCE] xfail ZEP async spark tests for release (#7038)
- [CONTRIB] expect_multicolumn_values_not_to_be_all_null (#6912) (thanks @yussaaa)
- [CONTRIB] Adding support for punctuation in column_value for the row_condition parser (#7018) (thanks @maayaniti)
0.15.46
- [BUGFIX] Disable
RendererConfiguration
constraint to support legacy renderer fallback behavior (#6938) - [DOCS] Remove the
great_expectations
path prefix for API docs (#6934) - [DOCS] Updates Custom Expectation docs w/ code snippets (#6365)
- [DOCS] Regex-Based, Set-Based, Query-Based, & Actions Docstrings (#6863)
- [DOCS] Documentation for classes and methods within ExecutionEngine class hierarchy (#6936)
0.15.45
- [FEATURE] Experimental datasources
batch.head()
(#6765) - [FEATURE] Add Validation Result URL to Checkpoint Result (#6908)
- [BUGFIX] Fix issues rendering code blocks in API docs (#6917)
- [BUGFIX] Fix list_keys method for TupleS3StoreBackend (#6901) (thanks @enagovitsyn)
- [BUGFIX] Fix rendering issue with api docs (#6924)
- [BUGFIX] Render bar graph with boolean values (#6910) (thanks @tmilitino)
- [BUGFIX] Capital one contrib/micdavis/import hotfix (#6922) (thanks @micdavis)
- [DOCS] Adding docstring for
Checkpoint.self_check()
(#6841) - [DOCS]
AbstractDataContext.add_store
docstring (#6851) - [DOCS] Doc Strings for ExpectationSuite Display Methods (#6856)
- [DOCS]
DataAssistantResult.get_expectation_suite()
docstring (#6862) - [DOCS] Misc docstrings around
DataAssistant
(#6866) - [DOCS] enable running
invoke docstrings
on select modules (#6868) - [DOCS] Adds docstring for expect_column_distinct_values_to_contain_set (#6855)
- [DOCS] Documentation Strings for Metric Domain Types and Metric Function Types (#6872)
- [DOCS] added docstrings for the public API (#6884)
- [DOCS] Add public API docstring for
expect_column_values_to_be_unique
validate_configuration
(#6897) - [DOCS] Miscellaneous docstrings for
DataContext
and utils (#6852) - [DOCS] Add public API docstring for
expect_column_values_to_be_of_type
validate_configuration
(#6896) - [DOCS] Add public API docstring for
JsonSchemaProflier.validate
(#6900) - [DOCS] Exclude DataAssistantRunner.run() (#6919)
- [DOCS] StoreValidationResultAction, StoreEvaluationParametersAction and StoreMetricsAction api docs (#6879)
- [DOCS] Add public API docstring for
expect_column_values_to_be_dateutil_parseable
validate_configuration
(#6864) - [DOCS] YAML docs (#6861)
- [DOCS] Add public API docstring for
expect_column_values_to_be_decreasing
validate_configuration
(#6865) - [DOCS] Docstrings for
Checkpoint
and related classes (#6882) - [DOCS] Add public API docstring for expect_table_row_count_to_be_between validate_configuration (#6883) (thanks @lockettks)
- [DOCS]
Validator.get_expectation_suite()
docstring (#6886) - [DOCS] Fix Checkpoint docstring whitespace (#6927)
- [DOCS]
DataAssistantResult
docstring (#6887) - [DOCS] Add public API docstring for
expect_column_values_to_be_in_set
validate_configuration
(#6890) - [DOCS] Add public API docstring for
expect_column_values_to_be_in_type_list
andexpect_column_values_to_be_increasing
validate_configuration
(#6891) - [DOCS] Deprecate
util.render_evaluation_parameter_string
function (#6894) - [DOCS] Add public API docstring for
Profiler.validate
(#6898) - [DOCS] Add public API docstring for
expect_column_values_to_be_between
validate_configuration
(#6858) - [DOCS] Add public API docstring for
expect_column_values_to_be_in_json_parseable
validate_configuration
(#6893) - [DOCS] Add public API docstring for
expect_column_values_to_be_null
validate_configuration
(#6895) - [DOCS] Update docstrings for some of actions.py (#6853)
- [DOCS] /typo correction (#6920) (thanks @mingyyy)
- [DOCS] DOC-417 How to use GX with AWS using Athena (#6828)
- [DOCS] Adding docstrings (#6854)
- [MAINTENANCE] Update
teams.yml
(#6839) - [MAINTENANCE] invoke 2.0 and
schema
task (for zep types) (#6836) - [MAINTENANCE] Build hierarchy in sidebars for API docs (#6842)
- [MAINTENANCE] Change public_api task name to avoid confusion (#6843)
- [MAINTENANCE] Add the fragment back to internal references (#6845)
- [MAINTENANCE] Clean up public_api excludes (#6846)
- [MAINTENANCE] Fix the error message for invalid batch request options (#6848)
- [MAINTENANCE] Standardize Store CRUD (#6826)
- [MAINTENANCE] Fix scripts not found error in invoke (#6867)
- [MAINTENANCE] Fix argument name typo (#6850) (thanks @KirillUlich)
- [MAINTENANCE] more clearly specifies range of supported python versions (#6870)
- [MAINTENANCE] add validate_configuration docstring (#6857)
- [MAINTENANCE] docstring for expect_column_values_to_not_be_null#validate_configuration (#6859)
- [MAINTENANCE] Standardize project config CRUD (#6837)
- [MAINTENANCE] update docstring in validator.py and checkpoint_result.py (#6875)
- [MAINTENANCE] updated docstring on validate configuration (#6871)
- [MAINTENANCE] Exclude unit tests from
comprehensive
stage ofdev
CI (#6903) - [MAINTENANCE] Refactor
file_relative_path
util (#6778) - [MAINTENANCE] switch to
ruff
linter (#6888) - [MAINTENANCE] Use docusaurus style code block in api docs (#6906)
- [MAINTENANCE] metrics linting (#6889)
- [MAINTENANCE] Add exception message to
RenderedAtomicContent
failure renderer (#6795) - [MAINTENANCE] Remove CloudNotificationAction (#6881)
- [MAINTENANCE] Use ruff linter for docstring linting (#6913)
- [MAINTENANCE] Add validate_configuration method docstrings (#6899)
- [MAINTENANCE] docstring for expect_column_values_to_not_match_like_pattern_list's validate_configuration (#6874)
- [MAINTENANCE] docstring for expect_column_values_to_not_match_like_pattern validate_configuration (#6876)
- [MAINTENANCE] docstring for expect_compound_columns_to_be_unique validate_configuration (#6878)
- [MAINTENANCE] Add docstrings for Validator and its save_expectation_suite and validate methods (#6885)
- [MAINTENANCE] Type Hints Correction in New Datasources; Additional DocStrings (#6918)
0.15.44
- [FEATURE] Add pandas datasource sorter by refactoring into DataAsset (#6787)
- [FEATURE] ID/PK Demo Files (#6833)
- [BUGFIX] Fix missing not operator ~ (#6808)
- [BUGFIX] Implemented lowercase function to check what type of file endswith (#6810) (thanks @tmilitino)
- [BUGFIX] : expect_day_count_to_be_close_to_equivalent_week_day_mean (#6811) (thanks @HadasManor)
- [BUGFIX] Pandas ID/PK query was causing DataDocs error (#6832)
- [DOCS] Link to gh issue #4152 for ruamel.yaml (#6799) (thanks @jamesmyatt)
- [DOCS] ExpectationSuite and remove_expectation api docs (#6785)
- [DOCS] Add GitHub PR links to changelogs (#6818)
- [DOCS] Update
yarn-snippet-check
to only target specific source code dirs (#6825) - [DOCS] Adding docstring for ExpectationSuite.add_expectation (#6829)
- [DOCS] DOC-394: Fix broken redirect links (#6835)
- [MAINTENANCE] Enable more backends for some contrib expectations (#6775)
- [MAINTENANCE] Change execution_engine_type from method to property. (#6788)
- [MAINTENANCE] More backends for expect_yesterday_count_compared_to_avg_equivalent_days_of_week (#6790)
- [MAINTENANCE] Update gallery pipeline to only have one scheduled run per day (early AM) (#6791)
- [MAINTENANCE] Convert the validation results to JSON serializable (#6776) (thanks @lu-lz)
- [MAINTENANCE] Propagate "runtime_configuration" argument throughout Validator flow (#6767)
- [MAINTENANCE] Only include relevant diagnostics info in gallery JSON (#6797)
- [MAINTENANCE] Clean up public api report part 1 (#6784)
- [MAINTENANCE] Clean up public api report part 2 (#6792)
- [MAINTENANCE] Shift daily gallery run by 6 hours (#6802)
- [MAINTENANCE] Misc docstrings in
AbstractDataContext
(#6801) - [MAINTENANCE] Add checkpoint and datadoc integration test for zep pandas datasource. (#6793)
- [MAINTENANCE] Use environment variables for expectation gallery data paths (#6805)
- [MAINTENANCE] Suppress 2 kl_divergence datasets for bigquery that took 90 minutes to insert (#6807)
- [MAINTENANCE] Improve type hints in ExecutionEngine.resolve_metrics() flow and delete unnecessary checks (#6804)
- [MAINTENANCE] Fixes for column_values_to_be_between tests (#6809)
- [MAINTENANCE] Clean up public api report part 3 (#6803)
- [MAINTENANCE] Add docstring for
AbstractDataContext.add_checkpoint
(#6728) - [MAINTENANCE] Use Enum classes for all metric name suffixes (#6819)
- [MAINTENANCE] Use shortened_dotted_paths in API docs (#6820)
- [MAINTENANCE] Update batch request option validation error message (#6821)
- [MAINTENANCE] Add docstring to DataAsset.add_sorters (#6822)
- [MAINTENANCE] Misc type cleanup within
checkpoint/
andvalidator/
(#6817) - [MAINTENANCE] Update algolia indexing (#6827)
- [MAINTENANCE] When running our test suite, suppress warnings
result_format
configuration in Expectations and Validators (#6823) - [MAINTENANCE] ZEP - lower logging levels from
INFO
->DEBUG
(#6830) - [MAINTENANCE] Use shortened dotted paths in api docs (#6831)
- [MAINTENANCE] Remove outdated refs to Superconductive (#6816)
- [CONTRIB] Improve contrib schwifty expectations (#6812) (thanks @mkopec87)
0.15.43
- [FEATURE] ZEP - Synchronize & save XDatasources (#6717)
- [FEATURE] Official Python 3.10 support (#6763)
- [FEATURE] F/great 1313/zep pandas poc (#6745)
- [FEATURE] Add GX Cloud hyperlink to slack notification (#6740)
- [FEATURE] Get ExpectationSuite, Checkpoint by name (#6774)
- [FEATURE] API docs (#6766)
- [BUGFIX] - Implementing deep copy of runtime_configuration variable (#6682) (thanks @tmilitino)
- [BUGFIX] Patch broken
test_data_context_ge_cloud_mode_with_incomplete_cloud_config_should_throw_error
(#6741) - [BUGFIX] reformatting
setup.py
(#6756) - [BUGFIX] Fix observed value (#6759) (thanks @itaise)
- [BUGFIX] fix comment stripping when saving a zep configuration (#6783)
- [DOCS] DOC-414: Remove guide for use of outdated docker images (#6718)
- [DOCS] Convert docs snippets to named snippets (#6735)
- [DOCS] Update documentation to reference
get_context
(#6738) - [DOCS] Convert remaining snippets to named snippets (#6736)
- [DOCS] Convert line number references to named references in docs (#6748)
- [DOCS] Doc-280 AWS golden path with S3 cloud storage and Pandas (#6618)
- [DOCS] Preparation for building api docs (#6737)
- [DOCS] Change prefix reference for tutorial folder/directory (#6751) (thanks @medeirosthiago)
- [DOCS] Fix line-links in 4th step's 5th and 6th block (#6752) (thanks @OnkarMadli)
- [DOCS] - fixed code reference in documentation (#6732) (thanks @tmilitino)
- [DOCS] validator.head docstring (#6762)
- [MAINTENANCE] Update docstrings for experimental SQL datasources. (#6714)
- [MAINTENANCE] update
cli
DataContext
types (#6703) - [MAINTENANCE] Fix missing exclamation marks in API docs admonitions (#6721)
- [MAINTENANCE] ID/PK Tests at Expectations-level with Warnings caught (#6713)
- [MAINTENANCE] Refactor tests to leverage
get_context
instead ofBaseDataContext
(#6720) - [MAINTENANCE] Update remaining atomic prescriptive templates (1 of 2) (#6696)
- [MAINTENANCE] Refactor tests to leverage
get_context
instead ofDataContext
(#6723) - [MAINTENANCE] Update remaining atomic prescriptive templates (2 of 2) (#6724)
- [MAINTENANCE]
execution_engine
typing (#6730) - [MAINTENANCE]
core/expectation_
type checking (#6731) - [MAINTENANCE] Remove printing of entire snippet map in
remark-named-snippet
hook (#6749) - [MAINTENANCE] Rename all instances of
ge_exceptions
togx_exceptions
(#6742) - [MAINTENANCE] Remove
base_data_context
mark in tests (#6750) - [MAINTENANCE] Consolidate different Metric Types definition Enums (#6746)
- [MAINTENANCE] exclude scripts directory from package (#6744) (thanks @cburroughs)
- [MAINTENANCE] Force cryptography version installed where snowflake runs to be v38.0.4 (#6755)
- [MAINTENANCE] ID/PK - Adding semi-colon to SQL Query output (#6743)
- [MAINTENANCE] ID/PK result_format documentation update (#6716)
- [MAINTENANCE] Consistent use of Metric Name Enum values (#6757)
- [MAINTENANCE] Fix min version test requirements install on azure async pipeline. (#6753)
- [MAINTENANCE] expectations linting & bug fixes (#6739)
- [MAINTENANCE] Add Python 3.10 to async Azure pipeline (#6760)
- [MAINTENANCE] Unset cloud vars when running pytest from a mac (#6747)
- [MAINTENANCE] Update CheckChanges for dev pipeline (#6768)
- [MAINTENANCE] Update contrib pipeline to track package changes (#6770)
- [MAINTENANCE] Update docstring for
AbstractDataContext.add_batch_kwargs_generator
(#6727) - [MAINTENANCE] Update docstring for add_datasource (#6729)
- [MAINTENANCE] Remove chunk of code in build_gallery.py to skip processing some Expectations (#6772)
- [MAINTENANCE] Update docstring for
AbstractDataContext.build_data_docs
(#6726) - [MAINTENANCE] Move
constraints-test
dir withinazure
dir to clean up project root (#6764) - [MAINTENANCE] Misc cleanup of
SerializableDataContext
(#6777) - [MAINTENANCE] Removed some 30 type hint violations (#6771)
- [MAINTENANCE] Adjust gallery schedule and timeout (#6781)
- [MAINTENANCE] type-checking -
metrics/util.py
(#6754) - [MAINTENANCE] Update
requirements.txt
to reflect 3.10 support (#6786) - [MAINTENANCE] Enable more backends for some contrib expectations (#6775)
- [CONTRIB] added condition to ExpectQueriedColumnListToBeUnique (#6702) (thanks @maayaniti)
- [CONTRIB] Implement Spark backend for several expectations (#6683) (thanks @mkopec87)
- [CONTRIB] Improve Spark backend support for contrib query based expectations (#6733) (thanks @mkopec87)
- [CONTRIB] Refactor ExpectColumnValuesToBeHexadecimal expectation to be RegexBased (#6734) (thanks @mkopec87)
- [CONTRIB] Fix regex based expectations for spark (#6725) (thanks @mkopec87)
0.15.42
- [FEATURE] ZEP - PG
BatchSorter
loading + dumping (#6580) - [FEATURE] Ensure
result_format
accessed is through Checkpoint, and warns users ifExpectation
orValidator
-level (#6562) - [FEATURE] ZEP - PG
SqlYearMonthSplitter
serialization + deserialization (#6595) - [FEATURE] ID/PK -
unexpected_index_list
updated to include actual unexpected value, and EVR to includeunexpected_index_column_names
(#6586) - [FEATURE] Atomic rendering of Evaluation Parameters (#6232)
- [FEATURE] Add zep datasource data assistant e2e tests. (#6612)
- [FEATURE] F/great 1400/sql datasource (#6623)
- [FEATURE] Accept a
pathlib.Path
context_root_dir
(#6613) - [FEATURE] Atomic rendering of meta notes (#6627)
- [FEATURE] Docstring linter for public api (#6638)
- [FEATURE] ZEP - Load/dump new style config from DataContext (#6631)
- [FEATURE] ID/PK Rendering in DataDocs (#6637)
- [FEATURE] Use docstring linter for public api to catch missing parameters (#6642)
- [FEATURE] ZEP - load from shared config (#6655)
- [FEATURE] Added new expectation: ExpectYesterdayCountComparedToAvgEquivalentDaysOfWeek… (#6622) (thanks @HadasManor)
- [FEATURE] Add sqlite datasource (#6657)
- [FEATURE] ExpectDaySumToBeCloseToEquivalentWeekDayMean (#6664) (thanks @HadasManor)
- [FEATURE] ID/PK flag
return_unexpected_index_query
that allowsQUERY
output to be suppressed (#6660) - [FEATURE] F/great 1400/postgres constr (#6674)
- [FEATURE] Ensure Pandas ID/PK can use named indices (#6656)
- [FEATURE] Support to include ID/PK in validation result for each row - Spark (#6676)
- [FEATURE] ID/PK Pandas query returned as all unexpected indices (#6692)
- [BUGFIX] Support non-string
datetime
evaluation parameters (#6571) - [BUGFIX] Use v3.3.6 or higher of google-cloud-bigquery (with shapely bugfix) (#6590)
- [BUGFIX] Remove rendered header from Cloud-rendering tests (#6597)
- [BUGFIX] Patch broken
Validator
test afterDataContext
refactor (#6605) - [BUGFIX] Column value counts metric
dtype
inference due tonumpy
deprecation ofobject
dtype
(#6604) - [BUGFIX]
delete_datasource()
was getting passed in incorrect parameter for Spark Docs test (#6601) - [BUGFIX] Stop overwriting query with static string in RuntimeBatchRequests for SQL (#6614)
- [BUGFIX] Simplify metric results processing and improve detection of Decimal types in columns (#6610)
- [BUGFIX] Do not round output of proportion computing metrics (#6619)
- [BUGFIX] Add sqrt on connect to sqlite database. (#6635)
- [BUGFIX]
RendererConfiguration
min_value
andmax_value
withdatetime
s (#6632) - [BUGFIX] Fix a typo in contrib queried expectation (and add type hints to its implementation) (#6650)
- [BUGFIX] Avoid key collisions between Rule-Based Profiler terminal literals and MetricConfiguration value keys (#6675)
- [BUGFIX] Add connect args to execution engine schema (#6663) (thanks @itaise)
- [BUGFIX] Increase minimum
numpy
versions for Python 3.8 and 3.9 due to support in latest release ofscipy
(#6704) - [BUGFIX] Format current branch name properly for tag branches in docker tests (#6711)
- [DOCS] Patch broken snippet around Validation Actions (#6606)
- [DOCS] Fix formatting of 0.15.35 and 0.15.36 in changelogs (#6603)
- [DOCS] Convert broken snippet to named snippets (#6611)
- [DOCS] - add anonymous_usage_statistics configutation in documentation (#6626) (thanks @tmilitino)
- [DOCS] fixing wrong line reference on docs (#6599) (thanks @wagneralbjr)
- [DOCS] Sidebar changes for Integrations and how-tos (#6649)
- [DOCS] edit term(data_conext, checkpoints)-link in with airflow (#6646) (thanks @jx2lee)
- [DOCS] DOC-400 remove an outdated video link (#6667)
- [DOCS] doc-356 importable prerequisite box with correct theme applied and default entries (#6666)
- [MAINTENANCE]
mypy
config update (#6589) - [MAINTENANCE] Small refactor of ExecutionEngine.resolve_metrics() for better code readability (and miscellaneous additional clean up) (#6587)
- [MAINTENANCE] Remove
ExplorerDataContext
(#6592) - [MAINTENANCE] Leverage
RendererConfiguration
in existing prescriptive templates (2 of 3) (#6488) - [MAINTENANCE] Leverage
RendererConfiguration
in existing prescriptive templates (3 of 3) (#6530) - [MAINTENANCE] Add docs snippet checker to
dev
CI (#6594) - [MAINTENANCE] Utilize a
StrEnum
forConfigPeer
modes (#6596) - [MAINTENANCE] Refactor
BaseDataContext
andDataContext
into factory functions (#6531) - [MAINTENANCE] flake8 coverage -
render
,juptyer_ux
,checkpoint
sub-packages (#6607) - [MAINTENANCE] filter
RemovedInMarshmallow4
warnings (#6602) - [MAINTENANCE] Generate public API candidates (#6600)
- [MAINTENANCE] partial
cli
+usage_stats
typing (#6335) - [MAINTENANCE] update pip installs in pipelines (#6609)
- [MAINTENANCE] create cache as part of
--ci
type-checking step (#6621) - [MAINTENANCE] Add
row_condition
logic toRendererConfiguration
and remove from atomic renderer implementations (#6616) - [MAINTENANCE] install
pydantic
in develop pipeline (#6624) - [MAINTENANCE] Fix develop static type-check stage (#6628)
- [MAINTENANCE] Unexpected Counts table in DataDocs able to show counts for sampled values (#6634)
- [MAINTENANCE] CI - install from
requirements-types.txt
(#6639) - [MAINTENANCE] Add docstring linter for public api to CI (#6641)
- [MAINTENANCE] Batch ID must incorporate batch_spec_passthrough. Instantiate Validator with DataContext in test modules. Query metrics/expectations types cleanup. (#6636)
- [MAINTENANCE] Skip postgres tests in spark. (#6643)
- [MAINTENANCE] Enrich
RendererConfiguration
primitive types (#6629) - [MAINTENANCE] M/great 1225/async builds branch (#6644)
- [MAINTENANCE] Comment out calling _disable_progress_bars in build_gallery.py (#6648)
- [MAINTENANCE] Update generate_expectation_tests func to only log an ERROR message if there is an error (#6651)
- [MAINTENANCE] Use the correct test for positive_test__exact_min_and_max on trino (#6652)
- [MAINTENANCE] Update evaluate_json_test_v3_api func to use the debug_logger with useful info when exception occurs (#6653)
- [MAINTENANCE] Make only column y data_alt different in col_pair_equal tests (#6661)
- [MAINTENANCE] Add separate tests for exact stdev for redshift and snowflake (#6658)
- [MAINTENANCE] Add redshift/snowflake casting for sample data on expect_column_values_to_be_of_type (#6659)
- [MAINTENANCE] Add redshift/snowflake casting for sample data on expect_column_values_to_be_in_type_list (#6668)
- [MAINTENANCE] Make only column y data_alt different in col_pair_in_set tests (#6670)
- [MAINTENANCE] Only spark v2 for special test in not_be_in_set (#6669)
- [MAINTENANCE] Don't attempt any regex Expectation tests with snowflake (#6672)
- [MAINTENANCE] Clean up returns style and type hints in CardinalityChecker utility (#6677)
- [MAINTENANCE] begin flake8 linting on tests (#6679)
- [MAINTENANCE] Clean up packaging & installation pipeline (#6687)
- [MAINTENANCE] Misc updates to dev Azure pipeline (#6686)
- [MAINTENANCE]
mypy
typing forcore/util.py
(#6617) - [MAINTENANCE] Update
get_context
return type (#6684) - [MAINTENANCE] Get datetime tests working for trino/snowflake/spark in values_to_be_in_set (#6671)
- [MAINTENANCE] Cleanup typing errors. (#6691)
- [MAINTENANCE] Remove unused Metric and BatchMetric classes and consolidate ValidationMetricIdentifier with other identifiers (#6693)
- [MAINTENANCE] Refactor
BaseDataContext
to leverageget_context
(#6689) - [MAINTENANCE] expect day count to be close to equivalent week day mean (#6680) (thanks @HadasManor)
- [MAINTENANCE] ID/PK squashed tests re-added (#6694)
- [MAINTENANCE] initial type checking for
rules_based_profiler
(#6681) - [MAINTENANCE] Improve type checking for Expectations with atomic renderers leveraging
RendererConfiguration
(#6633) - [MAINTENANCE] Add deprecated Cloud variables to
_CloudConfigurationProvider.get_values()
output (#6708) - [MAINTENANCE] Autobuild markdown stubs (#6700)
- [MAINTENANCE] API docs styling (#6712)
0.15.41
- [FEATURE] enable mostly for expect_compound_columns_to_be_unique (#6533) (thanks @kimfrie)
- [BUGFIX] Return unique list of batch_definitions (#6579) (thanks @tanelk)
- [BUGFIX] convert_to_json_serializable does not accept numpy datetime (#6553) (thanks @tmilitino)
- [DOCS] Clean up misc snippet violations (#6582)
- [MAINTENANCE] Update json schema validation on usage stats to filter based on format. (#6502)
- [MAINTENANCE] Renaming Metric Name Suffixes using Enum Values for Better Code Readability (#6575)
- [MAINTENANCE] M/great 1433/cloud tests to async (#6543)
- [MAINTENANCE] Add static type checking to
rule.py
andrule_based_profiler_result.py
(#6573) - [MAINTENANCE] Update most contrib Expectation docstrings to be consistent and decently formatted for gallery (#6577)
- [MAINTENANCE] Update changelogs to reflect PyPI yanks (0.15.37-0.15.39) (#6581)
- [MAINTENANCE] Refactor ExecutionEngine.resolve_metrics() for better code readability (#6578)
- [MAINTENANCE] adding googletag manager to docusaurus (#6584)
- [MAINTENANCE] typo in method name (#6585)
0.15.40
- [FEATURE] F/great 1397/zep checkpoints (#6525)
- [FEATURE] Add integration test for zep sqlalchemy datasource with renderering. (#6564)
- [BUGFIX] Patch additional deprecated call to
GXCloudIdentifier.ge_cloud_id
attr (#6555) - [BUGFIX] Patch
packaging_and_installation
Azure pipeline test failures (#6559) - [BUGFIX] Fix dependency issues to reenable RTD builds (#6560)
- [BUGFIX] Add missing
raise
statement inRuntimeDataConnector
logic (#6569) - [DOCS] doc 383: bring sql datasource configuration examples under test (#6466)
- [MAINTENANCE] Add error handling to docs snippet checker (#6556)
- [MAINTENANCE] ID/PK tests at
Checkpoint
-level (#6539) - [MAINTENANCE] Improve DataAssistant Parameter Builder Naming/Sanitization Mechanism and Enhance TableDomainBuilder (#6554)
- [MAINTENANCE] Simplify computational graph assembly from metric configurations (#6563)
- [MAINTENANCE] RTD Mobile header brand adjustment (#6557)
- [MAINTENANCE] Use MetricsCalculator methods for ValidationGraph construction and resolution operations in Validator (#6566)
- [MAINTENANCE] Cast type in
execution_environment.py
to bypass flakymypy
warnings (#6572) - [MAINTENANCE] Additional patch for
mypy
issue inexecution_environment.py
(#6574) - [MAINTENANCE] Clean up GX rename artifacts (#6561)
- [CONTRIB] fix observed value in custom expectation (#6515) (thanks @itaise)
0.15.39 - YANKED
- [BUGFIX] Patch faulty GX Cloud arg resolution logic (#6542)
- [BUGFIX] Fix resolution of cloud variables. (#6546)
- [DOCS] Fix line numbers in snippets part 2 (#6537)
- [DOCS] Convert nested snippets to named snippets (#6541)
- [DOCS] Simplify snippet checker logic to catch stray tags in CI (#6538)
- [MAINTENANCE] v2 Docs link (#6534)
- [MAINTENANCE] Fix logic around cloud_mode and ge_cloud_mode. (#6550)
0.15.38 - YANKED
- [BUGFIX] Patch broken Cloud E2E test around
Datasource
CRUD (#6520) - [BUGFIX] Patch outdated
ge_cloud_id
attribute call inValidationOperator
(#6529) - [BUGFIX] Revert refactor to
Datasource
instantiation logic inDataContext
(#6535) - [BUGFIX] Patch faulty GX Cloud arg resolution logic (#6542)
- [DOCS] Fix line numbers in snippets (#6536)
- [DOCS] Fix line numbers in snippets part 2 (#6537)
- [DOCS] Convert nested snippets to named snippets (#6541)
- [MAINTENANCE] Update Data Assistant plot images (#6521)
- [MAINTENANCE] Clean up type hints and make test generation more elegant (#6523)
- [MAINTENANCE] Clean up
Datasource
instantiation logic inDataContext
(#6517) - [MAINTENANCE] Update
Domain
computation in MetricConfiguration (#6528) - [MAINTENANCE] v2 Docs link (#6534)
0.15.37 - YANKED
- [FEATURE] Support to include ID/PK in validation result for each row - SQL (#6448)
- [FEATURE] Build process and example API docs (part 1) (#6474)
- [FEATURE] Add temp_table_schema_name support for BigQuery (#6303) (thanks @BobbyRyterski)
- [FEATURE] Decorators for API docs (part 2) (#6497)
- [FEATURE] Decorators for API docs (part 3) (#6504)
- [BUGFIX] Support slack channel name with webhook also (#6481) (thanks @Kozehh)
- [BUGFIX] Airflow operator package conflict for
jsonschema
(#6495) - [BUGFIX] Validator uses proper arguments to show progress bar at Metrics resolution-level (#6510) (thanks @tommy-watts-depop)
- [DOCS] Schedule Algolia Crawler daily at midnight (#6323)
- [DOCS] fix(gh-6512): fix rendering of Batch definition (#6513) (thanks @JoelGritter)
- [MAINTENANCE] Add pretty representations for zep pydantic models (#6472)
- [MAINTENANCE] Misc updates to PR template (#6479)
- [MAINTENANCE] Minor cleanup for better code readability (#6478)
- [MAINTENANCE] Move zep method from datasource to data asset. (#6477)
- [MAINTENANCE] Staging for build gallery (#6480)
- [MAINTENANCE] Reformat core expectation docstrings (#6423)
- [MAINTENANCE] Move "Domain" to "great_expectations/core" to avoid circular imports; also add MetricConfiguration tests; and other clean up. (#6484)
- [MAINTENANCE] Query the database for datetime column splitter defaults (#6482)
- [MAINTENANCE] Placing metrics test db under try-except (#6489)
- [MAINTENANCE] Clean up tests for more formal Batch and Validator instantiation (#6491)
- [MAINTENANCE] Rename
ge
togx
across the codebase (#6487) - [MAINTENANCE] Upgrade CodeSee workflow to version 2 (#6498) (thanks @codesee-maps[bot])
- [MAINTENANCE] Rename
GE
toGX
across codebase (GREAT-1352) (#6494) - [MAINTENANCE] Resolve
mypy
issues incli/docs.py
(#6500) - [MAINTENANCE] Increase timeout to 15 minutes for the 2 jobs in manual-staging-json-to-prod pipeline (#6509)
- [MAINTENANCE] Update Data Assistant plot color scheme and fonts (#6496)
- [MAINTENANCE] Update
RendererConfiguration
topydantic
model (#6452) - [MAINTENANCE] Message for how to install Great Expectations in Cloud Composer by pinning packages (#6492)
- [MAINTENANCE] Leverage
RendererConfiguration
in existing prescriptive templates (1 of 3) (#6460) - [MAINTENANCE] Clean up
teams.yml
(#6511) - [MAINTENANCE] Make Generated Integration Tests More Robust Using BatchDefinition and InMemoryDataContext In Validator and ExecutionEngine Instantiation (#6505)
- [MAINTENANCE] DO NOT MERGE UNTIL DEC 8: [MAINTENANCE] Brand changes in docs (#6427)
- [MAINTENANCE] fixed typo in nav (#6518)
- [MAINTENANCE] Clean up GX Cloud environment variable usage (GREAT-1352) (#6501)
- [MAINTENANCE] Update Data Assistant plot images (#6521)
- [CONTRIB] Add uniqueness expectation (#6473) (thanks @itaise)
0.15.36
- [BUGFIX] Contrib Expectation tracebacks (#6471)
- [BUGFIX] Add additional error checking to
ExpectationAnonymizer
(#6467) - [MAINTENANCE] Add docstring for context.sources.add_postgres (#6459)
- [MAINTENANCE] fixing type hints in metrics utils module (#6469)
- [MAINTENANCE] Moving tutorials to great-expectations repo (#6464)
0.15.35
- [FEATURE] add multiple input metric (#6373) (thanks @CarstenFrommhold)
- [FEATURE] add multiple column metric (#6372) (thanks @CarstenFrommhold)
- [FEATURE]: DataProfilerUnstructuredDataAssistant Integration (#6400) (thanks @micdavis)
- [FEATURE] add new metric - query template values (#5994) (thanks @itaise)
- [FEATURE] ZEP Config serialize as YAML (#6398)
- [BUGFIX] Patch issue with call to
ExpectationAnonymizer
to ensureDataContext
init events are captured (#6458) - [BUGFIX] Support Table and Column Names Case Non-Sensitivity Relationship Between Snowflake, Oracle, DB2, etc. DBMSs (Upper Case) and SQLAlchemy (Lower Case) Representations (#6450)
- [BUGFIX] Metrics return value no longer returns None for
unexpected_index_list
- Sql and Spark (#6392) - [BUGFIX] Fix for
mssql
tests that depend ondatetime
tostring
conversion (#6449) - [BUGFIX] issue-4295-fix-issue (#6164) (thanks @YevgeniyaLee)
- [BUGFIX] updated capitalone setup.py file (#6410) (thanks @micdavis)
- [BUGFIX] Patch key-generation issue with
DataContext.save_profiler()
(#6405) - [DOCS] add configuration of anonymous_usage_statistics for documentation (#6293) (thanks @milithino)
- [DOCS] add boto3 explanations on document (#6407) (thanks @tiruka)
- [MAINTENANCE] [CONTRIB] Multicolumns sum equal to single column (#6446) (thanks @asafla)
- [MAINTENANCE] [CONTRIB] add expectation - check gaps in SCD tables (#6433) (thanks @itaise)
- [MAINTENANCE] [CONTRIB] Add no days missing expectation (#6432) (thanks @itaise)
- [MAINTENANCE] [CONTRIB] Feature/add two tables expectation (#6429) (thanks @itaise)
- [MAINTENANCE] [CONTRIB] Add number of unique values expectation (#6425) (thanks @itaise)
- [MAINTENANCE] Add sorters to zep postgres datasource. (#6456)
- [MAINTENANCE] Bump ubuntu version in CI (#6457)
- [MAINTENANCE] Remove anticipatory multi-language support from renderers (#6426)
- [MAINTENANCE] Remove yaml user_flow_scripts (#6454)
- [MAINTENANCE] Additional
sqlite
database fixture fortaxi_data
- All 2020 data in single table (#6455) - [MAINTENANCE] Clean Up Variable Names In Test Modules, Type Hints, and Minor Refactoring For Better Code Elegance/Readability (#6444)
- [MAINTENANCE] Update and Simplify Pandas tests for MapMetrics (#6443)
- [MAINTENANCE] Add metadata to experimental datasource Batch class (#6442)
- [MAINTENANCE] Small refactor (#6422)
- [MAINTENANCE] Sorting batch IDs and typehints clean up (#6421)
- [MAINTENANCE] Clean Up Type Hints and Minor Refactoring For Better Code Elegance/Readability (#6418)
- [MAINTENANCE] Implement
RendererConfiguration
(#6412) - [MAINTENANCE] Cleanup For Better Code Elegance/Readability (#6406)
- [MAINTENANCE] ZEP -
GxConfig
cleanup (#6404) - [MAINTENANCE] Migrate remaining methods from
BaseDataContext
(#6403) - [MAINTENANCE] Migrate additional CRUD methods from
BaseDataContext
toAbstractDataContext
(#6395) - [MAINTENANCE] ZEP add yaml methods to all experimental models (#6401)
- [MAINTENANCE] Remove call to verify_library_dependent_modules for pybigquery (#6394)
- [MAINTENANCE] Make "IDDict.to_id()" serialization more efficient. (#6389)
0.15.34
- [BUGFIX] Ensure
packaging_and_installation
CI tests against latest tag (#6386) - [BUGFIX] Fixed missing comma in pydantic constraints (#6391) (thanks @awburgess)
- [BUGFIX] fix pydantic dev req file entries (#6396)
- [DOCS] DOC-379 bring spark datasource configuration example scripts under test (#6362)
- [MAINTENANCE] Handle both
ExpectationConfiguration
andExpectationValidationResult
in default Atomic renderers and cleanupinclude_column_name
(#6380) - [MAINTENANCE] Add type annotations to all existing atomic renderer signatures (#6385)
- [MAINTENANCE] move
zep
->experimental
package (#6378) - [MAINTENANCE] Migrate additional methods from
BaseDataContext
to other parts of context hierarchy (#6388)
0.15.33
- [FEATURE] POC ZEP Config Loading (#6320)
- [BUGFIX] Fix issue with misaligned indentation in docs snippets (#6339)
- [BUGFIX] Use
requirements.txt
file when installing linting/static check dependencies in CI (#6368) - [BUGFIX] Patch nested snippet indentation issues within
remark-named-snippets
plugin (#6376) - [BUGFIX] Ensure
packaging_and_installation
CI tests against latest tag (#6386) - [DOCS] DOC-308 update CLI command in docs when working with RBPs instead of Data Assistants (#6222)
- [DOCS] DOC-366 updates to docs in support of branding updates (#5766)
- [DOCS] Add
yarn snippet-check
command (#6351) - [MAINTENANCE] Add missing one-line docstrings and try to make the others consistent (#6340)
- [MAINTENANCE] Refactor variable aggregation/substitution logic into
ConfigurationProvider
hierarchy (#6321) - [MAINTENANCE] In ExecutionEngine: Make variable names and usage more descriptive of their purpose. (#6342)
- [MAINTENANCE] Move Cloud-specific enums to
cloud_constants.py
(#6349) - [MAINTENANCE] Refactor out
termcolor
dependency (#6348) - [MAINTENANCE] Zep PostgresDatasource returns a list of batches. (#6341)
- [MAINTENANCE] Refactor
usage_stats_opt_out
method in DataContext (#5339) - [MAINTENANCE] Fix computed metrics type hint in ExecutionEngine.resolve_metrics() method (#6347)
- [MAINTENANCE] Subject: Support to include ID/PK in validation result for each row t… (#5876) (thanks @abekfenn)
- [MAINTENANCE] Pin
mypy
to0.990
(#6361) - [MAINTENANCE] Misc cleanup of GX Cloud helpers (#6352)
- [MAINTENANCE] Update column_reflection_fallback to also use schema name for Trino (#6350)
- [MAINTENANCE] Bump version of
mypy
in contrib CLI (#6370) - [MAINTENANCE] Move config variable substitution logic into
ConfigurationProvider
(#6345) - [MAINTENANCE] Removes comment in code that was causing confusion to some users. (#6366)
- [MAINTENANCE] minor metrics typing (#6374)
- [MAINTENANCE] Make
ConfigurationProvider
andConfigurationSubstitutor
private (#6375) - [MAINTENANCE] Rename
GeCloudStoreBackend
toGXCloudStoreBackend
(#6377) - [MAINTENANCE] Cleanup Metrics and ExecutionEngine methods (#6371)
- [MAINTENANCE] F/great 1314/integrate zep in core (#6358)
- [MAINTENANCE] Loosen
pydantic
version requirement (#6384)
0.15.32
- [BUGFIX] Patch broken
CloudNotificationAction
tests (#6327) - [BUGFIX] add create_temp_table flag to ExecutionEngineConfigSchema (#6331) (thanks @tommy-watts-depop)
- [BUGFIX] MapMetrics now return
partial_unexpected
values forSUMMARY
format (#6334) - [DOCS] Re-writes "how to implement custom notifications" as "How to get Data Docs URLs for use in custom Validation Actions" (#6281)
- [DOCS] Removes deprecated expectation notebook exploration doc (#6298)
- [DOCS] Removes a number of unused & deprecated docs (#6300)
- [DOCS] Prioritizes Onboarding Data Assistant in ToC (#6302)
- [DOCS] Add ZenML into integration table in Readme (#6144) (thanks @dnth)
- [DOCS] add
pypi
release badge (#6324) - [MAINTENANCE] Remove unneeded
BaseDataContext.get_batch_list
(#6291) - [MAINTENANCE] Clean up implicit
Optional
errors flagged bymypy
(#6319) - [MAINTENANCE] Add manual prod flags to core Expectations (#6278)
- [MAINTENANCE] Fallback to isnot method if is_not is not available (old sqlalchemy) (#6318)
- [MAINTENANCE] Add ZEP postgres datasource. (#6274)
- [MAINTENANCE] Delete "metric_dependencies" from MetricConfiguration constructor arguments (#6305)
- [MAINTENANCE] Clean up
DataContext
(#6304) - [MAINTENANCE] Deprecate
save_changes
flag onDatasource
CRUD (#6258) - [MAINTENANCE] Deprecate
great_expectations.render.types
package (#6315) - [MAINTENANCE] Update range of allowable sqlalchemy versions (#6328)
- [MAINTENANCE] Fixing checkpoint types (#6325)
- [MAINTENANCE] Fix column_reflection_fallback for Trino and minor logging/testing improvements (#6218)
- [MAINTENANCE] Change the number of expected Expectations in the 'quick check' stage of build_gallery pipeline (#6333)
0.15.31
- [BUGFIX] Include all requirement files in the sdist (#6292) (thanks @xhochy)
- [DOCS] Updates outdated batch_request snippet in Terms (#6283)
- [DOCS] Update Conditional Expectations doc w/ current availability (#6279)
- [DOCS] Remove outdated Data Discovery page and all references (#6288)
- [DOCS] Remove reference/evaluation_parameters page and all references (#6294)
- [DOCS] Removing deprecated Custom Metrics doc (#6282)
- [DOCS] Re-writes "how to implement custom notifications" as "How to get Data Docs URLs for use in custom Validation Actions" (#6281)
- [DOCS] Removes deprecated expectation notebook exploration doc (#6298)
- [MAINTENANCE] Move RuleState into rule directory. (#6284)
0.15.30
- [FEATURE] Add zep datasources to data context. (#6255)
- [BUGFIX] Iterate through
GeCloudIdentifiers
to find the suite ID from the name (#6243) - [BUGFIX] Update default base url for cloud API (#6176)
- [BUGFIX] Pin
termcolor
to below2.1.0
due to breaking changes in lib's TTY parsing logic (#6257) - [BUGFIX]
InferredAssetSqlDataConnector
include_schema_name
introspection of identical table names in different schemas (#6166) - [BUGFIX] Fix
docs-integration
tests, and temporarily pinsqlalchemy
(#6268) - [BUGFIX] Fix serialization for contrib packages (#6266)
- [BUGFIX] Ensure that
Datasource
credentials are not persisted to Cloud/disk (#6254) - [DOCS] Updates package contribution references (#5885)
- [MAINTENANCE] Maintenance/great 1103/great 1318/alexsherstinsky/validation graph/refactor validation graph usage 2022 10 20 248 (#6228)
- [MAINTENANCE] Refactor instances of
noqa: F821
Flake8 directive (#6220) - [MAINTENANCE] Logo URI ref in
data_docs
(#6246) - [MAINTENANCE] fix typos in docstrings (#6247)
- [MAINTENANCE] Isolate Trino/MSSQL/MySQL tests in
dev
CI (#6231) - [MAINTENANCE] Split up
compatability
andcomprehensive
stages indev
CI to improve performance (#6245) - [MAINTENANCE] ZEP POC - Asset Type Registration (#6194)
- [MAINTENANCE] Add Trino CLI support and bump Trino version (#6215) (thanks @hovaesco)
- [MAINTENANCE] Delete unneeded Rule attribute property (#6264)
- [MAINTENANCE] Small clean-up of
Marshmallow
warnings (missing
parameter changed toload_default
as of 3.13) (#6213) - [MAINTENANCE] Move
.png
files out of project root (#6249) - [MAINTENANCE] Cleanup
expectation.py
attributes (#6265) - [MAINTENANCE] Further parallelize test runs in
dev
CI (#6267) - [MAINTENANCE] GCP Integration Pipeline fix (#6259)
- [MAINTENANCE] mypy
warn_unused_ignores
(#6270) - [MAINTENANCE] ZEP - Data Source base class (#6263)
- [MAINTENANCE] Reverting
marshmallow
version bump (#6271) - [MAINTENANCE] type hints cleanup in Rule-Based Profiler (#6272)
- [MAINTENANCE] Remove unused f-strings (#6248)
- [MAINTENANCE] Make ParameterBuilder.resolve_evaluation_dependencies() into instance (rather than utility) method (#6273)
- [MAINTENANCE] Test definition for
ExpectColumnValueZScoresToBeLessThan
(#6229) - [MAINTENANCE] Make RuleState constructor argument ordering consistent with standard pattern. (#6275)
- [MAINTENANCE] [REQUEST] Please allow Rachel to unblock blockers (#6253)
0.15.29
- [FEATURE] Add support to AWS Glue Data Catalog (#5123) (thanks @lccasagrande)
- [FEATURE] / Added pairwise expectation 'expect_column_pair_values_to_be_in_set' (#6097) (thanks @Arnavkar)
- [BUGFIX] Adjust condition in RenderedAtomicValueSchema.clean_null_attrs (#6168)
- [BUGFIX] Add
py
to dev dependencies to circumvent compatability issues withpytest==7.2.0
(#6202) - [BUGFIX] Fix
test_package_dependencies.py
to includepy
lib (#6204) - [BUGFIX] Fix logic in ExpectationDiagnostics._check_renderer_methods method (#6208)
- [BUGFIX] Patch issue with empty config variables file raising
TypeError
(#6216) - [BUGFIX] Release patch for Azure env vars (#6233)
- [BUGFIX] Cloud Data Context should overwrite existing suites based on
ge_cloud_id
instead of name (#6234) - [BUGFIX] Add env vars to Pytest min versions Azure stage (#6239)
- [DOCS] doc-297: update the create Expectations overview page for Data Assistants (#6212)
- [DOCS] DOC-378: bring example scripts for pandas configuration guide under test (#6141)
- [MAINTENANCE] Add unit test for MetricsCalculator.get_metric() Method -- as an example template (#6179)
- [MAINTENANCE] ZEP MetaDatasource POC (#6178)
- [MAINTENANCE] Update
scope_check
in Azure CI to trigger on changed.py
source code files (#6185) - [MAINTENANCE] Move test_yaml_config to a separate class (#5487)
- [MAINTENANCE] Changed profiler to Data Assistant in CLI, docs, and tests (#6189)
- [MAINTENANCE] Update default GE_USAGE_STATISTICS_URL in test docker image. (#6192)
- [MAINTENANCE] Re-add a renamed test definition file (#6182)
- [MAINTENANCE] Refactor method
parse_evaluation_parameter
(#6191) - [MAINTENANCE] Migrate methods from
BaseDataContext
toAbstractDataContext
(#6188) - [MAINTENANCE] Rename cfe to v3_api (#6190)
- [MAINTENANCE] Test Trino doc examples with test_script_runner.py (#6198)
- [MAINTENANCE] Cleanup of Regex ParameterBuilder (#6196)
- [MAINTENANCE] Apply static type checking to
expectation.py
(#6173) - [MAINTENANCE] Remove version matrix from
dev
CI pipeline to improve performance (#6203) - [MAINTENANCE] Rename
CloudMigrator.retry_unsuccessful_validations
(#6206) - [MAINTENANCE] Add validate_configuration method to expect_table_row_count_to_equal_other_table (#6209)
- [MAINTENANCE] Replace deprecated
iteritems
withitems
(#6205) - [MAINTENANCE] Add instructions for setting up the test_ci database (#6211)
- [MAINTENANCE] Add E2E tests for Cloud-backed
Datasource
CRUD (#6186) - [MAINTENANCE] Execution Engine linting & partial typing (#6210)
- [MAINTENANCE] Test definition for
ExpectColumnValuesToBeJsonParsable
, including a fix for Spark (#6207) - [MAINTENANCE] Port over usage statistics enabled methods from
BaseDataContext
toAbstractDataContext
(#6201) - [MAINTENANCE] Remove temporary dependency on
py
(#6217) - [MAINTENANCE] Adding type hints to DataAssistant implementations (#6224)
- [MAINTENANCE] Remove AWS config file dependencies and use existing env vars in CI/CD (#6227)
- [MAINTENANCE] Make
UsageStatsEvents
aStrEnum
(#6225) - [MAINTENANCE] Move all
requirements-dev*.txt
files to separate dir (#6223) - [MAINTENANCE] Maintenance/great 1103/great 1318/alexsherstinsky/validation graph/refactor validation graph usage 2022 10 20 248 (#6228)
0.15.28
- [FEATURE] Initial zep datasource protocol. (#6153)
- [FEATURE] Introduce BatchManager to manage Batch objects used by Validator and BatchData used by ExecutionEngine (#6156)
- [FEATURE] Add support for Vertica dialect (#6145) (thanks @viplazylmht)
- [FEATURE] Introduce MetricsCalculator and Refactor Redundant Code out of Validator (#6165)
- [BUGFIX] SQLAlchemy selectable Bug fix (#6159) (thanks @tommy-watts-depop)
- [BUGFIX] Parameterize usage stats endpoint in test dockerfile. (#6169)
- [BUGFIX] B/great 1305/usage stats endpoint (#6170)
- [BUGFIX] Ensure that spaces are recognized in named snippets (#6172)
- [DOCS] Clarify wording for interactive mode in databricks (#6154)
- [DOCS] fix source activate command (#6161) (thanks @JGrzywacz)
- [DOCS] Update version in
runtime.txt
to fix breaking Netlify builds (#6181) - [DOCS] Clean up snippets and line number validation in docs (#6142)
- [MAINTENANCE] Add Enums for renderer types (#6112)
- [MAINTENANCE] Minor cleanup in preparation for Validator refactoring into separate concerns (#6155)
- [MAINTENANCE] add the internal
GE_DATA_CONTEXT_ID
env var to the docker file (#6122) - [MAINTENANCE] Rollback setting GE_DATA_CONTEXT_ID in docker image. (#6163)
- [MAINTENANCE] disable ge_cloud_mode when specified, detect misconfiguration (#6162)
- [MAINTENANCE] Re-add missing Expectations to gallery and include package names (#6171)
- [MAINTENANCE] Use
from __future__ import annotations
to clean up type hints (#6127) - [MAINTENANCE] Make sure that quick stage check returns 0 if there are no problems (#6177)
- [MAINTENANCE] Remove SQL for expect_column_discrete_entropy_to_be_between (#6180)
0.15.27
- [FEATURE] Add logging/warnings to GX Cloud migration process (#6106)
- [FEATURE] Introduction of updated
gx.get_context()
method that returns correct DataContext-type (#6104) - [FEATURE] Contribute StatisticsDataAssistant and GrowthNumericDataAssistant (both experimental) (#6115)
- [BUGFIX] add OBJECT_TYPE_NAMES to the JsonSchemaProfiler - issue #6109 (#6110) (thanks @OphelieC)
- [BUGFIX] Fix example
Set-Based Column Map Expectation
template import (#6134) - [BUGFIX] Regression due to
GESqlDialect
Enum
for Hive (#6149) - [DOCS] Support for named snippets in documentation (#6087)
- [MAINTENANCE] Clean up
test_migrate=True
Cloud migrator output (#6119) - [MAINTENANCE] Creation of Hackathon Packages (#4587)
- [MAINTENANCE] Rename GCP Integration Pipeline (#6121)
- [MAINTENANCE] Change log levels used in
CloudMigrator
(#6125) - [MAINTENANCE] Bump version of
sqlalchemy-redshift
from0.7.7
to0.8.8
(#6082) - [MAINTENANCE] self_check linting & initial type-checking (#6126)
- [MAINTENANCE] Update per Clickhouse multiple same aliases Bug (#6128) (thanks @adammrozik)
- [MAINTENANCE] Only update existing
rendered_content
if rendering does not fail with newInlineRenderer
failure message (#6091)
0.15.26
- [FEATURE] Enable sending of
ConfigurationBundle
payload in HTTP request to Cloud backend (#6083) - [FEATURE] Send user validation results to Cloud backend during migration (#6102)
- [BUGFIX] Fix bigquery crash when using "in" with a boolean column (#6071)
- [BUGFIX] Fix serialization error when rendering kl_divergence (#6084) (thanks @roblim)
- [BUGFIX] Enable top-level parameters in Data Assistants accessed via dispatcher (#6077)
- [BUGFIX] Patch issue around
DataContext.save_datasource
not sendingclass_name
in result payload (#6108) - [DOCS] DOC-377 add missing dictionary in configured asset datasource portion of Pandas and Spark configuration guides (#6081)
- [DOCS] DOC-376 finalize definition for Data Assistants in technical terms (#6080)
- [DOCS] Update
docs-integration
test due to newwhole_table
splitter behavior (#6103) - [DOCS] How to create a Custom Multicolumn Map Expectation (#6101)
- [MAINTENANCE] Patch broken Cloud E2E test (#6079)
- [MAINTENANCE] Bundle data context config and other artifacts for migration (#6068)
- [MAINTENANCE] Add datasources to ConfigurationBundle (#6092)
- [MAINTENANCE] Remove unused config files from root of GX repo (#6090)
- [MAINTENANCE] Add
data_context_id
property toConfigurationBundle
(#6094) - [MAINTENANCE] Move all Cloud migrator logic to separate directory (#6100)
- [MAINTENANCE] Update aloglia scripts for new fields and replica indices (#6049) (thanks @winrp17)
- [MAINTENANCE] initial Data Source typings (#6099)
- [MAINTENANCE] Data context migrate to cloud event (#6095)
- [MAINTENANCE] Bundling tests with empty context configs (#6107)
- [MAINTENANCE] Fixing a typo (#6113)
0.15.25
- [FEATURE] Since value set in expectation kwargs is list of strings, do not emit expect_column_values_to_be_in_set for datetime valued columns (#6046)
- [FEATURE] add failed expectations list to slack message (#5812) (thanks @itaise)
- [FEATURE] Enable only ExactNumericRangeEstimator and QuantilesNumericRangeEstimator in "datetime_columns_rule" of OnboardingDataAssistant (#6063)
- [BUGFIX] numpy typing behind
if TYPE_CHECKING
(#6076) - [DOCS] Update "How to create an Expectation Suite with the Onboarding Data Assistant" (#6050)
- [DOCS] How to get one or more Batches of data from a configured Data Source (#6043)
- [DOCS] DOC-298 Data Assistant technical term page (#6057)
- [DOCS] Update OnboardingDataAssistant documentation (#6059)
- [MAINTENANCE] Clean up of DataAssistant tests that depend on Jupyter notebooks (#6039)
- [MAINTENANCE] AbstractDataContext.datasource_save() test simplifications (#6052)
- [MAINTENANCE] Rough architecture for cloud migration tool (#6054)
- [MAINTENANCE] Include git commit info when building docker image. (#6060)
- [MAINTENANCE] Allow
CloudDataContext
to retrieve and initialize its own project config (#6006) - [MAINTENANCE] Removing Jupyter notebook-based tests for DataAssistants (#6062)
- [MAINTENANCE] pinned dremio, fixed linting (#6067)
- [MAINTENANCE] usage-stats, & utils.py typing (#5925)
- [MAINTENANCE] Refactor external HTTP request logic into a
Session
factory function (#6007) - [MAINTENANCE] Remove tag validity stage from release pipeline (#6069)
- [MAINTENANCE] Remove unused test fixtures from test suite (#6058)
- [MAINTENANCE] Remove outdated release files (#6074)
0.15.24
- [FEATURE] context.save_datasource (#6009)
- [BUGFIX] Standardize
ConfiguredAssetSqlDataConnector
config indatasource new
CLI workflow (#6044) - [DOCS] DOC-371 update the getting started tutorial for data assistants (#6024)
- [DOCS] DOCS-369 sql data connector configuration guide (#6002)
- [MAINTENANCE] Remove outdated entry from release schedule JSON (#6032)
- [MAINTENANCE] Clean up Spark schema tests to have proper names (#6033)
0.15.23
- [FEATURE] do not require expectation_suite_name in DataAssistantResult.show_expectations_by...() methods (#5976)
- [FEATURE] Refactor PartitionParameterBuilder into dedicated ValueCountsParameterBuilder and HistogramParameterBuilder (#5975)
- [FEATURE] Implement default sorting for batches based on selected splitter method (#5924)
- [FEATURE] Make OnboardingDataAssistant default profiler in CLI SUITE NEW (#6012)
- [FEATURE] Enable omission of rounding of decimals in NumericMetricRangeMultiBatchParameterBuilder (#6017)
- [FEATURE] Enable non-default sorters for
ConfiguredAssetSqlDataConnector
(#5993) - [FEATURE] Data Assistant plot method indication of total metrics and expectations count (#6016)
- [BUGFIX] Addresses issue with ExpectCompoundColumnsToBeUnique renderer (#5970)
- [BUGFIX] Fix failing
run_profiler_notebook
test (#5983) - [BUGFIX] Handle case when only one unique "column.histogram" bin value is found (#5987)
- [BUGFIX] Update
get_validator
test assertions due to change in fixture batches (#5989) - [BUGFIX] Fix use of column.partition metric in HistogramSingleBatchParameterBuilder to more accurately handle errors (#5990)
- [BUGFIX] Make Spark implementation of "column.value_counts" metric more robust to None/NaN column values (#5996)
- [BUGFIX] Filter out np.nan values (just like None values) as part of ColumnValueCounts._spark() implementation (#5998)
- [BUGFIX] Handle case when only one unique "column.histogram" bin value is found with proper type casting (#6001)
- [BUGFIX] ColumnMedian._sqlalchemy() needs to handle case of single-value column (#6011)
- [BUGFIX] Patch broken
save_expectation_suite
behavior with Cloud-backedDataContext
(#6004) - [BUGFIX] Clean quantitative metrics DataFrames in Data Assistant plotting (#6023)
- [BUGFIX] Defer
pprint
inExpectationSuite.show_expectations_by_expectation_type()
due to Jupyter rate limit (#6026) - [BUGFIX] Use UTC TimeZone (rather than Local Time Zone) for Rule-Based Profiler DateTime Conversions (#6028)
- [DOCS] Update snippet refs in "How to create an Expectation Suite with the Onboarding Data Assistant" (#6014)
- [MAINTENANCE] Randomize the non-comprehensive tests (#5968)
- [MAINTENANCE] DatasourceStore refactoring (#5941)
- [MAINTENANCE] Expectation suite init unit tests + types (#5957)
- [MAINTENANCE] Expectation suite new unit tests for add_citation (#5966)
- [MAINTENANCE] Updated release schedule (#5977)
- [MAINTENANCE] Unit tests for
CheckpointStore
(#5967) - [MAINTENANCE] Enhance unit tests for ExpectationSuite.isEquivalentTo (#5979)
- [MAINTENANCE] Remove unused fixtures from test suite (#5965)
- [MAINTENANCE] Update to MultiBatch Notebook to include Configured - Sql (#5945)
- [MAINTENANCE] Update to MultiBatch Notebook to include Inferred - Sql (#5958)
- [MAINTENANCE] Add reverse assertion for isEquivalentTo tests (#5982)
- [MAINTENANCE] Unit test enhancements ExpectationSuite.eq() (#5984)
- [MAINTENANCE] Refactor
DataContext.__init__
to move Cloud-specific logic toCloudDataContext
(#5981) - [MAINTENANCE] Set up cloud integration tests with Azure Pipelines (#5995)
- [MAINTENANCE] Example of
splitter_method
atAsset
andDataConnector
level (#6000) - [MAINTENANCE] Replace
splitter_method
strings withSplitterMethod
Enum and leverageGESqlDialect
Enum where applicable (#5980) - [MAINTENANCE] Ensure that
DataContext.add_datasource
works with nestedDataConnector
ids (#5992) - [MAINTENANCE] Remove cloud integration tests from azure-pipelines.yml (#5997)
- [MAINTENANCE] Unit tests for
GeCloudStoreBackend
(#5999) - [MAINTENANCE] Parameterize pg hostname in jupyter notebooks (#6005)
- [MAINTENANCE] Unit tests for
Validator
(#5988) - [MAINTENANCE] Add unit tests for SimpleSqlalchemyDatasource (#6008)
- [MAINTENANCE] Remove
dgtest
from dev pipeline (#6003) - [MAINTENANCE] Remove deprecated
account_id
from GX Cloud integrations (#6010) - [MAINTENANCE] Added perf considerations to onboarding assistant notebook (#6022)
- [MAINTENANCE] Redshift specific temp table code path (#6021)
- [MAINTENANCE] Update
datasource new
workflow to enableConfiguredAssetDataConnector
usage with SQL-backedDatasources
(#6019)
0.15.22
- [FEATURE] Allowing
schema
to be passed in asbatch_spec_passthrough
in Spark (#5900) - [FEATURE] DataAssistants Example Notebook - Spark (#5919)
- [FEATURE] Improve slack error condition (#5818) (thanks @itaise)
- [BUGFIX] Ensure that ParameterBuilder implementations in Rule Based Profiler properly handle SQL DECIMAL type (#5896)
- [BUGFIX] Making an all-NULL column handling in RuleBasedProfiler more robust (#5937)
- [BUGFIX] Don't include abstract Expectation classes in _retrieve_expectations_from_module (#5947)
- [BUGFIX] Data Assistant plotting with zero expectations produced (#5934)
- [BUGFIX] prefix and suffix asset names are only relevant for InferredSqlAlchemyDataConnector (#5950)
- [BUGFIX] Prevent "division by zero" errors in Rule-Based Profiler calculations when Batch has zero rows (#5960)
- [BUGFIX] Spark column.distinct_values no longer returns entire table distinct values (#5969)
- [DOCS] DOC-368 spelling correction (#5912)
- [MAINTENANCE] Mark all tests within
tests/data_context/stores
dir (#5913) - [MAINTENANCE] Cleanup to allow docker test target to run tests in random order (#5915)
- [MAINTENANCE] Use datasource config in add_datasource support methods (#5901)
- [MAINTENANCE] Cleanup up some new datasource sql data connector tests. (#5918)
- [MAINTENANCE] Unit tests for
data_context/store
(#5923) - [MAINTENANCE] Mark all tests within
tests/validator
(#5926) - [MAINTENANCE] Certify InferredAssetSqlDataConnector and ConfiguredAssetSqlDataConnector (#5847)
- [MAINTENANCE] Mark DBFS tests with
@pytest.mark.integration
(#5931) - [MAINTENANCE] Reset globals modified in tests (#5936)
- [MAINTENANCE] Move
Store
test utils from source code to tests (#5932) - [MAINTENANCE] Mark tests within
tests/rule_based_profiler
(#5930) - [MAINTENANCE] Add missing import for ConfigurationIdentifier (#5943)
- [MAINTENANCE] Update to OnboardingDataAssistant Notebook - Sql (#5939)
- [MAINTENANCE] Run comprehensive tests in a random order (#5942)
- [MAINTENANCE] Unit tests for
ConfigurationStore
(#5948) - [MAINTENANCE] Add a dev-tools requirements option (#5944)
- [MAINTENANCE] Run spark and onboarding data assistant test in their own jobs. (#5951)
- [MAINTENANCE] Unit tests for
ValidationGraph
and related classes (#5954) - [MAINTENANCE] More unit tests for
Stores
(#5953) - [MAINTENANCE] Add x-fails to flaky Cloud tests for purposes of 0.15.22 (#5964)
- [MAINTENANCE] Bump
Marshmallow
upper bound to work with Airflow operator (#5952) - [MAINTENANCE] Use DataContext to ignore progress bars (#5959)
0.15.21
- [FEATURE] Add
include_rendered_content
toget_expectation_suite
andget_validation_result
(#5853) - [FEATURE] Add tags as an optional setting for the OpsGenieAlertAction (#5855) (thanks @stevewb1993)
- [BUGFIX] Ensure that
delete_expectation_suite
returns proper boolean result (#5878) - [BUGFIX] many small bugfixes (#5881)
- [BUGFIX] Fix typo in default value of "ignore_row_if" kwarg for MulticolumnMapExpectation (#5860) (thanks @mkopec87)
- [BUGFIX] Patch issue with
checkpoint_identifier
withinCheckpoint.run
workflow (#5894) - [BUGFIX] Ensure that
DataContext.add_checkpoint()
updates existing objects in GX Cloud (#5895) - [DOCS] DOC-364 how to configure a spark datasource (#5840)
- [MAINTENANCE] Unit Tests Pipeline step (#5838)
- [MAINTENANCE] Unit tests to ensure coverage over
Datasource
caching inDataContext
(#5839) - [MAINTENANCE] Add entries to release schedule (#5833)
- [MAINTENANCE] Properly label
DataAssistant
tests with@pytest.mark.integration
(#5845) - [MAINTENANCE] Add additional unit tests around
Datasource
caching (#5844) - [MAINTENANCE] Mark miscellaneous tests with
@pytest.mark.unit
(#5846) - [MAINTENANCE]
datasource
,data_context
,core
typing, lint fixes (#5824) - [MAINTENANCE] add --ignore-suppress and --ignore-only-for to build_gallery.py with bugfixes (#5802)
- [MAINTENANCE] Remove pyparsing pin for <3.0 (#5849)
- [MAINTENANCE] Finer type exclude (#5848)
- [MAINTENANCE] use
id
insteadid_
(#5775) - [MAINTENANCE] Add data connector names in datasource config (#5778)
- [MAINTENANCE] init tests for dict and json serializers (#5854)
- [MAINTENANCE] Remove Partitioning and Quantiles metrics computations from DateTime Rule of OnboardingDataAssistant (#5862)
- [MAINTENANCE] Update
ExpectationSuite
CRUD onDataContext
to recognize Cloud ids (#5836) - [MAINTENANCE] Handle Pandas warnings in Data Assistant plots (#5863)
- [MAINTENANCE] Misc cleanup of
test_expectation_suite_crud.py
(#5868) - [MAINTENANCE] Remove vendored
marshmallow__shade
(#5866) - [MAINTENANCE] don't force using the stand alone mock (#5871)
- [MAINTENANCE] Update expectation_gallery pipeline (#5874)
- [MAINTENANCE] run unit-tests on a target package (#5869)
- [MAINTENANCE] add
pytest-timeout
(#5857) - [MAINTENANCE] Label tests in
tests/core
with@pytest.mark.unit
and@pytest.mark.integration
(#5879) - [MAINTENANCE] new invoke test flags (#5880)
- [MAINTENANCE] JSON Serialize RowCondition and MetricBundle computation result to enable IDDict.to_id() for SparkDFExecutionEngine (#5883)
- [MAINTENANCE] increase the
pytest-timeout
timeout value during unit-testing step (#5884) - [MAINTENANCE] Add
@pytest.mark.slow
throughout test suite (#5882) - [MAINTENANCE] Add test_expectation_suite_send_usage_message (#5886)
- [MAINTENANCE] Mark existing tests as unit or integration (#5890)
- [MAINTENANCE] Convert integration tests to unit (#5891)
- [MAINTENANCE] Update distinct metric dependencies and implementations (#5811)
- [MAINTENANCE] Add slow pytest marker to config and sort them alphabetically. (#5892)
- [MAINTENANCE] Adding serialization tests for Spark (#5897)
- [MAINTENANCE] Improve existing expectation suite unit tests (phase 1) (#5898)
- [MAINTENANCE]
SqlAlchemyExecutionEngine
case for SQL AlchemySelect
andTextualSelect
due toSADeprecationWarning
(#5902)
0.15.20
- [FEATURE]
query.pair_column
Metric (#5743) - [FEATURE] Enhance execution time measurement utility, and save
DomainBuilder
execution time per Rule of Rule-Based Profiler (#5796) - [FEATURE] Support single-batch mode in MetricMultiBatchParameterBuilder (#5808)
- [FEATURE] Inline
ExpectationSuite
Rendering (#5726) - [FEATURE] Better error for missing expectation (#5750) (thanks @tylertrussell)
- [FEATURE] DataAssistants Example Notebook - Pandas (#5820)
- [BUGFIX] Ensure name not persisted (#5813)
- [DOCS] Change the selectable to a list (#5780) (thanks @itaise)
- [DOCS] Fix how to create custom table expectation (#5807) (thanks @itaise)
- [DOCS] DOC-363 how to configure a pandas datasource (#5779)
- [MAINTENANCE] Remove xfail markers on cloud tests (#5793)
- [MAINTENANCE] build-gallery enhancements (#5616)
- [MAINTENANCE] Refactor
save_profiler
to remove explicitname
andge_cloud_id
args (#5792) - [MAINTENANCE] Add v2_api flag for v2_api specific tests (#5803)
- [MAINTENANCE] Clean up
ge_cloud_id
reference fromDataContext
ExpectationSuite
CRUD (#5791) - [MAINTENANCE] Refactor convert_dictionary_to_parameter_node (#5805)
- [MAINTENANCE] Remove
ge_cloud_id
fromDataContext.add_profiler()
signature (#5804) - [MAINTENANCE] Remove "copy.deepcopy()" calls from ValidationGraph (#5809)
- [MAINTENANCE] Add vectorized is_between for common numpy dtypes (#5711)
- [MAINTENANCE] Make partitioning directives of PartitionParameterBuilder configurable (#5810)
- [MAINTENANCE] Write E2E Cloud test for
RuleBasedProfiler
creation and retrieval (#5815) - [MAINTENANCE] Change recursion to iteration for function in parameter_container.py (#5817)
- [MAINTENANCE] add
pytest-mock
&pytest-icdiff
plugins (#5819) - [MAINTENANCE] Surface cloud errors (#5797)
- [MAINTENANCE] Clean up build_parameter_container_for_variables (#5823)
- [MAINTENANCE] Bugfix/snowflake temp table schema name (#5814)
- [MAINTENANCE] Update
list_
methods onDataContext
to emit names along with object ids (#5826) - [MAINTENANCE] xfail Cloud E2E tests due to schema issue with
DataContextVariables
(#5828) - [MAINTENANCE] Clean up xfails in preparation for 0.15.20 release (#5835)
- [MAINTENANCE] Add back xfails for E2E Cloud tests that fail on env var retrieval in Docker (#5837)
0.15.19
- [FEATURE]
DataAssistantResult
plot multiple metrics per expectation (#5556) - [FEATURE] Enable passing "exact_estimation" boolean at
DataAssistant.run()
level (default value is True) (#5744) - [FEATURE] Example notebook for Onboarding DataAssistant -
postgres
(#5776) - [BUGFIX] dir update for data_assistant_result (#5751)
- [BUGFIX] Fix docs_integration pipeline (#5734)
- [BUGFIX] Patch flaky E2E Cloud test with randomized suite names (#5752)
- [BUGFIX] Fix RegexPatternStringParameterBuilder to use legal character repetition. Remove median, mean, and standard deviation features from OnboardingDataAssistant "datetime_columns_rule" definition. (#5757)
- [BUGFIX] Move
SuiteValidationResult.meta
validation id propogation beforeValidationOperator._run_action
(#5760) - [BUGFIX] Update "column.partition" Metric to handle DateTime Arithmetic Properly (#5764)
- [BUGFIX] JSON-serialize RowCondition and enable IDDict to support comparison operations (#5765)
- [BUGFIX] Insure all estimators properly handle datetime-float conversion (#5774)
- [BUGFIX] Return appropriate subquery type to Query Metrics for SA version (#5783)
- [DOCS] added guide how to use gx with emr serverless (#5623) (thanks @bvolodarskiy)
- [DOCS] DOC-362: how to choose between working with a single or multiple batches of data (#5745)
- [MAINTENANCE] Temporarily xfail E2E Cloud tests due to Azure env var issues (#5787)
- [MAINTENANCE] Add ids to
DataConnectorConfig
(#5740) - [MAINTENANCE] Rename GX Cloud "contract" resource to "checkpoint" (#5748)
- [MAINTENANCE] Rename GX Cloud "suite_validation_result" resource to "validation_result" (#5749)
- [MAINTENANCE] Store Refactor - cloud store return types & http-errors (#5730)
- [MAINTENANCE] profile_numeric_columns_diff_expectation (#5741) (thanks @stevensecreti)
- [MAINTENANCE] Clean up type hints around class constructors (#5738)
- [MAINTENANCE] invoke docker (#5703)
- [MAINTENANCE] Add plist to build docker test image daily. (#5754)
- [MAINTENANCE] opt-out type-checking (#5713)
- [MAINTENANCE] Enable Algolia UI (#5753)
- [MAINTENANCE] Linting & initial typing for data context (#5756)
- [MAINTENANCE] Update
oneshot
estimator toquantiles
estimator (#5737) - [MAINTENANCE] Update Auto-Initializing Expectations to use
exact
estimator by default (#5759) - [MAINTENANCE] Send a Gx-Version header set to version in requests to cloud (#5758) (thanks @wookasz)
- [MAINTENANCE] invoke docker --detach and more typing (#5770)
- [MAINTENANCE] In ParameterBuilder implementations, enhance handling of numpy.ndarray metric values, whose elements are or can be converted into datetime.datetime type. (#5771)
- [MAINTENANCE] Config/Schema round_tripping (#5697)
- [MAINTENANCE] Add experimental label to MetricStore Doc (#5782)
- [MAINTENANCE] Remove
GeCloudIdentifier
creation inCheckpoint.run()
(#5784)
0.15.18
- [FEATURE] Example notebooks for multi-batch Spark (#5683)
- [FEATURE] Introduce top-level
default_validation_id
inCheckpointConfig
(#5693) - [FEATURE] Pass down validation ids to
ExpectationSuiteValidationResult.meta
withinCheckpoint.run()
(#5725) - [FEATURE] Refactor data assistant runner to compute formal parameters for data assistant run method signatures (#5727)
- [BUGFIX] Restored sqlite database for tests (#5742)
- [BUGFIX] Fixing a typo in variable name for default profiler for auto-initializing expectation "expect_column_mean_to_be_between" (#5687)
- [BUGFIX] Remove
resource_type
from call toStoreBackend.build_key
(#5690) - [BUGFIX] Update how_to_use_great_expectations_in_aws_glue.md (#5685) (thanks @bvolodarskiy)
- [BUGFIX] Updated how_to_use_great_expectations_in_aws_glue.md again (#5696) (thanks @bvolodarskiy)
- [BUGFIX] Update how_to_use_great_expectations_in_aws_glue.md (#5722) (thanks @bvolodarskiy)
- [BUGFIX] Update aws_glue_deployment_patterns.py (#5721) (thanks @bvolodarskiy)
- [DOCS] added guide how to use great expectations with aws glue (#5536) (thanks @bvolodarskiy)
- [DOCS] Document the ZenML integration for Great Expectations (#5672) (thanks @stefannica)
- [DOCS] Converts broken ZenML md refs to Technical Tags (#5714)
- [DOCS] How to create a Custom Query Expectation (#5460)
- [MAINTENANCE] Pin makefun package to version range for support assurance (#5746)
- [MAINTENANCE] s3 link for logo (#5731)
- [MAINTENANCE] Assign
resource_type
inInlineStoreBackend
constructor (#5671) - [MAINTENANCE] Add mysql client to Dockerfile.tests (#5681)
- [MAINTENANCE]
RuleBasedProfiler
corner case configuration changes (#5631) - [MAINTENANCE] Update teams.yml (#5684)
- [MAINTENANCE] Utilize
e2e
mark on E2E Cloud tests (#5691) - [MAINTENANCE] pyproject.tooml build-system typo (#5692)
- [MAINTENANCE] expand flake8 coverage (#5676)
- [MAINTENANCE] Ensure Cloud E2E tests are isolated to
gx-cloud-e2e
stage of CI (#5695) - [MAINTENANCE] Add usage stats and initial database docker tests to CI (#5682)
- [MAINTENANCE] Add
e2e
mark topyproject.toml
(#5699) - [MAINTENANCE] Update docker readme to mount your repo over the builtin one. (#5701)
- [MAINTENANCE] Combine packages
rule_based_profiler
andrule_based_profiler.types
(#5680) - [MAINTENANCE] ExpectColumnValuesToBeInSetSparkOptimized (#5702)
- [MAINTENANCE] expect_column_pair_values_to_have_difference_of_custom_perc… (#5661) (thanks @exteli)
- [MAINTENANCE] Remove non-docker version of CI tests that are now running in docker. (#5700)
- [MAINTENANCE] Add back
integration
mark to tests intest_datasource_crud.py
(#5708) - [MAINTENANCE] DEVREL-2289/Stale/Triage (#5694)
- [MAINTENANCE] revert expansive flake8 pre-commit checking - flake8 5.0.4 (#5706)
- [MAINTENANCE] Bugfix for
cloud-db-integration-pipeline
(#5704) - [MAINTENANCE] Remove pytest-azurepipelines (#5716)
- [MAINTENANCE] Remove deprecation warning from
DataConnector
-levelbatch_identifiers
forRuntimeDataConnector
(#5717) - [MAINTENANCE] Refactor
AbstractConfig
to makename
andid_
consistent attrs (#5698) - [MAINTENANCE] Move CLI tests to docker (#5719)
- [MAINTENANCE] Leverage
DataContextVariables
inDataContext
hierarchy to automatically determine how to persist changes (#5715) - [MAINTENANCE] Refactor
InMemoryStoreBackend
out ofstore_backend.py
(#5679) - [MAINTENANCE] Move compatibility matrix tests to docker (#5728)
- [MAINTENANCE] Adds additional file extensions for Parquet assets (#5729)
- [MAINTENANCE] MultiBatch SqlExample notebook Update. (#5718)
- [MAINTENANCE] Introduce NumericRangeEstimator class hierarchy and encapsulate existing estimator implementations (#5735)
0.15.17
- [FEATURE] Improve estimation histogram computation in NumericMetricRangeMultiBatchParameterBuilder to include both counts and bin edges (#5628)
- [FEATURE] Enable retrieve by name for datasource with cloud store backend (#5640)
- [FEATURE] Update
DataContext.add_checkpoint()
to ensure validations withinCheckpointConfig
contain ids (#5638) - [FEATURE] Add
expect_column_values_to_be_valid_crc32
(#5580) (thanks @sp1thas) - [FEATURE] Enable showing expectation suite by domain and by expectation_type -- from DataAssistantResult (#5673)
- [BUGFIX] Patch flaky E2E GX Cloud tests (#5629)
- [BUGFIX] Pass
--cloud
flag todgtest-cloud-overrides
section of Azure YAML (#5632) - [BUGFIX] Remove datasource from config on delete (#5636)
- [BUGFIX] Patch issue with usage stats sync not respecting usage stats opt-out (#5644)
- [BUGFIX] SlackRenderer / EmailRenderer links to deprecated doc (#5648)
- [BUGFIX] Fix table.head metric issue when using BQ without temp tables (#5630)
- [BUGFIX] Quick bugfix on all profile numeric column diff bounds expectations (#5651) (thanks @stevensecreti)
- [BUGFIX] Patch bug with
id
vsid_
in Cloud integration tests (#5677) - [DOCS] Fix a typo in batch_request_parameters variable (#5612) (thanks @StasDeep)
- [MAINTENANCE] CloudDataContext add_datasource test (#5626)
- [MAINTENANCE] Update stale.yml (#5602)
- [MAINTENANCE] Add
id
toCheckpointValidationConfig
(#5603) - [MAINTENANCE] Better error message for RuntimeDataConnector for BatchIdentifiers (#5635)
- [MAINTENANCE] type-checking round 2 (#5576)
- [MAINTENANCE] minor cleanup of old comments (#5641)
- [MAINTENANCE] add
--clear-cache
flag forinvoke type-check
(#5639) - [MAINTENANCE] Install
dgtest
test runner utilizing Git URL in CI (#5645) - [MAINTENANCE] Make comparisons of aggregate values date aware (#5642) (thanks @jcampbell)
- [MAINTENANCE] Add E2E Cloud test for
DataContext.add_checkpoint()
(#5653) - [MAINTENANCE] Use docker to run tests in the Azure CI pipeline. (#5646)
- [MAINTENANCE] add new invoke tasks to
tasks.py
and create new fileusage_stats_utils.py
(#5593) - [MAINTENANCE] Don't include 'test-pipeline' in extras_require dict (#5659)
- [MAINTENANCE] move tool config to pyproject.toml (#5649)
- [MAINTENANCE] Refactor docker test CI steps into jobs. (#5665)
- [MAINTENANCE] Only run Cloud E2E tests in primary pipeline (#5670)
- [MAINTENANCE] Improve DateTime Conversion Candling in Comparison Metrics & Expectations and Provide a Clean Object Model for Metrics Computation Bundling (#5656)
- [MAINTENANCE] Ensure that
id_
fields in Marshmallow schema serialize asid
(#5660) - [MAINTENANCE] data_context initial type checking (#5662)
0.15.16
- [FEATURE] Multi-Batch Example Notebook - SqlDataConnector examples (#5575)
- [FEATURE] Implement "is_close()" for making equality comparisons "reasonably close" for each ExecutionEngine subclass (#5597)
- [FEATURE] expect_profile_numeric_columns_percent_diff_(inclusive bounds) (#5586) (thanks @stevensecreti)
- [FEATURE] DataConnector Query enabled for
SimpleSqlDatasource
(#5610) - [FEATURE] Implement the exact metric range estimate for NumericMetricRangeMultiBatchParameterBuilder (#5620)
- [FEATURE] Ensure that id propogates from RuleBasedProfilerConfig to RuleBasedProfiler (#5617)
- [BUGFIX] Pass cloud base url to datasource store (#5595)
- [BUGFIX] Temporarily disable Trino
0.315.0
from requirements (#5606) - [BUGFIX] Update _create_trino_engine to check for schema before creating it (#5607)
- [BUGFIX] Support
ExpectationSuite
CRUD atBaseDataContext
level (#5604) - [BUGFIX] Update test due to change in postgres stdev calculation method (#5624)
- [BUGFIX] Patch issue with
get_validator
on Cloud-backedDataContext
(#5619) - [MAINTENANCE] Add name and id to DatasourceConfig (#5560)
- [MAINTENANCE] Clear datasources in
test_data_context_datasources
to improve test performance and narrow test scope (#5588) - [MAINTENANCE] Fix tests that rely on guessing pytest generated random file paths. (#5589)
- [MAINTENANCE] Do not set google cloud credentials for lifetime of pytest process. (#5592)
- [MAINTENANCE] Misc updates to
Datasource
CRUD onDataContext
to ensure consistent behavior (#5584) - [MAINTENANCE] Add id to
RuleBasedProfiler
config (#5590) - [MAINTENANCE] refactor to enable customization of quantile bias correction threshold for bootstrap estimation method (#5587)
- [MAINTENANCE] Ensure that
resource_type
used inGeCloudStoreBackend
is converted toGeCloudRESTResource
enum as needed (#5601) - [MAINTENANCE] Create datasource with id (#5591)
- [MAINTENANCE] Enable Azure blob storage integration tests (#5594)
- [MAINTENANCE] Increase expectation kwarg line stroke width (#5608)
- [MAINTENANCE] Added Algolia Scripts (#5544) (thanks @devanshdixit)
- [MAINTENANCE] Handle
numpy
deprecation warnings (#5615) - [MAINTENANCE] remove approximate comparisons -- they will be replaced by estimator alternatives (#5618)
- [MAINTENANCE] Making the dependency on dev-lite clearer (#5514)
- [MAINTENANCE] Fix tests in tests/integration/profiling/rule_based_profiler/ and tests/render/renderer/ (#5611)
- [MAINTENANCE] DataContext in cloud mode test add_datasource (#5625)
0.15.15
- [FEATURE] Integrate
DataContextVariables
withDataContext
(#5466) - [FEATURE] Add mostly to MulticolumnMapExpectation (#5481)
- [FEATURE] [MAINTENANCE] Revamped expect_profile_numeric_columns_diff_between_exclusive_threshold_range (#5493) (thanks @stevensecreti)
- [FEATURE] [CONTRIB] expect_profile_numeric_columns_diff_(less/greater)_than_or_equal_to_threshold (#5522) (thanks @stevensecreti)
- [FEATURE] Provide methods for returning ExpectationConfiguration list grouped by expectation_type and by domain_type (#5532)
- [FEATURE] add support for Azure authentication methods (#5229) (thanks @sdebruyn)
- [FEATURE] Show grouped sorted expectations by Domain and by expectation_type (#5539)
- [FEATURE] Categorical Rule in VolumeDataAssistant Should Use Same Cardinality As Categorical Rule in OnboardingDataAssistant (#5551)
- [BUGFIX] Handle "division by zero" in "ColumnPartition" metric when all column values are NULL (#5507)
- [BUGFIX] Use string dialect name if not found in enum (#5546)
- [BUGFIX] Add
try/except
aroundDataContext._save_project_config
to mitigate issues with permissions (#5550) - [BUGFIX] Explicitly pass in mostly as 1 if not set in configuration. (#5548)
- [BUGFIX] Increase precision for categorical rule for fractional comparisons (#5552)
- [DOCS] DOC-340 partition local installation guide (#5425)
- [DOCS] Add DataHub Ingestion docs (#5330) (thanks @maggiehays)
- [DOCS] toc update for DataHub integration doc (#5518)
- [DOCS] Updating discourse to GitHub Discussions in Docs (#4953)
- [MAINTENANCE] Clean up payload for
/data-context-variables
endpoint to adhere to desired chema (#5509) - [MAINTENANCE] DataContext Refactor: DataAssistants (#5472)
- [MAINTENANCE] Ensure that validation operators are omitted from Cloud variables payload (#5510)
- [MAINTENANCE] Add end-to-end tests for multicolumn map expectations (#5517)
- [MAINTENANCE] Ensure that *_store_name attrs are omitted from Cloud variables payload (#5519)
- [MAINTENANCE] Refactor
key
arg out ofStore.serialize/deserialize
(#5511) - [MAINTENANCE] Fix links to documentation (#5177) (thanks @andyjessen)
- [MAINTENANCE] Readme Update (#4952)
- [MAINTENANCE] E2E test for
FileDataContextVariables
(#5516) - [MAINTENANCE] Cleanup/refactor prerequisite for group/filter/sort Expectations by domain (#5523)
- [MAINTENANCE] Refactor
GeCloudStoreBackend
to use PUT and DELETE HTTP verbs instead of PATCH (#5527) - [MAINTENANCE]
/profiler
Cloud endpoint support (#5499) - [MAINTENANCE] Add type hints to
Store
(#5529) - [MAINTENANCE] Move MetricDomainTypes to core (it is used more widely now than previously). (#5530)
- [MAINTENANCE] Remove dependency pins on pyarrow and snowflake-connector-python (#5533)
- [MAINTENANCE] use invoke for common contrib/dev tasks (#5506)
- [MAINTENANCE] Add snowflake-connector-python dependency lower bound. (#5538)
- [MAINTENANCE] enforce pre-commit in ci (#5526)
- [MAINTENANCE] Providing more robust error handling for determining
domain_type
of anExpectationConfiguration
object (#5542) - [MAINTENANCE] Remove extra indentation from store backend test (#5545)
- [MAINTENANCE] Plot-level dropdown for
DataAssistantResult
display charts (#5528) - [MAINTENANCE] Make DataAssistantResult.batch_id_to_batch_identifier_display_name_map private (in order to optimize auto-complete for ease of use) (#5549)
- [MAINTENANCE] Initial Dockerfile for running tests and associated README. (#5541)
- [MAINTENANCE] Other dialect test (#5547)
0.15.14
- [FEATURE] QueryExpectations (#5223)
- [FEATURE] Control volume of metadata output when running DataAssistant classes. (#5483)
- [BUGFIX] Snowflake Docs Integration Test Fix (#5463)
- [BUGFIX] DataProfiler Linting Fix (#5468)
- [BUGFIX] Update renderer snapshots with
None
values removed (#5474) - [BUGFIX] Rendering Test failures (#5475)
- [BUGFIX] Update
dependency-graph
pipeline YAML to ensure--spark
gets passed todgtest
(#5477) - [BUGFIX] Make sure the profileReport obj does not have defaultdicts (breaks gallery JSON) (#5491)
- [BUGFIX] Use Pandas.isnull() instead of NumPy.isnan() to check for empty values in TableExpectation._validate_metric_value_between(), due to wider types applicability. (#5502)
- [BUGFIX] Spark Schema has unexpected field for
spark.sql.warehouse.dir
(#5490) - [BUGFIX] Conditionally pop values from Spark config in tests (#5508)
- [DOCS] DOC-349 re-write and partition interactive mode expectations guide (#5448)
- [DOCS] DOC-344 partition data docs on s3 guide (#5437)
- [DOCS] DOC-342 partition how to configure a validation result store in amazon s3 guide (#5428)
- [DOCS] link fix in onboarding data assistant guide (#5469)
- [DOCS] Integrate great-expectation with ydata-synthetic (#4568) (thanks @arunnthevapalan)
- [DOCS] Add 'test' extra to setup.py with docs (#5415)
- [DOCS] DOC-343 partition how to configure expectation store for aws s3 guide (#5429)
- [DOCS] DOC-357 partition the how to create a new checkpoint guide (#5458)
- [DOCS] Remove outdated release process docs. (#5484)
- [MAINTENANCE] Update
teams.yml
(#5457) - [MAINTENANCE] Clean up GitHub Actions (#5461)
- [MAINTENANCE] Adds documentation and examples changes for snowflake connection string (#5447)
- [MAINTENANCE] DOC-345 partition the connect to s3 cloud storage with Pandas guide (#5439)
- [MAINTENANCE] Add unit and integration tests for Splitting on Mod Integer (#5452)
- [MAINTENANCE] Remove
InlineRenderer
invocation feature flag fromExpectationValidationResult
(#5441) - [MAINTENANCE]
DataContext
Refactor. Migration of datasource and store (#5404) - [MAINTENANCE] Add unit and integration tests for Splitting on Multi-Column Values (#5464)
- [MAINTENANCE] Refactor
DataContextVariables
to leverage@property
and@setter
(#5446) - [MAINTENANCE] expect_profile_numeric_columns_diff_between_threshold_range (#5467) (thanks @stevensecreti)
- [MAINTENANCE] Make
DataAssistantResult
fixtures module scoped (#5465) - [MAINTENANCE] Remove keyword arguments within table row count expectations (#4874) (thanks @andyjessen)
- [MAINTENANCE] Add unit tests for Splitting on Converted DateTime (#5470)
- [MAINTENANCE] Rearrange integration tests to insure categorization into proper deployment-style based lists (#5471)
- [MAINTENANCE] Provide better error messaging if batch_request is not supplied to DataAssistant.run() (#5473)
- [MAINTENANCE] Adds run time envvar for Snowflake Partner ID (#5485)
- [MAINTENANCE] fixed algolia search page (#5099)
- [MAINTENANCE] Remove pyspark<3.0.0 constraint for python 3.7 (#5496)
- [MAINTENANCE] Ensure that
parter-integration
pipeline only runs on cronjob (#5500) - [MAINTENANCE] Adding fixtures Query Expectations tests (#5486)
- [MAINTENANCE] Misc updates to
GeCloudStoreBackend
to better integrate with GE Cloud (#5497) - [MAINTENANCE] Update automated release schedule (#5488)
- [MAINTENANCE] Update core-team in
teams.yml
(#5489) - [MAINTENANCE] Update how_to_create_a_new_expectation_suite_using_rule_based_profile… (#5495)
- [MAINTENANCE] Remove pypandoc pin in constraints-dev.txt. (#5501)
- [MAINTENANCE] Ensure that
add_datasource
method onAbstractDataContext
does not persist by default (#5482)
0.15.13
- [FEATURE] Add atomic
rendered_content
toExpectationValidationResult
andExpectationConfiguration
(#5369) - [FEATURE] Add
DataContext.update_datasource
CRUD method (#5417) - [FEATURE] Refactor Splitter Testing Modules so as to Make them More General and Add Unit and Integration Tests for "split_on_whole_table" and "split_on_column_value" on SQLite and All Supported Major SQL Backends (#5430)
- [FEATURE] Support underscore in the
condition_value
of arow_condition
(#5393) (thanks @sp1thas) - [DOCS] DOC-322 update terminology to v3 (#5326)
- [MAINTENANCE] Change property name of TaxiSplittingTestCase to make it more general (#5419)
- [MAINTENANCE] Ensure that
BaseDataContext
does not persistDatasource
changes by default (#5423) - [MAINTENANCE] Migration of
project_config_with_variables_substituted
toAbstractDataContext
(#5385) - [MAINTENANCE] Improve type hinting in
GeCloudStoreBackend
(#5427) - [MAINTENANCE] Test serialization of text, table, and bulleted list
rendered_content
inExpectationValidationResult
(#5438) - [MAINTENANCE] Refactor
datasource_name
out ofDataContext.update_datasource
(#5440) - [MAINTENANCE] Add checkpoint name to validation results (#5442)
- [MAINTENANCE] Remove checkpoint from top level of schema since it is captured in
meta
(#5445) - [MAINTENANCE] Add unit and integration tests for Splitting on Divided Integer (#5449)
- [MAINTENANCE] Update cli with new default simple checkpoint name (#5450)
0.15.12
- [FEATURE] Add Rule Statistics to DataAssistantResult for display in Jupyter notebook (#5368)
- [FEATURE] Include detailed Rule Execution statistics in jupyter notebook "repr" style output (#5375)
- [FEATURE] Support datetime/date-part splitters on Amazon Redshift (#5408)
- [DOCS] Capital One DataProfiler Expectations README Update (#5365) (thanks @stevensecreti)
- [DOCS] Add Trino guide (#5287)
- [DOCS] DOC-339 remove redundant how-to guide (#5396)
- [DOCS] Capital One Data Profiler README update (#5387) (thanks @taylorfturner)
- [DOCS] Add sqlalchemy-redshfit to dependencies in redshift doc (#5386)
- [MAINTENANCE] Reduce output amount in Jupyter notebooks when displaying DataAssistantResult (#5362)
- [MAINTENANCE] Update linter thresholds (#5367)
- [MAINTENANCE] Move
_apply_global_config_overrides()
to AbstractDataContext (#5285) - [MAINTENANCE] WIP: [MAINTENANCE] stalebot configuration (#5301)
- [MAINTENANCE] expect_column_values_to_be_equal_to_or_greater_than_profile_min (#5372) (thanks @stevensecreti)
- [MAINTENANCE] expect_column_values_to_be_equal_to_or_less_than_profile_max (#5380) (thanks @stevensecreti)
- [MAINTENANCE] Replace string formatting with f-string (#5225) (thanks @andyjessen)
- [MAINTENANCE] Fix links in docs (#5340) (thanks @andyjessen)
- [MAINTENANCE] Caching of
config_variables
inDataContext
(#5376) - [MAINTENANCE] StaleBot Half DryRun (#5390)
- [MAINTENANCE] StaleBot DryRun 2 (#5391)
- [MAINTENANCE] file extentions applied to rel links (#5399)
- [MAINTENANCE] Allow installing jinja2 version 3.1.0 and higher (#5382)
- [MAINTENANCE] expect_column_values_confidence_for_data_label_to_be_less_than_or_equal_to_threshold (#5392) (thanks @stevensecreti)
- [MAINTENANCE] Add warnings to internal linters if actual error count does not match threshold (#5401)
- [MAINTENANCE] Ensure that changes made to env vars / config vars are recognized within subsequent calls of the same process (#5410)
- [MAINTENANCE] Stack
RuleBasedProfiler
progress bars for better user experience (#5400) - [MAINTENANCE] Keep all Pandas Splitter Tests in a Dedicated Module (#5411)
- [MAINTENANCE] Refactor DataContextVariables to only persist state to Store using explicit save command (#5366)
- [MAINTENANCE] Refactor to put tests for splitting and sampling into modules for respective ExecutionEngine implementation (#5412)
0.15.11
- [FEATURE] Enable NumericMetricRangeMultiBatchParameterBuilder to use evaluation dependencies (#5323)
- [FEATURE] Improve Trino Support (#5261) (thanks @aezomz)
- [FEATURE] added support to Aws Athena quantiles (#5114) (thanks @kuhnen)
- [FEATURE] Implement the "column.standard_deviation" metric for sqlite database (#5338)
- [FEATURE] Update
add_datasource
to leverage theDatasourceStore
(#5334) - [FEATURE] Provide ability for DataAssistant to return its effective underlying BaseRuleBasedProfiler configuration (#5359)
- [BUGFIX] Fix Netlify build issue that was being caused by entry in changelog (#5322)
- [BUGFIX] Numpy dtype.float64 formatted floating point numbers must be converted to Python float for use in SQLAlchemy Boolean clauses (#5336)
- [BUGFIX] Fix for failing Expectation test in
cloud_db_integration
pipeline (#5321) - [DOCS] revert getting started tutorial to RBP process (#5307)
- [DOCS] mark onboarding assistant guide as experimental and update cli command (#5308)
- [DOCS] Fix line numbers in getting started guide (#5324)
- [DOCS] DOC-337 automate updates to the version information displayed in the getting started tutorial. (#5348)
- [MAINTENANCE] Fix link in suite profile renderer (#5242) (thanks @andyjessen)
- [MAINTENANCE] Refactor of
_apply_global_config_overrides()
method to return config (#5286) - [MAINTENANCE] Remove "json_serialize" directive from ParameterBuilder computations (#5320)
- [MAINTENANCE] Misc cleanup post
0.15.10
release (#5325) - [MAINTENANCE] Standardize instantiation of NumericMetricRangeMultibatchParameterBuilder throughout the codebase. (#5327)
- [MAINTENANCE] Reuse MetricMultiBatchParameterBuilder computation results as evaluation dependencies for performance enhancement (#5329)
- [MAINTENANCE] clean up type declarations (#5331)
- [MAINTENANCE] Maintenance/great 761/great 1010/great 1011/alexsherstinsky/rule based profiler/data assistant/include only essential public methods in data assistant dispatcher class 2022 06 21 177 (#5351)
- [MAINTENANCE] Update release schedule JSON (#5349)
- [MAINTENANCE] Include only essential public methods in DataAssistantResult class (and its descendants) (#5360)
0.15.10
- [FEATURE]
DataContextVariables
CRUD forstores
(#5268) - [FEATURE]
DataContextVariables
CRUD fordata_docs_sites
(#5269) - [FEATURE]
DataContextVariables
CRUD foranonymous_usage_statistics
(#5271) - [FEATURE]
DataContextVariables
CRUD fornotebooks
(#5272) - [FEATURE]
DataContextVariables
CRUD forconcurrency
(#5273) - [FEATURE]
DataContextVariables
CRUD forprogress_bars
(#5274) - [FEATURE] Integrate
DatasourceStore
withDataContext
(#5292) - [FEATURE] Support both UserConfigurableProfiler and OnboardingDataAssistant in "CLI SUITE NEW --PROFILE name" command (#5306)
- [BUGFIX] Fix ColumnPartition metric handling of the number of bins (must always be integer). (#5282)
- [BUGFIX] Add new high precision rule for mean and stdev in
OnboardingDataAssistant
(#5276) - [BUGFIX] Warning in Getting Started Guide notebook. (#5297)
- [DOCS] how to create an expectation suite with the onboarding assistant (#5266)
- [DOCS] update getting started tutorial for onboarding assistant (#5294)
- [DOCS] getting started tutorial doc standards updates (#5295)
- [DOCS] Update standard arguments doc for Expectations to not reference datasets. (#5052)
- [MAINTENANCE] Add check to
check_type_hint_coverage
script to ensure propermypy
installation (#5291) - [MAINTENANCE]
DataAssistantResult
cleanup and extensibility enhancements (#5259) - [MAINTENANCE] Handle compare Expectation in presence of high precision floating point numbers and NaN values (#5298)
- [MAINTENANCE] Suppress persisting of temporary ExpectationSuite configurations in Rule-Based Profiler computations (#5305)
- [MAINTENANCE] Adds column values github user validation (#5302)
- [MAINTENANCE] Adds column values IATA code validation (#5303)
- [MAINTENANCE] Adds column values ARN validation (#5304)
- [MAINTENANCE] Fixing a typo in a comment (in several files) (#5310)
- [MAINTENANCE] Adds column scientific notation string validation (#5309)
- [MAINTENANCE] lint fixes (#5312)
- [MAINTENANCE] Adds column value JSON validation (#5313)
- [MAINTENANCE] Expect column values to be valid scientific notation (#5311)
0.15.9
- [FEATURE] Add new expectation: expect column values to match powers of a base g… (#5219) (thanks @rifatKomodoDragon)
- [FEATURE] Replace UserConfigurableProfiler with OnboardingDataAssistant in "CLI suite new --profile" Jupyter Notebooks (#5236)
- [FEATURE]
DatasourceStore
(#5206) - [FEATURE] add new expectation on validating hexadecimals (#5188) (thanks @andrewsx)
- [FEATURE] Usage Statistics Events for Profiler and DataAssistant "get_expectation_suite()" methods. (#5251)
- [FEATURE]
InlineStoreBackend
(#5216) - [FEATURE] The "column.histogram" metric must support integer values of the "bins" parameter for all execution engine options. (#5258)
- [FEATURE] Initial implementation of
DataContextVariables
accessors (#5238) - [FEATURE]
OnboardingDataAssistant
plots forexpect_table_columns_to_match_set
(#5208) - [FEATURE]
DataContextVariables
CRUD forconfig_variables_file_path
(#5262) - [FEATURE]
DataContextVariables
CRUD forplugins_directory
(#5263) - [FEATURE]
DataContextVariables
CRUD for store name accessors (#5264) - [BUGFIX] Hive temporary tables creation fix (#4956) (thanks @jaume-ferrarons)
- [BUGFIX] Provide error handling when metric fails for all Batch data samples (#5256)
- [BUGFIX] Patch automated release test date comparisons (#5278)
- [DOCS] How to compare two tables with the UserConfigurableProfiler (#5050)
- [DOCS] How to create a Custom Column Pair Map Expectation w/ supporting template & example (#4926)
- [DOCS] Auto API documentation script (#4964)
- [DOCS] Update formatting of links to public methods in class docs generated by auto API script (#5247)
- [DOCS] In the reference section of the ToC remove duplicates and update category pages (#5248)
- [DOCS] Update DataContext docstring (#5250)
- [MAINTENANCE] Add CodeSee architecture diagram workflow to repository (#5235) (thanks @codesee-maps[bot])
- [MAINTENANCE] Fix links to API docs (#5246) (thanks @andyjessen)
- [MAINTENANCE] Unpin cryptography upper bound (#5249)
- [MAINTENANCE] Don't use jupyter-client 7.3.2 (#5252)
- [MAINTENANCE] Re-introduce jupyter-client 7.3.2 (#5253)
- [MAINTENANCE] Add
cloud
mark topytest.ini
(#5254) - [MAINTENANCE] add partner integration framework (#5132)
- [MAINTENANCE]
DataContextVariableKey
for use in Stores (#5255) - [MAINTENANCE] Clarification of events in test with multiple checkpoint validations (#5257)
- [MAINTENANCE] Misc updates to improve security and automation of the weekly release process (#5244)
- [MAINTENANCE] show more test output and minor fixes (#5239)
- [MAINTENANCE] Add proper unit tests for Column Histogram metric and use Column Value Partitioner in OnboardingDataAssistant (#5267)
- [MAINTENANCE] Updates contributor docs to reflect updated linting guidance (#4909)
- [MAINTENANCE] Remove condition from
autoupdate
GitHub action (#5270) - [MAINTENANCE] Improve code readability in the processing section of "MapMetricColumnDomainBuilder". (#5279)
0.15.8
- [FEATURE]
OnboardingDataAssistant
plots forexpect_table_row_count_to_be_between
non-sequential batches (#5212) - [FEATURE] Limit sampling for spark and pandas (#5201)
- [FEATURE] Groundwork for DataContext Refactor (#5203)
- [FEATURE] Implement ability to change rule variable values through DataAssistant run() method arguments at runtime (#5218)
- [FEATURE] Plot numeric column domains in
OnboardingDataAssistant
(#5189) - [BUGFIX] Repair "CLI Suite --Profile" Operation (#5230)
- [DOCS] Remove leading underscore from sampling docs (#5214)
- [MAINTENANCE] suppressing type hints in ill-defined situations (#5213)
- [MAINTENANCE] Change CategoricalColumnDomainBuilder property name from "limit_mode" to "cardinality_limit_mode". (#5215)
- [MAINTENANCE] Update Note in BigQuery Docs (#5197)
- [MAINTENANCE] Sampling cleanup refactor (use BatchSpec in sampling methods) (#5217)
- [MAINTENANCE] Globally increase Azure timeouts to 120 mins (#5222)
- [MAINTENANCE] Comment out kl_divergence for build_gallery (#5196)
- [MAINTENANCE] Fix docstring on expectation (#5204) (thanks @andyjessen)
- [MAINTENANCE] Improve NaN handling in numeric ParameterBuilder implementations (#5226)
- [MAINTENANCE] Update type hint and docstring linter thresholds (#5228)
0.15.7
- [FEATURE] Add Rule for TEXT semantic domains within the Onboarding Assistant (#5144)
- [FEATURE] Helper method to determine whether Expectation is self-initializing (#5159)
- [FEATURE] OnboardingDataAssistantResult plotting feature parity with VolumeDataAssistantResult (#5145)
- [FEATURE] Example Notebook for self-initializing
Expectations
(#5169) - [FEATURE] DataAssistant: Enable passing directives to run() method using runtime_environment argument (#5187)
- [FEATURE] Adding DataAssistantResult.get_expectation_suite(expectation_suite_name) method (#5191)
- [FEATURE] Cronjob to automatically create release PR (#5181)
- [BUGFIX] Insure TABLE Domain Metrics Do Not Get Column Key From Column Type Rule Domain Builder (#5166)
- [BUGFIX] Update name for stdev expectation in
OnboardingDataAssistant
backend (#5193) - [BUGFIX] OnboardingDataAssistant and Underlying Metrics: Add Defensive Programming Into Metric Implementations So As To Avoid Warnings About Incompatible Data (#5195)
- [BUGFIX] Insure that Histogram Metric in Pandas operates on numerical columns that do not have NULL values (#5199)
- [BUGFIX] RuleBasedProfiler: Ensure that run() method runtime environment directives are handled correctly when existing setting is None (by default) (#5202)
- [BUGFIX] In aggregate metrics, Spark Implementation already gets Column type as argument -- no need for F.col() as the operand is not a string. (#5207)
- [DOCS] Update ToC with category links (#5155)
- [DOCS] update on availability and parameters of conditional expectations (#5150)
- [MAINTENANCE] Helper method for RBP Notebook tests that does clean-up (#5171)
- [MAINTENANCE] Increase timeout for longer stages in Azure pipelines (#5175)
- [MAINTENANCE] Rule-Based Profiler -- In ParameterBuilder insure that metrics are validated for conversion to numpy array (to avoid deprecation warnings) (#5173)
- [MAINTENANCE] Increase timeout in packaging & installation pipeline (#5178)
- [MAINTENANCE] OnboardingDataAssistant handle multiple expectations per domain (#5170)
- [MAINTENANCE] Update timeout in pipelines to fit Azure syntax (#5180)
- [MAINTENANCE] Error message when
Validator
is instantiated with IncorrectBatchRequest
(#5172) - [MAINTENANCE] Don't include infinity in rendered string for diagnostics (#5190)
- [MAINTENANCE] Mark Great Expectations Cloud tests and add stage to CI/CD (#5186)
- [MAINTENANCE] Trigger expectation gallery build with scheduled CI/CD runs (#5192)
- [MAINTENANCE]
expectation_gallery
Azure pipeline (#5194) - [MAINTENANCE] General cleanup/refactor of
DataAssistantResult
(#5198)
0.15.6
- [FEATURE]
NumericMetricRangeMultiBatchParameterBuilder
kernel density estimation (#5084) - [FEATURE] Splitters and limit sample work on AWS Athena (#5024)
- [FEATURE]
ColumnValuesLengthMin
andColumnValuesLengthMax
metrics (#5107) - [FEATURE] Use
batch_identifiers
in plot tooltips (#5091) - [FEATURE] Updated
DataAssistantResult
plotting API (#5117) - [FEATURE] Onboarding DataAssistant: Numeric Rules and Relevant Metrics (#5120)
- [FEATURE] DateTime Rule for OnboardingDataAssistant (#5121)
- [FEATURE] Categorical Rule is added to OnboardingDataAssistant (#5134)
- [FEATURE] OnboardingDataAssistant: Introduce MeanTableColumnsSetMatchMultiBatchParameterBuilder (to enable expect_table_columns_to_match_set) (#5135)
- [FEATURE] Giving the "expect_table_columns_to_match_set" Expectation Self-Initializing Capabilities. (#5136)
- [FEATURE] For OnboardingDataAssistant: Implement a TABLE Domain level rule to output "expect_table_columns_to_match_set" (#5137)
- [FEATURE] Enable self-initializing
ExpectColumnValueLengthsToBeBetween
(#4985) - [FEATURE]
DataAssistant
plotting for non-sequential batches (#5126) - [BUGFIX] Insure that Batch IDs are accessible in the order in which they were loaded in Validator (#5112)
- [BUGFIX] Update
DataAssistant
notebook for new plotting API (#5118) - [BUGFIX] For DataAssistants, added try-except for Notebook tests (#5124)
- [BUGFIX] CategoricalColumnDomainBuilder needs to accept limit_mode with dictionary type (#5127)
- [BUGFIX] Use
external_sqldialect
mark to skip during lightweight runs (#5139) - [BUGFIX] Use RANDOM_STATE in fixture to make tests deterministic (#5142)
- [BUGFIX] Read deployment_version instead of using versioneer in deprecation tests (#5147)
- [MAINTENANCE] DataAssistant: Refactoring Access to common ParameterBuilder instances (#5108)
- [MAINTENANCE] Refactor of
MetricTypes
andAttributedResolvedMetrics
(#5100) - [MAINTENANCE] Remove references to show_cta_footer except in schemas.py (#5111)
- [MAINTENANCE] Adding unit tests for sqlalchemy limit sampler part 1 (#5109)
- [MAINTENANCE] Don't re-raise connection errors in CI (#5115)
- [MAINTENANCE] Sqlite specific tests for splitting and sampling (#5119)
- [MAINTENANCE] Add Trino dialect in SqlAlchemyDataset (#5085) (thanks @ms32035)
- [MAINTENANCE] Move upper bound on sqlalchemy to <2.0.0. (#5140)
- [MAINTENANCE] Update primary pipeline to cut releases with tags (#5128)
- [MAINTENANCE] Improve handling of "expect_column_unique_values_count_to_be_between" in VolumeDataAssistant (#5146)
- [MAINTENANCE] Simplify DataAssistant Operation to not Depend on Self-Initializing Expectations (#5148)
- [MAINTENANCE] Improvements to Trino support (#5152)
- [MAINTENANCE] Update how_to_configure_a_new_checkpoint_using_test_yaml_config.md (#5157)
- [MAINTENANCE] Speed up the site builder (#5125) (thanks @tanelk)
- [MAINTENANCE] remove account id deprecation notice (#5158)
0.15.5
- [FEATURE] Add subset operation to Domain class (#5049)
- [FEATURE] In DataAssistant: Use Domain instead of domain_type as key for Metrics Parameter Builders (#5057)
- [FEATURE] Self-initializing
ExpectColumnStddevToBeBetween
(#5065) - [FEATURE] Enum used by DateSplitter able to be represented as YAML (#5073)
- [FEATURE] Implementation of auto-complete for DataAssistant class names in Jupyter notebooks (#5077)
- [FEATURE] Provide display ("friendly") names for batch identifiers (#5086)
- [FEATURE] Onboarding DataAssistant -- Initial Rule Implementations (Data Aspects) (#5101)
- [FEATURE] OnboardingDataAssistant: Implement Nullity/Non-nullity Rules and Associated Metrics (#5104)
- [BUGFIX]
self_check()
now also checks foraws_config_file
(#5040) - [BUGFIX]
multi_batch_rule_based_profiler
test up to date with RBP changes (#5066) - [BUGFIX] Splitting Support at Asset level (#5026)
- [BUGFIX] Make self-initialization in expect_column_values_to_be_between truly multi batch (#5068)
- [BUGFIX] databricks engine create temporary view (#4994) (thanks @gvillafanetapia)
- [BUGFIX] Patch broken Expectation gallery script (#5090)
- [BUGFIX] Sampling support at asset level (#5092)
- [DOCS] Update process and configurations in OpenLineage Action guide. (#5039)
- [DOCS] Update process and config examples in Opsgenie guide (#5037)
- [DOCS] Correct name of
openlineage-integration-common
package (#5041) (thanks @mobuchowski) - [DOCS] Remove reference to validation operator process from how to trigger slack notifications guide (#5034)
- [DOCS] Update process and configuration examples in email Action guide. (#5036)
- [DOCS] Update Docusaurus version (#5063)
- [MAINTENANCE] Saved output of usage stats schema script in repo (#5053)
- [MAINTENANCE] Apply Altair custom themes to return objects (#5044)
- [MAINTENANCE] Introducing RuleBasedProfilerResult -- neither expectation suite name nor expectation suite must be passed to RuleBasedProfiler.run() (#5061)
- [MAINTENANCE] Refactor
DataAssistant
plotting to leverage utility dataclasses (#5022) - [MAINTENANCE] Check that a passed string is parseable as an integer (mssql limit param) (#5071)
- [MAINTENANCE] Clean up mssql limit sampling code path and comments (#5074)
- [MAINTENANCE] Make saving bootstraps histogram for NumericMetricRangeMultiBatchParameterBuilder optional (absent by default) (#5075)
- [MAINTENANCE] Make self-initializing expectations return estimated kwargs with auto-generation timestamp and Great Expectation version (#5076)
- [MAINTENANCE] Adding a unit test for batch_id mapping to batch display names (#5087)
- [MAINTENANCE]
pypandoc
version constraint added (< 1.8
) (#5093) - [MAINTENANCE] Utilize Rule objects in Profiler construction in DataAssistant (#5089)
- [MAINTENANCE] Turn off metric calculation progress bars in
RuleBasedProfiler
andDataAssistant
workflows (#5080) - [MAINTENANCE] A small refactor of ParamerBuilder management used in DataAssistant classes (#5102)
- [MAINTENANCE] Convenience method refactor for Onboarding DataAssistant (#5103)
0.15.4
- [FEATURE] Enable self-initializing
ExpectColumnMeanToBeBetween
(#4986) - [FEATURE] Enable self-initializing
ExpectColumnMedianToBeBetween
(#4987) - [FEATURE] Enable self-initializing
ExpectColumnSumToBeBetween
(#4988) - [FEATURE] New MetricSingleBatchParameterBuilder for specifically single-Batch Rule-Based Profiler scenarios (#5003)
- [FEATURE] Enable Pandas DataFrame and Series as MetricValues Output of Metric ParameterBuilder Classes (#5008)
- [FEATURE] Notebook for
VolumeDataAssistant
Example (#5010) - [FEATURE] Histogram/Partition Single-Batch ParameterBuilder (#5011)
- [FEATURE] Update
DataAssistantResult.plot()
return value to emitPlotResult
wrapper dataclass (#4962) - [FEATURE] Limit samplers work with supported sqlalchemy backends (#5014)
- [FEATURE] trino support (#5021)
- [BUGFIX] RBP Profiling Dataset ProgressBar Fix (#4999)
- [BUGFIX] Fix DataAssistantResult serialization issue (#5020)
- [DOCS] Update slack notification guide to not use validation operators. (#4978)
- [MAINTENANCE] Update
autoupdate
GitHub action (#5001) - [MAINTENANCE] Move
DataAssistant
registry capabilities intoDataAssistantRegistry
to enable user aliasing (#4991) - [MAINTENANCE] Fix continuous partition example (#4939) (thanks @andyjessen)
- [MAINTENANCE] Preliminary refactors for data samplers. (#4996)
- [MAINTENANCE] Clean up unused imports and enforce through
flake8
in CI/CD (#5005) - [MAINTENANCE] ParameterBuilder tests should maximally utilize polymorphism (#5007)
- [MAINTENANCE] Clean up type hints in CLI (#5006)
- [MAINTENANCE] Making ParameterBuilder metric computations robust to failures through logging and exception handling (#5009)
- [MAINTENANCE] Condense column-level
vconcat
plots into one interactive plot (#5002) - [MAINTENANCE] Update version of
black
in pre-commit config (#5019) - [MAINTENANCE] Improve tooltips and formatting for distinct column values chart in VolumeDataAssistantResult (#5017)
- [MAINTENANCE] Enhance configuring serialization for DotDict type classes (#5023)
- [MAINTENANCE] Pyarrow upper bound (#5028)
0.15.3
- [FEATURE] Enable self-initializing capabilities for
ExpectColumnProportionOfUniqueValuesToBeBetween
(#4929) - [FEATURE] Enable support for plotting both Table and Column charts in
VolumeDataAssistant
(#4930) - [FEATURE] BigQuery Temp Table Support (#4925)
- [FEATURE] Registry for DataAssistant classes with ability to execute from DataContext by registered name (#4966)
- [FEATURE] Enable self-intializing capabilities for
ExpectColumnValuesToMatchRegex
/ExpectColumnValuesToNotMatchRegex
(#4958) - [FEATURE] Provide "estimation histogram" ParameterBuilder output details . (#4975)
- [FEATURE] Enable self-initializing
ExpectColumnValuesToMatchStrftimeFormat
(#4977) - [BUGFIX] check contrib requirements (#4922)
- [BUGFIX] Use
monkeypatch
to set a consistent bootstrap seed in tests (#4960) - [BUGFIX] Make all Builder Configuration classes of Rule-Based Profiler Configuration Serializable (#4972)
- [BUGFIX] extras_require (#4968)
- [BUGFIX] Fix broken packaging test and update
dgtest-overrides
(#4976) - [MAINTENANCE] Add timeout to
great_expectations
pipeline stages to prevent false positive build failures (#4957) - [MAINTENANCE] Defining Common Test Fixtures for DataAssistant Testing (#4959)
- [MAINTENANCE] Temporarily pin
cryptography
package (#4963) - [MAINTENANCE] Type annotate relevant functions with
-> None
(per PEP 484) (#4969) - [MAINTENANCE] Handle edge cases where
false_positive_rate
is not in range [0, 1] or very close to bounds (#4946) - [MAINTENANCE] fix a typo (#4974)
0.15.2
- [FEATURE] Split data assets using sql datetime columns (#4871)
- [FEATURE] Plot metrics with
DataAssistantResult.plot()
(#4873) - [FEATURE] RuleBasedProfiler/DataAssistant/MetricMultiBatchParameterBuilder: Enable Returning Metric Computation Results with batch_id Attribution (#4862)
- [FEATURE] Enable variables to be specified at both Profiler and its constituent individual Rule levels (#4912)
- [FEATURE] Enable self-initializing
ExpectColumnUniqueValueCountToBeBetween
(#4902) - [FEATURE] Improve diagnostic testing process (#4816)
- [FEATURE] Add Azure CI/CD action to aid with style guide enforcement (type hints) (#4878)
- [FEATURE] Add Azure CI/CD action to aid with style guide enforcement (docstrings) (#4617)
- [FEATURE] Use formal interfaces to clean up DataAssistant and DataAssistantResult modules/classes (#4901)
- [BUGFIX] fix validation issue for column domain type and implement expect_column_unique_value_count_to_be_between for VolumeDataAssistant (#4914)
- [BUGFIX] Fix issue with not using the generated table name on read (#4905)
- [BUGFIX] Add deprecation comment to RuntimeDataConnector
- [BUGFIX] Ensure proper class_name within all RuleBasedProfilerConfig instantiations
- [BUGFIX] fix rounding directive handling (#4887)
- [BUGFIX]
great_expectations
import fails when SQL Alchemy is not installed (#4880) - [MAINTENANCE] Altair types cleanup (#4916)
- [MAINTENANCE] test: update test time (#4911)
- [MAINTENANCE] Add module docstring and simplify access to DatePart (#4910)
- [MAINTENANCE] Chip away at type hint violations around data context (#4897)
- [MAINTENANCE] Improve error message outputted to user in DocstringChecker action (#4895)
- [MAINTENANCE] Re-enable bigquery tests (#4903)
- [MAINTENANCE] Unit tests for sqlalchemy splitter methods, docs and other improvements (#4900)
- [MAINTENANCE] Move plot logic from
DataAssistant
intoDataAssistantResult
(#4896) - [MAINTENANCE] Add condition to primary pipeline to ensure
import_ge
stage doesn't cause misleading Slack notifications (#4898) - [MAINTENANCE] Refactor
RuleBasedProfilerConfig
(#4882) - [MAINTENANCE] Refactor DataAssistant Access to Parameter Computation Results and Plotting Utilities (#4893)
- [MAINTENANCE] Update
dgtest-overrides
list to include all test files not captured by primary strategy (#4891) - [MAINTENANCE] Add dgtest-overrides section to dependency_graph Azure pipeline
- [MAINTENANCE] Data Source and DataContext-level tests for RuntimeDataConnector changes (#4866)
- [MAINTENANCE] Temporarily disable bigquery tests. (#4888)
- [MAINTENANCE] Import GE after running
ge init
in packaging CI pipeline (#4885) - [MAINTENANCE] Add CI stage importing GE with only required dependencies installed (#4884)
- [MAINTENANCE]
DataAssistantResult.plot()
conditional formatting and tooltips (#4881) - [MAINTENANCE] split data context files (#4879)
- [MAINTENANCE] Add Tanner to CODEOWNERS for schemas.py (#4875)
- [MAINTENANCE] Use defined constants for ParameterNode accessor keys (#4872)
0.15.1
- [FEATURE] Additional Rule-Based Profiler Parameter/Variable Access Methods (#4814)
- [FEATURE] DataAssistant and VolumeDataAssistant classes (initial implementation -- to be enhanced as part of subsequent work) (#4844)
- [FEATURE] Add Support for Returning Parameters and Metrics as DataAssistantResult class (#4848)
- [FEATURE] DataAssistantResult Includes Underlying Profiler Execution Time (#4854)
- [FEATURE] Add batch_id for every resolved metric_value to ParameterBuilder.get_metrics() result object (#4860)
- [FEATURE]
RuntimeDataConnector
able to specifyAssets
(#4861) - [BUGFIX] Linting error from hackathon automerge (#4829)
- [BUGFIX] Cleanup contrib (#4838)
- [BUGFIX] Add
notebook
toGE_REQUIRED_DEPENDENCIES
(#4842) - [BUGFIX] ParameterContainer return value formatting bug fix (#4840)
- [BUGFIX] Ensure that Parameter Validation/Configuration Dependency Configurations are included in Serialization (#4843)
- [BUGFIX] Correctly handle SQLA unexpected count metric for empty tables (#4618) (thanks @douglascook)
- [BUGFIX] Temporarily adjust Deprecation Warning Count (#4869)
- [DOCS] How to validate data with an in memory checkpoint (#4820)
- [DOCS] Update all tutorial redirect fix (#4841)
- [DOCS] redirect/remove dead links in docs (#4846)
- [MAINTENANCE] Refactor Rule-Based Profiler instantiation in Validator to make it available as a public method (#4823)
- [MAINTENANCE] String Type is not needed as Return Type from DomainBuilder.domain_type() (#4827)
- [MAINTENANCE] Fix Typo in Checkpoint Readme (#4835) (thanks @andyjessen)
- [MAINTENANCE] Modify conditional expectations readme (#4616) (thanks @andyjessen)
- [MAINTENANCE] Fix links within datasource new notebook (#4833) (thanks @andyjessen)
- [MAINTENANCE] Adds missing dependency, which is breaking CLI workflows (#4839)
- [MAINTENANCE] Update testing and documentation for
oneshot
estimation method (#4852) - [MAINTENANCE] Refactor
Datasource
tests that work withRuntimeDataConnector
by backend. (#4853) - [MAINTENANCE] Update DataAssistant interfaces (#4857)
- [MAINTENANCE] Improve types returned by DataAssistant interface methods (#4859)
- [MAINTENANCE] Refactor
DataContext
tests that work with RuntimeDataConnector by backend (#4858) - [HACKATHON] Hackathon PRs in this release
0.15.0
- [BREAKING] EOL Python 3.6 (#4567)
- [FEATURE] Implement Multi-Column Domain Builder for Rule-Based Profiler (#4604)
- [FEATURE] Update RBP notebook to include example for Multi-Column Domain Builder (#4606)
- [FEATURE] Rule-Based Profiler: ColumnPairDomainBuilder (#4608)
- [FEATURE] More package contrib info (#4693)
- [FEATURE] Introducing RuleState class and RuleOutput class for Rule-Based Profiler in support of richer use cases (such as DataAssistant). (#4704)
- [FEATURE] Add support for returning fully-qualified parameters names/values from RuleOutput object (#4773)
- [BUGFIX] Pass random seed to bootstrap estimator (#4605)
- [BUGFIX] Adjust output of
regex
ParameterBuilder to match Expectation (#4594) - [BUGFIX] Rule-Based Profiler: Only primitive type based BatchRequest is allowed for Builder classes (#4614)
- [BUGFIX] Fix DataContext templates test (#4678)
- [BUGFIX] update module_name in NoteBookConfigSchema from v2 path to v3 (#4589) (thanks @Josephmaclean)
- [BUGFIX] request S3 bucket location only when necessary (#4526) (thanks @error418)
- [DOCS] Update
ignored_columns
snippet in "Getting Started" (#4609) - [DOCS] Fixes import statement. (#4694)
- [DOCS] Update tutorial_review.md typo with intended word. (#4611) (thanks @cjbramble)
- [DOCS] Correct typo in url in docstring for set_based_column_map_expectation_template.py (example script) (#4817)
- [MAINTENANCE] Add retries to
requests
in usage stats integration tests (#4600) - [MAINTENANCE] Miscellaneous test cleanup (#4602)
- [MAINTENANCE] Simplify ParameterBuilder.build_parameter() interface (#4622)
- [MAINTENANCE] War on Warnings - DataContext (#4572)
- [MAINTENANCE] Update links within great_expectations.yml (#4549) (thanks @andyjessen)
- [MAINTENANCE] Provide cardinality limit modes from CategoricalColumnDomainBuilder (#4662)
- [MAINTENANCE] Rule-Based Profiler: Rename Rule.generate() to Rule.run() (#4670)
- [MAINTENANCE] Refactor ValidationParameter computation (to be more elegant/compact) and fix a type hint in SimpleDateFormatStringParameterBuilder (#4687)
- [MAINTENANCE] Remove
pybigquery
check that is no longer needed (#4681) - [MAINTENANCE] Rule-Based Profiler: Allow ExpectationConfigurationBuilder to be Optional (#4698)
- [MAINTENANCE] Slightly Clean Up NumericMetricRangeMultiBatchParameterBuilder (#4699)
- [MAINTENANCE] ParameterBuilder must not recompute its value, if it already exists in RuleState (ParameterContainer for its Domain). (#4701)
- [MAINTENANCE] Improve get validator functionality (#4661)
- [MAINTENANCE] Add checks for mostly=1.0 for all renderers (#4736)
- [MAINTENANCE] revert to not raising datasource errors on data context init (#4732)
- [MAINTENANCE] Remove unused bootstrap methods that were migrated to ML Flow (#4742)
- [MAINTENANCE] Update README.md (#4595) (thanks @andyjessen)
- [MAINTENANCE] Check for mostly equals 1 in renderers (#4815)
- [MAINTENANCE] Remove bootstrap tests that are no longer needed (#4818)
- [HACKATHON] ExpectColumnValuesToBeIsoLanguages (#4627) (thanks @szecsip)
- [HACKATHON] ExpectColumnAverageLatLonPairwiseDistanceToBeLessThan (#4559) (thanks @mmi333)
- [HACKATHON] ExpectColumnValuesToBeValidIPv6 (#4561) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidMac (#4562) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidMIME (#4563) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidHexColor (#4564) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidIban (#4565) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidIsoCountry (#4566) (thanks @voidforall)
- [HACKATHON] add expect_column_values_to_be_private_ipv4_class (#4656) (thanks @szecsip)
- [HACKATHON] Feature/expect column values url hostname match with cert (#4649) (thanks @szecsip)
- [HACKATHON] add expect_column_values_url_has_got_valid_cert (#4648) (thanks @szecsip)
- [HACKATHON] add expect_column_values_to_be_valid_us_state_or_territory (#4655) (thanks @Derekma73)
- [HACKATHON] ExpectColumnValuesToBeValidSsn (#4646) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidHttpStatusName (#4645) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidHttpStatusCode (#4644) (thanks @voidforall)
- [HACKATHON] Feature/expect column values to be daytime (#4643) (thanks @szecsip)
- [HACKATHON] add expect_column_values_ip_address_in_network (#4640) (thanks @szecsip)
- [HACKATHON] add expect_column_values_ip_asn_country_code_in_set (#4638) (thanks @szecsip)
- [HACKATHON] add expect_column_values_to_be_valid_us_state (#4654) (thanks @Derekma73)
- [HACKATHON] add expect_column_values_to_be_valid_us_state_or_territory_abbreviation (#4653) (thanks @Derekma73)
- [HACKATHON] add expect_column_values_to_be_weekday (#4636) (thanks @szecsip)
- [HACKATHON] add expect_column_values_to_be_valid_us_state_abbrevation (#4650) (thanks @Derekma73)
- [HACKATHON] ExpectColumnValuesGeometryDistanceToAddressToBeBetween (#4652) (thanks @pjdobson)
- [HACKATHON] ExpectColumnValuesToBeValidUdpPort (#4635) (thanks @voidforall)
- [HACKATHON] add expect_column_values_to_be_fibonacci_number (#4629) (thanks @szecsip)
- [HACKATHON] add expect_column_values_to_be_slug (#4628) (thanks @szecsip)
- [HACKATHON] ExpectColumnValuesGeometryToBeWithinPlace (#4626) (thanks @pjdobson)
- [HACKATHON] add expect_column_values_to_be_private_ipv6 (#4624) (thanks @szecsip)
- [HACKATHON] add expect_column_values_to_be_private_ip_v4 (#4623) (thanks @szecsip)
- [HACKATHON] ExpectColumnValuesToBeValidPrice (#4593) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidPhonenumber (#4592) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBePolygonAreaBetween (#4591) (thanks @mmi333)
- [HACKATHON] ExpectColumnValuesToBeValidTcpPort (#4634) (thanks @voidforall)
0.14.13
- [FEATURE] Convert Existing Self-Initializing Expectations to Make ExpectationConfigurationBuilder Self-Contained with its own validation_parameter_builder settings (#4547)
- [FEATURE] Improve diagnostic checklist details (#4548)
- [BUGFIX] Moves testing dependencies out of core reqs (#4522)
- [BUGFIX] Adjust output of datetime
ParameterBuilder
to match Expectation (#4590) - [DOCS] Technical term tags for Adding features to Expectations section of the ToC (#4462)
- [DOCS] Contributing integrations ToC update. (#4551)
- [DOCS] Update intro page overview image (#4540)
- [DOCS] clarifications on execution engines and scalability (#4539)
- [DOCS] technical terms for validate data advanced (#4535)
- [DOCS] technical terms for validate data actions docs (#4518)
- [DOCS] correct code reference line numbers and snippet tags for how to create a batch of data from an in memory data frame (#4573)
- [DOCS] Update links in page; fix markdown link in html block (#4585)
- [MAINTENANCE] Don't return from validate configuration methods (#4545)
- [MAINTENANCE] Rule-Based Profiler: Refactor utilities into appropriate modules/classes for better separation of concerns (#4553)
- [MAINTENANCE] Refactor global
conftest
(#4534) - [MAINTENANCE] clean up docstrings (#4554)
- [MAINTENANCE] Small formatting rearrangement for RegexPatternStringParameterBuilder (#4558)
- [MAINTENANCE] Refactor Anonymizer utilizing the Strategy design pattern (#4485)
- [MAINTENANCE] Remove duplicate
mistune
dependency (#4569) - [MAINTENANCE] Run PEP273 checks on a schedule or release cut (#4570)
- [MAINTENANCE] Package dependencies usage stats instrumentation - part 1 (#4546)
- [MAINTENANCE] Add DevRel team to GitHub auto-label action (#4575)
- [MAINTENANCE] Add GitHub action to conditionally auto-update PR's (#4574)
- [MAINTENANCE] Bump version of
black
in response to hotfix for Click v8.1.0 (#4577) - [MAINTENANCE] Update overview.md (#4556)
- [MAINTENANCE] Minor clean-up (#4571)
- [MAINTENANCE] Instrument package dependencies (#4583)
- [MAINTENANCE] Standardize DomainBuilder Constructor Arguments Ordering (#4599)
0.14.12
- [FEATURE] Enables Regex-Based Column Map Expectations (#4315)
- [FEATURE] Update diagnostic checklist to do linting checks (#4491)
- [FEATURE] format docstrings as markdown for gallery (#4502)
- [FEATURE] Introduces SetBasedColumnMapExpectation w/ supporting templates & doc (#4497)
- [FEATURE]
YAMLHandler
Class (#4510) - [FEATURE] Remove conflict between filter directives and row_conditions (#4488)
- [FEATURE] Add SNS as a Validation Action (#4519) (thanks @michael-j-thomas)
- [BUGFIX] Fixes ExpectColumnValuesToBeInSet to enable behavior indicated in Parameterized Expectations Doc (#4455)
- [BUGFIX] Fixes minor typo in custom expectation docs, adds missing link (#4507)
- [BUGFIX] Removes validate_config from RegexBasedColumnMap templates & doc (#4506)
- [BUGFIX] Update ExpectColumnValuesToMatchRegex to support parameterized expectations (#4504)
- [BUGFIX] Add back
nbconvert
to dev dependencies (#4515) - [BUGFIX] Account for case where SQLAlchemy dialect is not downloaded when masking a given URL (#4516)
- [BUGFIX] Fix failing test for
How to Configure Credentials
(#4525) - [BUGFIX] Remove Temp Dir (#4528)
- [BUGFIX] Add pin to Jinja 2 due to API changes in v3.1.0 release (#4537)
- [BUGFIX] Fixes broken links in How To Write A How-To Guide (#4536)
- [BUGFIX] Removes cryptography upper bound for general reqs (#4487)
- [BUGFIX] Don't assume boto3 is installed (#4542)
- [DOCS] Update tutorial_review.md (#3981)
- [DOCS] Update AUTHORING_INTRO.md (#4470) (thanks @andyjessen)
- [DOCS] Add clarification (#4477) (thanks @strickvl)
- [DOCS] Add missing word and fix wrong dataset reference (#4478) (thanks @strickvl)
- [DOCS] Adds documentation on how to use Great Expectations with Prefect (#4433) (thanks @desertaxle)
- [DOCS] technical terms validate data checkpoints (#4486)
- [DOCS] How to use a Custom Expectation (#4467)
- [DOCS] Technical Terms for Validate Data: Overview and Core Skills docs (#4465)
- [DOCS] technical terms create expectations advanced skills (#4441)
- [DOCS] Integration documentation (#4483)
- [DOCS] Adding Meltano implementation pattern to docs (#4509) (thanks @pnadolny13)
- [DOCS] Update tutorial_create_expectations.md (#4512) (thanks @andyjessen)
- [DOCS] Fix relative links on github (#4479) (thanks @andyjessen)
- [DOCS] Update README.md (#4533) (thanks @andyjessen)
- [HACKATHON] ExpectColumnValuesToBeValidIPv4 (#4457) (thanks @voidforall)
- [HACKATHON] ExpectColumnValuesToBeValidIanaTimezone (#4532) (thanks @lucasasmith)
- [MAINTENANCE] Clean up
Checkpoints
documentation and addsnippet
(#4474) - [MAINTENANCE] Finalize Great Expectations contrib JSON structure (#4482)
- [MAINTENANCE] Update expectation filenames to match snake_case of their defined Expectations (#4484)
- [MAINTENANCE] Clean Up Types and Rely on "to_json_dict()" where appropriate (#4489)
- [MAINTENANCE] type hints for Batch Request to be string (which leverages parameter/variable resolution) (#4494)
- [MAINTENANCE] Insure consistent ordering of arguments to ParameterBuilder instantiations (#4496)
- [MAINTENANCE] Refactor build_gallery.py script (#4493)
- [MAINTENANCE] Feature/cloud 385/mask cloud creds (#4444)
- [MAINTENANCE] Enforce consistent JSON schema through usage stats (#4499)
- [MAINTENANCE] Applies
camel_to_snake
util toRegexBasedColumnMapExpectation
(#4511) - [MAINTENANCE] Removes unused dependencies (#4508)
- [MAINTENANCE] Revert changes made to dependencies in #4508 (#4520)
- [MAINTENANCE] Add
compatability
stage todependency_graph
pipeline (#4514) - [MAINTENANCE] Add prod metadata and remove package attribute from library_metadata (#4517)
- [MAINTENANCE] Move builder instantiation methods to utility module for broader usage among sub-components within Rule-Based Profiler (#4524)
- [MAINTENANCE] Update package info for Capital One DataProfiler (#4523)
- [MAINTENANCE] Remove tag 'needs migration to modular expectations api' for some Expectations (#4521)
- [MAINTENANCE] Add type hints and PyCharm macros in a test module for DefaultExpectationConfigurationBuilder (#4529)
- [MAINTENANCE] Continue War on Warnings (#4500)
0.14.11
- [FEATURE] Script to validate docs snippets line number refs (#4377)
- [FEATURE] GitHub action to auto label
core-team
(#4382) - [FEATURE]
add_rule()
method for RuleBasedProfilers and tests (#4358) - [FEATURE] Enable the passing of an existing suite to
RuleBasedProfiler.run()
(#4386) - [FEATURE] Impose Ordering on Marshmallow Schema validated Rule-Based Profiler Configuration fields (#4388)
- [FEATURE] Use more granular requirements-dev-xxx.txt files (#4327)
- [FEATURE] Rule-Based Profiler: Implement Utilities for getting all available parameter node names and objects resident in memory (#4442)
- [BUGFIX] Minor Serialization Correction for MeanUnexpectedMapMetricMultiBatchParameterBuilder (#4385)
- [BUGFIX] Fix CategoricalColumnDomainBuilder to be compliant with serialization / instantiation interfaces (#4395)
- [BUGFIX] Fix bug around
get_parent
usage stats utility intest_yaml_config
(#4410) - [BUGFIX] Adding
--spark
flag back toazure-pipelines.yml
compatibility_matrix stage. (#4418) - [BUGFIX] Remove remaining usage of --no-spark and --no-postgresql flags for pytest (#4425)
- [BUGFIX] Insure Proper Indexing of Metric Computation Results in ParameterBuilder (#4426)
- [BUGFIX] Include requirements-dev-contrib.txt in dev-install-matrix.yml for lightweight (#4430)
- [BUGFIX] Remove
pytest-azurepiplines
usage fromtest_cli
stages in Azure pipelines (#4432) - [BUGFIX] Updates or deletes broken and deprecated example notebooks (#4404)
- [BUGFIX] Add any dependencies we import directly, but don't have as explicit requirements (#4447)
- [BUGFIX] Removes potentially sensitive webhook URLs from logging (#4440)
- [BUGFIX] Fix packaging test (#4452)
- [DOCS] Fix typo in how_to_create_custom_metrics (#4379)
- [DOCS] Add
snippet
tag to gcs data docs (#4383) - [DOCS] adjust lines for py reference (#4390)
- [DOCS] technical tags for connecting to data: core skills docs (#4403)
- [DOCS] technical term tags for connect to data database documents (#4413)
- [DOCS] Technical term tags for documentation under Connect to data: Filesystem (#4411)
- [DOCS] Technical term tags for setup pages (#4392)
- [DOCS] Technical term tags for Connect to Data: Advanced docs. (#4406)
- [DOCS] Technical tags: Connect to data:In memory docs (#4405)
- [DOCS] Add misc
snippet
tags to existing documentation (#4397) - [DOCS] technical terms create expectations: core skills (#4435)
- [DOCS] Creates Custom Table Expectation How-To (#4399)
- [HACKATHON] ExpectTableLinearFeatureImportancesToBe (#4400)
- [MAINTENANCE] Group MAP_SERIES and MAP_CONDITION_SERIES with VALUE-type metrics (#3286)
- [MAINTENANCE] minor imports cleanup (#4381)
- [MAINTENANCE] Change schedule for
packaging_and_installation
pipeline to run at off-hours (#4384) - [MAINTENANCE] Implicitly anonymize object based on module (#4387)
- [MAINTENANCE] Preparatory cleanup refactoring of get_compute_domain (#4371)
- [MAINTENANCE] RBP -- make parameter builder configurations for self initializing expectations consistent with ParameterBuilder class interfaces (#4398)
- [MAINTENANCE] Refactor
ge_class
attr out of Anonymizer and related child classes (#4393) - [MAINTENANCE] Removing Custom Expectation Renderer docs from sidebar (#4401)
- [MAINTENANCE] Enable "rule_based_profiler.run()" Method to Accept Batch Data Arguments Directly (#4409)
- [MAINTENANCE] Refactor out unnecessary Anonymizer child classes (#4408)
- [MAINTENANCE] Replace "sampling_method" with "estimator" in Rule-Based Profiler code (#4420)
- [MAINTENANCE] Add docstrings and type hints to
Anonymizer
(#4419) - [MAINTENANCE] Continue chipping away at warnings (#4422)
- [MAINTENANCE] Rule-Based Profiler: Standardize on Include/Exclude Column Names List (#4424)
- [MAINTENANCE] Set upper bound on number of allowed warnings in snippet validation script (#4434)
- [MAINTENANCE] Clean up of
RegexPatternStringParameterBuilder
tests to use unittests (#4436)
0.14.10
- [FEATURE] ParameterBuilder for Computing Average Unexpected Values Fractions for any Map Metric (#4340)
- [FEATURE] Improve bootstrap quantile method accuracy (#4270)
- [FEATURE] Decorate RuleBasedProfiler.run() with usage statistics (#4321)
- [FEATURE] MapMetricColumnDomainBuilder for Rule-Based Profiler (#4353)
- [FEATURE] Enable expect_column_min/_max_to_be_between expectations to be self-initializing (#4363)
- [FEATURE] Azure pipeline to perform nightly CI/CD runs around packaging/installation (#4274)
- [BUGFIX] Fix
IndexError
around data asset pagination from CLI (#4346) - [BUGFIX] Upper bound pyathena to <2.5.0 (#4350)
- [BUGFIX] Fixes PyAthena type checking for core expectations & tests (#4359)
- [BUGFIX] BatchRequest serialization (CLOUD-743) (#4352)
- [BUGFIX] Update the favicon on docs site (#4376)
- [BUGFIX] Fix issue with datetime objects in expecatation args (#2652) (thanks @jstammers)
- [DOCS] Universal map TOC update (#4292)
- [DOCS] add Config section (#4355)
- [DOCS] Deployment Patterns to Reference Architectures (#4344)
- [DOCS] Fixes tutorial link in reference architecture prereqs component (#4360)
- [DOCS] Tag technical terms in getting started tutorial (#4354)
- [DOCS] Update overview pages to link to updated tutorial pages. (#4378)
- [HACKATHON] ExpectColumnValuesToBeValidUUID (#4322)
- [HACKATHON] add expectation core (#4357)
- [HACKATHON] ExpectColumnAverageToBeWithinRangeOfGivenPoint (#4356)
- [MAINTENANCE] rule based profiler minor clean up of ValueSetParameterBuilder (#4332)
- [MAINTENANCE] Adding tests that exercise single and multi-batch BatchRequests (#4330)
- [MAINTENANCE] Formalize ParameterBuilder contract API usage in ValueSetParameterBuilder (#4333)
- [MAINTENANCE] Rule-Based Profiler: Create helpers directory; use column domain generation convenience method (#4335)
- [MAINTENANCE] Deduplicate table domain kwargs splitting (#4338)
- [MAINTENANCE] Update Azure CI/CD cron schedule to run more frequently (#4345)
- [MAINTENANCE] Optimize CategoricalColumnDomainBuilder to compute metrics in a single method call (#4348)
- [MAINTENANCE] Reduce tries to 2 for probabilistic tests (#4351)
- [MAINTENANCE] Refactor Checkpoint toolkit (#4342)
- [MAINTENANCE] Refactor all uses of
format
in favor of f-strings (#4347) - [MAINTENANCE] Update great_expectations_contrib CLI tool to use existing diagnostic classes (#4316)
- [MAINTENANCE] Setting stage for removal of
--no-postgresql
and--no-spark
flags frompytest
. Enable--postgresql
and--spark
(#4309) - [MAINTENANCE] convert unexpected_list contents to hashable type (#4336)
- [MAINTENANCE] add operator and func handling to stores urns (#4334)
- [MAINTENANCE] Refactor ParameterBuilder classes to extend parent class where possible; also, minor cleanup (#4375)
0.14.9
- [FEATURE] Enable Simultaneous Execution of all Metric Computations for ParameterBuilder implementations in Rule-Based Profiler (#4282)
- [FEATURE] Update print_diagnostic_checklist with an option to show any failed tests (#4288)
- [FEATURE] Self-Initializing Expectations (implemented for three example expectations). (#4258)
- [FEATURE] ValueSetMultiBatchParameterBuilder and CategoricalColumnDomainBuilder (#4269)
- [FEATURE] Remove changelog-bot GitHub Action (#4297)
- [FEATURE] Add requirements-dev-lite.txt and update tests/docs (#4273)
- [FEATURE] Enable All ParameterBuilder and DomainBuilder classes to accept batch_list generically (#4302)
- [FEATURE] Enable Probabilistic Tests To Retry upon Assertion Failure (#4308)
- [FEATURE] Update usage stats schema to account for RBP's run() payload (#4266)
- [FEATURE] ProfilerRunAnonymizer (#4264)
- [FEATURE] Enable Expectation "expect_column_values_to_be_in_set" to be Self-Initializing (#4318)
- [BUGFIX] Add redirect for removed Spark EMR page (#4280)
- [BUGFIX]
ConfiguredAssetSqlDataConnector
now correctly handlesschema
andprefix
/suffix
(#4268) - [BUGFIX] Fixes Expectation Diagnostics failing on multi-line docstrings with leading linebreaks (#4286)
- [BUGFIX] Respect test backends (#4287)
- [BUGFIX] Skip test__generate_expectations_tests__xxx tests when sqlalchemy isn't there (#4300)
- [BUGFIX] test_backends integration test fix and supporting docs code ref fixes (#4306)
- [BUGFIX] Update
deep_filter_properties_iterable
to ensure that empty values are cleaned (#4298) - [BUGFIX] Fixes validate_configuration checking in diagnostics (#4307)
- [BUGFIX] Update test output that should be returned from generate_diagnostic_checklist (#4317)
- [BUGFIX] Standardizes imports in expectation templates and examples (#4320)
- [BUGFIX] Only validate row_condition if not None (#4329)
- [BUGFIX] Fix PEP273 Windows issue (#4328)
- [DOCS] Fixes misc. verbiage & typos in new Custom Expectation docs (#4283)
- [DOCS] fix formatting in configuration details block of Getting Started (#4289) (thanks @afeld)
- [DOCS] Fixes imports and code refs to expectation templates (#4314)
- [DOCS] Update creating_custom_expectations/overview.md (#4278) (thanks @binarytom)
- [CONTRIB] CapitalOne Dataprofiler expectations (#4174) (thanks @taylorfturner)
- [HACKATHON] ExpectColumnValuesToBeLatLonCoordinatesInRangeOfGivenPoint (#4284)
- [HACKATHON] ExpectColumnValuesToBeValidDegreeDecimalCoordinates (#4319)
- [MAINTENANCE] Refactor parameter setting for simpler ParameterBuilder interface (#4299)
- [MAINTENANCE] SimpleDateTimeFormatStringParameterBuilder and general RBP example config updates (#4304)
- [MAINTENANCE] Make adherence to Marshmallow Schema more robust (#4325)
- [MAINTENANCE] Refactor rule based profiler to keep objects/utilities within intended scope (#4331)
- [MAINTENANCE] Dependabot version upgrades (#4253, #4231, #4058, #4041, #3916, #3886, #3583, #2856, #3370, #3216, #2935, #2855, #3302, #4008, #4252)
0.14.8
- [FEATURE] Add
run_profiler_on_data
method to DataContext (#4190) - [FEATURE]
RegexPatternStringParameterBuilder
forRuleBasedProfiler
(#4167) - [FEATURE] experimental column map expectation checking for vectors (#3102) (thanks @manyshapes)
- [FEATURE] Pre-requisites in Rule-Based Profiler for Self-Estimating Expectations (#4242)
- [FEATURE] Add optional parameter
condition
to DefaultExpectationConfigurationBuilder (#4246) - [BUGFIX] Ensure that test result for
RegexPatternStringParameterBuilder
is deterministic (#4240) - [BUGFIX] Remove duplicate RegexPatternStringParameterBuilder test (#4241)
- [BUGFIX] Improve pandas version checking in test_expectations[_cfe].py files (#4248)
- [BUGFIX] Ensure
test_script_runner.py
actually raises AssertionErrors correctly (#4239) - [BUGFIX] Check for pandas>=024 not pandas>=24 (#4263)
- [BUGFIX] Add support for SqlAlchemyQueryStore connection_string credentials (#4224) (thanks @davidvanrooij)
- [BUGFIX] Remove assertion (#4271)
- [DOCS] Hackathon Contribution Docs (#3897)
- [MAINTENANCE] Rule-Based Profiler: Fix Circular Imports; Configuration Schema Fixes; Enhanced Unit Tests; Pre-Requisites/Refactoring for Self-Estimating Expectations (#4234)
- [MAINTENANCE] Reformat contrib expectation with black (#4244)
- [MAINTENANCE] Resolve cyclic import issue with usage stats (#4251)
- [MAINTENANCE] Additional refactor to clean up cyclic imports in usage stats (#4256)
- [MAINTENANCE] Rule-Based Profiler prerequisite: fix quantiles profiler configuration and add comments (#4255)
- [MAINTENANCE] Introspect Batch Request Dictionary for its kind and instantiate accordingly (#4259)
- [MAINTENANCE] Minor clean up in style of an RBP test fixture; making variables access more robust (#4261)
- [MAINTENANCE] define empty sqla_bigquery object (#4249)
0.14.7
- [FEATURE] Support Multi-Dimensional Metric Computations Generically for Multi-Batch Parameter Builders (#4206)
- [FEATURE] Add support for sqlalchemy-bigquery while falling back on pybigquery (#4182)
- [BUGFIX] Update validate_configuration for core Expectations that don't return True (#4216)
- [DOCS] Fixes two references to the Getting Started tutorial (#4189)
- [DOCS] Deepnote Deployment Pattern Guide (#4169)
- [DOCS] Allow Data Docs to be rendered in night mode (#4130)
- [DOCS] Fix datepicker filter on data docs (#4217)
- [DOCS] Deepnote Deployment Pattern Image Fixes (#4229)
- [MAINTENANCE] Refactor RuleBasedProfiler toolkit pattern (#4191)
- [MAINTENANCE] Revert
dependency_graph
pipeline changes to ensureusage_stats
runs in parallel (#4198) - [MAINTENANCE] Refactor relative imports (#4195)
- [MAINTENANCE] Remove temp file that was accidently committed (#4201)
- [MAINTENANCE] Update default candidate strings SimpleDateFormatString parameter builder (#4193)
- [MAINTENANCE] minor type hints clean up (#4214)
- [MAINTENANCE] RBP testing framework changes (#4184)
- [MAINTENANCE] add conditional check for 'expect_column_values_to_be_in_type_list' (#4200)
- [MAINTENANCE] Allow users to pass in any set of polygon points in expectation for point to be within region (#2520) (thanks @ryanlindeborg)
- [MAINTENANCE] Better support Hive, better support BigQuery. (#2624) (thanks @jacobpgallagher)
- [MAINTENANCE] move process_evaluation_parameters into conditional (#4109)
- [MAINTENANCE] Type hint usage stats (#4226)
0.14.6
- [FEATURE] Create profiler from DataContext (#4070)
- [FEATURE] Add read_sas function (#3972) (thanks @andyjessen)
- [FEATURE] Run profiler from DataContext (#4141)
- [FEATURE] Instantiate Rule-Based Profiler Using Typed Configuration Object (#4150)
- [FEATURE] Provide ability to instantiate Checkpoint using CheckpointConfig typed object (#4166)
- [FEATURE] Misc cleanup around CLI
suite
command and related utilities (#4158) - [FEATURE] Add scheduled runs for primary Azure pipeline (#4117)
- [FEATURE] Promote dependency graph test strategy to production (#4124)
- [BUGFIX] minor updates to test definition json files (#4123)
- [BUGFIX] Fix typo for metric name in expect_column_values_to_be_edtf_parseable (#4140)
- [BUGFIX] Ensure that CheckpointResult object can be pickled (#4157)
- [BUGFIX] Custom notebook templates (#2619) (thanks @luke321321)
- [BUGFIX] Include public fields in property_names (#4159)
- [DOCS] Reenable docs-under-test for RuleBasedProfiler (#4149)
- [DOCS] Provided details for using GE_HOME in commandline. (#4164)
- [MAINTENANCE] Return Rule-Based Profiler base.py to its dedicated config subdirectory (#4125)
- [MAINTENANCE] enable filter properties dict to handle both inclusion and exclusion lists (#4127)
- [MAINTENANCE] Remove unused Great Expectations imports (#4135)
- [MAINTENANCE] Update trigger for scheduled Azure runs (#4134)
- [MAINTENANCE] Maintenance/upgrade black (#4136)
- [MAINTENANCE] Alter
great_expectations
pipeline trigger to be more consistent (#4138) - [MAINTENANCE] Remove remaining unused imports (#4137)
- [MAINTENANCE] Remove
class_name
as mandatory field fromRuleBasedProfiler
(#4139) - [MAINTENANCE] Ensure
AWSAthena
does not create temporary table as part of processing Batch by default, which is currently not supported (#4103) - [MAINTENANCE] Remove unused
Exception as e
instances (#4143) - [MAINTENANCE] Standardize DictDot Method Behaviors Formally for Consistent Usage Patterns in Subclasses (#4131)
- [MAINTENANCE] Remove unused f-strings (#4142)
- [MAINTENANCE] Minor Validator code clean up -- for better code clarity (#4147)
- [MAINTENANCE] Refactoring of
test_script_runner.py
. Integration and Docs tests (#4145) - [MAINTENANCE] Remove
compatability
stage fromdependency-graph
pipeline (#4161) - [MAINTENANCE] CLOUD-618: GE Cloud "account" to "organization" rename (#4146)
0.14.5
- [FEATURE] Delete profilers from DataContext (#4067)
- [FEATURE] [BUGFIX] Support nullable int column types (#4044) (thanks @scnerd)
- [FEATURE] Rule-Based Profiler Configuration and Runtime Arguments Reconciliation Logic (#4111)
- [BUGFIX] Add default BIGQUERY_TYPES (#4096)
- [BUGFIX] Pin
pip --upgrade
to a specific version for CI/CD pipeline (#4100) - [BUGFIX] Use
pip==20.2.4
for usage statistics stage of CI/CD (#4102) - [BUGFIX] Fix shared state issue in renderer test (#4000)
- [BUGFIX] Missing docstrings on validator expect_ methods (#4062) (#4081)
- [BUGFIX] Fix s3 path suffix bug on windows (#4042) (thanks @scnerd)
- [MAINTENANCE] fix typos in changelogs (#4093)
- [MAINTENANCE] Migration of GCP tests to new project (#4072)
- [MAINTENANCE] Refactor Validator methods (#4095)
- [MAINTENANCE] Fix Configuration Schema and Refactor Rule-Based Profiler; Initial Implementation of Reconciliation Logic Between Configuration and Runtime Arguments (#4088)
- [MAINTENANCE] Minor Cleanup -- remove unnecessary default arguments from dictionary cleaner (#4110)
0.14.4
- [BUGFIX] Fix typing_extensions requirement to allow for proper build (#4083) (thanks @vojtakopal and @Godoy)
- [DOCS] data docs action rewrite (#4087)
- [DOCS] metric store how to rewrite (#4086)
- [MAINTENANCE] Change
logger.warn
tologger.warning
to remove deprecation warnings (#4085)
0.14.3
- [FEATURE] Profiler Store (#3990)
- [FEATURE] List profilers from DataContext (#4023)
- [FEATURE] add bigquery json credentials kwargs for sqlalchemy connect (#4039)
- [FEATURE] Get profilers from DataContext (#4033)
- [FEATURE] Add RuleBasedProfiler to
test_yaml_config
utility (#4038) - [BUGFIX] Checkpoint Configurator fix to allow notebook logging suppression (#4057)
- [DOCS] Created a page containing our glossary of terms and definitions. (#4056)
- [DOCS] swap of old uri for new in data docs generated (#4013)
- [MAINTENANCE] Refactor
test_yaml_config
(#4029) - [MAINTENANCE] Additional distinction made between V2 and V3 upgrade script (#4046)
- [MAINTENANCE] Correcting Checkpoint Configuration and Execution Implementation (#4015)
- [MAINTENANCE] Update minimum version for SQL Alchemy (#4055)
- [MAINTENANCE] Refactor RBP constructor to work with **kwargs instantiation pattern through config objects (#4043)
- [MAINTENANCE] Remove unnecessary metric dependency evaluations and add common table column types metric. (#4063)
- [MAINTENANCE] Clean up new RBP types, method signatures, and method names for the long term. (#4064)
- [MAINTENANCE] fixed broken function call in CLI (#4068)
0.14.8
- [FEATURE] Add
run_profiler_on_data
method to DataContext (#4190) - [FEATURE]
RegexPatternStringParameterBuilder
forRuleBasedProfiler
(#4167) - [FEATURE] experimental column map expectation checking for vectors (#3102) (thanks @manyshapes)
- [FEATURE] Pre-requisites in Rule-Based Profiler for Self-Estimating Expectations (#4242)
- [FEATURE] Add optional parameter
condition
to DefaultExpectationConfigurationBuilder (#4246) - [BUGFIX] Ensure that test result for
RegexPatternStringParameterBuilder
is deterministic (#4240) - [BUGFIX] Remove duplicate RegexPatternStringParameterBuilder test (#4241)
- [BUGFIX] Improve pandas version checking in test_expectations[_cfe].py files (#4248)
- [BUGFIX] Ensure
test_script_runner.py
actually raises AssertionErrors correctly (#4239) - [BUGFIX] Check for pandas>=024 not pandas>=24 (#4263)
- [BUGFIX] Add support for SqlAlchemyQueryStore connection_string credentials (#4224) (thanks @davidvanrooij)
- [BUGFIX] Remove assertion (#4271)
- [DOCS] Hackathon Contribution Docs (#3897)
- [MAINTENANCE] Rule-Based Profiler: Fix Circular Imports; Configuration Schema Fixes; Enhanced Unit Tests; Pre-Requisites/Refactoring for Self-Estimating Expectations (#4234)
- [MAINTENANCE] Reformat contrib expectation with black (#4244)
- [MAINTENANCE] Resolve cyclic import issue with usage stats (#4251)
- [MAINTENANCE] Additional refactor to clean up cyclic imports in usage stats (#4256)
- [MAINTENANCE] Rule-Based Profiler prerequisite: fix quantiles profiler configuration and add comments (#4255)
- [MAINTENANCE] Introspect Batch Request Dictionary for its kind and instantiate accordingly (#4259)
- [MAINTENANCE] Minor clean up in style of an RBP test fixture; making variables access more robust (#4261)
- [MAINTENANCE] define empty sqla_bigquery object (#4249)
0.14.7
- [FEATURE] Support Multi-Dimensional Metric Computations Generically for Multi-Batch Parameter Builders (#4206)
- [FEATURE] Add support for sqlalchemy-bigquery while falling back on pybigquery (#4182)
- [BUGFIX] Update validate_configuration for core Expectations that don't return True (#4216)
- [DOCS] Fixes two references to the Getting Started tutorial (#4189)
- [DOCS] Deepnote Deployment Pattern Guide (#4169)
- [DOCS] Allow Data Docs to be rendered in night mode (#4130)
- [DOCS] Fix datepicker filter on data docs (#4217)
- [DOCS] Deepnote Deployment Pattern Image Fixes (#4229)
- [MAINTENANCE] Refactor RuleBasedProfiler toolkit pattern (#4191)
- [MAINTENANCE] Revert
dependency_graph
pipeline changes to ensureusage_stats
runs in parallel (#4198) - [MAINTENANCE] Refactor relative imports (#4195)
- [MAINTENANCE] Remove temp file that was accidently committed (#4201)
- [MAINTENANCE] Update default candidate strings SimpleDateFormatString parameter builder (#4193)
- [MAINTENANCE] minor type hints clean up (#4214)
- [MAINTENANCE] RBP testing framework changes (#4184)
- [MAINTENANCE] add conditional check for 'expect_column_values_to_be_in_type_list' (#4200)
- [MAINTENANCE] Allow users to pass in any set of polygon points in expectation for point to be within region (#2520) (thanks @ryanlindeborg)
- [MAINTENANCE] Better support Hive, better support BigQuery. (#2624) (thanks @jacobpgallagher)
- [MAINTENANCE] move process_evaluation_parameters into conditional (#4109)
- [MAINTENANCE] Type hint usage stats (#4226)
0.14.6
- [FEATURE] Create profiler from DataContext (#4070)
- [FEATURE] Add read_sas function (#3972) (thanks @andyjessen)
- [FEATURE] Run profiler from DataContext (#4141)
- [FEATURE] Instantiate Rule-Based Profiler Using Typed Configuration Object (#4150)
- [FEATURE] Provide ability to instantiate Checkpoint using CheckpointConfig typed object (#4166)
- [FEATURE] Misc cleanup around CLI
suite
command and related utilities (#4158) - [FEATURE] Add scheduled runs for primary Azure pipeline (#4117)
- [FEATURE] Promote dependency graph test strategy to production (#4124)
- [BUGFIX] minor updates to test definition json files (#4123)
- [BUGFIX] Fix typo for metric name in expect_column_values_to_be_edtf_parseable (#4140)
- [BUGFIX] Ensure that CheckpointResult object can be pickled (#4157)
- [BUGFIX] Custom notebook templates (#2619) (thanks @luke321321)
- [BUGFIX] Include public fields in property_names (#4159)
- [DOCS] Reenable docs-under-test for RuleBasedProfiler (#4149)
- [DOCS] Provided details for using GE_HOME in commandline. (#4164)
- [MAINTENANCE] Return Rule-Based Profiler base.py to its dedicated config subdirectory (#4125)
- [MAINTENANCE] enable filter properties dict to handle both inclusion and exclusion lists (#4127)
- [MAINTENANCE] Remove unused Great Expectations imports (#4135)
- [MAINTENANCE] Update trigger for scheduled Azure runs (#4134)
- [MAINTENANCE] Maintenance/upgrade black (#4136)
- [MAINTENANCE] Alter
great_expectations
pipeline trigger to be more consistent (#4138) - [MAINTENANCE] Remove remaining unused imports (#4137)
- [MAINTENANCE] Remove
class_name
as mandatory field fromRuleBasedProfiler
(#4139) - [MAINTENANCE] Ensure
AWSAthena
does not create temporary table as part of processing Batch by default, which is currently not supported (#4103) - [MAINTENANCE] Remove unused
Exception as e
instances (#4143) - [MAINTENANCE] Standardize DictDot Method Behaviors Formally for Consistent Usage Patterns in Subclasses (#4131)
- [MAINTENANCE] Remove unused f-strings (#4142)
- [MAINTENANCE] Minor Validator code clean up -- for better code clarity (#4147)
- [MAINTENANCE] Refactoring of
test_script_runner.py
. Integration and Docs tests (#4145) - [MAINTENANCE] Remove
compatability
stage fromdependency-graph
pipeline (#4161) - [MAINTENANCE] CLOUD-618: GE Cloud "account" to "organization" rename (#4146)
0.14.5
- [FEATURE] Delete profilers from DataContext (#4067)
- [FEATURE] [BUGFIX] Support nullable int column types (#4044) (thanks @scnerd)
- [FEATURE] Rule-Based Profiler Configuration and Runtime Arguments Reconciliation Logic (#4111)
- [BUGFIX] Add default BIGQUERY_TYPES (#4096)
- [BUGFIX] Pin
pip --upgrade
to a specific version for CI/CD pipeline (#4100) - [BUGFIX] Use
pip==20.2.4
for usage statistics stage of CI/CD (#4102) - [BUGFIX] Fix shared state issue in renderer test (#4000)
- [BUGFIX] Missing docstrings on validator expect_ methods (#4062) (#4081)
- [BUGFIX] Fix s3 path suffix bug on windows (#4042) (thanks @scnerd)
- [MAINTENANCE] fix typos in changelogs (#4093)
- [MAINTENANCE] Migration of GCP tests to new project (#4072)
- [MAINTENANCE] Refactor Validator methods (#4095)
- [MAINTENANCE] Fix Configuration Schema and Refactor Rule-Based Profiler; Initial Implementation of Reconciliation Logic Between Configuration and Runtime Arguments (#4088)
- [MAINTENANCE] Minor Cleanup -- remove unnecessary default arguments from dictionary cleaner (#4110)
0.14.4
- [BUGFIX] Fix typing_extensions requirement to allow for proper build (#4083) (thanks @vojtakopal and @Godoy)
- [DOCS] data docs action rewrite (#4087)
- [DOCS] metric store how to rewrite (#4086)
- [MAINTENANCE] Change
logger.warn
tologger.warning
to remove deprecation warnings (#4085)
0.14.3
- [FEATURE] Profiler Store (#3990)
- [FEATURE] List profilers from DataContext (#4023)
- [FEATURE] add bigquery json credentials kwargs for sqlalchemy connect (#4039)
- [FEATURE] Get profilers from DataContext (#4033)
- [FEATURE] Add RuleBasedProfiler to
test_yaml_config
utility (#4038) - [BUGFIX] Checkpoint Configurator fix to allow notebook logging suppression (#4057)
- [DOCS] Created a page containing our glossary of terms and definitions. (#4056)
- [DOCS] swap of old uri for new in data docs generated (#4013)
- [MAINTENANCE] Refactor
test_yaml_config
(#4029) - [MAINTENANCE] Additional distinction made between V2 and V3 upgrade script (#4046)
- [MAINTENANCE] Correcting Checkpoint Configuration and Execution Implementation (#4015)
- [MAINTENANCE] Update minimum version for SQL Alchemy (#4055)
- [MAINTENANCE] Refactor RBP constructor to work with **kwargs instantiation pattern through config objects (#4043)
- [MAINTENANCE] Remove unnecessary metric dependency evaluations and add common table column types metric. (#4063)
- [MAINTENANCE] Clean up new RBP types, method signatures, and method names for the long term. (#4064)
- [MAINTENANCE] fixed broken function call in CLI (#4068)
0.14.2
- [FEATURE] Marshmallow schema for Rule Based Profiler (#3982)
- [FEATURE] Enable Rule-Based Profile Parameter Access To Collection Typed Values (#3998)
- [BUGFIX] Docs integration pipeline bugfix (#3997)
- [BUGFIX] Enables spark-native null filtering (#4004)
- [DOCS] Gtm/cta in docs (#3993)
- [DOCS] Fix incorrect variable name in how_to_configure_an_expectation_store_in_amazon_s3.md (#3971) (thanks @moritzkoerber)
- [DOCS] update custom docs css to add a subtle border around tabbed content (#4001)
- [DOCS] Migration Guide now includes example for Spark data (#3996)
- [DOCS] Revamp Airflow Deployment Pattern (#3963) (thanks @denimalpaca)
- [DOCS] updating redirects to reflect a moved file (#4007)
- [DOCS] typo in gcp + bigquery tutorial (#4018)
- [DOCS] Additional description of Kubernetes Operators in GCP Deployment Guide (#4019)
- [DOCS] Migration Guide now includes example for Databases (#4005)
- [DOCS] Update how to instantiate without a yml file (#3995)
- [MAINTENANCE] Refactor of
test_script_runner.py
to break-up test list (#3987) - [MAINTENANCE] Small refactor for tests that allows DB setup to be done from all tests (#4012)
0.14.1
- [FEATURE] Add pagination/search to CLI batch request listing (#3854)
- [BUGFIX] Safeguard against using V2 API with V3 Configuration (#3954)
- [BUGFIX] Bugfix and refactor for
cloud-db-integration
pipeline (#3977) - [BUGFIX] Fixes breaking typo in expect_column_values_to_be_json_parseable (#3983)
- [BUGFIX] Fixes issue where nested columns could not be addressed properly in spark (#3986)
- [DOCS] How to connect to your data in
mssql
(#3950) - [DOCS] MigrationGuide - Adding note on Migrating Expectation Suites (#3959)
- [DOCS] Incremental Update: The Universal Map's Getting Started Tutorial (#3881)
- [DOCS] Note about creating backup of Checkpoints (#3968)
- [DOCS] Connecting to BigQuery Doc line references fix (#3974)
- [DOCS] Remove RTD snippet about comments/suggestions from Docusaurus docs (#3980)
- [DOCS] Add howto for the OpenLineage validation operator (#3688) (thanks @rossturk)
- [DOCS] Updates to README.md (#3964)
- [DOCS] Update migration guide (#3967)
- [MAINTENANCE] Refactor docs dependency script (#3952)
- [MAINTENANCE] Use Effective SQLAlchemy for Reflection Fallback Logic and SQL Metrics (#3958)
- [MAINTENANCE] Remove outdated scripts (#3953)
- [MAINTENANCE] Add pytest opt to improve collection time (#3976)
- [MAINTENANCE] Refactor
render
method in PageRenderer (#3962) - [MAINTENANCE] Standardize rule based profiler testing directories organization (#3984)
- [MAINTENANCE] Metrics Cleanup (#3989)
- [MAINTENANCE] Refactor
render
method of Content Block Renderer (#3960)
0.14.0
- [BREAKING] Change Default CLI Flag To V3 (#3943)
- [FEATURE] Cloud-399/Cloud-519: Add Cloud Notification Action (#3891)
- [FEATURE]
great_expectations_contrib
CLI tool (#3909) - [FEATURE] Update
dependency_graph
pipeline to usedgtest
CLI (#3912) - [FEATURE] Incorporate updated dgtest CLI tool in experimental pipeline (#3927)
- [FEATURE] Add YAML config option to disable progress bars (#3794)
- [BUGFIX] Fix internal links to docs that may be rendered incorrectly (#3915)
- [BUGFIX] Update SlackNotificationAction to send slack_token and slack_channel to send_slack_notification function (#3873) (thanks @Calvo94)
- [BUGFIX]
CheckDocsDependenciesChanges
to only handle.py
files (#3936) - [BUGFIX] Provide ability to capture schema_name for SQL-based datasources; fix method usage bugs. (#3938)
- [BUGFIX] Ensure that Jupyter Notebook cells convert JSON strings to Python-compliant syntax (#3939)
- [BUGFIX] Cloud-519/cloud notification action return type (#3942)
- [BUGFIX] Fix issue with regex groups in
check_docs_deps
(#3949) - [DOCS] Created link checker, fixed broken links (#3930)
- [DOCS] adding the link checker to the build (#3933)
- [DOCS] Add name to link checker in build (#3935)
- [DOCS] GCP Deployment Pattern (#3926)
- [DOCS] remove v3api flag in documentation (#3944)
- [DOCS] Make corrections in HOWTO Guides for Getting Data from SQL Sources (#3945)
- [DOCS] Tiny doc fix (#3948)
- [MAINTENANCE] Fix breaking change caused by the new version of ruamel.yaml (#3908)
- [MAINTENANCE] Drop extraneous print statement in self_check/util.py. (#3905)
- [MAINTENANCE] Raise exceptions on init in cloud mode (#3913)
- [MAINTENANCE] removing commented requirement (#3920)
- [MAINTENANCE] Patch for atomic renderer snapshot tests (#3918)
- [MAINTENANCE] Remove types/expectations.py (#3928)
- [MAINTENANCE] Tests/test data class serializable dot dict (#3924)
- [MAINTENANCE] Ensure that concurrency is backwards compatible (#3872)
- [MAINTENANCE] Fix issue where meta was not recognized as a kwarg (#3852)
0.13.49
- [FEATURE] PandasExecutionEngine is able to instantiate Google Storage client in Google Cloud Composer (#3896)
- [BUGFIX] Revert change to ExpectationSuite constructor (#3902)
- [MAINTENANCE] SQL statements that are of TextClause type expressed as subqueries (#3899)
0.13.48
- [DOCS] Updates to configuring credentials (#3856)
- [DOCS] Add docs on creating suites with the UserConfigurableProfiler (#3877)
- [DOCS] Update how to configure an expectation store in GCS (#3874)
- [DOCS] Update how to configure a validation result store in GCS (#3887)
- [DOCS] Update how to host and share data docs on GCS (#3889)
- [DOCS] Organize metadata store sidebar category by type of store (#3890)
- [MAINTENANCE]
add_expectation()
inExpectationSuite
supports usage statistics for GE. (#3824) - [MAINTENANCE] Clean up Metrics type usage, SQLAlchemyExecutionEngine and SQLAlchemyBatchData implementation, and SQLAlchemy API usage (#3884)
0.13.47
- [FEATURE] Add support for named groups in data asset regex (#3855)
- [BUGFIX] Fix issue where dependency graph tester picks up non *.py files and add test file (#3830)
- [BUGFIX] Ensure proper exit code for dependency graph script (#3839)
- [BUGFIX] Allows GE to work when installed in a zip file (PEP 273). Fixes issue #3772 (#3798) (thanks @joseignaciorc)
- [BUGFIX] Update conditional for TextClause isinstance check in SQLAlchemyExecutionEngine (#3844)
- [BUGFIX] Fix usage stats events (#3857)
- [BUGFIX] Make ExpectationContext optional and remove when null to ensure backwards compatability (#3859)
- [BUGFIX] Fix sqlalchemy expect_compound_columns_to_be_unique (#3827) (thanks @harperweaver-dox)
- [BUGFIX] Ensure proper serialization of SQLAlchemy Legacy Row (#3865)
- [DOCS] Update migration_guide.md (#3832)
- [MAINTENANCE] Remove the need for DataContext registry in the instrumentation of the Legacy Profiler profiling method. (#3836)
- [MAINTENANCE] Remove DataContext registry (#3838)
- [MAINTENANCE] Refactor cli suite conditionals (#3841)
- [MAINTENANCE] adding hints to stores in data context (#3849)
- [MAINTENANCE] Improve usage stats testing (#3858, #3861)
- [MAINTENANCE] Make checkpoint methods in DataContext pass-through (#3860)
- [MAINTENANCE] Data Source and ExecutionEngine Anonymizers handle missing module_name (#3867)
- [MAINTENANCE] Add logging around DatasourceInitializationError in DataContext (#3846)
- [MAINTENANCE] Use f-string to prevent string concat issue in Evaluation Parameters (#3864)
- [MAINTENANCE] Test for errors / invalid messages in logs & fix various existing issues (#3875)
0.13.46
- [FEATURE] Instrument Runtime DataConnector for Usage Statistics: Add "checkpoint.run" Event Schema (#3797)
- [FEATURE] Add suite creation type field to CLI SUITE "new" and "edit" Usage Statistics events (#3810)
- [FEATURE] [EXPERIMENTAL] Dependency graph based testing strategy and related pipeline (#3738, #3815, #3818)
- [FEATURE] BaseDataContext registry (#3812, #3819)
- [FEATURE] Add usage statistics instrumentation to Legacy UserConfigurableProfiler execution (#3828)
- [BUGFIX] CheckpointConfig.deepcopy() must copy all fields, including the null-valued fields (#3793)
- [BUGFIX] Fix issue where configuration store didn't allow nesting (#3811)
- [BUGFIX] Fix Minor Bugs in and Clean Up UserConfigurableProfiler (#3822)
- [BUGFIX] Ensure proper replacement of nulls in Jupyter Notebooks (#3782)
- [BUGFIX] Fix issue where configuration store didn't allow nesting (#3811)
- [DOCS] Clean up TOC (#3783)
- [DOCS] Update Checkpoint and Actions Reference with testing (#3787)
- [DOCS] Update How to install Great Expectations locally (#3805)
- [DOCS] How to install Great Expectations in a hosted environment (#3808)
- [MAINTENANCE] Make BatchData Serialization More Robust (#3791)
- [MAINTENANCE] Refactor SiteIndexBuilder.build() (#3789)
- [MAINTENANCE] Update ref to ge-cla-bot in PR template (#3799)
- [MAINTENANCE] Anonymizer clean up and refactor (#3801)
- [MAINTENANCE] Certify the expectation "expect_table_row_count_to_equal_other_table" for V3 API (#3803)
- [MAINTENANCE] Refactor to enable broader use of event emitting method for usage statistics (#3825)
- [MAINTENANCE] Clean up temp file after CI/CD run (#3823)
- [MAINTENANCE] Raising exceptions for misconfigured datasources in cloud mode (#3866)
0.13.45
- [FEATURE] Feature/render validation metadata (#3397) (thanks @vshind1)
- [FEATURE] Added expectation expect_column_values_to_not_contain_special_characters() (#2849, #3771) (thanks @jaibirsingh)
- [FEATURE] Like and regex-based expectations in Athena dialect (#3762) (thanks @josges)
- [FEATURE] Rename
deep_filter_properties_dict()
todeep_filter_properties_iterable()
- [FEATURE] Extract validation result failures (#3552) (thanks @BenGale93)
- [BUGFIX] Allow now() eval parameter to be used by itself (#3719)
- [BUGFIX] Fixing broken logo for legacy RTD docs (#3769)
- [BUGFIX] Adds version-handling to sqlalchemy make_url imports (#3768)
- [BUGFIX] Integration test to avoid regression of simple PandasExecutionEngine workflow (#3770)
- [BUGFIX] Fix copying of CheckpointConfig for substitution and printing purposes (#3759)
- [BUGFIX] Fix evaluation parameter usage with Query Store (#3763)
- [BUGFIX] Feature/fix row condition quotes (#3676) (thanks @benoitLebreton-perso)
- [BUGFIX] Fix incorrect filling out of anonymized event payload (#3780)
- [BUGFIX] Don't reset_index for conditional expectations (#3667) (thanks @abekfenn)
- [DOCS] Update expectations gallery link in V3 notebook documentation (#3747)
- [DOCS] Correct V3 documentation link in V2 notebooks to point to V2 documentation (#3750)
- [DOCS] How to pass an in-memory DataFrame to a Checkpoint (#3756)
- [MAINTENANCE] Fix typo in Getting Started Guide (#3749)
- [MAINTENANCE] Add proper docstring and type hints to Validator (#3767)
- [MAINTENANCE] Clean up duplicate logging statements about optional
black
dep (#3778)
0.13.44
- [FEATURE] Add new result_format to include unexpected_row_list (#3346)
- [FEATURE] Implement "deep_filter_properties_dict()" method (#3703)
- [FEATURE] Create Constants for GETTING_STARTED Entities (e.g., datasource_name, expectation_suite_name, etc.) (#3712)
- [FEATURE] Add usage statistics event for DataContext.get_batch_list() method (#3708)
- [FEATURE] Add data_context.run_checkpoint event to usage statistics (#3721)
- [FEATURE] Add event_duration to usage statistics events (#3729)
- [FEATURE] InferredAssetSqlDataConnector's introspection can list external tables in Redshift Spectrum (#3646)
- [BUGFIX] Using a RuntimeBatchRequest in a Checkpoint with a top-level batch_request instead of validations (#3680)
- [BUGFIX] Using a RuntimeBatchRequest in a Checkpoint at runtime with Checkpoint.run() (#3713)
- [BUGFIX] Using a RuntimeBatchRequest in a Checkpoint at runtime with context.run_checkpoint() (#3718)
- [BUGFIX] Use SQLAlchemy make_url helper where applicable when parsing URLs (#3722)
- [BUGFIX] Adds check for quantile_ranges to be ordered or unbounded pairs (#3724)
- [BUGFIX] Updates MST renderer to return JSON-parseable boolean (#3728)
- [BUGFIX] Removes sqlite suppression for expect_column_quantile_values_to_be_between test definitions (#3735)
- [BUGFIX] Handle contradictory configurations in checkpoint.yml, checkpoint.run(), and context.run_checkpoint() (#3723)
- [BUGFIX] fixed a bug where expectation metadata doesn't appear in edit template for table-level expectations (#3129) (thanks @olechiw)
- [BUGFIX] Added temp_table creation for Teradata in SqlAlchemyBatchData (#3731) (thanks @imamolp)
- [DOCS] Add Databricks video walkthrough link (#3702, #3704)
- [DOCS] Update the link to configure a MetricStore (#3711, #3714) (thanks @txblackbird)
- [DOCS] Updated code example to remove deprecated "File" function (#3632) (thanks @daccorti)
- [DOCS] Delete how_to_add_a_validation_operator.md as OBE. (#3734)
- [DOCS] Update broken link in FOOTER.md to point to V3 documentation (#3745)
- [MAINTENANCE] Improve type hinting (using Optional type) (#3709)
- [MAINTENANCE] Standardize names for assets that are used in Getting Started Guide (#3706)
- [MAINTENANCE] Clean up remaining improper usage of Optional type annotation (#3710)
- [MAINTENANCE] Refinement of Getting Started Guide script (#3715)
- [MAINTENANCE] cloud-410 - Support for Column Descriptions (#3707)
- [MAINTENANCE] Types Clean Up in Checkpoint, Batch, and DataContext Classes (#3737)
- [MAINTENANCE] Remove DeprecationWarning for validator.remove_expectation (#3744)
0.13.43
- [FEATURE] Enable support for Teradata SQLAlchemy dialect (#3496) (thanks @imamolp)
- [FEATURE] Dremio connector added (SQLalchemy) (#3624) (thanks @chufe-dremio)
- [FEATURE] Adds expect_column_values_to_be_string_integers_increasing (#3642)
- [FEATURE] Enable "column.quantile_values" and "expect_column_quantile_values_to_be_between" for SQLite; add/enable new tests (#3695)
- [BUGFIX] Allow glob_directive for DBFS Data Connectors (#3673)
- [BUGFIX] Update black version in pre-commit config (#3674)
- [BUGFIX] Make sure to add "mostly_pct" value if "mostly" kwarg present (#3661)
- [BUGFIX] Fix BatchRequest.to_json_dict() to not overwrite original fields; also type usage cleanup in CLI tests (#3683)
- [BUGFIX] Fix pyfakefs boto / GCS incompatibility (#3694)
- [BUGFIX] Update prefix attr assignment in cloud-based DataConnector constructors (#3668)
- [BUGFIX] Update 'list_keys' signature for all cloud-based tuple store child classes (#3669)
- [BUGFIX] evaluation parameters from different expectation suites dependencies (#3684) (thanks @OmriBromberg)
- [DOCS] Databricks deployment pattern documentation (#3682)
- [DOCS] Remove how_to_instantiate_a_data_context_on_databricks_spark_cluster (#3687)
- [DOCS] Updates to Databricks doc based on friction logging (#3696)
- [MAINTENANCE] Fix checkpoint anonymization and make BatchRequest.to_json_dict() more robust (#3675)
- [MAINTENANCE] Update kl_divergence domain_type (#3681)
- [MAINTENANCE] update filter_properties_dict to use set for inclusions and exclusions (instead of list) (#3698)
- [MAINTENANCE] Adds CITATION.cff (#3697)
0.13.42
- [FEATURE] DBFS Data connectors (#3659)
- [BUGFIX] Fix "null" appearing in notebooks due to incorrect ExpectationConfigurationSchema serialization (#3638)
- [BUGFIX] Ensure that result_format from saved expectation suite json file takes effect (#3634)
- [BUGFIX] Allowing user specified run_id to appear in WarningAndFailureExpectationSuitesValidationOperator validation result (#3386) (thanks @wniroshan)
- [BUGFIX] Update black dependency to ensure passing Azure builds on Python 3.9 (#3664)
- [BUGFIX] fix Issue #3405 - gcs client init in pandas engine (#3408) (thanks @dz-1)
- [BUGFIX] Recursion error when passing RuntimeBatchRequest with query into Checkpoint using validations (#3654)
- [MAINTENANCE] Cloud 388/supported expectations query (#3635)
- [MAINTENANCE] Proper separation of concerns between specific File Path Data Connectors and corresponding ExecutionEngine objects (#3643)
- [MAINTENANCE] Enable Docusaurus tests for S3 (#3645)
- [MAINTENANCE] Formalize Exception Handling Between DataConnector and ExecutionEngine Implementations, and Update DataConnector Configuration Usage in Tests (#3644)
- [MAINTENANCE] Adds util for handling SADeprecation warning (#3651)
0.13.41
- [FEATURE] Support median calculation in AWS Athena (#3596) (thanks @persiyanov)
- [BUGFIX] Be able to use spark execution engine with spark reuse flag (#3541) (thanks @fep2)
- [DOCS] punctuation how_to_contribute_a_new_expectation_to_great_expectations.md (#3484) (thanks @plain-jane-gray)
- [DOCS] Update next_steps.md (#3483) (thanks @plain-jane-gray)
- [DOCS] Update how_to_configure_a_validation_result_store_in_gcs.md (#3482) (thanks @plain-jane-gray)
- [DOCS] Choosing and configuring DataConnectors (#3533)
- [DOCS] Remove --no-spark flag from docs tests (#3625)
- [DOCS] DevRel - docs fixes (#3498)
- [DOCS] Adding a period (#3627) (thanks @plain-jane-gray)
- [DOCS] Remove comments that describe Snowflake parameters as optional (#3639)
- [MAINTENANCE] Update CODEOWNERS (#3604)
- [MAINTENANCE] Fix logo (#3598)
- [MAINTENANCE] Add Expectations to docs navbar (#3597)
- [MAINTENANCE] Remove unused fixtures (#3218)
- [MAINTENANCE] Remove unnecessary comment (#3608)
- [MAINTENANCE] Superconductive Warnings hackathon (#3612)
- [MAINTENANCE] Bring Core Skills Doc for Creating Batch Under Test (#3629)
- [MAINTENANCE] Refactor and Clean Up Expectations and Metrics Parts of the Codebase (better encapsulation, improved type hints) (#3633)
0.13.40
- [FEATURE] Retrieve data context config through Cloud API endpoint #3586
- [FEATURE] Update Batch IDs to match name change in paths included in batch_request #3587
- [FEATURE] V2-to-V3 Upgrade/Migration #3592
- [FEATURE] table and graph atomic renderers #3595
- [FEATURE] V2-to-V3 Upgrade/Migration (Sidebar.js update) #3603
- [DOCS] Fixing broken links and linking to Expectation Gallery #3591
- [MAINTENANCE] Get TZLocal back to its original version control. #3585
- [MAINTENANCE] Add tests for datetime evaluation parameters #3601
- [MAINTENANCE] Removed warning for pandas option display.max_colwidth #3606
0.13.39
- [FEATURE] Migration of Expectations to Atomic Prescriptive Renderers (#3530, #3537)
- [FEATURE] Cloud: Editing Expectation Suites programmatically (#3564)
- [BUGFIX] Fix deprecation warning for importing from collections (#3546) (thanks @shpolina)
- [BUGFIX] SQLAlchemy version 1.3.24 compatibility in map metric provider (#3507) (thanks @shpolina)
- [DOCS] Clarify how to configure optional Snowflake parameters in CLI datasource new notebook (#3543)
- [DOCS] Added breaks to code snippets, reordered guidance (#3514)
- [DOCS] typo in documentation (#3542) (thanks @DanielEdu)
- [DOCS] Update how_to_configure_a_new_data_context_with_the_cli.md (#3556) (thanks @plain-jane-gray)
- [DOCS] Improved installation instructions, included in-line installation instructions to getting started (#3509)
- [DOCS] Update contributing_style.md (#3521) (thanks @plain-jane-gray)
- [DOCS] Update contributing_test.md (#3519) (thanks @plain-jane-gray)
- [DOCS] Revamp style guides (#3554)
- [DOCS] Update contributing.md (#3523, #3524) (thanks @plain-jane-gray)
- [DOCS] Simplify getting started (#3555)
- [DOCS] How to introspect and partition an SQL database (#3465)
- [DOCS] Update contributing_checklist.md (#3518) (thanks @plain-jane-gray)
- [DOCS] Removed duplicate prereq, how_to_instantiate_a_data_context_without_a_yml_file.md (#3481) (thanks @plain-jane-gray)
- [DOCS] fix link to expectation glossary (#3558) (thanks @sephiartlist)
- [DOCS] Minor Friction (#3574)
- [MAINTENANCE] Make CLI Check-Config and CLI More Robust (#3562)
- [MAINTENANCE] tzlocal version fix (#3565)
0.13.38
- [FEATURE] Atomic Renderer: Initial framework and Prescriptive renderers (#3529)
- [FEATURE] Atomic Renderer: Diagnostic renderers (#3534)
- [BUGFIX] runtime_parameters: {batch_data: Spark DF} serialization (#3502)
- [BUGFIX] Custom query in RuntimeBatchRequest for expectations using table.row_count metric (#3508)
- [BUGFIX] Transpose \n and , in notebook (#3463) (thanks @mccalluc)
- [BUGFIX] Fix contributor link (#3462) (thanks @mccalluc)
- [DOCS] How to introspect and partition a files based data store (#3464)
- [DOCS] fixed duplication of text in code example (#3503)
- [DOCS] Make content better reflect the document organization. (#3510)
- [DOCS] Correcting typos and improving the language. (#3513)
- [DOCS] Better Sections Numbering in Documentation (#3515)
- [DOCS] Improved wording (#3516)
- [DOCS] Improved title wording for section heading (#3517)
- [DOCS] Improve Readability of Documentation Content (#3536)
- [MAINTENANCE] Content and test script update (#3532)
- [MAINTENANCE] Provide Deprecation Notice for the "parse_strings_as_datetimes" Expectation Parameter in V3 (#3539)
0.13.37
- [FEATURE] Implement CompoundColumnsUnique metric for SqlAlchemyExecutionEngine (#3477)
- [FEATURE] add get_available_data_asset_names_and_types (#3476)
- [FEATURE] add s3_put_options to TupleS3StoreBackend (#3470) (Thanks @kj-9)
- [BUGFIX] Fix TupleS3StoreBackend remove_key bug (#3489)
- [DOCS] Adding Flyte Deployment pattern to docs (#3383)
- [DOCS] g_e docs branding updates (#3471)
- [MAINTENANCE] Add type-hints; add utility method for creating temporary DB tables; clean up imports; improve code readability; and add a directory to pre-commit (#3475)
- [MAINTENANCE] Clean up for a better code readability. (#3493)
- [MAINTENANCE] Enable SQL for the "expect_compound_columns_to_be_unique" expectation. (#3488)
- [MAINTENANCE] Fix some typos (#3474) (Thanks @mohamadmansourX)
- [MAINTENANCE] Support SQLAlchemy version 1.3.24 for compatibility with Airflow (Airflow does not currently support later versions of SQLAlchemy). (#3499)
- [MAINTENANCE] Update contributing_checklist.md (#3478) (Thanks @plain-jane-gray)
- [MAINTENANCE] Update how_to_configure_a_validation_result_store_in_gcs.md (#3480) (Thanks @plain-jane-gray)
- [MAINTENANCE] update implemented_expectations (#3492)
0.13.36
- [FEATURE] GREAT-3439 extended SlackNotificationsAction for slack app tokens (#3440) (Thanks @psheets)
- [FEATURE] Implement Integration Test for "Simple SQL Data Source" with Partitioning, Splitting, and Sampling (#3454)
- [FEATURE] Implement Integration Test for File Path Data Connectors with Partitioning, Splitting, and Sampling (#3452)
- [BUGFIX] Fix Incorrect Implementation of the "_sample_using_random" Sampling Method in SQLAlchemyExecutionEngine (#3449)
- [BUGFIX] Handle RuntimeBatchRequest passed to Checkpoint programatically (without yml) (#3448)
- [DOCS] Fix typo in command to create new checkpoint (#3434) (Thanks @joeltone)
- [DOCS] How to validate data by running a Checkpoint (#3436)
- [ENHANCEMENT] cloud-199 - Update Expectation and ExpectationSuite classes for GE Cloud (#3453)
- [MAINTENANCE] Does not test numpy.float128 when it doesn't exist (#3460)
- [MAINTENANCE] Remove Unnecessary SQL OR Condition (#3469)
- [MAINTENANCE] Remove validation playground notebooks (#3467)
- [MAINTENANCE] clean up type hints, API usage, imports, and coding style (#3444)
- [MAINTENANCE] comments (#3457)
0.13.35
- [FEATURE] Create ExpectationValidationGraph class to Maintain Relationship Between Expectation and Metrics and Use it to Associate Exceptions to Expectations (#3433)
- [BUGFIX] Addresses issue #2993 (#3054) by using configuration when it is available instead of discovering keys (listing keys) in existing sources. (#3377)
- [BUGFIX] Fix Data asset name rendering (#3431) (Thanks @shpolina)
- [DOCS] minor fix to syntax highlighting in how_to_contribute_a_new_expectation… (#3413) (Thanks @edjoesu)
- [DOCS] Fix broken links in how_to_create_a_new_expectation_suite_using_rule_based_profile… (#3410) (Thanks @edjoesu)
- [ENHANCEMENT] update list_expectation_suite_names and ExpectationSuiteValidationResult payload (#3419)
- [MAINTENANCE] Clean up Type Hints, JSON-Serialization, ID Generation and Logging in Objects in batch.py Module and its Usage (#3422)
- [MAINTENANCE] Fix Granularity of Exception Handling in ExecutionEngine.resolve_metrics() and Clean Up Type Hints (#3423)
- [MAINTENANCE] Fix broken links in how_to_create_a_new_expectation_suite_using_rule_based_profiler (#3441)
- [MAINTENANCE] Fix issue where BatchRequest object in configuration could cause Checkpoint to fail (#3438)
- [MAINTENANCE] Insure consistency between implementation of overriding Python hash() and internal ID property value (#3432)
- [MAINTENANCE] Performance improvement refactor for Spark unexpected values (#3368)
- [MAINTENANCE] Refactor MetricConfiguration out of validation_graph.py to Avoid Future Circular Dependencies in Python (#3425)
- [MAINTENANCE] Use ExceptionInfo to encapsulate common expectation validation result error information. (#3427)
0.13.34
- [FEATURE] Configurable multi-threaded checkpoint speedup (#3362) (Thanks @jdimatteo)
- [BUGFIX] Insure that the "result_format" Expectation Argument is Processed Properly (#3364)
- [BUGFIX] fix error getting validation result from DataContext (#3359) (Thanks @zachzIAM)
- [BUGFIX] fixed typo and added CLA links (#3347)
- [DOCS] Azure Data Connector Documentation for Pandas and Spark. (#3378)
- [DOCS] Connecting to GCS using Spark (#3375)
- [DOCS] Docusaurus - Deploying Great Expectations in a hosted environment without file system or CLI (#3361)
- [DOCS] How to get a batch from configured datasource (#3382)
- [MAINTENANCE] Add Flyte to README (#3387) (Thanks @samhita-alla)
- [MAINTENANCE] Adds expect_table_columns_to_match_set (#3329) (Thanks @viniciusdsmello)
- [MAINTENANCE] Bugfix/skip substitute config variables in ge cloud mode (#3393)
- [MAINTENANCE] Clean Up ValidationGraph API Usage, Improve Exception Handling for Metrics, Clean Up Type Hints (#3399)
- [MAINTENANCE] Clean up ValidationGraph API and add Type Hints (#3392)
- [MAINTENANCE] Enhancement/update _set methods with kwargs (#3391) (Thanks @roblim)
- [MAINTENANCE] Fix incorrect ToC section name (#3395)
- [MAINTENANCE] Insure Correct Processing of the catch_exception Flag in Metrics Resolution (#3360)
- [MAINTENANCE] exempt batch_data from a deep_copy operation on RuntimeBatchRequest (#3388)
- [MAINTENANCE] [WIP] Enhancement/cloud 169/update checkpoint.run for ge cloud (#3381)
0.13.33
- [FEATURE] Add optional ge_cloud_mode flag to DataContext to enable use with Great Expectations Cloud.
- [FEATURE] Rendered Data Doc JSONs can be uploaded and retrieved from GE Cloud
- [FEATURE] Implement InferredAssetAzureDataConnector with Support for Pandas and Spark Execution Engines (#3372)
- [FEATURE] Spark connecting to Google Cloud Storage (#3365)
- [FEATURE] SparkDFExecutionEngine can load data accessed by ConfiguredAssetAzureDataConnector (integration tests are included). (#3345)
- [FEATURE] [MER-293] GE Cloud Mode for DataContext (#3262) (Thanks @roblim)
- [BUGFIX] Allow for RuntimeDataConnector to accept custom query while suppressing temp table creation (#3335) (Thanks @NathanFarmer)
- [BUGFIX] Fix issue where multiple validators reused the same execution engine, causing a conflict in active batch (GE-3168) (#3222) (Thanks @jcampbell)
- [BUGFIX] Run batch_request dictionary through util function convert_to_json_serializable (#3349) (Thanks @NathanFarmer)
- [BUGFIX] added casting of numeric value to fix redshift issue #3293 (#3338) (Thanks @sariabod)
- [DOCS] Docusaurus - How to connect to an MSSQL database (#3353) (Thanks @NathanFarmer)
- [DOCS] GREAT-195 Docs remove all stubs and links to them (#3363)
- [MAINTENANCE] Update azure-pipelines-docs-integration.yml for Azure Pipelines
- [MAINTENANCE] Update implemented_expectations.md (#3351) (Thanks @spencerhardwick)
- [MAINTENANCE] Updating to reflect current Expectation dev state (#3348) (Thanks @spencerhardwick)
- [MAINTENANCE] docs: Clean up Docusaurus refs (#3371)
0.13.32
- [FEATURE] Add Performance Benchmarks Using BigQuery. (Thanks @jdimatteo)
- [WIP] [FEATURE] add backend args to run_diagnostics (#3257) (Thanks @edjoesu)
- [BUGFIX] Addresses Issue 2937. (#3236) (Thanks @BenGale93)
- [BUGFIX] SQL dialect doesn't register for BigQuery for V2 (#3324)
- [DOCS] "How to connect to data on GCS using Pandas" (#3311)
- [MAINTENANCE] Add CODEOWNERS with a single check for sidebars.js (#3332)
- [MAINTENANCE] Fix incorrect DataConnector usage of _get_full_file_path() API method. (#3336)
- [MAINTENANCE] Make Pandas against S3 and GCS integration tests more robust by asserting on number of batches returned and row counts (#3341)
- [MAINTENANCE] Make integration tests of Pandas against Azure more robust. (#3339)
- [MAINTENANCE] Prepare AzureUrl to handle WASBS format (for Spark) (#3340)
- [MAINTENANCE] Renaming default_batch_identifier in examples #3334
- [MAINTENANCE] Tests for RuntimeDataConnector at DataContext-level (#3304)
- [MAINTENANCE] Tests for RuntimeDataConnector at DataContext-level (Spark and Pandas) (#3325)
- [MAINTENANCE] Tests for RuntimeDataConnector at Data Source-level (Spark and Pandas) (#3318)
- [MAINTENANCE] Various doc patches (#3326)
- [MAINTENANCE] clean up imports and method signatures (#3337)
0.13.31
- [FEATURE] Enable
GCS DataConnector
integration withPandasExecutionEngine
(#3264) - [FEATURE] Enable column_pair expectations and tests for Spark (#3294)
- [FEATURE] Implement
InferredAssetGCSDataConnector
(#3284) - [FEATURE]/CHANGE run time format (#3272) (Thanks @serialbandicoot)
- [DOCS] Fix misc errors in "How to create renderers for Custom Expectations" (#3315)
- [DOCS] GDOC-217 remove stub links (#3314)
- [DOCS] Remove misc TODOs to tidy up docs (#3313)
- [DOCS] Standardize capitalization of various technologies in
docs
(#3312) - [DOCS] Fix broken link to Contributor docs (#3295) (Thanks @discdiver)
- [MAINTENANCE] Additional tests for RuntimeDataConnector at Data Source-level (query) (#3288)
- [MAINTENANCE] Update GCSStoreBackend + tests (#2630) (Thanks @hmandsager)
- [MAINTENANCE] Write integration/E2E tests for
ConfiguredAssetAzureDataConnector
(#3204) - [MAINTENANCE] Write integration/E2E tests for both
GCSDataConnectors
(#3301)
0.13.30
- [FEATURE] Implement Spark Decorators and Helpers; Demonstrate on MulticolumnSumEqual Metric (#3289)
- [FEATURE] V3 implement expect_column_pair_values_to_be_in_set for SQL Alchemy execution engine (#3281)
- [FEATURE] Implement
ConfiguredAssetGCSDataConnector
(#3247) - [BUGFIX] Fix import issues around cloud providers (GCS/Azure/S3) (#3292)
- [MAINTENANCE] Add force_reuse_spark_context to DatasourceConfigSchema (#3126) (thanks @gipaetusb and @mbakunze)
0.13.29
- [FEATURE] Implementation of the Metric "select_column_values.unique.within_record" for SQLAlchemyExecutionEngine (#3279)
- [FEATURE] V3 implement ColumnPairValuesInSet for SQL Alchemy execution engine (#3278)
- [FEATURE] Edtf with support levels (#2594) (thanks @mielvds)
- [FEATURE] V3 implement expect_column_pair_values_to_be_equal for SqlAlchemyExecutionEngine (#3267)
- [FEATURE] add expectation for discrete column entropy (#3049) (thanks @edjoesu)
- [FEATURE] Add SQLAlchemy Provider for the column_pair_values.a_greater_than_b Metric (#3268)
- [FEATURE] Expectations tests for BigQuery backend (#3219) (Thanks @jdimatteo)
- [FEATURE] Add schema validation for different GCS auth methods (#3258)
- [FEATURE] V3 - Implement column_pair helpers/providers for SqlAlchemyExecutionEngine (#3256)
- [FEATURE] V3 implement expect_column_pair_values_to_be_equal expectation for PandasExecutionEngine (#3252)
- [FEATURE] GCS DataConnector schema validation (#3253)
- [FEATURE] Implementation of the "expect_select_column_values_to_be_unique_within_record" Expectation (#3251)
- [FEATURE] Implement the SelectColumnValuesUniqueWithinRecord metric (for PandasExecutionEngine) (#3250)
- [FEATURE] V3 - Implement ColumnPairValuesEqual for PandasExecutionEngine (#3243)
- [FEATURE] Set foundation for GCS DataConnectors (#3220)
- [FEATURE] Implement "expect_column_pair_values_to_be_in_set" expectation (support for PandasExecutionEngine) (#3242)
- [BUGFIX] Fix deprecation warning for importing from collections (#3228) (thanks @ismaildawoodjee)
- [DOCS] Document BigQuery test dataset configuration (#3273) (Thanks @jdimatteo)
- [DOCS] Syntax and Link (#3266)
- [DOCS] API Links and Supporting Docs (#3265)
- [DOCS] redir and search (#3249)
- [MAINTENANCE] Update azure-pipelines-docs-integration.yml to include env vars for Azure docs integration tests
- [MAINTENANCE] Allow Wrong ignore_row_if Directive from V2 with Deprecation Warning (#3274)
- [MAINTENANCE] Refactor test structure for "Connecting to your data" cloud provider integration tests (#3277)
- [MAINTENANCE] Make test method names consistent for Metrics tests (#3254)
- [MAINTENANCE] Allow
PandasExecutionEngine
to acceptAzure DataConnectors
(#3214) - [MAINTENANCE] Standardize Arguments to MetricConfiguration Constructor; Use instead of dict(). (#3246)
0.13.28
- [FEATURE] Implement ColumnPairValuesInSet metric for PandasExecutionEngine
- [BUGFIX] Wrap optional azure imports in data_connector setup
0.13.27
- [FEATURE] Accept row_condition (with condition_parser) and ignore_row_if parameters for expect_multicolumn_sum_to_equal (#3193)
- [FEATURE] ConfiguredAssetDataConnector for Azure Blob Storage (#3141)
- [FEATURE] Replace MetricFunctionTypes.IDENTITY domain type with convenience method get_domain_records() for SparkDFExecutionEngine (#3226)
- [FEATURE] Replace MetricFunctionTypes.IDENTITY domain type with convenience method get_domain_records() for SqlAlchemyExecutionEngine (#3215)
- [FEATURE] Replace MetricFunctionTypes.IDENTITY domain type with convenience method get_full_access_compute_domain() for PandasExecutionEngine (#3210)
- [FEATURE] Set foundation for Azure-related DataConnectors (#3188)
- [FEATURE] Update ExpectCompoundColumnsToBeUnique for V3 API (#3161)
- [BUGFIX] Fix incorrect schema validation for Azure data connectors (#3200)
- [BUGFIX] Fix incorrect usage of "all()" in the comparison of validation results when executing an Expectation (#3178)
- [BUGFIX] Fixes an error with expect_column_values_to_be_dateutil_parseable (#3190)
- [BUGFIX] Improve parsing of .ge_store_backend_id (#2952)
- [BUGFIX] Remove fixture parameterization for Cloud DBs (Snowflake and BigQuery) (#3182)
- [BUGFIX] Restore support for V2 API style custom expectation rendering (#3179) (Thanks @jdimatteo)
- [DOCS] Add
conda
as installation option in README (#3196) (Thanks @rpanai) - [DOCS] Standardize capitalization of "Python" in "Connecting to your data" section of new docs (#3209)
- [DOCS] Standardize capitalization of Spark in docs (#3198)
- [DOCS] Update BigQuery docs to clarify the use of temp tables (#3184)
- [DOCS] Create _redirects (#3192)
- [ENHANCEMENT] RuntimeDataConnector messaging is made more clear for
test_yaml_config()
(#3206) - [MAINTENANCE] Add
credentials
YAML key support forDataConnectors
(#3173) - [MAINTENANCE] Fix minor typo in S3 DataConnectors (#3194)
- [MAINTENANCE] Fix typos in argument names and types (#3207)
- [MAINTENANCE] Update changelog. (#3189)
- [MAINTENANCE] Update documentation. (#3203)
- [MAINTENANCE] Update validate_your_data.md (#3185)
- [MAINTENANCE] update tests across execution engines and clean up coding patterns (#3223)
0.13.26
- [FEATURE] Enable BigQuery tests for Azure CI/CD (#3155)
- [FEATURE] Implement MulticolumnMapExpectation class (#3134)
- [FEATURE] Implement the MulticolumnSumEqual Metric for PandasExecutionEngine (#3130)
- [FEATURE] Support row_condition and ignore_row_if Directives Combined for PandasExecutionEngine (#3150)
- [FEATURE] Update ExpectMulticolumnSumToEqual for V3 API (#3136)
- [FEATURE] add python3.9 to python versions (#3143) (Thanks @dswalter)
- [FEATURE]/MER-16/MER-75/ADD_ROUTE_FOR_VALIDATION_RESULT (#3090) (Thanks @rreinoldsc)
- [BUGFIX] Enable
--v3-api suite edit
to proceed without selecting DataConnectors (#3165) - [BUGFIX] Fix error when
RuntimeBatchRequest
is passed toSimpleCheckpoint
withRuntimeDataConnector
(#3152) - [BUGFIX] allow reader_options in the CLI so can read
.csv.gz
files (#2695) (Thanks @luke321321) - [DOCS] Apply Docusaurus tabs to relevant pages in new docs
- [DOCS] Capitalize python to Python in docs (#3176)
- [DOCS] Improve Core Concepts - Expectation Concepts (#2831)
- [MAINTENANCE] Error messages must be friendly. (#3171)
- [MAINTENANCE] Implement the "compound_columns_unique" metric for PandasExecutionEngine (with a unit test). (#3159)
- [MAINTENANCE] Improve Coding Practices in "great_expectations/expectations/expectation.py" (#3151)
- [MAINTENANCE] Update test_script_runner.py (#3177)
0.13.25
- [FEATURE] Pass on meta-data from expectation json to validation result json (#2881) (Thanks @sushrut9898)
- [FEATURE] Add sqlalchemy engine support for
column.most_common_value
metric (#3020) (Thanks @shpolina) - [BUGFIX] Added newline to CLI message for consistent formatting (#3127) (Thanks @ismaildawoodjee)
- [BUGFIX] fix pip install snowflake build error with Python 3.9 (#3119) (Thanks @jdimatteo)
- [BUGFIX] Populate (data) asset name in data docs for RuntimeDataConnector (#3105) (Thanks @ceshine)
- [DOCS] Correct path to docs_rtd/changelog.rst (#3120) (Thanks @jdimatteo)
- [DOCS] Fix broken links in "How to write a 'How to Guide'" (#3112)
- [DOCS] Port over "How to add comments to Expectations and display them in DataDocs" from RTD to Docusaurus (#3078)
- [DOCS] Port over "How to create a Batch of data from an in memory Spark or Pandas DF" from RTD to Docusaurus (#3099)
- [DOCS] Update CLI codeblocks in create_your_first_expectations.md (#3106) (Thanks @ories)
- [MAINTENANCE] correct typo in docstring (#3117)
- [MAINTENANCE] DOCS/GDOC-130/Add Changelog (#3121)
- [MAINTENANCE] fix docstring for expectation "expect_multicolumn_sum_to_equal" (previous version was not precise) (#3110)
- [MAINTENANCE] Fix typos in docstrings in map_metric_provider partials (#3111)
- [MAINTENANCE] Make sure that all imports use column_aggregate_metric_provider (not column_aggregate_metric). (#3128)
- [MAINTENANCE] Rename column_aggregate_metric.py into column_aggregate_metric_provider.py for better code readability. (#3123)
- [MAINTENANCE] rename ColumnMetricProvider to ColumnAggregateMetricProvider (with DeprecationWarning) (#3100)
- [MAINTENANCE] rename map_metric.py to map_metric_provider.py (with DeprecationWarning) for a better code readability/interpretability (#3103)
- [MAINTENANCE] rename table_metric.py to table_metric_provider.py with a deprecation notice (#3118)
- [MAINTENANCE] Update CODE_OF_CONDUCT.md (#3066)
- [MAINTENANCE] Upgrade to modern Python syntax (#3068) (Thanks @cclauss)
0.13.24
- [FEATURE] Script to automate proper triggering of Docs Azure pipeline (#3003)
- [BUGFIX] Fix an undefined name that could lead to a NameError (#3063) (Thanks @cclauss)
- [BUGFIX] fix incorrect pandas top rows usage (#3091)
- [BUGFIX] Fix parens in Expectation metric validation method that always returned True assertation (#3086) (Thanks @morland96)
- [BUGFIX] Fix run_diagnostics for contrib expectations (#3096)
- [BUGFIX] Fix typos discovered by codespell (#3064) (Thanks cclauss)
- [BUGFIX] Wrap get_view_names in try clause for passing the NotImplemented error (#2976) (Thanks @kj-9)
- [DOCS] Ensuring consistent style of directories, files, and related references in docs (#3053)
- [DOCS] Fix broken link to example DAG (#3061) (Thanks fritz-astronomer)
- [DOCS] GDOC-198 cleanup TOC (#3088)
- [DOCS] Migrating pages under guides/miscellaneous (#3094) (Thanks @spbail)
- [DOCS] Port over “How to configure a new Checkpoint using test_yaml_config” from RTD to Docusaurus
- [DOCS] Port over “How to configure an Expectation store in GCS” from RTD to Docusaurus (#3071)
- [DOCS] Port over “How to create renderers for custom Expectations” from RTD to Docusaurus
- [DOCS] Port over “How to run a Checkpoint in Airflow” from RTD to Docusaurus (#3074)
- [DOCS] Update how-to-create-and-edit-expectations-in-bulk.md (#3073)
- [MAINTENANCE] Adding a comment explaining the IDENTITY metric domain type. (#3057)
- [MAINTENANCE] Change domain key value from “column” to “column_list” in ExecutionEngine implementations (#3059)
- [MAINTENANCE] clean up metric errors (#3085)
- [MAINTENANCE] Correct the typo in the naming of the IDENTIFICATION semantic domain type name. (#3058)
- [MAINTENANCE] disable snowflake tests temporarily (#3093)
- [MAINTENANCE] [DOCS] Port over “How to host and share Data Docs on GCS” from RTD to Docusaurus (#3070)
- [MAINTENANCE] Enable repr for MetricConfiguration to assist with troubleshooting. (#3075)
- [MAINTENANCE] Expand test of a column map metric to underscore functionality. (#3072)
- [MAINTENANCE] Expectation anonymizer supports v3 expectation registry (#3092)
- [MAINTENANCE] Fix -- check for column key existence in accessor_domain_kwargsn for condition map partials. (#3082)
- [MAINTENANCE] Missing import of SparkDFExecutionEngine was added. (#3062)
Older Changelist
Older changelist can be found at https://github.com/great-expectations/great_expectations/blob/develop/docs_rtd/changelog.rst