`great_expectations.experimental.datasources.postgres_datasource`¶

Module Contents¶

Classes¶

`ColumnSplitter`()
`SqlYearMonthSplitter`()
`BatchSorter`()
`TableAsset`(**kwargs)
`PostgresDatasource`(**kwargs)	Postgres datasource

Functions¶

`_batch_sorter_from_list`(sorters: BatchSortersDefinition)
`_batch_sorter_from_str`(sort_key: str)	Convert a list of strings to BatchSorters

exception great_expectations.experimental.datasources.postgres_datasource.PostgresDatasourceError¶

Bases: Exception

Common base class for all non-exit exceptions.

exception great_expectations.experimental.datasources.postgres_datasource.BatchRequestError¶

Bases: Exception

Common base class for all non-exit exceptions.

class great_expectations.experimental.datasources.postgres_datasource.ColumnSplitter¶

column_name :str¶

method_name :str¶

param_names :List[str]¶

abstract param_defaults(self, data_asset: DataAsset)¶

class great_expectations.experimental.datasources.postgres_datasource.SqlYearMonthSplitter¶

Bases: great_expectations.experimental.datasources.postgres_datasource.ColumnSplitter

method_name :str = split_on_year_and_month¶

param_names :List[str]¶

param_defaults(self, data_asset: DataAsset)¶

Query the database to get the years and months to split over.

Parameters: data_asset – A TableAsset over which we want to split the data.

class great_expectations.experimental.datasources.postgres_datasource.BatchSorter¶

metadata_key :str¶

reverse :bool = False¶

great_expectations.experimental.datasources.postgres_datasource.BatchSortersDefinition :TypeAlias¶

great_expectations.experimental.datasources.postgres_datasource._batch_sorter_from_list(sorters: BatchSortersDefinition) → List[BatchSorter]¶

great_expectations.experimental.datasources.postgres_datasource._batch_sorter_from_str(sort_key: str) → BatchSorter¶

Convert a list of strings to BatchSorters

Parameters: sort_key – A batch metadata key which will be used to sort batches on a data asset. This can be prefixed with a + or - to indicate increasing or decreasing sorting. If not specified, defaults to increasing order.

class great_expectations.experimental.datasources.postgres_datasource.TableAsset(**kwargs)¶

Bases: great_expectations.experimental.datasources.interfaces.DataAsset

type :Literal['table'] = table¶

table_name :str¶

column_splitter :Optional[ColumnSplitter]¶

name :str¶

_order_by :List[BatchSorter]¶

property order_by(self)¶

get_batch_request(self, options: Optional[BatchRequestOptions] = None)¶

A batch request that can be used to obtain batches for this DataAsset.

Parameters: options – A dict that can be used to limit the number of batches returned from the asset. The dict structure depends on the asset type. A template of the dict can be obtained by calling batch_request_options_template.
Returns: A BatchRequest object that can be used to obtain a batch list from a Datasource by calling the get_batch_list_from_batch_request method.

_valid_batch_request_options(self, options: BatchRequestOptions)¶

_validate_batch_request(self, batch_request: BatchRequest)¶

Validates the batch_request has the correct form.

Parameters: batch_request – A batch request object to be validated.

batch_request_options_template(self)¶

A BatchRequestOptions template for get_batch_request.

Returns: A BatchRequestOptions dictionary with the correct shape that get_batch_request will understand. All the option values are defaulted to None.

add_sorters(self, sorters: BatchSortersDefinition)¶

add_year_and_month_splitter(self, column_name: str)¶

Associates a year month splitter with this DataAsset

Parameters: column_name – A column name of the date column where year and month will be parsed out.
Returns: This TableAsset so we can use this method fluently.

_fully_specified_batch_requests(self, batch_request)¶: Populates a batch requests unspecified params producing a list of batch requests.

_sort_batches(self, batch_list: List[Batch])¶

Sorts batch_list in place.

Parameters: batch_list – The list of batches to sort in place.

get_batch_list_from_batch_request(self, batch_request: BatchRequest)¶

A list of batches that match the BatchRequest.

Parameters: batch_request – A batch request for this asset. Usually obtained by calling get_batch_request on the asset.
Returns: A list of batches that match the options specified in the batch request.

class great_expectations.experimental.datasources.postgres_datasource.PostgresDatasource(**kwargs)¶

Bases: great_expectations.experimental.datasources.interfaces.Datasource

Postgres datasource

Parameters

name – The name of this datasource
connection_str – The SQLAlchemy connection string used to connect to the database. For example: “postgresql+psycopg2://postgres:@localhost/test_database”
assets – An optional dictionary whose keys are table asset names and whose values are TableAsset objects.

asset_types :ClassVar[List[Type[DataAsset]]]¶

type :Literal['postgres'] = postgres¶

connection_string :str¶

assets :Dict[str, TableAsset]¶

execution_engine_type(self)¶: Returns the default execution engine type.

add_table_asset(self, name: str, table_name: str, order_by: Optional[BatchSortersDefinition] = None)¶

Adds a table asset to this datasource.