Skip to main content
Version: 0.18.21

SqlAlchemyExecutionEngine

class great_expectations.execution_engine.SqlAlchemyExecutionEngine(name: Optional[str] = None, credentials: Optional[dict] = None, data_context: Optional[Any] = None, engine: Optional[SaEngine] = None, connection_string: Optional[str] = None, url: Optional[str] = None, batch_data_dict: Optional[dict] = None, create_temp_table: bool = True, concurrency: Optional[ConcurrencyConfig] = None, **kwargs)#

SparkDFExecutionEngine instantiates the ExecutionEngine API to support computations using Spark platform.

Constructor builds a SqlAlchemyExecutionEngine, using a provided connection string/url/engine/credentials to access the desired database.

Also initializes the dialect to be used.

Parameters

  • name (str) – The name of the SqlAlchemyExecutionEngine

  • credentials – If the Execution Engine is not provided, the credentials can be used to build the ExecutionEngine object. If the Engine is provided, it will be used instead.

  • data_context (DataContext) – An object representing a Great Expectations project that can be used to access ExpectationSuite objects and the Project Data itself.

  • engine (Engine) – A SqlAlchemy Engine used to set the SqlAlchemyExecutionEngine being configured, useful if an Engine has already been configured and should be reused. Will override Credentials if provided. If you are passing an engine that requires a single connection e.g. if temporary tables are not persisted if the connection is closed (e.g. sqlite, mssql) then you should create the engine with a StaticPool e.g. engine = sa.create_engine(connection_url, poolclass=sa.pool.StaticPool)

  • connection_string (string) – If neither the engines nor the credentials have been provided, a connection string can be used to access the data. This will be overridden by both the engine and credentials if those are provided.

  • url (string) – If neither the engines, the credentials, nor the connection_string have been provided, a URL can be used to access the data. This will be overridden by all other configuration options if any are provided.

  • concurrency (ConcurrencyConfig) – Concurrency config used to configure the sqlalchemy engine.

  • kwargs (dict) – These will be passed as optional parameters to the SQLAlchemy engine, not the ExecutionEngine

For example:

    execution_engine: ExecutionEngine = SqlAlchemyExecutionEngine(connection_string="dbmstype://user:password@host:5432/database_name")
execute_query(query: sqlalchemy.sql.selectable.Selectable)→ sqlalchemy.engine.cursor.CursorResult | sqlalchemy.engine.cursor.LegacyCursorResult#

Execute a query using the underlying database engine.

New in version 0.16.14.

Parameters

query – Sqlalchemy selectable query.

Returns

CursorResult for sqlalchemy 2.0+ or LegacyCursorResult for earlier versions.

execute_query_in_transaction(query: sqlalchemy.sql.selectable.Selectable)→ sqlalchemy.engine.cursor.CursorResult | sqlalchemy.engine.cursor.LegacyCursorResult#

Execute a query using the underlying database engine within a transaction that will auto commit.

New in version 0.16.14.

Begin once: Connections.Html

Parameters

query – Sqlalchemy selectable query.

Returns

CursorResult for sqlalchemy 2.0+ or LegacyCursorResult for earlier versions.

get_compute_domain(domain_kwargs: dict, domain_type: Union[str, great_expectations.core.metric_domain_types.MetricDomainTypes], accessor_keys: Optional[Iterable[str]] = None) → Tuple[sqlalchemy.sql.selectable.Selectable, dict, dict]#

Uses a given batch dictionary and Domain kwargs to obtain a SqlAlchemy column object.

Parameters

  • domain_kwargs (dict) – a dictionary consisting of the Domain kwargs specifying which data to obtain

  • domain_type (str or MetricDomainTypes) – an Enum value indicating which metric Domain the user would like to be using, or a corresponding string value representing it. String types include "identity", "column", "column_pair", "table" and "other". Enum types include capitalized versions of these from the class MetricDomainTypes.

  • accessor_keys (str iterable) – keys that are part of the compute Domain but should be ignored when describing the Domain and simply transferred with their associated values into accessor_domain_kwargs.

Returns

SqlAlchemy column

get_domain_records(domain_kwargs: dict)→ sqlalchemy.sql.selectable.Selectable#

Uses the given Domain kwargs (which include row_condition, condition_parser, and ignore_row_if directives) to obtain and/or query a Batch of data.

Parameters

domain_kwargs (dict) –

Returns

An SqlAlchemy table/column(s) (the selectable object for obtaining data on which to compute returned in the format of an SqlAlchemy table/column(s) object)