SqlAlchemyExecutionEngine
class great_expectations.execution_engine.SqlAlchemyExecutionEngine(name: Optional[str] = None, credentials: Optional[dict] = None, data_context: Optional[Any] = None, engine: Optional[SaEngine] = None, connection_string: Optional[str] = None, url: Optional[str] = None, batch_data_dict: Optional[dict] = None, create_temp_table: bool = True, concurrency: Optional[ConcurrencyConfig] = None, **kwargs)#
SparkDFExecutionEngine instantiates the ExecutionEngine API to support computations using Spark platform.
Constructor builds a SqlAlchemyExecutionEngine, using a provided connection string/url/engine/credentials to access the desired database.
Also initializes the dialect to be used.
- Parameters
name (str) – The name of the SqlAlchemyExecutionEngine
credentials – If the Execution Engine is not provided, the credentials can be used to build the ExecutionEngine object. If the Engine is provided, it will be used instead.
data_context (DataContext) – An object representing a Great Expectations project that can be used to access ExpectationSuite objects and the Project Data itself.
engine (Engine) – A SqlAlchemy Engine used to set the SqlAlchemyExecutionEngine being configured, useful if an Engine has already been configured and should be reused. Will override Credentials if provided. If you are passing an engine that requires a single connection e.g. if temporary tables are not persisted if the connection is closed (e.g. sqlite, mssql) then you should create the engine with a StaticPool e.g. engine = sa.create_engine(connection_url, poolclass=sa.pool.StaticPool)
connection_string (string) – If neither the engines nor the credentials have been provided, a connection string can be used to access the data. This will be overridden by both the engine and credentials if those are provided.
url (string) – If neither the engines, the credentials, nor the connection_string have been provided, a URL can be used to access the data. This will be overridden by all other configuration options if any are provided.
concurrency (ConcurrencyConfig) – Concurrency config used to configure the sqlalchemy engine.
kwargs (dict) – These will be passed as optional parameters to the SQLAlchemy engine, not the ExecutionEngine
For example:
execution_engine: ExecutionEngine = SqlAlchemyExecutionEngine(connection_string="dbmstype://user:password@host:5432/database_name")
- execute_query(query: sqlalchemy.sql.selectable.Selectable)sqlalchemy.engine.cursor.CursorResult | sqlalchemy.engine.cursor.LegacyCursorResult #
Execute a query using the underlying database engine.
New in version 0.16.14.
- Parameters
query – Sqlalchemy selectable query.
- Returns
CursorResult for sqlalchemy 2.0+ or LegacyCursorResult for earlier versions.
- execute_query_in_transaction(query: sqlalchemy.sql.selectable.Selectable)sqlalchemy.engine.cursor.CursorResult | sqlalchemy.engine.cursor.LegacyCursorResult #
Execute a query using the underlying database engine within a transaction that will auto commit.
New in version 0.16.14.
Begin once: Connections.Html
- Parameters
query – Sqlalchemy selectable query.
- Returns
CursorResult for sqlalchemy 2.0+ or LegacyCursorResult for earlier versions.
get_compute_domain(domain_kwargs: dict, domain_type: Union[str, great_expectations.core.metric_domain_types.MetricDomainTypes], accessor_keys: Optional[Iterable[str]] = None) Tuple[sqlalchemy.sql.selectable.Selectable, dict, dict] #
Uses a given batch dictionary and Domain kwargs to obtain a SqlAlchemy column object.
- Parameters
domain_kwargs (dict) – a dictionary consisting of the Domain kwargs specifying which data to obtain
domain_type (str or MetricDomainTypes) – an Enum value indicating which metric Domain the user would like to be using, or a corresponding string value representing it. String types include "identity", "column", "column_pair", "table" and "other". Enum types include capitalized versions of these from the class MetricDomainTypes.
accessor_keys (str iterable) – keys that are part of the compute Domain but should be ignored when describing the Domain and simply transferred with their associated values into accessor_domain_kwargs.
- Returns
SqlAlchemy column
- get_domain_records(domain_kwargs: dict)sqlalchemy.sql.selectable.Selectable #
Uses the given Domain kwargs (which include row_condition, condition_parser, and ignore_row_if directives) to obtain and/or query a Batch of data.
- Parameters
domain_kwargs (dict) –
- Returns
An SqlAlchemy table/column(s) (the selectable object for obtaining data on which to compute returned in the format of an SqlAlchemy table/column(s) object)