great_expectations.datasource.data_connector.configured_asset_azure_data_connector

Module Contents

Classes

ConfiguredAssetAzureDataConnector(name: str, datasource_name: str, container: str, assets: dict, execution_engine: Optional[ExecutionEngine] = None, default_regex: Optional[dict] = None, sorters: Optional[list] = None, name_starts_with: str = ‘’, delimiter: str = ‘/’, azure_options: Optional[dict] = None, batch_spec_passthrough: Optional[dict] = None)

Extension of ConfiguredAssetFilePathDataConnector used to connect to Azure

great_expectations.datasource.data_connector.configured_asset_azure_data_connector.logger
great_expectations.datasource.data_connector.configured_asset_azure_data_connector.BlobServiceClient
class great_expectations.datasource.data_connector.configured_asset_azure_data_connector.ConfiguredAssetAzureDataConnector(name: str, datasource_name: str, container: str, assets: dict, execution_engine: Optional[ExecutionEngine] = None, default_regex: Optional[dict] = None, sorters: Optional[list] = None, name_starts_with: str = '', delimiter: str = '/', azure_options: Optional[dict] = None, batch_spec_passthrough: Optional[dict] = None)

Bases: great_expectations.datasource.data_connector.configured_asset_file_path_data_connector.ConfiguredAssetFilePathDataConnector

Extension of ConfiguredAssetFilePathDataConnector used to connect to Azure

DataConnectors produce identifying information, called “batch_spec” that ExecutionEngines can use to get individual batches of data. They add flexibility in how to obtain data such as with time-based partitioning, splitting and sampling, or other techniques appropriate for obtaining batches of data.

The ConfiguredAssetAzureDataConnector is one of two classes (InferredAssetAzureDataConnector being the other one) designed for connecting to data on Azure.

A ConfiguredAssetAzureDataConnector requires an explicit specification of each DataAsset you want to connect to. This allows more fine-tuning, but also requires more setup. Please note that in order to maintain consistency with Azure’s official SDK, we utilize terms like “container” and “name_starts_with”.

As much of the interaction with the SDK is done through a BlobServiceClient, please refer to the official docs if a greater understanding of the supported authentication methods and general functionality is desired. Source: https://docs.microsoft.com/en-us/python/api/azure-storage-blob/azure.storage.blob.blobserviceclient?view=azure-python

build_batch_spec(self, batch_definition: BatchDefinition)

Build BatchSpec from batch_definition by calling DataConnector’s build_batch_spec function.

Parameters

batch_definition (BatchDefinition) – to be used to build batch_spec

Returns

BatchSpec built from batch_definition

_get_data_reference_list_for_asset(self, asset: Optional[Asset])
_get_full_file_path_for_asset(self, path: str, asset: Optional[Asset] = None)