great_expectations.datasource.data_connector.inferred_asset_file_path_data_connector

Module Contents

Classes

InferredAssetFilePathDataConnector(name: str, datasource_name: str, execution_engine: Optional[ExecutionEngine] = None, default_regex: Optional[dict] = None, sorters: Optional[list] = None, batch_spec_passthrough: Optional[dict] = None)

The InferredAssetFilePathDataConnector is one of two classes (ConfiguredAssetFilePathDataConnector being the

great_expectations.datasource.data_connector.inferred_asset_file_path_data_connector.logger

class great_expectations.datasource.data_connector.inferred_asset_file_path_data_connector.InferredAssetFilePathDataConnector(name: str, datasource_name: str, execution_engine: Optional[ExecutionEngine] = None, default_regex: Optional[dict] = None, sorters: Optional[list] = None, batch_spec_passthrough: Optional[dict] = None)

Bases: great_expectations.datasource.data_connector.file_path_data_connector.FilePathDataConnector

The InferredAssetFilePathDataConnector is one of two classes (ConfiguredAssetFilePathDataConnector being the other one) designed for connecting to filesystem-like data. This includes files on disk, but also things like S3 object stores, etc:

InferredAssetFilePathDataConnector is a base class that operates on file paths and determines the data_asset_name implicitly (e.g., through the combination of the regular expressions pattern and group names)

Note: InferredAssetFilePathDataConnector is not meant to be used on its own, but extended. Currently InferredAssetFilesystemDataConnector, InferredAssetS3DataConnector, InferredAssetAzureDataConnector, and InferredAssetGCSDataConnector are subclasses of InferredAssetFilePathDataConnector.

_refresh_data_references_cache(self)

refreshes data_reference cache

get_data_reference_list_count(self)

Returns the list of data_references known by this DataConnector by looping over all data_asset_names in _data_references_cache

Returns

number of data_references known by this DataConnector

get_unmatched_data_references(self)

Returns the list of data_references unmatched by configuration by looping through items in _data_references_cache and returning data_references that do not have an associated data_asset.

Returns

list of data_references that are not matched by configuration.

get_available_data_asset_names(self)

Return the list of asset names known by this DataConnector

Returns

A list of available names

build_batch_spec(self, batch_definition: BatchDefinition)

Build BatchSpec from batch_definition by calling DataConnector’s build_batch_spec function.

Parameters

batch_definition (BatchDefinition) – to be used to build batch_spec

Returns

BatchSpec built from batch_definition

_get_batch_definition_list_from_cache(self)

_get_regex_config(self, data_asset_name: Optional[str] = None)