great_expectations.datasource.data_connector.inferred_asset_filesystem_data_connector

Module Contents

Classes

InferredAssetFilesystemDataConnector(name: str, datasource_name: str, base_directory: str, execution_engine: Optional[ExecutionEngine] = None, default_regex: Optional[dict] = None, glob_directive: str = ‘*’, sorters: Optional[list] = None, batch_spec_passthrough: dict = None)

Extension of InferredAssetFilePathDataConnector used to connect to data on a filesystem.

great_expectations.datasource.data_connector.inferred_asset_filesystem_data_connector.logger
class great_expectations.datasource.data_connector.inferred_asset_filesystem_data_connector.InferredAssetFilesystemDataConnector(name: str, datasource_name: str, base_directory: str, execution_engine: Optional[ExecutionEngine] = None, default_regex: Optional[dict] = None, glob_directive: str = '*', sorters: Optional[list] = None, batch_spec_passthrough: dict = None)

Bases: great_expectations.datasource.data_connector.InferredAssetFilePathDataConnector

Extension of InferredAssetFilePathDataConnector used to connect to data on a filesystem.

The InferredAssetFilesystemDataConnector is one of two classes (ConfiguredAssetFilesystemDataConnector being the other one) designed for connecting to data on a filesystem. It connects to assets inferred from directory and file name by default_regex and glob_directive.

InferredAssetFilesystemDataConnector that operates on file paths and determines the data_asset_name implicitly (e.g., through the combination of the regular expressions pattern and group names)

_get_data_reference_list(self, data_asset_name: Optional[str] = None)

List objects in the underlying data store to create a list of data_references.

This method is used to refresh the cache.

_get_full_file_path(self, path: str, data_asset_name: Optional[str] = None)
property base_directory(self)

Accessor method for base_directory. If directory is a relative path, interpret it as relative to the root directory. If it is absolute, then keep as-is.

build_batch_spec(self, batch_definition: BatchDefinition)

Build BatchSpec from batch_definition by calling DataConnector’s build_batch_spec function.

Parameters

batch_definition (BatchDefinition) – to be used to build batch_spec

Returns

BatchSpec built from batch_definition