great_expectations.datasource.data_connector.inferred_asset_filesystem_data_connector
¶
Module Contents¶
Classes¶
|
Extension of InferredAssetFilePathDataConnector used to connect to data on a filesystem. |
-
great_expectations.datasource.data_connector.inferred_asset_filesystem_data_connector.
logger
¶
-
class
great_expectations.datasource.data_connector.inferred_asset_filesystem_data_connector.
InferredAssetFilesystemDataConnector
(name: str, datasource_name: str, base_directory: str, execution_engine: Optional[ExecutionEngine] = None, default_regex: Optional[dict] = None, glob_directive: str = '*', sorters: Optional[list] = None, batch_spec_passthrough: dict = None)¶ Bases:
great_expectations.datasource.data_connector.InferredAssetFilePathDataConnector
Extension of InferredAssetFilePathDataConnector used to connect to data on a filesystem.
The InferredAssetFilesystemDataConnector is one of two classes (ConfiguredAssetFilesystemDataConnector being the other one) designed for connecting to data on a filesystem. It connects to assets inferred from directory and file name by default_regex and glob_directive.
InferredAssetFilesystemDataConnector that operates on file paths and determines the data_asset_name implicitly (e.g., through the combination of the regular expressions pattern and group names)
-
_get_data_reference_list
(self, data_asset_name: Optional[str] = None)¶ List objects in the underlying data store to create a list of data_references.
This method is used to refresh the cache.
-
_get_full_file_path
(self, path: str, data_asset_name: Optional[str] = None)¶
-
property
base_directory
(self)¶ Accessor method for base_directory. If directory is a relative path, interpret it as relative to the root directory. If it is absolute, then keep as-is.
-
build_batch_spec
(self, batch_definition: BatchDefinition)¶ Build BatchSpec from batch_definition by calling DataConnector’s build_batch_spec function.
- Parameters
batch_definition (BatchDefinition) – to be used to build batch_spec
- Returns
BatchSpec built from batch_definition
-