great_expectations.datasource.data_connector.inferred_asset_dbfs_data_connector

Module Contents

Classes

InferredAssetDBFSDataConnector(name: str, datasource_name: str, base_directory: str, execution_engine: ExecutionEngine, default_regex: Optional[dict] = None, glob_directive: str = ‘*’, sorters: Optional[list] = None, batch_spec_passthrough: Optional[dict] = None)

Extension of InferredAssetFilesystemDataConnector used to connect to data on a DBFS filesystem.

great_expectations.datasource.data_connector.inferred_asset_dbfs_data_connector.logger
class great_expectations.datasource.data_connector.inferred_asset_dbfs_data_connector.InferredAssetDBFSDataConnector(name: str, datasource_name: str, base_directory: str, execution_engine: ExecutionEngine, default_regex: Optional[dict] = None, glob_directive: str = '*', sorters: Optional[list] = None, batch_spec_passthrough: Optional[dict] = None)

Bases: great_expectations.datasource.data_connector.InferredAssetFilesystemDataConnector

Extension of InferredAssetFilesystemDataConnector used to connect to data on a DBFS filesystem. Note: This works for the current implementation of DBFS. If in the future DBFS diverges from a Filesystem-like implementation, we should instead inherit from InferredAssetFilePathDataConnector or another DataConnector.

The InferredAssetDBFSDataConnector is one of two classes (ConfiguredAssetDBFSDataConnector being the other one) designed for connecting to data on a DBFS filesystem. It connects to assets inferred from directory and file name by default_regex and glob_directive.

InferredAssetDBFSDataConnector that operates on file paths and determines the data_asset_name implicitly (e.g., through the combination of the regular expressions pattern and group names)

_get_full_file_path(self, path: str, data_asset_name: Optional[str] = None)