great_expectations.datasource.batch_kwargs_generator.manual_batch_kwargs_generator
¶
Module Contents¶
Classes¶
|
ManualBatchKwargsGenerator returns manually-configured batch_kwargs for named data assets. It provides a |
-
great_expectations.datasource.batch_kwargs_generator.manual_batch_kwargs_generator.
logger
¶
-
class
great_expectations.datasource.batch_kwargs_generator.manual_batch_kwargs_generator.
ManualBatchKwargsGenerator
(name='default', datasource=None, assets=None)¶ Bases:
great_expectations.datasource.batch_kwargs_generator.batch_kwargs_generator.BatchKwargsGenerator
ManualBatchKwargsGenerator returns manually-configured batch_kwargs for named data assets. It provides a convenient way to capture complete batch definitions without requiring the configuration of a more fully-featured batch kwargs generator.
A fully configured ManualBatchKwargsGenerator in yml might look like the following:
my_datasource: class_name: PandasDatasource batch_kwargs_generators: my_generator: class_name: ManualBatchKwargsGenerator assets: asset1: - partition_id: 1 path: /data/file_1.csv reader_options: sep: ; - partition_id: 2 path: /data/file_2.csv reader_options: header: 0 logs: path: data/log.csv
-
recognized_batch_parameters
¶
-
property
assets
(self)¶
-
get_available_data_asset_names
(self)¶ Return the list of asset names known by this batch kwargs generator.
- Returns
A list of available names
-
_get_data_asset_config
(self, data_asset_name)¶
-
_get_iterator
(self, data_asset_name, **kwargs)¶
-
get_available_partition_ids
(self, generator_asset=None, data_asset_name=None)¶ Applies the current _partitioner to the batches available on data_asset_name and returns a list of valid partition_id strings that can be used to identify batches of data.
- Parameters
data_asset_name – the data asset whose partitions should be returned.
- Returns
A list of partition_id strings
-
_build_batch_kwargs
(self, batch_parameters)¶ Build batch kwargs from a partition id.
-