great_expectations.rule_based_profiler

Subpackages

Package Contents

Classes

RuleBasedProfiler(name: str, config_version: float, variables: Optional[Dict[str, Any]] = None, rules: Optional[Dict[str, Dict[str, Any]]] = None, data_context: Optional[‘DataContext’] = None)

RuleBasedProfiler object serves to profile, or automatically evaluate a set of rules, upon a given

class great_expectations.rule_based_profiler.RuleBasedProfiler(name: str, config_version: float, variables: Optional[Dict[str, Any]] = None, rules: Optional[Dict[str, Dict[str, Any]]] = None, data_context: Optional['DataContext'] = None)

Bases: great_expectations.rule_based_profiler.rule_based_profiler.BaseRuleBasedProfiler

RuleBasedProfiler object serves to profile, or automatically evaluate a set of rules, upon a given batch / multiple batches of data.

Feature Maturity

icon-98d85a4e95bc11ecb09f0242ac110002 Rule-Based Profiler - How-to Guide
Use YAML to configure a flexible Profiler engine, which will then generate an ExpectationSuite for a data set
Maturity: Experimental
Details:
API Stability: Low (instantiation of Profiler and the signature of the run() method will change)
Implementation Completeness: Moderate (some augmentation and/or growth in capabilities is to be expected)
Unit Test Coverage: High (but not complete – additional unit tests will be added, commensurate with the upcoming new functionality)
Integration Infrastructure/Test Coverage: N/A -> TBD
Documentation Completeness: Moderate
Bug Risk: Low/Moderate
Expectation Completeness: Moderate
icon-98d85d3c95bc11ecb09f0242ac110002 Domain Builders - How-to Guide
Use YAML to build domains for ExpectationConfiguration generator (table, column, semantic types, etc.)
Maturity: Experimental
Details:
API Stability: Moderate
Implementation Completeness: Moderate (additional DomainBuilder classes will be developed)
Unit Test Coverage: High (but not complete – additional unit tests will be added, commensurate with the upcoming new functionality)
Integration Infrastructure/Test Coverage: N/A -> TBD
Documentation Completeness: Moderate
Bug Risk: Low/Moderate
Expectation Completeness: Moderate
icon-98d85eea95bc11ecb09f0242ac110002 Parameter Builders - How-to Guide
Use YAML to configure single and multi batch based parameter computation modules for the use by ExpectationConfigurationBuilder classes
Maturity: Experimental
Details:
API Stability: Moderate
Implementation Completeness: Moderate (additional ParameterBuilder classes will be developed)
Unit Test Coverage: High (but not complete – additional unit tests will be added, commensurate with the upcoming new functionality)
Integration Infrastructure/Test Coverage: N/A -> TBD
Documentation Completeness: Moderate
Bug Risk: Low/Moderate
Expectation Completeness: Moderate
icon-98d8607095bc11ecb09f0242ac110002 ExpectationConfiguration Builders - How-to Guide
Use YAML to configure ExpectationConfigurationBuilder classes, which emit lists of ExpectationConfiguration objects (e.g., as kwargs and meta arguments)
Maturity: Experimental
Details:
API Stability: Moderate
Implementation Completeness: Moderate (additional ExpectationConfigurationBuilder classes might be developed)
Unit Test Coverage: High (but not complete – additional unit tests will be added, commensurate with the upcoming new functionality)
Integration Infrastructure/Test Coverage: N/A -> TBD
Documentation Completeness: Moderate
Bug Risk: Low/Moderate
Expectation Completeness: Moderate
static run_profiler(data_context: DataContext, profiler_store: ProfilerStore, name: Optional[str] = None, ge_cloud_id: Optional[str] = None, variables: Optional[dict] = None, rules: Optional[dict] = None, expectation_suite_name: Optional[str] = None, include_citation: bool = True)
static run_profiler_on_data(data_context: DataContext, profiler_store: ProfilerStore, batch_request: Union[BatchRequest, RuntimeBatchRequest, dict], name: Optional[str] = None, ge_cloud_id: Optional[str] = None, expectation_suite_name: Optional[str] = None, include_citation: bool = True)
_generate_rule_overrides_from_batch_request(self, batch_request: Union[BatchRequest, RuntimeBatchRequest, dict])

Iterates through the profiler’s builder attributes and generates a set of Rules that contain overrides from the input batch request. This only applies to ParameterBuilder and any DomainBuilder with a COLUMN MetricDomainType.

Note that we are passing all batches, corresponding to the specified batch_request, to ParameterBuilder objects. If not used carefully, bias may creep in to the resulting estimates, computed by these ParameterBuilder objects.

Users of this override should be aware that a batch request should either have no notion of “current/active” batch or it is excluded.

Parameters

batch_request – Data used to override builder attributes

Returns

The dictionary representation of the Rules used as runtime arguments to run()

static add_profiler(config: RuleBasedProfilerConfig, data_context: DataContext, profiler_store: ProfilerStore, ge_cloud_id: Optional[str] = None)
static _check_validity_of_batch_requests_in_config(config: RuleBasedProfilerConfig)
static get_profiler(data_context: DataContext, profiler_store: ProfilerStore, name: Optional[str] = None, ge_cloud_id: Optional[str] = None)
static delete_profiler(profiler_store: ProfilerStore, name: Optional[str] = None, ge_cloud_id: Optional[str] = None)
static list_profilers(profiler_store: ProfilerStore, ge_cloud_mode: bool)