great_expectations.profile.base
¶
Module Contents¶
Classes¶
Generic enumeration. |
|
Generic enumeration. |
|
Useful data types for building profilers. |
|
Useful cardinality categories for building profilers. |
|
Useful backend type mapping for building profilers. |
|
|
Profiler creates suites from various sources of truth. |
-
great_expectations.profile.base.
logger
¶
-
class
great_expectations.profile.base.
OrderedEnum
¶ Bases:
enum.Enum
Generic enumeration.
Derive from this class to define new enumerations.
-
__ge__
(self, other)¶ Return self>=value.
-
__gt__
(self, other)¶ Return self>value.
-
__le__
(self, other)¶ Return self<=value.
-
__lt__
(self, other)¶ Return self<value.
-
-
class
great_expectations.profile.base.
OrderedProfilerCardinality
¶ Bases:
great_expectations.profile.base.OrderedEnum
Generic enumeration.
Derive from this class to define new enumerations.
-
NONE
= 0¶
-
ONE
= 1¶
-
TWO
= 2¶
-
VERY_FEW
= 3¶
-
FEW
= 4¶
-
MANY
= 5¶
-
VERY_MANY
= 6¶
-
UNIQUE
= 7¶
-
classmethod
get_basic_column_cardinality
(cls, num_unique=0, pct_unique=0)¶ Takes the number and percentage of unique values in a column and returns the column cardinality. If you are unexpectedly returning a cardinality of “None”, ensure that you are passing in values for both num_unique and pct_unique. :param num_unique: The number of unique values in a column :param pct_unique: The percentage of unique values in a column
- Returns
The column cardinality
-
-
class
great_expectations.profile.base.
ProfilerDataType
¶ Bases:
enum.Enum
Useful data types for building profilers.
-
INT
= int¶
-
FLOAT
= float¶
-
NUMERIC
= numeric¶
-
STRING
= string¶
-
BOOLEAN
= boolean¶
-
DATETIME
= datetime¶
-
UNKNOWN
= unknown¶
-
-
class
great_expectations.profile.base.
ProfilerCardinality
¶ Bases:
enum.Enum
Useful cardinality categories for building profilers.
-
NONE
= none¶
-
ONE
= one¶
-
TWO
= two¶
-
FEW
= few¶
-
VERY_FEW
= very few¶
-
MANY
= many¶
-
VERY_MANY
= very many¶
-
UNIQUE
= unique¶
-
-
class
great_expectations.profile.base.
ProfilerTypeMapping
¶ Useful backend type mapping for building profilers.
-
INT_TYPE_NAMES
= ['INTEGER', 'integer', 'int', 'int_', 'int8', 'int16', 'int32', 'int64', 'uint8', 'uint16', 'uint32', 'uint64', 'INT', 'INTEGER', 'INT64', 'TINYINT', 'BYTEINT', 'SMALLINT', 'BIGINT', 'IntegerType', 'LongType']¶
-
FLOAT_TYPE_NAMES
= ['FLOAT', 'FLOAT4', 'FLOAT8', 'FLOAT64', 'DOUBLE', 'DOUBLE_PRECISION', 'NUMERIC', 'FloatType', 'DoubleType', 'float_', 'float16', 'float32', 'float64', 'number', 'DECIMAL', 'REAL']¶
-
STRING_TYPE_NAMES
= ['CHAR', 'NCHAR', 'VARCHAR', 'NVARCHAR', 'TEXT', 'NTEXT', 'STRING', 'StringType', 'string', 'str', 'object', "dtype('O')"]¶
-
BOOLEAN_TYPE_NAMES
= ['BOOLEAN', 'boolean', 'BOOL', 'TINYINT', 'BIT', 'bool', 'BooleanType']¶
-
DATETIME_TYPE_NAMES
= ['DATE', 'TIME', 'DATETIME', 'DATETIME2', 'DATETIME64', 'SMALLDATETIME', 'DATETIMEOFFSET', 'TIMESTAMP', 'Timestamp', 'TimestampType', 'DateType', 'datetime64', 'datetime64[ns]', 'timedelta[ns]', '<M8[ns]']¶
-
BINARY_TYPE_NAMES
= ['BINARY', 'binary', 'VARBINARY', 'varbinary', 'IMAGE', 'image']¶
-
CURRENCY_TYPE_NAMES
= ['MONEY', 'money', 'SMALLMONEY', 'smallmoney']¶
-
IDENTIFIER_TYPE_NAMES
= ['UNIQUEIDENTIFIER', 'uniqueidentifier']¶
-
MISCELLANEOUS_TYPE_NAMES
= ['SQL_VARIANT', 'sql_variant']¶
-
RECORD_TYPE_NAMES
= ['JSON', 'json']¶
-
-
great_expectations.profile.base.
profiler_data_types_with_mapping
¶
-
great_expectations.profile.base.
profiler_semantic_types
¶
-
class
great_expectations.profile.base.
Profiler
(configuration: dict = None)¶ Profiler creates suites from various sources of truth.
These sources of truth can be data or non-data sources such as DDLs.
When implementing a Profiler ensure that you: - Implement a . _profile() method - Optionally implement .validate() method that verifies you are running on the right
kind of object. You should raise an appropriate Exception if the object is not valid.
-
validate
(self, item_to_validate: Any)¶
-
profile
(self, item_to_profile: Any, suite_name: str = None)¶
-
abstract
_profile
(self, item_to_profile: Any, suite_name: str = None)¶
-
-
class
great_expectations.profile.base.
DatasetProfiler
¶ Bases:
great_expectations.profile.base.DataAssetProfiler
-
classmethod
validate
(cls, dataset)¶
-
classmethod
add_expectation_meta
(cls, expectation)¶
-
classmethod
add_meta
(cls, expectation_suite, batch_kwargs=None)¶
-
classmethod
profile
(cls, data_asset, run_id=None, profiler_configuration=None, run_name=None, run_time=None)¶
-
abstract classmethod
_profile
(cls, dataset, configuration=None)¶
-
classmethod