Simple data validations

Nullity and Uniqueness

Column must not have null values

The indicated column must not have null values in any row.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.

Examples

Consider the next table:

ID	ANIMAL	AGE
1	DOG	3
2	CAT	NULL
3	COW	4

• Column must not have null values on the ANIMAL column will return Success ✅
• Column must not have null values on the AGE column will return Error ❌

Column values must be null

Column values must be null.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.

Examples

Consider the next table:

ID	ANIMAL	AGE
1	DOG	NULL
2	CAT	NULL
3	COW	NULL

• Column values must be null on the AGE column will return Success ✅
• Column values must be null on the ANIMAL column will return Error ❌

Column values must be unique

Column values must be unique across all rows.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.

Examples

Consider the next table:

ID	ANIMAL	AGE
1	DOG	2
2	CAT	3
3	COW	3

• Column values must be unique on the ID column will return Success ✅
• Column values must be unique on the AGE column will return Error ❌

Set of column values must be unique

Given two or more columns the set of values must be unique across all rows.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column list	Set	Set of comma separated values. Strings should be surrounded by quotes. For example: "column_a","column_b"

Examples

Consider the next table:

ID	ANIMAL	AGE	OWNER
1	DOG	2	Joe
2	DOG	2	Mary
3	COW	3	Alex

• Set of column values must be unique on the set "ID","ANIMAL","AGE" column will return Success ✅
• Set of column values must be unique on the set "ANIMAL","AGE" column will return Error ❌

Column values must be unique in the same row

Values for the given columns must be unique for each row.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column list	Set	Set of comma separated values. Strings should be surrounded by quotes. For example: "column_a","column_b"

Examples

Consider the next table:

ID	ANIMAL	AGE	OWNER
1	DOG	2	Joe
2	DOG	2	Mary
3	COW	3	Alex

• Column values must be unique in the same row on the set "ID","ANIMAL" column will return Success ✅
• Column values must be unique in the same row on the set "ID","AGE" column will return Error ❌

Set of values

Column values must be in set

Every distinct column value must be contained in the given set.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.
Value set	Array	Set of comma separated values. Strings should be surrounded by quotes. For example: "large","small","medium"

Examples

Consider the next table:

ID	ANIMAL	AGE	OWNER
1	DOG	2	Joe
2	CAT	2	Mary
3	COW	3	Alex

• Column values must be in set with the set "DOG","CAT","COW" on the ANIMAL column will return Success ✅
• Column values must be in set with the set "JOE","MARY" on the column OWNER will return Error ❌

Column distinct values must contain set

The given set must be contained in the set of distinct column values.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.
Value set	Set	Set of comma separated values. Strings should be surrounded by quotes. For example: "large","small","medium"

Examples

Consider the next table:

ID	ANIMAL	AGE	OWNER
1	DOG	2	Joe
2	CAT	2	Mary
3	COW	3	Alex

• Column distinct values must contain set with the set "DOG","CAT" on the ANIMAL column will return Success ✅
• Column distinct values must contain set with the set "JOE","PAUL" on the column OWNER will return Error ❌

Column distinct values must be equal to set

Set of distinct column values must be equal to the given set.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.
Value set	Set	Set of comma separated values. Strings should be surrounded by quotes. For example: "large","small","medium"

Examples

Consider the next table:

ID	ANIMAL	AGE	OWNER
1	DOG	2	Joe
2	CAT	2	Mary
3	COW	3	Alex

• Column distinct values must be equal to set with the set "DOG","CAT","COW" on the ANIMAL column will return Success ✅
• Column distinct values must be equal to set with the set "JOE","MARY" on the column OWNER will return Error ❌

Column most common value must be in set

The most common value in the column must be in the given set.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.
Value set	Set	Set of comma separated values. Strings should be surrounded by quotes. For example: "large","small","medium"

Examples

Consider the next table:

ID	ANIMAL	AGE	OWNER
1	DOG	2	Mary
2	DOG	2	Mary
3	COW	3	Alex

• Column most common value must be in set with the set "DOG","CAT" on the ANIMAL column will return Success ✅
• Column most common value must be in set with the set "JOE","PAUL" on the column OWNER will return Error ❌

Values in range

Column maximum value must be in range

The column maximum value must be between a minimum value and a maximum value. This validation is by default inclusive.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.
Minimum value	Number	Minimum allowed value.
Exclude minimum value	Boolean	If you check this option the minimum value must be strictly larger than the specified minimum value.
Maximum value	Number	Maximum allowed value.
Exclude maximum value	Boolean	If you check this option the maximum value must be strictly smaller than the specified maximum value.

Examples

Consider the next table:

ID	ANIMAL	AGE	OWNER
1	DOG	1	Mary
2	CAT	2	Joe
3	COW	3	Alex

• Column maximum value must be in range with a minimum value of 0 and a maximum value of 20 on the AGE column will return Success ✅
• Column maximum value must be in range with a minimum value of 3 (excluding the minimum value) and a maximum value of 20 on the AGE column will return Error ❌

Column mean value must be in range

Column mean value must be between a minimum value and a maximum value. This validation is by default inclusive.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.
Minimum value	Number	The minimum value for the column mean.
Exclude minimum value	Boolean	If you check this option the mean value must be strictly larger than the specified minimum value.
Maximum value	Number	The maximum value for the column mean.
Exclude maximum value	Boolean	If you check this option the mean value must be strictly smaller than the specified maximum value.

Examples

Consider the next table:

ID	ANIMAL	AGE	OWNER
1	DOG	1	Mary
2	CAT	2	Joe
3	COW	3	Alex

• Column mean value must be in range with a minimum value of 2 and a maximum value of 10 on the AGE column will return Success ✅
• Column mean value must be in range with a minimum value of 1 and a maximum value of 3 (excluding the maximum value) on the AGE column will return Error ❌

Column minimum value must be in range

Column minimum value must be between a minimum value and a maximum value. This validation is by default inclusive.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.
Minimum value	Number	Minimum allowed value.
Exclude minimum value	Boolean	If you check this option the min value must be strictly larger than the specified minimum value.
Maximum value	Number	Maximum allowed value.
Exclude maximum value	Boolean	If you check this option the min value must be strictly smaller than the specified maximum value.

Examples

Consider the next table:

ID	ANIMAL	AGE	OWNER
1	DOG	1	Mary
2	CAT	2	Joe
3	COW	3	Alex

• Column minimum value must be in range with a minimum value of 0 and a maximum value of 20 on the AGE column will return Success ✅
• Column minimum value must be in range with a minimum value of 1 (excluding the minimum value) and a maximum value of 20 on the AGE column will return Error ❌

Column proportion of unique values must be in range

Proportion of unique values must be between a minimum value and a maximum value.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.
Minimum value	Number	The minimum proportion of unique values. (Proportions are on the range 0 to 1)
Exclude minimum value	Boolean	If you check this option the proportion of unique values must be strictly greater than the specified minimum value.
Maximum value	Number	The maximum proportion of unique values. (Proportions are on the range 0 to 1)
Exclude maximum value	Boolean	If you check this option the proportion of unique values must be strictly smaller than the specified maximum value

Examples

Consider the next table:

ID	ANIMAL	AGE	OWNER
1	DOG	1	Mary
2	CAT	2	Joe
3	CAT	3	Alex

• Column proportion of unique values must be in range with a minimum value of 0.5 and a maximum value of 0.8 on the ANIMAL column will return Success ✅.
• Column proportion of unique values must be in range with a minimum value of 0.8 and a maximum value of 1 on the ANIMAL column will return Error ❌

In this case the ANIMAL column has 3 total values and 2 unique values so the proportion of unique values will be 0.66.

Column total unique values must be in range

The total number of unique values must be between a minimum value and a maximum value. This validation is by default inclusive.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.
Minimum value	Number	The minimum number of unique values allowed.
Maximum value	Number	The maximum number of unique values allowed.

Examples

Consider the next table:

ID	ANIMAL	AGE	OWNER
1	DOG	1	Mary
2	CAT	2	Joe
3	COW	3	Alex

• Column total unique values must be in range with a minimum value of 3 and a maximum value of 5 on the ANIMAL column will return Success ✅.
• Column total unique values must be in range with a minimum value of 10 and a maximum value of 20 on the ANIMAL column will return Error ❌

Multiple columns

Column A must be greater than column B

Every value in column A must be greater than the value in column B for the same row.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column A	Column	The column A name.
Column B	Column	The column B name.
Values can be equal	Boolean	If you check this option, the values in column A can be equal to column B and not strictly greater.

Examples

Consider the next table:

ID	ANIMAL	ANIMAL_AGE	OWNER	OWNER_AGE
1	DOG	1	Mary	23
2	CAT	2	Joe	30
3	COW	3	Alex	42

• Column A must be greater than column B with column A OWNER_AGE and column B ANIMAL_AGE column will return Success ✅.
• Column A must be greater than column B with column A ANIMAL_AGE and column B OWNER_AGE column will return Error ❌.

String values

Strings length must be in range

Column values must be strings with length between a minimum value and a maximum value. This validation is by default inclusive.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.
Minimum value	Number	The minimum value for the string length.
Maximum value	Number	The maximum value for the string length.

Examples

Consider the next table:

ID	ANIMAL	AGE	OWNER
1	DOG	1	Mary
2	CAT	2	Joe
3	COW	3	Alex

• Strings length must be in range with a minimum value of 3 and a maximum value of 15 on the ANIMAL column will return Success ✅.
• Strings length must be in range with a minimum value of 5 and a maximum value of 20 on the OWNER column will return Error ❌

Strings length must be equal to

Column values must be strings with length equal to the given value.

Available for

Connector	Available
Google BigQuery	✅
MySQL	✅
PostgreSQL	✅
Snowflake	✅
Amazon Redshift	✅

Parameters

Name	Type	Description
Column	Column	The column name to validate.
Value	Number	String length value.

Examples

Consider the next table:

ID	ANIMAL	AGE	OWNER
1	DOG	1	Mary
2	CAT	2	Joe
3	COW	3	Alex

• Strings length must be in range with a value of 3 on the ANIMAL column will return Success ✅.
• Strings length must be in range with a value of 4 on the OWNER column will return Error ❌