processed?

A dataset has been defined using Delta Live Tables and includes an expectations clause:

CONSTRAINT valid_timestamp EXPECT (timestamp > ‘2020-01-01’) ON VIOLATION DROP ROW

What is the expected behavior when a batch of data containing data that violates these constraints is

processed?
A . Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table.
B . Records that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.
C . Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
D . Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
E . Records that violate the expectation cause the job to fail.

View Answer

Answer: C

Explanation:

Delta Live Tables expectations are optional clauses that apply data quality checks on each record passing through a query. An expectation consists of a description, a boolean statement, and an action to take when a record fails the expectation. The ON VIOLATION clause specifies the action to take, which can be one of the following: warn, drop, or fail. The drop action means that invalid records are dropped from the target dataset before data is written to the target. The failure is reported as a metric for the dataset, which can be viewed by querying the Delta Live Tables event log. The event log contains information such as the number of records that violate an expectation, the number of records dropped, and the number of records written to the target dataset.

Reference: Manage data quality with Delta Live Tables

Monitor Delta Live Tables pipelines

Delta Live Tables SQL language reference