Where are Interactive notebook results stored in Databricks product architecture?

Where are Interactive notebook results stored in Databricks product architecture?
A . Data plane
B . Control plane
C . Data and Control plane
D . JDBC data source
E . Databricks web application

View Answer

Answer: C

Explanation:

The answer is Data and Control plane,

Only Job results are stored in Data Plane (your storage), Interactive notebook results are stored in a combination of the control plane (partial results for presentation in the UI) and customer storage.

https://docs.microsoft.com/en-us/azure/databricks/getting-started/overview#–high-level-architecture

Snippet from the above documentation,

Graphical user

interface, application

Description automatically generated

How to change this behavior?

You can change this behavior using Workspace/Admin Console settings for that workspace, once enabled all of the interactive results are stored in the customer account(data plane) except the new notebook visualization feature Databricks has recently introduced, this still stores some metadata in the control pane irrespective of the below settings. please refer to the documentation for more details.

Graphical user

interface, text, application, email

Description automatically generated

Why is this important to know?

I recently worked on a project where we had to deal with sensitive information of customers and we had a security requirement that all of the data need to be stored in the data plane including notebook results.