Why does AUTO LOADER require schema location?

Why does AUTO LOADER require schema location?
A . Schema location is used to store user provided schema
B . Schema location is used to identify the schema of target table
C . AUTO LOADER does not require schema location, because its supports Schema evolution
D . Schema location is used to store schema inferred by AUTO LOADER
E . Schema location is used to identify the schema of target table and source table

View Answer

Answer: D

Explanation:

The answer is, Schema location is used to store schema inferred by AUTO LOADER, so the next time AUTO LOADER runs faster as does not need to infer the schema every single time by trying to use the last known schema.

Auto Loader samples the first 50 GB or 1000 files that it discovers, whichever limit is crossed first. To avoid incurring this inference cost at every stream start up, and to be able to provide a stable schema across stream restarts, you must set the option cloudFiles.schemaLocation. Auto Loader creates a hidden directory _schemas at this location to track schema changes to the input data over time.

The below link contains detailed documentation on different options Auto Loader options | Databricks on AWS