You are tasked to set up a set notebook as a job for six departments and each department can run the task parallelly, the notebook takes an input parameter dept number to process the data by department, how do you go about to setup this up in job?

You are tasked to set up a set notebook as a job for six departments and each department can run the task parallelly, the notebook takes an input parameter dept number to process the data by department, how do you go about to setup this up in job?
A . Use a single notebook as task in the job and use dbutils.notebook.run to run each note-book with parameter in a different cell
B. A task in the job cannot take an input parameter, create six notebooks with hardcoded dept number and setup six tasks with linear dependency in the job
C. A task accepts key-value pair parameters, creates six tasks pass department number as parameter foreach task with no dependency in the job as they can all run in parallel. (Correct)
D. A parameter can only be passed at the job level, create six jobs pass department number to each job with linear job dependency
E. A parameter can only be passed at the job level, create six jobs pass department number to each job with no job dependency

View Answer

Answer: C

Explanation:

Here is how you setup

Create a single job and six tasks with the same notebook and assign a different parameter for each task,

Graphical user interface, text, application, email

Description automatically generated

All tasks are added in a single job and can run parallel either using single shared cluster or with individual clusters.

Graphical user

interface, application, Teams

Description automatically generated