In this lecture.

I’ll show you how to use parameters, specifically how to pass parameters and variables between tasks.

So I’ve created a new folder called O3 Parameters and Task Values.

It contains two notebooks.

Notebook zero one.

Contains some code where I’ve created a variable called name that stores the value.

James.

name = ‘James’

Using DB utils, specifically the jobs and task values utilities.

I can set a task value.

The task value is.

James, As you can see here in this value argument, it’s referencing the name variable which stores

James And I’ve set the key for this task value to be name, as you can see here in Notebook two.

I’m retrieving the task value that was set in notebook one by using the get method in the same utilities,

which is the task values and jobs.

Utilities.

For the task key You specify the name of the task where you execute the job in which you are setting

the task value, and that task will be called Notebook zero one.

So this isn’t the name of the notebook.

This is the name of the task.

For the key.

This is name.

So this will be the name of the key that you’ve set.

So as you can see, its name.

And then this debug value is optional.

This is what is returned if the key cannot be found.

It avoids a type error in case the value we’re trying to return is from a notebook that’s running outside

of this job.

I’ll leave this link about task values in a notebook called Links and Resources.

It basically contains the code that I’ve shown you on how you can set your task values and how you can

get your task values.

It also contains additional details on all the arguments.

So please review this in your own time.

Okay.

So I’ve created a job called Demo Job zero three that’s already successfully executed.

It contains two tasks.

Notebook zero one and notebook zero two.

So as you can see here in the Tasks Notebook, one just runs notebook one and notebook two runs.

Notebook zero two.

So if I go back to runs and then click here, if I open up notebook one, you can see we’ve successfully.

Set the task value name.

Toby James.

And then.

For this task.

We’ve returned it and we’ve printed out James.

And note, the task itself is called Notebook oh one.

So please, no.

That there is a size limit for the task values.

You can’t pass a large data frame or anything like that.

It’s limited to around 100 KB.

S, so you can only really pass in small values between tasks.

If you check out this link, which I’ll also provide in the course resources, you can see we have a

specific task parameter variables.

You have the job ID run ID start date start time.

Haven one task.

Key and so on.

You reference them with these double curly braces.

So back in the task creation, let me pass one of the task variables into the notebook one task.

So I’ll go to tasks and for notebook one.

I will click add on parameters.

And I’ll specify a key for the parameter, which will be job ID and I’ll reference the job ID.

By using these double curly braces.

So let’s save that task.

And then go back to workspace.

And then in notebook one.

I’ll retrieve the value for that task parameter.

So what we can do is type DB utils dot widgets, dot get and then just specify the parameter.

This is the name of the parameter that will be passed into the task.

And then we can wrap this around the print statement.

I’ve discussed widget utilities in the DB utils module in a previous section, but you can refer to

this documentation here which contains additional information on Databricks widgets.

And I’ll leave this as a link in the course resources.

But essentially this is used to retrieve the parameter that we’ve defined in the task itself.

So it’s used.

To retrieve this parameter, which will be the job ID.

So let me run the job.

So the job has been triggered.

So the job run was successful and all the tasks have executed without any issues.

So if I click on the latest run.

Go on.

Notebook one.

You can see.

This is the job ID?

And it matches the job here.

So essentially.

When we look at the tasks.

We’ve created a parameter here called job ID, which is passed into the task.

And that stores the value, which is a task variable which contains the job ID for the job that is executing

that task.

This is then passed into the notebook and we’ve referenced it inside the notebook.

By using the debug tools widgets dot get method by specifying the parameter.

So that’s how you can pass parameters between notebooks and even into tasks that can then be retrieved

inside your notebook for that specific task.

This can be useful, especially in terms of passing in metadata, such as the job ID run ID and other

bits of information which you can then add on to your dataframes or other objects in your workflows.

This can be for audit purposes, tracking purposes as well as other use cases such as implementing control

flow.