Jobs¶
Jobs are a way for users to execute custom logic on demand from within the Nautobot UI. Jobs can interact directly with Nautobot data to accomplish various data creation, modification, and validation tasks, such as:
- Automatically populate new devices and cables in preparation for a new site deployment
- Create a range of new reserved prefixes or IP addresses
- Fetch data from an external source and import it to Nautobot
- Check and report whether all top-of-rack switches have a console connection
- Check and report whether every router has a loopback interface with an assigned IP address
- Check and report whether all IP addresses have a parent prefix
...and so on. Jobs are Python code and exist outside of the official Nautobot code base, so they can be updated and changed without interfering with the core Nautobot installation. And because they're completely customizable, there's practically no limit to what a job can accomplish.
Note
Jobs unify and supersede the functionality previously provided in NetBox by "custom scripts" and "reports". Jobs are backwards-compatible for now with the Script
and Report
class APIs, but you are urged to move to the new Job
class API described below. Jobs may be optionally marked as read-only which equates to the Report
functionally, but in all cases, user input is supported via job variables.
Writing Jobs¶
Jobs may be manually installed as files in the JOBS_ROOT
path (which defaults to $NAUTOBOT_ROOT/jobs/
). Each file created within this path is considered a separate module. Each module holds one or more Jobs (Python classes), each of which serves a specific purpose. The logic of each job can be split into a number of distinct methods, each of which performs a discrete portion of the overall job logic.
Warning
The jobs path must include a file named __init__.py
, which registers the path as a Python module. Do not delete this file.
As an alternative to manually managing job files, you can store job files in an external Git repository. The actual content of the files will be the same either way.
For example, we can create a module named devices.py
to hold all of our jobs which pertain to devices in Nautobot. Within that module, we might define several jobs. Each job is defined as a Python class inheriting from extras.jobs.Job
, which provides the base functionality needed to accept user input and log activity.
from nautobot.extras.jobs import Job
class CreateDevices(Job):
...
class DeviceConnectionsReport(Job):
...
class DeviceIPsReport(Job):
...
Each job class will implement some or all of the following components:
- Module and class attributes, mostly for documentation and human readability
- a set of variables for user input via the Nautobot UI (if your job requires any user inputs)
- a
run()
method, which is executed first and receives the user input values, if any - any number of
test_*()
methods, which will be invoked next in order of declaration. Log messages generated by the job will be grouped together by the test method they were invoked from. - a
post_run()
method, which is executed last and can be used to handle any necessary cleanup or final events (such as sending an email or triggering a webhook). The status of the overall job is available at this time asself.failed
and theJobResult
data object is available asself.result
.
You can implement the entire job within the run()
function, but for more complex jobs, you may want to provide more granularity in the output and logging of activity. For this purpose, you can implement portions of the logic as test_*()
methods (i.e., methods whose name begins with test_*
) and/or a post_run()
method. Log messages generated by the job logging APIs (more below on this topic) will be grouped together according to their base method (run
, test_a
, test_b
, ..., post_run
) which can aid in understanding the operation of the job.
Note
Your job can of course define additional Python methods to compartmentalize and reuse logic as required; however the run
, test_*
, and post_run
methods are the only ones that will be automatically invoked by Nautobot.
It's important to understand that jobs execute on the server asynchronously as background tasks; they log messages and report their status to the database as JobResult
records.
Note
When actively developing a Job utilizing a development environment it's important to understand that the "reload on code changes" debugging functionality does not automatically restart the nautobot_worker
; therefore, it is required to restart the worker
after each update to your Job source code.
Module Attributes¶
name
¶
You can define name
within a job module (the Python file which contains one or more job classes) to set the name that will be displayed in the Nautobot UI. If this value is not defined, the module's file name will be used.
Note
In some UI elements and API endpoints, the module file name is displayed in addition to or in place of this attribute, so even if defining this attribute, you should still choose an appropriately explanatory file name as well.
Class Attributes¶
Job-specific attributes may be defined under a class named Meta
within each job class you implement. All of these are optional, but encouraged.
name
¶
This is the human-friendly name of your job, as will be displayed in the Nautobot UI. If not set, the class name will be used.
Note
In some UI elements and API endpoints, the class name is displayed in addition to or in place of this attribute, so even if defining this attribute, you should still choose an appropriately explanatory class name as well.
description
¶
A human-friendly description of what this job does.
commit_default
¶
The checkbox to commit database changes when executing a job is checked by default in the Nautobot UI. You can set commit_default
to False
under the Meta
class if you want this option to instead be unchecked by default.
class MyJob(Job):
class Meta:
commit_default = False
field_order
¶
A list of strings (field names) representing the order your form fields should appear. If not defined, fields will appear in order of their definition in the code.
read_only
¶
A boolean that designates whether the job is able to make changes to data in the database. The value defaults to False
but when set to True
, any data modifications executed from the job's code will be automatically aborted at the end of the job. The job input form is also modified to remove the commit
checkbox as it is irrelevant for read-only jobs. When a job is marked as read-only, log messages that are normally automatically emitted about the DB transaction state are not included because no changes to data are allowed. Note that user input may still be optionally collected with read-only jobs via job variables, as described below.
Variables¶
Variables allow your job to accept user input via the Nautobot UI, but they are optional; if your job does not require any user input, there is no need to define any variables. Conversely, if you are making use of user input in your job, you must also implement the run()
method, as it is the only entry point to your job that has visibility into the variable values provided by the user.
from nautobot.extras.jobs import Job, StringVar, IntegerVar, ObjectVar
class CreateDevices(Job):
var1 = StringVar(...)
var2 = IntegerVar(...)
var3 = ObjectVar(...)
def run(self, data, commit):
...
The remainder of this section documents the various supported variable types and how to make use of them.
Default Variable Options¶
All job variables support the following default options:
default
- The field's default valuedescription
- A brief user-friendly description of the fieldlabel
- The field name to be displayed in the rendered formrequired
- Indicates whether the field is mandatory (all fields are required by default)widget
- The class of form widget to use (see the Django documentation)
StringVar
¶
Stores a string of characters (i.e. text). Options include:
min_length
- Minimum number of charactersmax_length
- Maximum number of charactersregex
- A regular expression against which the provided value must match
Note that min_length
and max_length
can be set to the same number to effect a fixed-length field.
TextVar
¶
Arbitrary text of any length. Renders as a multi-line text input field.
IntegerVar
¶
Stores a numeric integer. Options include:
min_value
- Minimum valuemax_value
- Maximum value
BooleanVar
¶
A true/false flag. This field has no options beyond the defaults listed above.
ChoiceVar
¶
A set of choices from which the user can select one.
choices
- A list of(value, label)
tuples representing the available choices. For example:
CHOICES = (
('n', 'North'),
('s', 'South'),
('e', 'East'),
('w', 'West')
)
direction = ChoiceVar(choices=CHOICES)
In the example above, selecting the choice labeled "North" will submit the value n
.
MultiChoiceVar
¶
Similar to ChoiceVar
, but allows for the selection of multiple choices.
ObjectVar
¶
A particular object within Nautobot. Each ObjectVar must specify a particular model, and allows the user to select one of the available instances. ObjectVar accepts several arguments, listed below.
model
- The model classdisplay_field
- The name of the REST API object field to display in the selection list (default:'display'
)query_params
- A dictionary of query parameters to use when retrieving available options (optional)null_option
- A label representing a "null" or empty choice (optional)
The display_field
argument is useful in cases where using the display
API field is not desired for referencing the object. For example, when displaying a list of IP Addresses, you might want to use the dns_name
field:
device_type = ObjectVar(
model=IPAddress,
display_field="dns_name",
)
To limit the selections available within the list, additional query parameters can be passed as the query_params
dictionary. For example, to show only devices with an "active" status:
device = ObjectVar(
model=Device,
query_params={
'status': 'active'
}
)
Multiple values can be specified by assigning a list to the dictionary key. It is also possible to reference the value of other fields in the form by prepending a dollar sign ($
) to the variable's name.
region = ObjectVar(
model=Region
)
site = ObjectVar(
model=Site,
query_params={
'region_id': '$region'
}
)
MultiObjectVar
¶
Similar to ObjectVar
, but allows for the selection of multiple objects.
FileVar
¶
An uploaded file. Note that uploaded files are present in memory only for the duration of the job's execution: They will not be automatically saved for future use. The job is responsible for writing file contents to disk where necessary.
IPAddressVar
¶
An IPv4 or IPv6 address, without a mask. Returns a netaddr.IPAddress
object.
IPAddressWithMaskVar
¶
An IPv4 or IPv6 address with a mask. Returns a netaddr.IPNetwork
object which includes the mask.
IPNetworkVar
¶
An IPv4 or IPv6 network with a mask. Returns a netaddr.IPNetwork
object. Two attributes are available to validate the provided mask:
min_prefix_length
- Minimum length of the maskmax_prefix_length
- Maximum length of the mask
The run()
Method¶
The run()
method, if you choose to implement it, should accept two arguments:
data
- A dictionary which will contain all of the variable data passed in by the user (via the web UI or REST API)commit
- A boolean indicating whether database changes should be committed.
from nautobot.extras.jobs import Job, StringVar, IntegerVar, ObjectVar
class CreateDevices(Job):
var1 = StringVar(...)
var2 = IntegerVar(...)
var3 = ObjectVar(...)
def run(self, data, commit):
...
Again, defining user variables is totally optional; you may create a job with just a run()
method if no user input is needed, in which case data
will be an empty dictionary.
Note
The test_*()
and post_run()
methods do not accept any arguments; if you need to access user data
or the commit
flag, your run()
method is responsible for storing these values in the job instance, such as:
python
def run(self, data, commit):
self.data = data
self.commit = commit
Warning
When writing Jobs that create and manipulate data it is recommended to make use of the validated_save()
convenience method which exists on all core models. This method saves the instance data but first enforces model validation logic. Simply calling save()
on the model instance does not enforce validation automatically and may lead to bad data. See the development best practices.
Warning
The Django ORM provides methods to create/edit many objects at once, namely bulk_create()
and update()
. These are best avoided in most cases as they bypass a model's built-in validation and can easily lead to database corruption if not used carefully.
The test_*()
Methods¶
If your job class defines any number of methods whose names begin with test_
, these will be automatically invoked after the run()
method (if any) completes. These methods must take no arguments (other than self
).
Log messages generated by any of these methods will be automatically grouped together by the test method they were invoked from, which can be helpful for readability.
The post_run()
Method¶
If your job class implements a post_run()
method (which must take no arguments other than self
), this method will be automatically invoked after the run()
and test_*()
methods (if any). It will be called even if one of the other methods raises an exception, so this method can be used to handle any necessary cleanup or final events (such as sending an email or triggering a webhook). The status of the overall job is available at this time as self.failed
and the JobResult
data object is available as self.result
.
Logging¶
The following instance methods are available to log results from an executing job to be stored into the associated JobResult
record:
self.log(message)
self.log_debug(message)
self.log_success(obj=None, message=None)
self.log_info(obj=None, message=None)
self.log_warning(obj=None, message=None)
self.log_failure(obj=None, message=None)
Messages recorded with log()
or log_debug()
will appear in a job's results but are never associated with a particular object; the other log_*
functions may be invoked with or without a provided object to associate the message with.
It is advised to log a message for each object that is evaluated so that the results will reflect how many objects are being manipulated or reported on.
Markdown rendering is supported for log messages.
Note
Using self.log_failure()
, in addition to recording a log message, will flag the overall job as failed, but it will not stop the execution of the job. To end a job early, you can use a Python raise
or return
as appropriate.
Accessing Request Data¶
Details of the current HTTP request (the one being made to execute the job) are available as the instance attribute self.request
. This can be used to infer, for example, the user executing the job and their client IP address:
username = self.request.user.username
ip_address = self.request.META.get('HTTP_X_FORWARDED_FOR') or \
self.request.META.get('REMOTE_ADDR')
self.log_info(f"Running as user {username} (IP: {ip_address})...")
For a complete list of available request parameters, please see the Django documentation.
Reading Data from Files¶
The Job
class provides two convenience methods for reading data from files:
load_yaml
load_json
These two methods will load data in YAML or JSON format, respectively, from files within the local path (i.e. JOBS_ROOT/
).
Running Jobs¶
Note
To run any job, a user must be assigned the extras.run_job
permission. This is achieved by assigning the user (or group) a permission on the extras > job
object and specifying the run
action in the admin UI as shown below.
Jobs and class_path
¶
It is a key concept to understand the 3 class_path
elements:
grouping_name
: which can be one oflocal
,git
, orplugin
- depending on where theJob
has been defined.module_name
: which is the Python path to the job definition file, for a plugin-provided job, this might be something likemy_plugin_name.jobs.my_job_filename
ornautobot_golden_config.jobs
and is the importable Python path name (which would not include the.py
extension, as per Python syntax standards).JobClassName
: which is the name of the class inheriting fromnautobot.extras.jobs.Job
contained in the above file.
The class_path
is often represented as a string in the format of <grouping_name>/<module_name>/<JobClassName>
, such as
local/example/MyJobWithNoVars
or plugins/nautobot_golden_config.jobs/BackupJob
. Understanding the definitions of these
elements will be important in running jobs programmatically.
Via the Web UI¶
Jobs can be run via the web UI by navigating to the job, completing any required form data (if any), and clicking the "Run Job" button.
Once a job has been run, the latest JobResult
for that job will be summarized in the job list view.
Via the API¶
To run a job via the REST API, issue a POST request to the job's endpoint /api/extras/jobs/<class_path>/run/
. You can optionally provide JSON data to set the commit
flag and/or specify any required user input data
.
Note
See above for information on constructing the class_path
for any given Job.
For example, to run a job with no user inputs and without committing any anything to the database:
curl -X POST \
-H "Authorization: Token $TOKEN" \
-H "Content-Type: application/json" \
-H "Accept: application/json; indent=4" \
http://nautobot/api/extras/jobs/local/example/MyJobWithNoVars/run/
Or to run a job that expects user inputs, and commit changes to the database:
curl -X POST \
-H "Authorization: Token $TOKEN" \
-H "Content-Type: application/json" \
-H "Accept: application/json; indent=4" \
http://nautobot/api/extras/jobs/local/example/MyJobWithVars/run/ \
--data '{"data": {"string_variable": "somevalue", "integer_variable": 123}, "commit": true}'
When providing input data, it is possible to specify complex values contained in ObjectVar
s, MultiObjectVar
s, and IPAddressVar
s.
ObjectVar
s can be specified by either using their primary key directly as the value, or as a dictionary containing a more complicated query that gets passed into the Django ORM as keyword arguments.MultiObjectVar
s can be specified as a list of primary keys.IPAddressVar
s can be provided as strings in CIDR notation.
Via the CLI¶
Jobs that do not require user input can be run from the CLI by invoking the management command:
nautobot-server runjob [--username <username>] [--commit] <class_path>
Note
See above for class_path
definitions.
Using the same example shown in the API:
nautobot-server runjob --username myusername local/example/MyJobWithNoVars
Provision of user input (data
values) via the CLI is not supported at this time.
Warning
The --username <username>
parameter can be used to specify the user that will be identified as the requester of the job. It is optional if the job will not be modifying the database, but is mandatory if you are running with --commit
, as the specified user will own any resulting database changes.
Note that nautobot-server
commands, like all management commands and other direct interactions with the Django database, are not gated by the usual Nautobot user authentication flow. It is possible to specify any existing --username
with the nautobot-server runjob
command in order to impersonate any defined user in Nautobot. Use this power wisely and be cautious who you allow to access it.
Example Jobs¶
Creating objects for a planned site¶
This job prompts the user for three variables:
- The name of the new site
- The device model (a filtered list of defined device types)
- The number of access switches to create
These variables are presented as a web form to be completed by the user. Once submitted, the job's run()
method is called to create the appropriate objects, and it returns simple CSV output to the user summarizing the created objects.
from django.utils.text import slugify
from nautobot.dcim.models import Device, DeviceRole, DeviceType, Manufacturer, Site
from nautobot.extras.models import Status
from nautobot.extras.jobs import *
class NewBranch(Job):
class Meta:
name = "New Branch"
description = "Provision a new branch site"
field_order = ['site_name', 'switch_count', 'switch_model']
site_name = StringVar(
description="Name of the new site"
)
switch_count = IntegerVar(
description="Number of access switches to create"
)
manufacturer = ObjectVar(
model=Manufacturer,
required=False
)
switch_model = ObjectVar(
description="Access switch model",
model=DeviceType,
query_params={
'manufacturer_id': '$manufacturer'
}
)
def run(self, data, commit):
STATUS_PLANNED = Status.objects.get(slug='planned')
# Create the new site
site = Site(
name=data['site_name'],
slug=slugify(data['site_name']),
status=STATUS_PLANNED,
)
site.validated_save()
self.log_success(obj=site, message="Created new site")
# Create access switches
switch_role = DeviceRole.objects.get(name='Access Switch')
for i in range(1, data['switch_count'] + 1):
switch = Device(
device_type=data['switch_model'],
name=f'{site.slug}-switch{i}',
site=site,
status=STATUS_PLANNED,
device_role=switch_role
)
switch.validated_save()
self.log_success(obj=switch, message="Created new switch")
# Generate a CSV table of new devices
output = [
'name,make,model'
]
for switch in Device.objects.filter(site=site):
attrs = [
switch.name,
switch.device_type.manufacturer.name,
switch.device_type.model
]
output.append(','.join(attrs))
return '\n'.join(output)
Device validation¶
A job to perform various validation of Device data in Nautobot. As this job does not require any user input, it does not define any variables, nor does it implement a run()
method.
from nautobot.dcim.models import ConsolePort, Device, PowerPort
from nautobot.extras.models import Status
from nautobot.extras.jobs import Job
class DeviceConnectionsReport(Job):
description = "Validate the minimum physical connections for each device"
def test_console_connection(self):
STATUS_ACTIVE = Status.objects.get(slug='active')
# Check that every console port for every active device has a connection defined.
for console_port in ConsolePort.objects.prefetch_related('device').filter(device__status=STATUS_ACTIVE):
if console_port.connected_endpoint is None:
self.log_failure(
obj=console_port.device,
message="No console connection defined for {}".format(console_port.name)
)
elif not console_port.connection_status:
self.log_warning(
obj=console_port.device,
message="Console connection for {} marked as planned".format(console_port.name)
)
else:
self.log_success(obj=console_port.device)
def test_power_connections(self):
STATUS_ACTIVE = Status.objects.get(slug='active')
# Check that every active device has at least two connected power supplies.
for device in Device.objects.filter(status=STATUS_ACTIVE):
connected_ports = 0
for power_port in PowerPort.objects.filter(device=device):
if power_port.connected_endpoint is not None:
connected_ports += 1
if not power_port.connection_status:
self.log_warning(
obj=device,
message="Power connection for {} marked as planned".format(power_port.name)
)
if connected_ports < 2:
self.log_failure(
obj=device,
message="{} connected power supplies found (2 needed)".format(connected_ports)
)
else:
self.log_success(obj=device)