21 Nov Using the JAMF Pro API and Python to detect duplicated attributes
No matter where you go, there they are…
We’ve been attempting to develop a unified inventory solution for all of our computers using JAMF Pro. While building out our database, we conducted a large number of tests and mistakes were made while adding machines. These errors resulted in duplicate and incomplete entries scattered among the legitimate machines. Compounding this was the discovery that the Window’s computers we were adding had motherboards with identical information (UUIDs and serial numbers, for example) making it difficult or impossible for the JAMF database to distinguish one machine from another.
Unfortunately, the JAMF’s Windows client tools don’t offer the ability to define an alternate UUID. If you’ve experienced this issue in your deployments, we invite you to visit and vote for our feature request here: Add flag to jamf.exe to specify UUID of my choosing during enrollment. This issue required us to use the JAMF API and script a solution to add these machines with software-defined UUIDs.
During our cleanup of the database we discovered that while the machine may have been successfully added to the database, it may have additional database fields that duplicated other machines information and cause HTTP 409 errors when attempting to modify their records using the API. A 409 error denotes a resource conflict, usually caused by multiple machines sharing an identical value for a field which should hold a unique value. At this time the JAMF Pro server does not offer much in its UI to track down these duplicates. But by combining the JAMF Pro API and the power of Python, we can quickly find these offending entries.
This article builds on topics covered in Fetching Data From the JSS using API calls & Python. We have additional related posts coming soon that look at searching and writing data to the JAMF Pro database using Python.
Python’s dictionaries
This script makes good use of Python’s dictionaries or associative arrays. A dictionary is a list of key and value pairs. Each key-value entry has a single key. This singular key can be any of a number of different data types. In the image below, the keys are book title, or strings. They could just as easily be phone numbers or dates, for example.
The value portion of the pair can be any kind and quantity of data. In the case of the example image, authors. While you normally wouldn’t change the content of the key, changing the value field is a very powerful feature.
This script stores the attribute (computer name, serial number, etc) as the key, and the value is a list of the clients that share that value. For example, if we have a machine with a computer name “MacBook Pro”, we’ll store the name as the key and the first client ID is stored as the value. The next time we find a machine named “MacBook Pro”, we’ll add it’s ID to the list of IDs stored in the value field.
Breaking down the code
In broad strokes here is how the script works:
- Ask the JAMF server for the list of computers and their identifiers.
- For each client, ask the JAMF server for specific pieces of information.
- Define a dictionary for each of those pieces.
- For each client and each dictionary do the following:
- Does this piece of information exist in the dictionary?
- If it does, add the client identifier to the value for that piece.
- If it doesn’t, create a new pair in the dictionary with the piece as the key and the client identifier as the value.
- Print out a summary for each dictionary by checking each key for values with more than one client ID.
Let’s take a closer look at the major sections of the overall script.
The computer object
This is an object that we can pass back and forth between functions. It stores the variables that we will compare later. The __init__ section is called when the object is created and sets the initial state of the variables. The print_myself function is not used as in the script and is left for future usage.
class Computer:
# simple reusable object to define a Computer
def __init__(self, id):
self.identifier = id
self.computername = None
self.serialnumber = None
self.udid = None
self.mac_address = None
self.alt_mac_address = None
def print_myself(self):
print("id: %s" % self.identifier)
print("hostname: %s" % self.computername)
print("serial: %s" % self.serialnumber)
print("udid: %s" % self.udid)
print("MAC: %s" % self.mac_address)
print("alt MAC: %s" % self.alt_mac_address)
Grabbing the clients
This function asks the JAMF Pro API to return the client identifiers and computer names of all of the machines stored in the JAMF database. This information is then reformatted into a data structure we can use later to ask the JAMF server about specific machines.
def build_client_list(jss_url, jss_user, jss_pwrd):
# query jss for list of id's and computers names
request = urllib2.Request(jss_url)
request.add_header('Accept', 'application/json')
request.add_header('Authorization', 'Basic ' + base64.b64encode(jss_user + ':' + jss_pwrd))
response = urllib2.urlopen(request)
response_json = json.loads(response.read())
clients_raw = response_json['computers']
id_list = []
for client in clients_raw:
id_list.append( (client['id'], client['name']) )
return id_list
Inspecting individual clients
This function requests more information about individual machines. The API call we are using returns the least amount of extra data beyond what we need. Much of the function involves handling potential errors that could occur when interacting with the JAMF Pro API and smoothly handling those events. The last few lines of the function store the actual values we’re interested in.
def query_jss(this_client, jss_url, jss_user, jss_pwrd):
# query jss for individual client
try:
url = jss_url + '/id/' + str(this_client.identifier) + "/subset/general&location"
request = urllib2.Request(url)
request.add_header('Accept', 'application/json')
request.add_header('Authorization', 'Basic ' + base64.b64encode(jss_user + ':' + jss_pwrd))
response = urllib2.urlopen(request)
response_json = json.loads(response.read())
if response.code is not 200:
print("%i returned." % response.code)
return
# handle errors
except urllib2.HTTPError, error:
contents = error.read()
if error.code == 400:
print("HTTP code %i: %s " % (error.code, "Request error."))
return
elif error.code == 401:
print("HTTP code %i: %s " % (error.code, "Authorization error."))
return
elif error.code == 403:
print("HTTP code %i: %s " % (error.code, "Permissions error."))
return
elif error.code == 404:
print("HTTP code %i: %s " % (error.code, "Resource not found."))
return
elif error.code == 409:
print("HTTP code %i: %s " % (error.code, "Resource conflict."))
return
else:
print("HTTP code %i: %s " % (error.code, "Generic error."))
return
except urllib2.URLError, error:
print("Error contacting JSS.")
return
except:
print("Error querying JSS.")
return
# assign values
this_client.serialnumber = (response_json['computer']['general']['serial_number'])
this_client.udid = (response_json['computer']['general']['udid'])
this_client.mac_address = (response_json['computer']['general']['mac_address'])
this_client.alt_mac_address = (response_json['computer']['general']['alt_mac_address'])
this_client.computername = (response_json['computer']['general']['name'])
Building dictionaries and handling collisions
This section is the heart of the script. We request the list of machines and specific individual info, and then check for collisions. Each of the separate try/except blocks checks for a different type of attribute. Try/excepts work in the following way: if an error occurs in the try block, execution of the script moves to the associated except block. In our script, if we attempt to access a preexisting attribute key, we append it to the existing value list, otherwise the key doesn’t exist and we encounter an error switch to the except block and create a new key-value pair.
# individual dictionaries to store attributes
serial_dict = {}
udid_dict = {}
mac_dict = {}
amac_dict = {}
name_dict = {}
# build the list of clients
jss_clients = build_client_list(jss_url, jss_user, jss_pwrd)
# check each client
for client in jss_clients:
print("\n >>> Processing %s %s" % (client[0], client[1]))
# instantiate computer object
this_client = Computer(client[0])
# request specific ifno for client and populate object
query_jss(this_client, jss_url, jss_user, jss_pwrd)
# check serial number
try:
if serial_dict[this_client.serialnumber]:
serial_dict[this_client.serialnumber].append(this_client.identifier)
except:
serial_dict[this_client.serialnumber] = [this_client.identifier]
# check UDID
try:
if udid_dict[this_client.udid]:
udid_dict[this_client.udid].append(this_client.identifier)
except:
udid_dict[this_client.udid] = [this_client.identifier]
# check MAC
try:
if mac_dict[this_client.mac_address]:
mac_dict[this_client.mac_address].append(this_client.identifier)
except:
mac_dict[this_client.mac_address] = [this_client.identifier]
# check alternate MAC
try:
if amac_dict[this_client.alt_mac_address]:
amac_dict[this_client.alt_mac_address].append(this_client.identifier)
except:
amac_dict[this_client.alt_mac_address] = [this_client.identifier]
# check computer name
try:
if name_dict[this_client.computername]:
name_dict[this_client.computername].append(this_client.identifier)
except:
name_dict[this_client.computername] = [this_client.identifier]
Displaying the results
Here we display the results of the script. This is done by checking each dictionary for items with more than one identifier in its value and printing those items.
print("\nSummary")
print("Total clients: %i\n" % len(jss_clients))
print("Serial Collisions:")
for key, value in serial_dict.items():
if len(value) > 1:
print("[%i] %r %r" % (len(value), key, value))
print("\nUDID Collisions:")
for key, value in udid_dict.items():
if len(value) > 1:
print("[%i] %r %r" % (len(value), key, value))
print("\nMAC Collisions:")
for key, value in mac_dict.items():
if len(value) > 1:
print("[%i] %r %r" % (len(value), key, value))
print("\nAlt MAC Collisions:")
for key, value in amac_dict.items():
if len(value) > 1:
print("[%i] %r %r" % (len(value), key, value))
print("\nName Collisions:")
for key, value in name_dict.items():
if len(value) > 1:
print("[%i] %r %r" % (len(value), key, value)
Output
Here is a sample of the output you can expect from the full script. I’ve broken down each section below. I’ve also edited the content with ellipsis’ (‘…’) for brevity. Using the client identifiers you can begin to address the impact they have on your deployment and actions that need to be taken to correct the issue.
1 – Scanning clients
Here the script notes each client as it is parsed.
2 – Summary
Shows the total number of clients in the JAMF database.
3 – Serial number collisions
This sections shows where duplicate serial numbers have been found. The output is read as follows: number of clients, the matching value, and a list of the JAMF client IDs sharing the same value. The u’value’ format means the script is printing a raw, unformatted unicode string, the quote marks denote the beginning and end of the string. This will allow use to see the difference between an empty string, like this one, and a string consisting of a single space.
Take a close look at the line showing 336 duplicates:
[336] u” [1326, 301, 378, 1397, 783, 791, 796, 798, …]
This line is actually showing that there are 336 clients with no serial number at all.
The following line shows 3 clients with Not Available as the serial number.
The 4 following lines with Apple-like serial numbers are likely indicating duplicated machines.
4 – UDID collisions
The UDID is the used by the JAMF server as the primary means of differentiating between clients. It will not allow multiple identical values.
5 – MAC collisions
Media access controller (MAC) address collisions. This is the unique address for the primary ethernet port.
6 – Alternate MAC collisions
Alternate (secondary) MAC address collisions, if the machine has multiple ethernet ports.
7 – Computer name collisions
Duplicated computer names will be listed here.
The complete script
To make the script operable you will need to provide the address the your JAMF Pro server.
#!/usr/bin/python
import urllib2
import xml.etree.cElementTree as ET
import base64
import json
import getpass
class Computer:
# simple reusable object to define a Computer
def __init__(self, id):
self.identifier = id
self.computername = None
self.serialnumber = None
self.udid = None
self.mac_address = None
self.alt_mac_address = None
def print_myself(self):
print("id: %i" % self.identifier)
print("hostname: %s" % self.computername)
print("serial: %s" % self.serialnumber)
print("udid: %s" % self.udid)
print("MAC: %s" % self.mac_address)
print("alt MAC: %s" % self.alt_mac_address)
def build_client_list(jss_url, jss_user, jss_pwrd):
# query jss for list of id's and computers names
request = urllib2.Request(jss_url)
request.add_header('Accept', 'application/json')
request.add_header('Authorization', 'Basic ' + base64.b64encode(jss_user + ':' + jss_pwrd))
response = urllib2.urlopen(request)
response_json = json.loads(response.read())
clients_raw = response_json['computers']
id_list = []
for client in clients_raw:
id_list.append( (client['id'], client['name']) )
return id_list
def query_jss(this_client, jss_url, jss_user, jss_pwrd):
# query jss for individual client
try:
url = jss_url + '/id/' + str(this_client.identifier) + "/subset/general&location"
request = urllib2.Request(url)
request.add_header('Accept', 'application/json')
request.add_header('Authorization', 'Basic ' + base64.b64encode(jss_user + ':' + jss_pwrd))
response = urllib2.urlopen(request)
response_json = json.loads(response.read())
if response.code is not 200:
print("%i returned." % response.code)
return
# handle errors
except urllib2.HTTPError, error:
contents = error.read()
if error.code == 400:
print("HTTP code %i: %s " % (error.code, "Request error."))
return
elif error.code == 401:
print("HTTP code %i: %s " % (error.code, "Authorization error."))
return
elif error.code == 403:
print("HTTP code %i: %s " % (error.code, "Permissions error."))
return
elif error.code == 404:
print("HTTP code %i: %s " % (error.code, "Resource not found."))
return
elif error.code == 409:
print("HTTP code %i: %s " % (error.code, "Resource conflict."))
return
else:
print("HTTP code %i: %s " % (error.code, "Generic error."))
return
except urllib2.URLError, error:
print("Error contacting JSS.")
return
except:
print("Error querying JSS.")
return
# assign values
this_client.serialnumber = (response_json['computer']['general']['serial_number'])
this_client.udid = (response_json['computer']['general']['udid'])
this_client.mac_address = (response_json['computer']['general']['mac_address'])
this_client.alt_mac_address = (response_json['computer']['general']['alt_mac_address'])
this_client.computername = (response_json['computer']['general']['name'])
def main():
# acquire the account info for a user with API access privileges
jss_user = raw_input("JSS username? ")
jss_pwrd = getpass.getpass("JSS password?")
# the root API url to your JAMF pro server
jss_url = 'https://your_jamfpro_server:8443/JSSResource/computers'
# individual dictionaries to store attributes
serial_dict = {}
udid_dict = {}
mac_dict = {}
amac_dict = {}
name_dict = {}
jss_clients = build_client_list(jss_url, jss_user, jss_pwrd)
for client in jss_clients:
print("\n >>> Processing %s %s" % (client[0], client[1]))
this_client = Computer(client[0])
query_jss(this_client, jss_url, jss_user, jss_pwrd)
try:
if serial_dict[this_client.serialnumber]:
serial_dict[this_client.serialnumber].append(this_client.identifier)
except:
serial_dict[this_client.serialnumber] = [this_client.identifier]
try:
if udid_dict[this_client.udid]:
udid_dict[this_client.udid].append(this_client.identifier)
except:
udid_dict[this_client.udid] = [this_client.identifier]
try:
if mac_dict[this_client.mac_address]:
mac_dict[this_client.mac_address].append(this_client.identifier)
except:
mac_dict[this_client.mac_address] = [this_client.identifier]
try:
if amac_dict[this_client.alt_mac_address]:
amac_dict[this_client.alt_mac_address].append(this_client.identifier)
except:
amac_dict[this_client.alt_mac_address] = [this_client.identifier]
try:
if name_dict[this_client.computername]:
name_dict[this_client.computername].append(this_client.identifier)
except:
name_dict[this_client.computername] = [this_client.identifier]
print("\nSummary")
print("Total clients: %i\n" % len(jss_clients))
print("Serial Collisions:")
for key, value in serial_dict.items():
if len(value) > 1:
print("[%i] %r %r" % (len(value), key, value))
print("\nUDID Collisions:")
for key, value in udid_dict.items():
if len(value) > 1:
print("[%i] %r %r" % (len(value), key, value))
print("\nMAC Collisions:")
for key, value in mac_dict.items():
if len(value) > 1:
print("[%i] %r %r" % (len(value), key, value))
print("\nAlt MAC Collisions:")
for key, value in amac_dict.items():
if len(value) > 1:
print("[%i] %r %r" % (len(value), key, value))
print("\nName Collisions:")
for key, value in name_dict.items():
if len(value) > 1:
print("[%i] %r %r" % (len(value), key, value))
if __name__ == '__main__':
main()
Additional resources
Here are links to sites offering more in-depth discussions of Python’s dictionaries.
No Comments