Thursday, July 14, 2016

Writing a Palo Alto Firewall REST API Client in Python

While working on code to configure my PaloAlto instances automatically in Amazon AWS, I needed to write functions that would interact with the Palo Alto gateway (add/remove rules, create objects, commit changes, etc.). Palo Alto makes available a number of documents available to help with this, but I didn’t find any one source that would explain the process completely of how to send commands, interpret return codes and then parse outputs returned, and so I wanted to document this process.
Palo Alto gateways have a REST API available which allows you to send commands over HTTPS (or HTTP), and then returns the output in XML format making it easy to parse through and extract the information.


REST Request and Response Structure


The structure for the API requests is:
http(s)://hostname/api/?request-parameters-values
  • hostname: hostname or IP address of the Palo Alto gateway.
  • request: Can be one of 9 different request types, we will mainly use: keygen, config, op, and commit. There are others that allow you to export/import configuration or logs and other information. The request has to be specified with the 'type' paramater, for example: 'type=keygen'.
  • parameters: Each request type has different parameters available to it. For example, the 'config' request has the 'action' parameter which can be set to values such as edit,set,delete,etc. in the format (action=set,action=edit,action=delete,etc.). The op (operation) request has commands associated with it such as save, show, request, etc. 
  • values: Some parameter would require values that need to be provided, and these go here. For example, adding a new rule would require specifying parameters such as source, destination, service,etc.
Once a request is sent, the gateway will send back a response which consists of two parts: 
  1. Response status: This includes a ‘status’ such as ‘success’ or ‘error’, and a ‘code’ which is a numerical value. For example error codes 19 and 20 mean success, and other codes would specify different error reasons (For example, internal errors, invalid objects, etc.).
  2. Results. Certain commands would require a response from the gateway such as listing specific rules or objects, and these would be shown under the results section. 
Both these parts are provided in an XML tree format:

<response status="success" code="20">
<msg>command succeeded</msg>
</response>

For more details on the REST API structure, a list of all different requests and responses, refer to the PAN-OS XML API Usage Guide.

Determining Command Syntax


The XML API guide from Palo Alto is very helpful in finding out the request types, parameters available, as well as error codes and notes on the response structures. However, there are some things that might not be included in the guide, such as the structure of the values field that needs to be passed for certain commands. The easiest way to find that out is to enable debugging in the CLI, and then execute the command that would achieve the result you are looking for. Palo Alto will then show you the syntax it passed, and you can use that as a model. 
For example, to get the syntax for adding a security rule:
  1. Turn on debugging “debug cli on”
  2. Go into configure mode by running “configure”
  3. Run the command to add a rule:
set rulebase security rules "new" from untrust to trust source 1.1.1.1 destination 2.2.2.2 service tcp_1234 application any profile-setting group strict_spg

      4. From the output, the parts highlighted in red are what you would need to carry:

<request cmd="set" obj="/config/devices/entry[@name='localhost.localdomain']/vsys/entry[@name='vsys1']/rulebase/security/rules/entry[@name='new']" cookie="2476837088324585"><from><member>untrust</member></from><to><member>trust</member></to><source><member>1.1.1.1</member></source><destination><member>2.2.2.2</member></destination><service><member>tcp_1234</member></service><application><member>any</member></application><profile-setting><group><member>strict_spg</member></group></profile-setting></request>

     5. The XML API guide will also provide guidance on the values it is expecting. For the “set” action under the “config” type, it tells you that it expects two parts for values: xpath and element.

The full request would be as follows:
https:///api/?type=config&action=set&Key=&xpath="/config/devices/entry[@name='localhost.localdomain']/vsys/entry[@name='vsys1']/rulebase/security/rules/entry[@name='sg-b0f7ddcb']&element=trustuntrustany172.20.200.225anytcp_465allowstrict_spg

Authentication


To be able to send requests to the gateway, you need an access key to be included in each request. Once you have the key, the key should be included in every request and is listed right after the request, for example:

http(s)://hostname/api/?type=config&action=set&Key=KEYVALUE&cmd=


To generate a key, send the following request to the gateway:

http(s)://hostname/api/?type=keygen&user=username&password=password 

(Where username and password are the credentials of an already configured administrator account on the gateway).

The response will contain the access key. Sample response:
<response status="success"> 
 <result> 
  <key>gJlQWE56987nBxIqsdflkjsdf234ASo2BgzEA9UOnlZBhU</key> 
 </result> 
</response>

Sending a request in Python


With this information in mind, we can now turn to Python to write the code that will send requests to Palo Alto gateways and then interpret the responses. Let’s walk through the example of writing a function which will let us add security rules on a Palo Alto gateway.

Constructing a request

The request we will be sending will be as follows (which covers all parts in the request syntax mentioned above):

url = "https://"+pa_ip+cmd+"Key="+pa_key+"&"+urllib.urlencode(parameters)

Each of the items highlighted in red are variables that need to be filled in to complete the request: 
  • pa_ip: holds the IP address (or hostname) of the gateway.
  • cmd: type of request and parameter associated with the function we are trying to perform. In this case, we are writing a function to add security rules to the gateway, and so the type is config and action is set, so:
cmd = "/api/?type=config&action=set&"
  • pa_key: access key we obtained for the gateway.
  • urllib.urlencode(parameters): there are two parts here to consider:
    • parameters: holds the last part of the request which are the values required for the command or request we are sending to the gateway. In case of adding a rule, there are two values that need to be sent:
      1. xpath: path to the item we are modifying on the Palo Alto gateway.
      2. element: contains the values of the different options in the rule (source/destination IP, action, service, etc.).
    • urllib.urlencode: since we are sending these requests over HTTP, some characters need to be encoded so that the receiving end can interpret them properly. Characters such as space cannot be sent as is but have to be changed to a supported format. urlencode function from the urllib library allows us to do this, and requires that values passed to it are passed in a python dictionary format. 
parameters = {'xpath': "/config/devices/entry[@name='localhost.localdomain']/vsys/entry[@name='vsys1']/rulebase/security/rules/entry[@name='sg-b0f7ddcb']", 'element': '<to><member>trust</member></to><from><member>untrust</member></from><source><member>any</member></source><destination><member>172.20.200.225</member></destination><application><member>any</member></application><service><member>tcp_465</member></service><action>allow</action><profile-setting><group><member>strict_spg</member></group></profile-setting>'}


Sending the request


Once we have constructed the request, we can send it using the urlopen function from urllib2 library:

response = urllib2.urlopen(url)

In case you are connecting to a gateway with an untrusted SSL certificate, you will need to tell the urlopen command to ignore the SSL certificate check, otherwise the command will throw an error. To do this you can use the ssl library in python to create a context that ignores the certificate check and then pass it to the urlopen function:

ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
response = urllib2.urlopen(url, context=ctx)

Parsing the Response in Python


In the previous section, we sent a request using the urlopen function, and specified the response variable to hold the return values from urllopen. ‘response’ in this case would hold the values returned from the Palo Alto gateway as well as details on the response on the HTTP level. So we can use response.code for example to print of the HTTP status of our request (200 for Okay, and almost everything else would indicate errors). 
If the HTTP status returned is 200, then we can proceed to analyze the response from the Palo Alto gateway. To do this, we need to read this as XML. Response.read() allows us to read the contents of the response from Palo Alto, and the fromstring function allows us to parse this in XML format (Need to import xml.etree.ElementTree for this function, and I have used the syntax ‘import xml.etree.ElementTree as ET’ to make it easier to reference):

contents= ET.fromstring(response.read())

contents now will hold the Palo Alto response, in an XML tree format which we can parse easily. ‘contents’ on its own access the first level, ‘contents[0]’ access the first member on the second level, contents[0][0] access the first member in the second level of the first member on the first level, and so on.

On each level, there are three values we can access: tag, attrib, and text. Consider the example of querying the Palo Alto gateway to list all configured service objects: 

Request:

https:///api/?type=config&action=get&Key=&xpath=/config/devices/entry[@name='localhost.localdomain']/vsys/entry[@name='vsys1']/service

Output:

<response status="success" code="19">
 <result total-count="1" count="1">
  <service admin="admin" dirtyId="14" time="2016/07/12 19:12:50">
   <entry name="tcp_81">
    <protocol>
     <tcp>
      <port>81</port>
     </tcp>
    </protocol>
   </entry>
   <entry name="tcp_8000">
    <protocol>
     <tcp>
      <source-port>1-65535</source-port>
      <port>8000</port>
     </tcp>
    </protocol>
   </entry>
   <entry name="tcp_465" admin="admin" dirtyId="12" time="2016/07/12 19:01:49">
    <protocol admin="admin" dirtyId="12" time="2016/07/12 19:01:49">
     <tcp admin="admin" dirtyId="12" time="2016/07/12 19:01:49">
      <port admin="admin" dirtyId="12" time="2016/07/12 19:01:49">465</port>
     </tcp>
    </protocol>
   </entry>
  </service>
 </result>
</response>

Code: 

def paloalto_service_find(pa_ip,pa_key,protocol,port):
 # Find if there are service objects that match a certain port and protocol type
 # Input: Palo Alto gateway IP, Palo Alto Access Key, IP protocol type, and port number
 # Output: Returns service object name if found or "" if there are no matches
 
 ctx = ssl.create_default_context()
 ctx.check_hostname = False
 ctx.verify_mode = ssl.CERT_NONE

 cmd = "/api/?type=config&action=get&"
 parameters = {'xpath':"/config/devices/entry[@name=\'localhost.localdomain\']/vsys/entry[@name=\'vsys1\']/service"}
 url = "https://"+pa_ip+cmd+"Key="+pa_key+"&"+urllib.urlencode(parameters)

 response = urllib2.urlopen(url, context=ctx)
 contents= ET.fromstring(response.read())

 result = ""

 for i in contents[0][0]:
  if i[0][0].tag == protocol:
   for j in i[0][0]:
    if j.tag == 'port' and j.text == port:
     result = i.attrib['name']

 return result

the response is first formatted in XML so that it can be parsed, and saved in variable 'contents'. Afterwards, we iterate through all the entries two levels down using the variable 'i', and subsequently two more levels down from i with the variable j. Throughout this process, we are comparing the protocol and port values we are reading with the port and protocol variables provided to the function. The first of these entries would be:

<entry name="tcp_81">
 <protocol>
  <tcp>
   <port>81</port>
  </tcp>
 </protocol>
</entry>

In this case, the tag is ‘entry’, attrib is a dictionary {'name':'tcp_81'}, so to access the value of 'name' we can do i. If we wanted to go one level below, we would have to reference i[0], and then for example i[0].tag would return ‘protocol’. A level below that would be i[0][0], and that would give us access to <tcp>, and so on.


Download


You can download the code from my github page.
To use it, you only need to import the paloalto.py in your code and then call the functions. The functions will continue to be updated as I work to add more functionality. 

13 comments:

  1. Hi Nik
    Can you help me to write code to import bulk of object groups using excel or csv file.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. Unfortunately I have moved on to a different position and I don't work with Palo Alto devices as much or easily have access.
      However wIth some notepad/excel work you should be able to create the commands you need to import these objects using the CLI (Check out the following link https://live.paloaltonetworks.com/t5/Configuration-Articles/How-to-Import-and-Export-Address-and-Address-Objects/ta-p/56124).

      Hope this helps

      Delete
  2. Bro, can you please write a program for pulling serial number, version, and hostnames of the firewalls connected to panorama and post it to a CSV File??

    Many thanks in advance

    Regards
    Sridhar Modali

    ReplyDelete
  3. Bro, can you please write a program for pulling serial number, version, and hostnames of the firewalls connected to panorama and post it to a CSV File??

    Many thanks in advance

    Regards
    Sridhar Modali

    ReplyDelete
    Replies
    1. Hey Sridhar,
      I think I can get that for you. Will work with a buddy of mine who is a guru with Palo Alto and scripting. Look out for a future post on this topic.

      Stay safe!

      Delete
    2. Brilliant!! Highly appreciate your reply... Waiting for your post

      Thanks again!!

      Delete
    3. Hey Sridhar,
      Just a heads-up this post is ready and should be posted this weekend.

      Delete
  4. Perfect example, you helped me to configure the decryptrules. Kudos!

    ReplyDelete
  5. Hi, if i want to make a (Https) GET request with a self signed certificate which created from palo-alto device certificate how it would be. If you can give a example with requests lib also it will be help full.

    ReplyDelete