Sunday, July 10, 2016

Building a monitoring solution - Parsing Palo Alto 200 - TRAFFIC (Firewall) Logs



In this post, we will leverage Splunk – which I installed previously – to build a dashboard that allows us to get a quick overview of our Palo Alto “Traffic” Logs.

One of first things to do when building a Splunk dashboard or parsing events from any log source, is to understand the data within the events. 

Below is a sample of our Palo Alto “TRAFFIC” events as seen by Splunk

Jul 8 20:52:09 192.168.1.5 Jul 8 08:49:07 1,2016/07/08 08:49:07,001606042988,TRAFFIC,end,1,2016/07/08 08:49:05,192.168.1.30,8.8.8.8,99.229.40.63,8.8.8.8,Web Traffic,,,dns,vsys1,TRUSTED_LAN_L3,INTERNET,vlan,ethernet1/1,Security Monitoring,2016/07/08 08:49:06,45165,1,31249,53,3123,53,0x400000,udp,allow,203,64,139,2,2016/07/08 08:48:36,0,any,0,4811734,0x0,192.168.0.0-192.168.255.255,United States,0,1,1


By looking at the Palo Alto documents in the reference section, you would see the position of each value. This information is helpful to contribute to successful parsing. However, it is not absolutely needed. However, I do recommend you look at it.

Now that we have our logs, let’s build out all of the fields that are of importance to us using the "rex" command.
* | rex field=_raw ".*,TRAFFIC,.*?,\d{1,5},.*?,(?<fw_srcIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),(?<fw_dstIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),.*?,.*?,(?<fw_rule>.*?),.*?,.*?,(?<fw_application>.*?),.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,(?<fw_srcPort>\w+),(?<fw_dstPort>\w+),.*?,.*?,.*?,(?<fw_protocol>.*?),(?<fw_action>\w+),(?<fw_total_bytes>\d+),(?<fw_bytes_sent>\d+),(?<fw_bytes_received>\d+),(?<fw_total_packets>\d+),.*?,(?<fw_duration>\d+),.*?,.*?,.*?,.*?,.*?,(?<fw_geolocation>.*?)," | stats count by fw_srcIP, fw_dstIP, fw_rule, fw_application, fw_srcPort, fw_dstPort, fw_protocol, fw_action, fw_total_bytes, fw_bytes_sent, fw_bytes_received, fw_total_packets, fw_duration, fw_geolocation | sort count | reverse

To extract the fields above, the following “rex” command was used.
* | rex field=_raw ".*,TRAFFIC,.*?,\d{1,5},.*?,(?<fw_srcIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),(?<fw_dstIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),.*?,.*?,(?<fw_rule>.*?),.*?,.*?,(?<fw_application>.*?),.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,(?<fw_srcPort>\w+),(?<fw_dstPort>\w+),.*?,.*?,.*?,(?<fw_protocol>.*?),(?<fw_action>\w+),(?<fw_total_bytes>\d+),(?<fw_bytes_sent>\d+),(?<fw_bytes_received>\d+),(?<fw_total_packets>\d+),.*?,(?<fw_duration>\d+),.*?,.*?,.*?,.*?,.*?,(?<fw_geolocation>.*?)," | stats count by fw_srcIP, fw_dstIP, fw_rule, fw_application, fw_srcPort, fw_dstPort, fw_protocol, fw_action, fw_total_bytes, fw_bytes_sent, fw_bytes_received, fw_total_packets, fw_duration, fw_geolocation | sort count | reverse



Below shows an examples of all the important fields build out using the search query above.

By building out the fields above we can now leverage this information to pick specific fields to gain greater insights.

Looking at all traffic sourced from the local network let's look at the geolocation which were seen.

* | rex field=_raw ".*,TRAFFIC,.*?,\d{1,5},.*?,(?<fw_srcIP>192.168\.\d{1,3}\.\d{1,3}),(?<fw_dstIP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}),.*?,.*?,(?<fw_rule>.*?),.*?,.*?,(?<fw_application>.*?),.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,(?<fw_srcPort>\w+),(?<fw_dstPort>\w+),.*?,.*?,.*?,(?<fw_protocol>.*?),(?<fw_action>\w+),(?<fw_total_bytes>\d+),(?<fw_bytes_sent>\d+),(?<fw_bytes_received>\d+),(?<fw_total_packets>\d+),.*?,(?<fw_duration>\d+),.*?,.*?,.*?,.*?,.*?,(?<fw_geolocation>.*?)," | stats count by fw_geolocation | sort count | reverse



Below we see the results of this command above.




No comments:

Post a Comment