Security Incidents
Metrics Parser

Platform

Python

Year

Autumn 2022

Metrics. If you bother with tracking Security Incidents and Alerts, you need metrics. At the end of the month, you need to know exactly:

What happened?
What *exactly* happened... Was it bad? was it very bad?
What *exactly* happened... Was it a DDOS? Was it a malware?
Was our response adequate? Timely?
Compared to the previous months, what are the trends?
Say, for the past year?

And these are important questions that require you to look at your dataset and extract the answers.
At the beginning of the month, every month until the end of cybercrime. Fortunately, you'll have most of this data or can calculate it. It just takes precious time which we don't have. Another story of how Python can come to the rescue and return to us the most precious of commodities...

Prelude

10/10/2022

My first approach was to employ the out-of-the-box reporting features provided by the IT Service Management solution: Service-Now. These proved more or less adequate for basic queries like: "How many incidents occurred in the last month and of what type?" It allows further breakdown into the impact of the incident (low, moderate, ...) as well as control over the representation. It was, unfortunately, lacking in enterprise-grade reporting like also offering the data as well... maybe a combined export of the visualisation along with the data in seperate tabs (e.g. by month) in a spreadsheet format.

The latter could also be obtained from Service-Now and quite precisely so using custom filters, but only from the module itself used to track the incidents. Which already requires a different set of access privileges and would still mean a manual operation to be commenced each month.

So automation seemed the best solution here, while still leveraging the Service-Now filter feature. The anologies with the SIEM parser (see LINK) were definetly present here. Just like in that case, I (can) dispose over a generated CSV export having all the data. And what is more, being able to access the webclient to the ITSM solution, enables me to have this CSV generated on-demand. I would have LOVED to be able to address the solution with an API, alas, i was not so lucky due to the customer setup. But, this is just an extension on the chase, it will not prevent the catch ;-) (but i'll get to this).

Just like with the SIEM tool, a model can be constructed on which questions can be posed. It becomes a matter than to export the model and the answers to the questions. Without too much detail, i needed at least 3 different forms of export.

The first export was for auditors, tailoring to very specific (and complex) questions in a certain answering format. The focus here was to get accurate and precise data over various timeframes. Definetly not befitting standardised reporting expecting weeks, months, years, etc...
The second export was for internal use, it mostly focuses on key performance indicators and workload. This was complicated because of a transition in ways these KPI's were calculated and i needed to support both ways for a transition period.
The third export, the most verbose, was to report to the hierarchy as well as controlling functions on the security operations teams. The data here needed to be complete, but also be represented in a concise way... both tabular as well as using graphs.

So the problem boils down to 3 things:

A) having an intelligent datamodel that questions can be asked off...
B) keeping the formatting logic seperate, regardless of what is needing to be exported, the datamodel must not require alterations.

C) because i don't have an API, i need to get the data in real time. The solution that popped into my mind was test automation.... but then not to actually test the application, but to get the data out and into my tool.

A is addressed by Python Core programming. I need to read in the data and use object oriented coding to make the necessary calculations.

B is addressed by PythonPyxl, a library set that allows data to be written in spreadsheet formats. I require only to iterate my objects and foresee the literals & formatting

C is addressed by Selenium, a python library for test automation.

I quickly realised that although this *might* be programmed as a console like the arcsight parser, but it would not be befitting my use cases. The arcsight parser was a clear 1 CSV goes in and 1 parsed extract comes out, along with a few tweaks and corrections. And to be used by 1 person only.

This tool also had 1 CSV that goes in, but depending on what is needed, different exports were needed. I also foresee additional expansions that may need to be added in the future, making this a more modular type of coding. Also, multiple people would need to use this, each getting the data they specifically need. And then there are the "flags" to consider, some may want only data of the same year, others may want to the full bag, and even others may be interested in just the actual KPI data.

This means, to provide a user-friendly tool while still offering these different features and bells in an overview, i need a GUI. This brings me to problem/aspect D: a GUI, which can be addressed with Python Tkinter.

Selenium Bridge: Getting the data

Using a script to get data from an application? Thàt has "Application Programming Interface" written all over it. As mentioned earlier, it is not an option because the environment doesn't allow it. Yet the lure of always having the fresh data, in real time, from the main repository is too strong to ignore. As such, i considered another way to get this (rather than doing manual exports). I figured that test automation might be a way to deal with this, even if not testing the webclient, but leveraging it to make simple & repeatable steps to export the data I need in a handsoff manner.

Selenium is an opensource python library that is ideal for this task. It works by exposing API's on a webdriver for your native browsers.

You can read more about it here: https://selenium-python.readthedocs.io/

And from this page:
"Selenium Python bindings provides a simple API to write functional/acceptance tests using Selenium WebDriver. Through Selenium Python API you can access all functionalities of Selenium WebDriver in an intuitive way. Selenium Python bindings provide a convenient API to access Selenium WebDrivers like Firefox, Ie, Chrome, Remote etc. The current supported Python versions are 3.5 and above."

On most sites, you would identify hookpoints on a website that you interact with... like a input field or a hyperlink or a button or a form. And you would use XPATH to locate these preferably, or base yourself of class or css element identifiers. Unfortunately, the Service-Now webapplication is highly dynamic, showing you content depending on access control, activated modules, customizations, .... and what is worse, i noticed that on every instantiation of this webclient, the ID tags on the elements differ every time! An otherwise ideal hook point.

To solve this, i identified specific calls to the web application that i know would always give the same response with the focus on the same element for everybody. Because from this point i can use Selenium to provide key commands like "Tab" or "down, left, right, enter, ... .

This does imply that if the websites changes design, like a reordering of elements, the selenium code would have to be redesigned. But this would have been the case with hard webhooks on XPATH as well so... no big change there.

Tkinter GUI

GUI implementations are typically based on frames and putting objects therein. While there a few GUI frameworks, Tkinter is a well-known and used for this program. It works very similar to constructing GUI's in Java using widgets (buttons, checkboxes, labels, textfields, ...)

I am in no way versed in GUI development, so i tried to follow some basic design principles:

A menu is used to show the main features the tool is offering, enabling or disabling menu options depending on the state of the workflow (for example: you cannot unload data if none is loaded).
A label field is used as "system message" whose aim is to guide the user to the next steps to take or to inform the user in case errors have happened (for example, a notification of invalidity if a random file is uploaded.
There is a section with checkboxes that implement flags, allowing the user to influence the behaviour of the application.
There is a section giving various meta-data (like how many rows and columns had been read in, if the expected data is present, the amount of months and years that has been spanned, etc...
And a section that shows the calculated data on a monthly basis

(*: Image redacted. Numbers are altered. Reference to Client removed)

Tkinter is very easy to use to build exactly this, however, while i'm putting all the data underneath each other, a fancier approach would be to use a tabbed viewpanel... but this would be overkill as the current presentation is adequate for it's purpose.
There is some care required, though, because the user selection (like determining which KPI calculation type to use, or to limit the data to e.g. the current year) might require the GUI to be rebuild. There are a few ways to handle this, but in my code i used to put the GUI code in a function that is called when state-changes have happened.

Python Core

This the bread and butter of the code. It works by reading in a CSV file that contains all data we need to calculate the metrics we are looking for. Each line in this CSV file is a ticket; so the CSV file is read in line by line (and therefore ticket by ticket). This will form an object on which methods can be invoked. Methods that allow sorting the data (chronological order is not guaranteed), ordering the data by month and calculate how many tickets had been opened by category and by severity and using this as properties for that month. Because we consider tickets and order them by month they are created, since there closure time is recorded in the ticket we can know for any given month in what month it's tickets were closed.

Something we can't easily know, is the overview of tickets closed in a given month. Using code, you can find this. A nice example of how code can provide a much richer view than what you get by tabular data in spreadsheets alone (unless you also provide another set of tabs to show this).

Because this data is in memory, and complete, starting the exports means handpicking the data we need to export.

OpenPyxl Exports

And that write out brings us to OpenPyxl. While the tool shows the data in a tabular way in a way I find particularly pleasing, the final audience of this data are auditors, managers and control entities that overwatch the operations of (amongst others) the CSIRT & the SOC operations. And they expect spreadsheets. There are 3 report types to consider, each with a different angle.

Report 1: For Control & Compliance. This report verifies the output of highly specific queries that come in various timeframes and ticket properties.
Report 2: Key Performance Indicators. The original report, with a focus on the metrics of tickets managed by the security operational teams. The report provides a KPI summary, followed by the data aggregated by month (which as mentioned, the ITSM tool simply can't generate in one go)
Report 3: A full report for the customer management. It is the largest of the dataset, and due to the audience will contain, besides the data aggregated by month, contains a concise tabular data representation showing incidents by category and by impact. In order to visualise the data, an image is generated using the MatPlotLib (<check) library which is written on the first sheet of the report.

Openpyxl is a community favorite due to it's ease of use, allowing to fill an in-memory representation of a spreadsheet with finally writing this datamodel into a file.

CLI possibilities

The core of the tool is still a commandline invoked binary file that can be passed parameters, e.g. via the use of a LNK file. The main usage is to allow the end-user to start the tool and invoke obtaining the data via selenium right away, write the 3 export files and then close the program. Additional parameters had been provided to provide control on the wait times to be allowed for the service-now webapplication to obtain & export the data.

Trivia

Writing this program took about 1 month in "hobby" time, on & off for times.
Was my first experience with Gui's in Python, as well as using Selenium and OpenPyxl.
It has roughly 2500 lines of code

Security Incidents Metrics Parser