Tuesday 10 September 2019

Security Metrics: Persuading and Influencing the Board



Written by Chris Jeffers

The Importance of Security Metrics


A few years ago, I attended a meeting with senior management reviewing several security related initiatives. I was prepared as I knew I’d be asked to provide rationalization for these new projects. I talked through my justification process by first identifying the problem, then the need to address this problem, and finally how this new solution would resolve it. All was going as planned until one of them asked the question, “How will this align with the organization’s business initiatives?”.  Before I could answer, another senior leader asked me “Chris, how do we know what’s currently deployed is keeping us secure?”.

My first thought was you have to be kidding! I don’t have time and what’s the point? There’s a new story everyday about a data breach and an organizations’ data getting encrypted because a user clicked on a ransomware email; we should purchase the solution and get it deployed so we aren’t the next organization hitting the headlines - however, I realized that response would get me nowhere; in fact, that response could throw me out the door looking for a new job! The reality was those questions were valid and caused me to start thinking, so I responded by saying I’d get back to them with some answers. Now, I had to figure out how to do just that.


Using Metrics


I understood the primary reason for having these various security tools, processes and staff - to reduce the risks to the organization - but how can I show they’re accomplishing that? How can I provide evidence that illustrates risk is being reduced?

This is achieved by establishing items to measure and produce metrics. The metrics are numerical data that represent what is occurring and provide the means to tell the organization how the risk is being reduced. These metrics are not based on subjective judgment or interpretation, such as using a low, medium or high rating, but rather presented as a percentage or numerical value.

To better understand what these metrics are and some possible data sources, I started out with the following
  • Anti-virus code and definition versions currently deployed
  • Vulnerabilities discovered in the network and grouped based on CVS score
  • Monthly incidents
  • Email-related malicious events
  • Days from missing patch discovered to actual deployment

I recognized this as a good starting point, understanding we should have enough initial coverage to illustrate to senior management how we are reducing risk to the business:
  • Collecting the current anti-virus versions and definitions helped to identify whether our updating process was progressing or needed attention
  • Understanding the current vulnerabilities and their severity presented the assets at higher risk
  • The monthly incident data was used to understand the types of security incidents, resources required and realizing how effective the processes and staff were. This helped us to understand the type of additional training needed
  • Email-related metrics was all about understanding if the organization had been targeted and the effectiveness of our spam filtering and phishing awareness training
  • Reporting metrics on patching would enable us to understand the amount of time it was taking to get the correct patches deployed. From the point of identifying the patch(s) required, to the point of being applied, this was used to track how well the mandated SLAs were being addressed, as well as the amount of time the asset was left in a higher risk state


Advice for Establishing Your Security Metrics


When considering which data sources to use for your security metrics, try to avoid collecting from a source that presents a long and difficult process and attempt to implement an automated method of data collection over a manual process. The issue with the manual approach is it increases the risk of human error and it becomes harder collecting in a timely manner - timely collection being important to allow for current metrics and trending.

Now that you are collecting metrics from meaningful data sources, you need to put together the report to present to senior leadership. In doing so, be sure to follow some basic rules to help make your presentation well received.
  • Be sure you understand your audience and the strategic objectives of the business. Actually, as the one responsible for driving the security direction and operation for the organization, it is imperative that you understand the strategic business objectives. It’s very difficult to have a clear understanding of the risks to the organization without understanding the business and leaderships’ tolerance to risks.
  • The metrics data being presented must be relevant and meaningful to senior management.  Avoid using many IT abbreviations, jargon, and expressions which make it hard to understand. Ideally, the metrics should be self-explanatory or, if required, include a straight-forward definition. Consider including colorful, visual graphs which make information easier to absorb than text.
  • Lastly, you want to create a situation that will encourage conversation between yourself and the leaders. The goal is to provide information and insights into how risk is truly being reduced, whilst staying in line with the business’s objectives.


Summary


To summarize, security metrics are used for providing evidence that security tools, processes, and people are reducing risk in the organization. The metrics are objective numerical data, presented as a percentage or numerical value. In data collection, an automated process is preferable over a manual process, to avoid the risk of human error and to ensure the process of reporting is efficient. When reporting the findings to leadership, ensure you understand your audience and business objectives, and ensure the insights provided are clear.