- Are you performing well in your current job?
- How do you know?
But what got me re-engaged on this topic now was a just-released Cyber Defense Magazine article by Dmytro Tereshchenko called "Evaluating the CISO." I will come back to why it grabbed my attention later in this blog, but first I want to take readers through a quick survey that I did regarding recent attempts at “grading CISOs” from a variety of sources and perspectives over the past seven years since my first blog on this topic came out.
METRICS AND BEHAVIORS
Many articles on this CISO performance topic focus mainly on team metrics, key performance indicators (KPIs) and behaviors that are most important.
To start, SecurityScorecard offers 6 Cybersecurity Metrics Every CISO Should Monitor:
“Cybersecurity monitoring is not a one-and-done, as attack surfaces and the methods used by malicious actors are constantly changing. By tracking the right cybersecurity metrics, Chief Information Security Officers (CISOs) can monitor the effectiveness of security controls over time, evaluate team performance, and show return on investment (ROI) of cybersecurity investments at the board level.”
The six metrics include:
- Third-Party Risk
- Benchmarking
- Training
- Incident Response
- Personnel
- Return on Investment (ROI)
“An organization’s cybersecurity is a complex and always evolving beast. But one thing that never changes?
“It is the constant misalignment between CISOs, CIOs, and stakeholders over what cybersecurity metrics are actually important.
“In this blog we rundown what security metrics are critical to measure in real-time, ad-hoc, and what you can side-step. We’ll also be covering some extra KPIs that can help cement smoother relationships with your board of directors and non technical reports.”
It then goes on to list 30 KPI; see the article for details.
Third, Gartner offers several studies that examine their “CISO Effectiveness Diagnostic.” Gartner raised some eyebrows when, using their methodology, they proclaimed that only 12 percent of CISOs are considered highly effective. They offer plenty of guidance on the four facets of effective CISOs and plenty of details on how to improve in each area. They also examine the importance of continued professional development.
The chart below describes the five Gartner key behaviors for effective CISOs.
Here is how that piece ends: “As a new CISO, you have a lot of great responsibility. However, it's important you don't forget that your first responsibility is to have all the details that you need — business objectives, an inventory of assets, allies in the company and more — to move forward knowledgeably and confidently in your new role.”
Fifth, take a look at this UpGard article entitled: "What Makes an Effective and Successful CISO?"
The article does a nice job of defining terms; listing role differences between CISOs and CSOs; offering the specific skills that successful CISOs have, along with primary responsibilities; and finally closing with the eight most important qualities and qualifications that make an effective and successful CISO.
Sixth, this article from The Driz Group lays out ways of "Measuring the Success of Your Virtual CISO - Key Performance Indicators (KPIs)."
Finally, for this industry overview section, read this article covering "How do you measure a CISO's job performance?" This very honest and brief blog shows the frustration that many CISOs and others in the security field are feeling regarding this topic, and basically just says measuring is fruitless.
Here is how the blog ends: “Security ROI remains unmeasurable for the same reason that we cannot measure security risk. But that does not change the nature of the job or the strategic need for a CISO to focus always on Security ROI. Finding creative ways to solve this problem is the true test of a CISO’s performance.”
LOHRMANN'S VIEW ON MEASURING EFFECTIVE CISOs
In the section above, I attempted to provide a sample of what many others are saying about CISO evaluation. My view is that, while there is plenty of good advice in these articles, most of the sincere measurements are way too complex, with far too many KPIs, to track effectively over an extended time. Other approaches cover only a fraction of the CISO job and duties, are unmanageable, or don’t even address grading CISOs at all. Rather, many approaches measure project deliverables, “to do” boxes that must be checked within security programs, continuous learning needs, security budget ROI or other things.
Which brings me back to where I started this blog and Dmytro Tereshchenko’s article, "Evaluating the CISO."
I am very honored that Tereshchenko, who is the chief information security officer at Sigma Software Group, based in Lisbon, Portugal, wrote these words for his Cyber Defense Magazine feature:
“Daniel Lohrmann’s 2018 article sparked an important conversation about how to assess CISOs in this broader role. Drawing on years of experience as a CISO and mentor for other security and risk leaders, I’ve slightly adapted Lohrmann’s ideas. In this article, I reflect on five key groups with whom CISOs should build relationships, presented in a specific order.”
Most of all, I am glad to see that relationship-focused methodology described has gained global benefit for CISOs as a tool to measure, improve, grow and mentor others. While I won’t repeat his approach here, I encourage you to read his piece (along with my original approach identified) and apply the system to aid in your CISO (or other CxO) leadership journey.
But before I leave this topic, I do want to mention that Mr. Tereshchenko’s article identifies one major difference from mine. He drops external customers and adds in security programs and projects.
He writes: “This area is not addressed in Lohrmann’s article, but I consider it to be one of the most important. A CISO is hired to lead, manage, and support specific projects or programs such as migrating to a cloud or hybrid infrastructure, implementing zero-trust principles, launching security awareness initiatives, or assessing risks and creating a roadmap for post-quantum cryptography implementation. The success of these initiatives ultimately falls under the CISO’s responsibility.
“To execute these programs effectively, the CISO relies heavily on its team and internal organizational peers. As such, building strong relationships with both is essential for successfully delivering projects. Below are examples of projects and programs a CISO may undertake after excelling in the first two areas:
- Zero Trust Initiatives
- Migration to cloud or hybrid infrastructure
- Configuration and roll-out of EDR (Endpoint Detection and Response) and MDM (Mobile Device Management) tools
- Improvement of the Vulnerability and Patch Management program
- Security Awareness Program
- Enhancement of Application Security Program
- and more”
Yes, I certainly agree with the importance of each of these cybersecurity and team project items (and more when you add in incident response, AI, quantum computing, identity management, threat intelligence, etc.).
Put another way, these specific projects, technologies and priorities will change in your annual performance appraisals. They will be covered by my first three groups: staff/team relationships, technology peer relationships and senior management’s view of your performance.
Nevertheless, I will stop any further criticism of this revised approach. I admire Tereshchenko for using and finding such value in the 2018 method of evaluating CISOs, and I am certainly glad that he has adapted my approach to meet his specific needs and the needs of the CISOs that he mentors.
FINAL THOUGHTS
There is plenty of great CISO career advice, top tips from other CISOs, white papers and more available online for free for CISOs to improve in their roles. Here is one piece for government CISOs: "Guidance for Chief Information Security Officers (CISO)." And the Federal CIO Council CISO Handbook can be found here.
Last year, I wrote this blog asking "Can CISOs Meet Expectations?"
Finally, as I have written many times in multiple blogs, and is also implied by Dmytro Tereshchenko’s article, I think every CISO should have a mentor, be a mentor or both to address blind spots.
