IT Security Metrics in Software

Painting QR Code ISO/IEC18004 in Software IT Security Metrics

formal SIP in support of your security metrics program is to provide the necessary governance structures to help guarantee that the SMPs you undertake will support more than just the tactical goals and questions that make up the projects

If your SMPs were about collecting and analyzing data in support of the goals and questions that you established for each project, then the SIP is about making that data more useful to more people in more contexts IT security metrics have at least two values: The first value is to the immediate measurement project that needs the data to meet a project goal The second value is to the project teams, managers, and others who will benefit from the metrics data later when they replicate the project, conduct a related project, or seek to understand broader security issues by examining case studies and historical evidence

In many cases, SMPs are not one-time projects, but are repeated on a regular basis, such as in the case of vulnerability or risk assessments, monthly or quarterly reviews, or decision support projects around budget or staffing You would think that, of all the examples here, these types of repeated projects would benefit from governance structures of the sort proposed by the SIP After all, these projects are expected and scheduled and often are conducted by the same people over some time period Unfortunately, even these projects are all too often treated as stand-alone efforts, more or less disconnected from what went before or what may come in the future Part of the problem can be a checklist approach to security, in which a list of annual activities exists, based either on a formal compliance requirement or on various definitions of best practice that mandate certain activities will be completed regularly When projects are conducted for these reasons, the motivation to understand what the project actually accomplished (knowing what tasks have been completed and the resulting changes, as opposed to completing a task and checking off the box) is far less than if the project were part of a security improvement strategy I ve seen many examples of repeated security assessments in which the final deliverable each time is virtually the same as the previous versions, indicating that the real security benefit was the ability to say an assessment had been completed A SIP approach, on the other hand, would focus not on the immediate findings of any single project, but on the attempt to determine whether or not security was changing as a result of all SMP efforts By measuring the lack of progress in correcting or improving security problems among projects, the SIP can provide valuable insight into the real functions of your security operations

An SMP, particularly one that is bounded and specific, will often lead to questions that are obvious, but that are not addressed directly by the metrics and data that emerge from that particular SMP effort Several examples of this sort of follow-on project

11:

opportunity existed in the projects discussed in earlier chapters In these situations, two capabilities need to be in place if the opportunity is to be effectively addressed: A capability for driving the questions and requirements from the first SMP into a new, separate SMP A capability for aligning and mapping the results of the related SMP among projects

The need for effective measurement governance in these situations is particularly important, because the projects in question may cross functional or organizational boundaries If a penetration test, for instance, discovered widespread availability of intellectual property on user laptops or workstations, then an obvious follow-on question might be this: What process deficiencies were contributing to this lack of protection of sensitive information The network security team, however, might have no authority or ability to drive a security measurement project through other business units to determine why this information was so prevalent In these types of situations, a dedicated SIP capability with appropriate management support could step in to ensure that the appropriate actions were taken