"Security Failures": Is the sky REALLY falling?

Blogger: Pete Lindstrom

It is fairly common in the security community to lament that security is failing. In fact, it is so common as to essentially be conventional wisdom. I was on a panel in early August when a question arose that suggested this failure as some sort of fact. When I suggested that it wasn’t clear to me that security actually was failing, it made for one of the more humorous moments of the workshop… at least for the audience who laughed in unison at my apparently ridiculous statement.

The notion that security is failing is an interesting one simply because it is a matter of opinion. There are a handful of ways to measure and then evaluate this hypothesis that are more exact and provide insight into the nature of risk management and differences in operational objectives for security. Here are some of these, taken from the position of defining success (we security folks are much too pessimistic for our own good ;-)):

"Success is the complete absence of failure" – given that anecdotal evidence of compromises is used to suggest failure, it only takes one or a handful of incidents to support a hypothesis revolving around success as the complete absence of failure. This appears to be the most prevalent condition described by security professionals to support their belief that security is failing.

"Success means that benefits are greater than the costs" – at the opposite extreme from absolute success is a minimum requirement for success that allows for a high degree of failure. In this case, an environment can fail to the extent that the costs associated with failure do not exceed the benefits associated with IT to begin with.

These two extremes create ends on spectrum that may be used to evaluate success. On the total success side, security professionals provide a critical view of information systems risk that is unattainable (consider an analog in the physical world where the police force in New York City decided to eradicate murders and that a single murder was unacceptable). The real challenge here is that this type of attitude fosters a level of extreme risk aversion and may create the perception of security professional as Chicken Little. While the sky may be falling to us, it is not necessarily so to the rest of the world.

At this other end of the spectrum is where I believe most IT professionals and savvy businesspeople are. That is, they are willing to tolerate some level of "failure" – that is, a number of incidents whose costs are considered reasonable within the scope of the benefits provided by IT. There are many real world analogs once again – for example, shrinkage in the retail world measures the level of theft of inventory, and credit card fraud may be rampant to some, but the credit card industry is very content to make a lot of money even in the face of this fraud. Of course, it is even easier to sustain losses when they are borne by the merchants, but you also don’t see a lot of merchants abandoning credit cards altogether (some do) because the benefits outweigh the risks.

It seems fairly clear that most organizations (i.e. decisionmakers outside of the security profession) favor this latter extreme. The evidence is simple – the growth of IT budgets overall. It would be irrational to continue spending on IT if it were failing. So, either all decisionmakers are irrational (perhaps plausible ;-)) or security is succeeding at least to a level where it is still useful to use information systems.

The notion that security professionals are at one end of the spectrum and decision-makers are at the other is an important one. It highlights the possibility that for all of our talk about synchronizing with businesses, we really are often quite different. This notion hit home personally and ironically during that workshop panel incident mentioned earlier – while everyone laughed at the notion that perhaps security wasn’t failing, at the same workshop the mantra was always about thinking, acting, and speaking in conformity with the business.

Practically speaking, maintaining status quo should never really be an objective, especially in the face of some level of failure. So what is the option? The third alternative is the service-level agreement:

"Success is meeting a specified service-level agreement (SLA)" – an SLA typically recognizes that absolute goals may not be attainable and so we strive to maximize effectiveness while being practical about the existence of some small tolerable of failure. While this is the most common way to measure other IT operational characteristics, it is an atypical one in security. This is not due to inappropriateness, it is simply because there are no universally-agreed upon measures that could make up an SLA at this time. The most useful candidates are not heavily used at this stage. 

In order to create an SLA, we must come up with objectively measurable items. Presumably, these items should also make sense. The most likely candidate for SLAs with respect to security is a measure around control success. Makes sense, doesn’t it, to have an SLA that cuts to the heart of our protection mechanisms?

There is good news here – the health care community has already created a paradigm for measuring the outcome of tests (in our world, "controls"). That is, every test takes event inputs that are either "good" or "bad" by our definitions (i.e. lead to an unwanted outcome) and our measures of control will either recognize this or not. A good event that is recognized as such is a success, as is a bad event that is recognized as bad. These are true positives and true negatives as determined by the test itself. Control failures are well-defined, too – false positives and false negatives are created from good events identified as bad and vice versa.

We are much better off as a profession considering a more tolerant view of information security. That doesn't mean we shouldn't strive for the best, it simply factors in some level of incident occurrence. While this might seem like we are giving ourselves a break, it really means that we are less likely to promote policies and seek controls that are impractical or worse. But the opposite extreme is no panacea – we certainly should be looking for ways to get better, as it almost always appears obvious what could have been done differently to prevent any specific incident after the fact.

The real challenge for the security profession is to create practical SLAs that are both tolerant in a small way yet challenging nonetheless. There is plenty of room in between the two extremes.