Security and Risk Management Strategies Blog

Vendor Product Recommendations - Whatchya Really Really Want?

Blogger: Diana Kelley

Back when I was working for a Big 4 consulting company, I was assigned to a project that included an analysis of which PKI solution was right for one of our clients. This was in 1997 when there was a lot of buzz and confusion around PKI but not a lot of deep understanding regarding what it could and could not do for organizations. The plan was for me to complete the requirements portion of the engagement, select a product, and then the company I worked for would, if all went well, sell a follow-on engagement to implement the PKI. Straight forward, right?

The problem was that as I completed the requirements gathering process something unexpected was uncovered. Based on the needs of the organization, a PKI was not the right solution. Not that one PKI was less right than another - once the requirements and legacy constraints were considered, it was clear that PKI was not the right solution at that time. So I documented my findings and presented them to the customer. Though it meant the company I worked for did not get that portion of the follow-on work, it also meant the customer did not spend millions on a solution that would not meet their needs.

Many analysts are asked a variation on the PKI question I got. Boiled down it's essentially, "We want a product solution, which 3 or 4 vendors should we shortlist?"

On the surface this seems like a simple, logical request. It’s not. The reason it's extremely complicated is that there's an underlying implication that some requirements gathering have been completed, analysis of technical and process options have been completed, and priorities and "must haves" for the solution have been determined. Very often, however this is not the case. It wasn't the case when I set off to pick a PKI solution for that customer.

As an example, let's take a product selection almost all of us has made at some point - purchasing a car.

If you asked someone the question, "I want a car, which 3 or 4 models should I shortlist?" imagine the wide variety of responses you could receive. Some answers, well-researched, solid answers, could well be given in good faith but be absolutely wrong for the person asking the question. One of the reasons for this is that proper requirements gathering is a tricky business.

I purchased two very different cars in the last 10 years. A Mazda Miata, two-seater convertible and a Toyota RAV4 SUV. Both great cars, and I was (am) very happy with both choices. But the requirements that led me to select the RAV4 were very different than those that led to the Miata.

Some of my requirements were stable:

• Good gas mileage
• High reliability
• Responsive handling
• Only two seats required (what can I say, I'm anti-social when it comes to having people in my car)

Some changed:

• Very small for ease of use in an urban environment v. big enough to haul around two shelter-shepherd mixes and weekly loads of cruft to the transfer station
• Front wheel drive for city driving v. four wheel drive for wintry, rural roads

Obviously, there were more requirements involved, but the point here is that the stable requirements would have short listed both the Miata and the RAV4 (not sure about newer models, but older RAVs had removable back seats, so I have a two-seater RAV and the dogs have more space in back). The ones that changed were the ones that would have led to very different recommendations.

You probably know people that purchased cars because they came in a certain color, or imply status, or have higher safety ratings. Knowing your requirements and the trade-offs, if any, involved with selecting for one over another are the key to getting the right solution.

Back to the world of IT - "What are the top 3-4 web filtering solutions?" Based on what requirements? If it's pure market share, well that's an easy one to ascertain. But will it lead to selection of the best product for your organization? Flipping to cars again, the most popular car colors for 2006 were white/silver - that wouldn't work for my personal car requirements, but if I wanted to drive the most popular car color, then white/silver would be right.

Selecting the right web filtering solution, actually, deciding if implementing is the right choice at all, will, for most organizations entail a lot more than knowing what everyone else is buying. Requirements that would impact the decision for most large organizations include, ease of use of UI, ability to support a mobile/remote workforce, configurability based on roles/groups, ability to integrate the solution with a user directory store, etc. Recommendations are only as useful as the requirements they're based on.

So at this point, if you're still reading, you may be thinking, "Thanks for stating the obvious, freaky dog person!" But if what I just wrote is so obvious, then why are so many of us looking for a shortlist of the top 3-4 vendors in a product space? We all know how to get market share numbers, so we're looking for more than that. I think we're all busy and we'd like to look to analysts to do the research and the work and come up with a quick answer. But I also think there are no easy answers. Analysts can't make critical requirements decisions, and therefore product selection decisions, for external organizations in a vacuum. But we can do the research that normalizes the discussion, takes the marketing hype out and puts the technology specifics and the security and risk trade-offs back in. We can empower your organization with the unbiased information necessary to make the decisions.

To me, that's a lot more compelling and useful than having someone arbitrarily say, there are 3-4 solutions, pick one.

The alternative is ending up with a multi-million dollar PKI solution that doesn't solve the problem it was meant to. Or driving around in a white/silver Toyota Camry when the right car for you may be a dark green, two-seater, RAV4 with a couple of shepherd mutts hanging out the windows.

An Auditing Standard: Has this rough beast's hour come round at last?

Photo (C) Ian Glazer All Rights Reserved

Blogger: Bob Blakley

During the Q&A following an SRMS panel I moderated at our recent Catalyst conference, an attendee asked me “What new standard would you most like to see?”.  My answer was “A standard defining mechanisms for generation, collection, and disposition of audit events.”

I’ve wanted this for years, but I’ve not held out a lot of hope.  Auditing has always been the red-headed stepchild of security; everyone grudgingly agrees that it’s important but no one really wants to work on it, because it’s not cool like cryptography (you’ll never see a black syslog t-shirt declaring itself to be a munition).

Imagine my surprise, then, when my proposal drew a warm and enthusiastic response from the audience. The new emphasis on compliance seems to have made the value of good event auditing much clearer than it was fifteen years ago when I first worked on auditing technology.

Clearer to the users, that is… after listening to the audience express its desire for the sort of standard I’d described, the vendors on the panel all donned serious expressions and talked about how hard it would be to get “the vendors” (as if they were talking about someone else!) to agree on such a standard.  One vendor suggested the burden of developing the standard should be handed to NIST, on the grounds that government purchasing could be an adoption driver.

I think that’s a cop-out.  I don’t see any reason – commercial or technical – why an excellent distributed auditing standard couldn’t be developed quickly and adopted widely without resorting to government purchasing schedules and mandates.  I can see lots of reasons why such a standard should be developed, the most important of which is that the standard would supply one of the key missing pieces of the compliance reporting puzzle.

There is some movement in the direction of such a standard.  ArcSight has developed a proposed event record format called “Common Event Format” (CEF for short) and appears to want to standardize it.  It’s a decent start, but it will need more work before it becomes the standard we need.  Most notably, CEF defines only a record format; it doesn’t define service interfaces to allow event producers to notify event consumers that an event has been created and is ready to be processed.  There are also technical deficiencies in the record format itself; it does not contain any mechanism for dealing with clock synchronization issues in distributed environments in which multiple systems are producing events.  This omission makes CEF unsuitable for creating audit records which support forensic determination of the ordering of events which occur on different systems.  Finally, CEF leaves the definition of event types (which are called “Signature IDs” in the specification, in a nod toward the intrusion-detection world) up to the individual event producers, thus inviting both ID conflict issues and proliferation of different names for events of the same type in different systems.  You can get CEF from ArcSight here: http://www.arcsight.com/solutions_cef.htm.

MITRE is apparently starting an effort address some of the issues CEF leaves open through a standardization effort called “Common Event Expression” (CEE).  This effort was announced on several blogs (including here: http://raffy.ch/blog/2007/04/19/standard-logging-format-common-event-exchange-cee/) in April.  But no detailed information about CEE seems to have been published by MITRE yet.

In the meantime, an old standard I worked on all those years ago has been resurrected in not one but two places.  At about the time X/Open and OSF merged to create The Open Group, a standard called “X/Open Distributed Audit Standard” (XDAS) was advanced to “preliminary” status.  XDAS was a great piece of work; it addressed all the concerns necessary to build a robust distributed security auditing system in a mature and complete way.  But it was a child of its time; it was an interface standard for C programs running on UNIX machines.

The Open Group moved on from X/Open’s mission of standardizing UNIX interfaces and OSF’s mission of building distributed system code which could be shared by many vendors, and XDAS got left behind.

But sometimes ideas are too good to die, and it seems that like King Arthur, XDAS was not dead but only sleeping.  The Open Group Security Forum has recognized the need for an audit standard for the modern world, and is revising the specification to remove all the C and UNIX anachronisms and bring the functionality up to date.  The landing page for this effort is here: http://www.opengroup.org/security/.  And Novell, whose Bandit Project incorporates XDAS, has contributed source code to a new open-source project called OpenXDAS (http://openxdas.sourceforge.net/) which makes the XDAS implementation widely available.

Can XDAS be modernized? Can CEF be extended?  What will CEE look like? Which of these efforts should you support?  Or should you press NIST for a standard instead? 

Frankly, I think you should support all of these options.  We’ve been wandering in the audit desert for too long.  Three standards would be infinitely better than none.  One broadly agreed standard would be best of all – especially if that one standard exploited Web 2.0 information distribution channels like RSS.  If we all start going to the meetings, my bet is that the existing efforts will converge pretty quickly on something well-integrated into the Web 2.0 architecture which is lighter than XDAS, more functional than CEF, and more broadly supported than either; this should be the goal.

I’m putting my money (or at least my time) where my mouth is here.  I’ll be at the Open Group Security Forum meeting in Austin next week, working on XDAS again for the first time in a decade.  I’ll let you know how it goes.

Catalyst Clarifies Information Security Challenges

Blogger: Dan Blum

The theme for security at Burton Group’s Catalyst conference was this: successful security requires a proactive approach. We focused on many aspects of proactivity, but a few points jumped out pretty clearly:

Data and Risk cannot be governed, but the responsible persons can be – if we have the metrics. Could IT security artifacts be managed through market mechanisms similar to those that drive the business?

Creating open security management frameworks, or ecosystems, could be a win-win for security platform vendors and enterprise customers. Why is this, and what will it take to realize the promise?

My “successful security” presentation proposed the model for proactive security shown below. One of the notions in the model is that organizations should “get stronger sooner” by addressing risks earlier in the IT lifecycle by becoming involved in business planning and risk management.

We researched methodologies for risk management, but they either address low level security project blocking and tackling - or require that business executives meet IT security staff halfway by developing more understanding of the technology in order to set direction and take accountability for it. And unfortunately it can be rather difficult for even the senior managers in IT security to change the behavior of business executives and get them to do this.

At Catalyst, however, IBM’s Steve Adler gave us some exciting ideas in his “Six Questions to Ask About Data Governance” presentation. Among other things, this presentation discussed opportunities for using Utility Theory to derive a value for data, so that investments in managing, protecting or controlling data could be more properly calibrated. The corollary to valuing data is valuing risk – a tough problem. Here Adler referred us to Wikipedia’s coverage of Alternative Risk Transfer (ART).

We’ve written on how to measure Return on Security Investment (ROSI) in one of our reports. And I thought about the Basel II regulation, which requires that banks measure risk and set aside a capital reserve to fund recovery operations should risks materialize.

We’re still early in this line of research, but I find these ideas exciting because they could take risk away from being an externality and put it in terms any executive can understand. And while this may strike you as an overly theoretical point, isn’t the idea akin to pollution credits – which are very real today – and the notion of carbon debt, which may soon also start to impact business?

Thus, data and risk values could appear on a balance sheet that gets rolled up to top executives just like the monthly sales and expense forecasts. Executives wouldn’t have to understand the details of the technology creating the risk anymore than they have to understand every detail of the expense to research some of their rocket science products. But they can understand numbers, trends of numbers, and thresholds for how big those numbers should be. And by managing those numbers, could they give IT the guidance it needs for risk management tradeoffs?

Of course, the smart executive with time on his hands may drill down into any number at any time to spot check it or understand it. The valuation of risk has to be realistic and defensible. Where actuarial evidence is not available (as is so often the case) one might start this exercise with very conservative numbers, explain why they might be understated, and increase those numbers over time as incidents or losses provide more real-world evidence.

If your organization has been valuing data or risk successfully, or has studied the idea seriously, we’d like to talk to you. Please contact us or leave a comment.

And concerning the second major insight about security management frameworks, stay tuned: I’ll cover it in the next week or two. Keep coming back – this blog works!