Security and Risk Management Strategies Blog

Is Microsoft’s SDL Working?

Blogger: Pete Lindstrom

Microsoft’s Security Development Lifecycle (SDL) is the main product of its Trustworthy Computing Initiative, launched from the now-famous Bill Gates memo in 2002. Six years into the initiative, Microsoft surely must be reaping the benefits of, for example, the well-publicized security training every developer went through.

So, how do we determine whether the SDL is working? Microsoft suggests that this is a simple exercise – simply compare the number of public vulnerabilities disclosed for products prior to SDL with similar products developed after SDL. The most recent case was comparing Windows XP SP2 to Vista vulnerabilities in the first year. The count is down and Microsoft provides a quick and easy example of the logical fallacy “post hoc ergo propter hoc” which in this case means “public perception is ripe for deception.”

The biggest problem with Microsoft’s assertion is simply that there are too many variables that are uncontrolled and could just as easily be making the difference. There are too many unknowns related to effort of independent researchers and focus on a specific Microsoft platform. At the very least, Microsoft has done an admirable job in making people feel more secure. (I happen to believe the SDL is working as well, but that belief is a matter of conjecture without strong evidence).

If Microsoft wants to use public vulnerability counts as the ultimate arbiter, it needs to create an environment where independent researchers are encouraged to find bugs. Creating a controlled bounty program for a limited time period would increase incentives and at least provide circumstantial evidence of SDL effectiveness. Interestingly, if the number of vuln counts was higher, it still wouldn’t mean SDL is ineffective,  but the framing of the conversation would be entirely different.

The plot thickens when Microsoft makes claims that spending more time and leveraging external resources are a part of SDL. Whether they are or not, there is a big difference between making programmers more secure developers and simply spending more money on a problem. You don’t really need SDL if the latter is more beneficial.

But if public vulnerability counts are not the answer, what should Microsoft be doing to demonstrate the effectiveness of its SDL? Well, it is much easier to determine causality by controlling for all other variables, and conducting a test of two groups – one with SDL training and one without. Comparing vulnerability creation rates per unit output (either developer-hours or lines of code, for example) would go a long way to answering the effectiveness question.

At this stage, it might be difficult to find a group of developers in-house that aren’t SDL trained, and Microsoft is fully vested in the program such that it wouldn’t allow an untrained developer on a real project, so a new experiment may need to be set up using some arbitrary project created solely for the experiment. Alternatively, Microsoft could measure the differences in development skills after an acquisition and during the transition to SDL-trained developers. Or a final option is to conduct a private benchmarking exercise where the effectiveness is compared among multiple groups.

At this stage, it may be even harder to figure out the effectiveness of an SDL-trained QA group. Presumably, QA training will help the group find more bugs earlier, but if the developers are getting better, then the rate of finding vulnerabilities will go down. There are techniques associated with defect density that could be leveraged to determine this effectiveness level as well.

Creating fewer bugs and finding more bugs early, I believe, are the real expectations of SDL, and finding those numbers would provide much stronger evidence for or against its effectiveness. Not only that, but this information would better frame discussions around ultimate effectiveness of software development: Microsoft is likely to have spent more money than anyone else on its SDL efforts, so the benchmarks provided by the company would serve as an upper limit for expectations.

Jericho Forum and the Collaboration Oriented Architecture (COA) position paper

Blogger: Dan Blum

After discussing the concept of collaboration oriented architecture (COA) for some time, Jericho Forum released its COA position paper last month at the RSA and Infosecurity Europe conferences. The paper is now posted at http://www.opengroup.org/jericho/COA_v1.0.pdf.

For those who may be unfamiliar with Jericho Forum, it started as a user forum for discussing the problem of deperimeterization, wherein centralized firewalls become less effective as the mainstay of corporate security due to mobility, partnering, outsourcing, telecommuting and all those good things that happen as organizations become more geographically distributed and virtual.

The COA paper focuses on the need for business processes to operate across and between multiple organizations, potentially over untrusted networks such as the Internet. Users and endpoints must securely interact with services and applications controlled by multiple security domains.

The COA position paper builds on the Jericho Forum commandments, which are published at http://www.opengroup.org/jericho/commandments_v1.2.pdf. When reading the commandments, by the way, I find it helps to ignore the explanatory paragraphs, and just focus on the 11 statements of principle. This gets me away from nitpicking the explanations to death and into a state where I just accept them as a very good list of principles for operating securely over open networks.

The COA position paper spends much of its space describing the need for secure, open collaboration as well as principles, processes, standards and frameworks through which the collaboration might be achieved. Most of this doesn’t convey much new information to persons who already grasp the notion of deperimeterization and understand that security is about people, process and technology. But there were some really interesting bits in the section Recommended Solution/Response:

"The COA framework generalizes conventional architectures as follows. It provides:

• increased emphasis on the requirements listed under ‘principles’ below. These are traditionally only seen as external or ‘boundary’ interface concerns in enterprise architectures.
• a user repository (keyed on people identifiers) is generalized into a contract repository (keyed on relationship, or obligation identifiers). A contract repository records agreements, and the obligations and capabilities that ensue from them.
• an accounting log (keyed on system events) is generalized into a reputation repository (keyed on business events). A reputation repository records user actions and compares them to applicable contracts, and, depending on whether or not the actions are in accordance with the contract, upgrades or downgrades a reputation.

The architecture formed by combining SOA (Service Oriented Architecture) with available security protocols (SAML or other XML) is insufficient to support COA. The following elements are also valuable:  [Here, I shorten and paraphrase the list of bullet points]

• attribute brokers
• access brokers
• contract brokers
• policy language (like XACML 3.0)
• performance manager (builds audit logs and reputation systems)”

I wish that the COA position paper had spent more space discussing some of its recommended solutions. The notion of a reputation system (not just a repository) is something we’re hearing more and more about. There is also a growing awareness of the importance of intermediaries, or brokers, that can fairly represent the interests of multiple parties. Perhaps we’ll see this covered in some future Jericho Forum work.

PS: The last bit of COA, in the conclusion, was quite entertaining: “A fundamental shift in thinking is required to implement a COA, moving from the thinking of a hedgehog, an animal that rolls into a tight ball at any sign of threat, to that of a Strawberry Plant, which puts all its key genetic material securely on its outside, as well as sending out suckers to extend the plant’s domain

In the Eye of Malware’s Hurricane

Blogger: Dan Blum

In the eye of the hurricane, the helmsman rests; sun glitters through windswept clouds but the sea is deceptively calm as far as the eye can see. He shades his gaze from the glare and wonders from which point the storm will strike again…

Like the helmsman, security staff at a number of large companies we visited are wondering where the malware went. After years of battling spreading worms and viruses, the alarms are quiet, the infections relatively rare. What happened? Did firewalls, patch management and anti-virus tame the beast? Or has the nature of the beast changed?

A bit of both, we think. Countermeasures successfully block most self-propagating malware, especially (as we predicted) in large enterprise environments with locked down systems. And there are fewer propagating worms and viruses out there than there were in their heyday.

But malware isn’t going away, its just changing. McAfee told us the amount of samples they see has increased many fold over the last two years. According to the Internet Security Threat Report Volume XIII: April, 2008 Symantec saw 499,811 new malicious code threats in the second half of 2007, a 136% increase over the first half of the year. Symantec also measured over 61,000 active botnet machines per day, and Websense found thousands of legitimate websites infected with links to malicious sites.

There are a growing number of underground economy servers selling everything from financial information to increasingly sophisticated crimeware toolkits. Finjan predicts that in the future, the underground economy will mature to point where already-prevalent crimeware as a service (CaaS) style offerings will enable cybercriminals to tap straight into a data feed from victim machines belonging to an individual or organizational target of interest – no need to bother with much up front work. That’s a scary thought.

Its not just vendors hyping the stats to drum up business, we’ve heard many inklings of targeted malware from large enterprise customers (not that they like to talk about getting hit). Some of this showed up in my blog entry Financial Services Roundtable Promotes Information Sharing. I’ve also heard from an impeccable source that industrial espionage is rife in the aerospace and defense sector; DoD has told aerospace and defense companies the Chinese are eating their unclassified data lunch and hauled staff to Washington for meetings to consult on what new compliance requirements to put into future defense contracts. Information warfare isn’t science fiction anymore. We saw it in Serbia, we saw it in Iraq, and most recently there were denial of service attacks in Estonia.

When you’re in the eye of a hurricane, it’s hard to tell where the storm will hit next, but you know isn’t over. While financial services firms and defense/aerospace companies take the brunt of targeted attacks today, government is also under heavy attack and according to Symantec, yielded the lion’s share of identity data breaches in 2H 2007. ISPs get phished a lot, medical identity fraud is increasing, and any organization charged with safeguarding critical infrastructure is also under a cloud. No organization is immune to one of their employees randomly tripping across a malicious website and getting rooted, defrauded, and botted; possibly to its embarrassment, lawsuit, loss and potentially getting targeted for additional exploitation.

For organizations in some industries, the direct targeted attacks may not be a major concern, while the others I’ve identified that have a higher profile are more likely to attract the attention of disgruntled individuals or professional crackers looking for secrets or economic gain. Those organizations need to be wary and maintain high vigilance and counter-measures. Others can relax a bit more, but still need to run a tight ship and monitor the weather map, so to speak.

If you’re lucky enough to have calm skies, do as the wise mariner would: batten down the hatches, rest and feed the crew, then take out the charts. Make sure the basic stuff like anti-malware and patch management are working. Lock down the desktops, tighten up the security zones, track down and corral your critical data and then start thinking about additional architectural strategies to get ahead and stay ahead of the threat. We like inherent protections of the endpoint such as data execution protection (DEP), address space layout randomization (ASLR) and trusted platform modules (TPM) as well application whitelisting in addition to basic system hardening configuration measures. But that’s another post.