Thursday, July 02, 2009

Traffic Talk 6 Posted

My 6th edition of Traffic Talk, titled Wireshark 1.2 tutorial: Open source network analyzer's new features has been posted. From the article:

Wireshark is a staple of any network administrator's toolkit, and it can be equally useful for any network solution providers or consultants who troubleshoot business networks. Most of the readers of this tutorial have probably used Gerald Combs' open source protocol analyzer for years. In this edition of Traffic Talk, I'd like to discuss a few new features of Wireshark as present in the 1.2 version released on June 15, 2009. I use Windows XP SP3 as my test platform.

If you have any questions on the article, please post them here. Thank you.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Still Blogging

When I announced I would join General Electric as Director of Incident Response in June 2007, I had to post a follow-up titled I'm Not Dead. That issue even made it onto Bill Brenner's radar. Two years later I'm still at GE, glad that as of 1 January this year we have a functional and growing Computer Incident Response Team (CIRT) manned by the best incident handlers and support staff you'll find anywhere.

Sometimes work occupies time I would have previously spent blogging, reading, or writing. That's why you'll often see a flurry of blog posts when I have time on a weekend (or now, before a Company holiday). I've fallen far behind in my reading, and my writing is limited to articles. However, I will be collaborating with Keith Jones and team for Real Digital Forensics Volume 2, which should be cool. I don't have a schedule for other books beyond RDF2 at the moment.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Bejtlich on Black Hat Briefings Panel

The registration process for my TCP/IP Weapons School 2.0 class at Black Hat USA 2009 continues to be active, with seats almost gone in the weekday version. The weekend version has open seats. If you'd like more details, please see my post Black Hat Class Outline Posted.

I was invited to be a panelist for The Laws of Vulnerabilities Research Version 2.0: Comparing Critical Infrastructure Industries, a description of which is posted at the Black Hat Briefings speaker list. Because I'm busy during the 10 am panel time on day 1, I won't have to make the decision about which great talk I'll miss at that time! I mean, Billy Hoffman, FX, Rod Beckstrom, Dino Dai Zovi, and Chris Gates all at the same time?


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Review of Hacking Exposed: Windows, 3rd Ed Posted

Amazon.com just posted my four star review of Hacking Exposed: Windows, 3rd Ed. Better late than never! From the review:

I've been reading and reviewing Hacking Exposed (HE) books since 1999, and I reviewed the two previous Windows books. Hacking Exposed: Windows, 3rd Ed (HEW3E) is an excellent addition to the HE series. I agree with Chris Gates' review, but I'd like to add a few of my own points. The bottom line is that if you need a solid book on Windows technologies and how to attack and defend them, HEW3E is the right resource.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

NSA to "Screen" .gov Now, I Predict .com Later

In my Predictions for 2008 I wrote Expect greater military involvement in defending private sector networks. Today I read a great Washington Post story titled Obama Administration to Involve NSA in Defending Civilian Agency Networks. It says in part:

The Obama administration will proceed with a Bush-era plan to use National Security Agency assistance in screening government computer traffic on private-sector networks, with AT&T as the likely test site...

President Obama said in May that government efforts to protect computer systems from attack would not involve "monitoring private sector networks or Internet traffic" and Department of Homeland Security officials say that the new program will only scrutinize data going to or from government systems...

Under a classified pilot program approved during the Bush administration, NSA data and hardware would be used to protect the networks of some civilian government agencies. Part of an initiative known as Einstein 3, the pilot called for telecommunications companies to route the Internet traffic of civilian government agencies through a monitoring box that would search for and block malicious computer codes...

The internal controversy reflects the central tension in the debate over how best to defend the nation's mostly private system of computer networks. The most effective techniques, experts say, require the automated scrutiny of e-mail and other electronic communications content -- something that commercial providers already do.

Proponents of involving the government said such efforts should harness the NSA's resources, especially its database of computer codes, or signatures, that have been linked to cyberattacks or known adversaries. The NSA has compiled the cache by, for example, electronically observing hackers trying to gain access to U.S. military systems, the officials said.

"That's the secret sauce," one official said. "It's the stuff they have that the private sector doesn't."

But it is also the prospect of NSA involvement in cybersecurity that fuels concerns of unwarranted government snooping into private communications...

The classified NSA system, known as Tutelage, has the ability to decide how to handle malicious intrusions -- to block them or watch them closely to better assess the threat, sources said. It is currently used to defend military networks.


You're thinking, "this article says NSA will not monitor purely private networks. What's the fuss?" Imagine you're the CEO, CIO/CTO, or CISO of a big company. You say "why is my company and our employees paying taxes so that the government can protect itself while my company is left outside the circled wagons?" The higher you go in corporate management, the more likely the only "security" that will be recognized will be "firewalls." So, you're going to have big-league corporate leaders telling the government that they want their companies "protected" too. This isn't really what is happening, but at that level it really doesn't matter.

The bottom line is that first the military protected itself, and now the military is going to help protect civilian government agencies. Critical private infrastructure will be next, followed by economically important companies -- think "too big to be 0wned." This will be interesting.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Late Las Vegas registration ends 22 July.

Sunday, June 28, 2009

Simpler IP Range Matching with Tshark Display Filters

In today's SANS ISC journal, the story IP Address Range Search with libpcap wonders how to accomplish the following:

...how to find SYN packets directed to natted addresses where an attempt was made to connect or scan a service natted to an internal resource. I used this filter for addresses located in the range 192.168.25.6 to 192.168.25.35.

The proposed answer is this:

tcpdump -nr file '((ip[16:2] = 0xc0a8 and ip[18] = 0x19 and ip[19] > 0x06)\
and (ip[16:2] = 0xc0a8 and ip[18] = 0x19 and ip[19] < 0x23) and tcp[13] = 0x02)'

I am sure it's clear to everyone what that means!

Given my low success rate in getting comments posted to the SANS ISC blog, I figured I would reply here.

Last fall I wrote Using Wireshark and Tshark display filters for troubleshooting. Wireshark display filters make writing such complex Berkeley Packet Filter syntax a thing of the past.

Using Wireshark display filters, a mere mortal could write the following:

tshark -nr file 'tcp.flags.syn and (ip.dst > 192.168.25.6 and ip.dst < 192.168.25.35)'

Note that if you want to be inclusive, change the > to >= and the < to <= .

To show that my filter works, I ran the filter against a file with traffic on my own 192.168.2.0/24 network, so I altered the last two octets to match my own traffic.

$ tshark -nr test.pcap 'tcp.flags.syn and (ip.dst > 192.168.2.103 and ip.dst < 192.168.2.106)'

137 2009-06-28 16:21:44.195504 74.125.115.100 -> 192.168.2.104 HTTP Continuation or non-HTTP traffic

You have plenty of other options, such as ip.src and ip.addr.

Which one do you think is faster to write and easier to understand?


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Effective Digital Security Preserves Long-Term Competitiveness

Yesterday I mentioned a speech by my CEO, Jeff Immelt. Charlie Rose also interviewed Mr Immelt last week. In both scenarios Mr Immelt talked about preserving long-term competitiveness. Two of his themes were funding research and development and ensuring the native capability to perform technical tasks.

It occurred to me that digital security is reflected in both themes. In Crisis 0: Game Over I asked I'm sure some savvy reader knows of some corporate espionage case that ended badly for the victim, i.e., bankruptcy or the like? I got a few interesting cases, but I believe the net result is that it is difficult to find examples where an intrusion or breach was so devastating that it ended up destroying the victim organization.

This makes sense once you reflect on it. Why would a mature, thoughtful intruder seek to destroy his victim, if the purpose of his mission is to conduct espionage on behalf of a competitor or intelligence service? Destroying the victim renders it useless as a source for stealing intellectual property gained by the victim's research and development. In the foreign intelligence case, almost all operators prefer to keep a source active, even in wartime when you might think that destruction is the ultimate goal.

Taking this line of reasoning to its natural conclusion, we can see that digital security can be considered a means to preserve long-term competitiveness, particularly for organizations that seek to drive internal growth via investing in research and development. Such an organization is a natural target for competitors who find it immensely cheaper to steal intellectual property, rather than fund their own.

The problem is showing those who make budgetary and management decisions that digital security has a real role in loss prevention. I've written a lot about intellectual property and digital security, but it is exceptionally difficult to tie individual intrusions to real impact. How does pervasive theft of intellectual property (IP) manifest itself? In commercial cases, perhaps it would appear as a loss of sales to rivals who make similar or duplicate products based on stolen IP. Would the victim organization even know these lost or declining sales were the result of IP theft?

Even if the victim identified the stolen IP, could it be traced back to one or more intrusions, or would it be considered the consequences of product reverse engineering by competitors? The bottom line could be that the victim is still in business, but the double-digit growth and expanding market share it craves are reduced to single-digit growth and eroding market share.

It's a waste of time to use terms like "ROI" or "ROSI" when talking to managers or business people. It is usually impossible to fully explain, from loss to impact, the IP theft cases like the one I described in Intellectual Property: Develop or Steal, i.e., spend $10 million over 10 years on a product, then watch the Chinese duplicate it for $1.4 million in 6 months after stealing the IP. More often than not, the victim of IP theft simple whithers, wondering why its competitive advantage is not what it expected it to be. It's time to get managers and business people to think in terms of long-term competitiveness.

Clearly Mr Immelt has determined that it is not in his company's best interest, nor in the interests of the country, for the US to be underfunding R&D or outsourcing everything overseas. We security professionals need to adopt this line of reasoning to emphasize how effective digital security preserves long-term competitiveness.

By the way, you might be wondering if I can prove there is an impact to IP theft. I look at the question this way. If there were no impact to IP theft, why would economic and national competitors fund teams to steal IP? You might argue that IP thieves can duplicate and sell products at prices lower than the IP owner could afford, thereby serving a new market. If that were true, why would IP owners file patents? Clearly there is value in IP, so stealing it lessens the value available to the IP owner.

I use a variant of this argument when I encounter asset owners who claim there is no impact associated with an intrusion. My reply is usually this: If there is no impact, then why operate the asset? Retire it.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Saturday, June 27, 2009

Posts to Read Elsewhere

I'm not a big fan of just publishing links to other people's stories, but there's a few that I really like this week. Please consider checking these out:



Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Black Hat Budgeting

Earlier this month I wondered How much to spend on digital security. I'd like to put that question in a different light by imagining what a black hat could do with a $1 million budget.

The ideas in this post are rough approximations. They certainly aren't a black hat business plan. I don't recommend anyone follow through on this, although I am sure there are shops our there who do this work already.

Let's start by defining the mission of this organization, called Project Intrusion (PI). PI is in "business" to steal intellectual property from organizations and sell it to the highest bidders. In the course of accomplishing that mission, PI may develop tools and techniques that it could sell down the food chain, once PI determines their utility to PI has sufficiently decreased.

With $1 million in funding, let's allocate some resources.

  • Staff. Without people, this business goes nowhere. We allocate $750,000 of our budget to salaries and benefits to hire the following people.


    • The team leader should have experience as a vulnerability researcher, exploit developer, penetration tester, enterprise defender, and preferably an intelligence operative. The leader can be very skilled in at least one speciality (say Web apps or Windows services) but should be familiar with all of the team's roles. The team leader needs a vision for the team while delivering value to clients. $120,000.

    • The team needs at least one attack tool and technique developer for each target platform or technology that PI intends to exploit. PI hires three. One focuses on Windows OS and client apps, one on Web apps, and one on Unix and network infrastructure. $330,000.

    • The team hires two penetration operators who execute the team leader's mission directives by using the attack tools and techniques supplied by the developers. The operators penetrate the target and establish the persistence required to acquire the desired intellectual property. $180,000.

    • The team hires one intelligence operative to direct the penetration operators attention toward information of value, and then assess the value of exfiltrated data. The intel operative interfaces with clients to make deals. $120,000.


  • Technology. The team will need the following, for a total of $200,000.


    • Lab computers running the software likely to be attacked during operations.

    • Operations computers from which the penetration operators run attacks.

    • Network connectivity and hosting for the lab computers and operations computers, dispersed around the world.

    • Software required by the team, since many good attack tools are commercial. MSDN licenses are needed too. There's no need to steal these; we have the budget!


  • Miscellaneous. The last $50,000 could be spent on incidentals, bribes, team awards, travel, or whatever else the group might require in start-up mode.


If the attack developers manage to make enough extra money by selling original exploits, I would direct the funds to additional penetration operators. It would take about six of them to support a sustainable 24x7 operation. With only two they would need to be careful and operate within certain time windows.

So what is the point of this exercise? I submit that for $1 million per year an adversary could fund a Western-salaried black hat team that could penetrate and persist in roughly any target it chose to attack. This team has the structure and expertise to develop its own attack methods, execute them, and sell the results of its efforts to the highest bidders.

This should be a fairly scary concept to my readers. Why? Think about what $1 million buys in your security organization. If your company is small, $1 million could go a long way. However, when you factor in all of the defensive technology you buy, and the salaries of your staff, and the scope of your responsibilities, and so on, quickly you realize you are probably out-gunned by Project Intrusion. PI has the in-house expertise to develop its own exploits, keep intruders on station, and assess and sell the information it steals.

Worse, PI can reap economies of scale by attacking multiple targets for that same $1 million. Why? Everyone runs Windows. Everyone uses the same client software. Everyone's enterprise tends to have the same misconfigurations, missing patches, overworked staff, and other problems. The tools and techniques that penetrate company A are likely to work against company B.

This is why I've always considered it folly to praise the Air Force for standardizing its Windows deployment with supposedly secure configurations. If PI looks at its targets and sees Windows, Windows, some other OS that might be Linux or BSD or who knows what, Windows, Windows, who do you think PI will avoid?

It's all about cost, on the part of the attacker or defender. Unfortunately for defenders, it's only intruders who can achieve "return on investment" when it comes to exploiting digital security.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Being a Critic Is Easy, So What Would I Do?

After my last post, some of you are probably thinking that it's easy to be a critic, but what would I suggest instead? The answer is simple to name but difficult to implement.

  1. Operate a defensible network architecture. Hardly anyone does. I don't need to explain all of the reasons why here; they could occupy a series of posts, or maybe even a book.

  2. Once the DNA is operating, detect and respond to failures. The nice aspect of operating a DNA is that the number of failures should be lower but of higher complexity. Unfortunately at the moment almost all of the world's detection and response teams have to deal with the entire spectrum of security incidents. These range from the most mundane to the most complex. Too often the mundane hide the complex, or at the very least divert resources and attention.

  3. Use the knowledge learned from failures (either caused by adversaries or adversary simulation) to guide the next version of the DNA. Since most enterprises are not operating a DNA, they never get to work on the next version anyway.


I know other people think this way too. Harlan Carvey is one. He is also an incident responder and he finds so many clients that are not doing the basics anywhere remotely right.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Ugly Security

I read Anton Chuvakin's post MUST READ: Best Chapter From “Beautiful Security” Downloadable! with some interest. He linked to a post by Mark Curphey pointing out that Mark's chapter from O'Reilly's new book Beautiful Security was available free for download in .pdf format. O'Reilly had been kind enough to send me a copy of the book, so I decided to read Mark's chapter today.

I found the following excerpts interesting.

Builders Versus Breakers

Security people fall into two main categories:

  • Builders usually represent the glass as half full. While recognizing the seriousness of vulnerabilities and dangers in current practice, they are generally optimistic people who believe that by advancing the state they can change the world for the better.

  • Breakers usually represent the glass as half empty, and are often so pessimistic that you wonder, when listening to some of them, why the Internet hasn’t totally collapsed already and why any of us have money left unpilfered in our bank accounts. Their pessimism leads them to apply the current state of the art to exposing weaknesses and failures in current approaches.


I remembered I had seen something like this before and wrote On Breakership in response. However, back then the debate seemed to center around calling people who helped create and defend systems as "builders, while labeling people who exploited or at least tested systems as "breakers." Mark seems to have dismissed people who "break" systems in order to improve security, while praising builders as people who stay "optimistic." I don't think this is fair. My post Response to Is Vulnerability Research Ethical? explains my position, which is essentially that Offense and Defense Inform Each Other.

Next, in a section titled Clouds and Web Services to the Rescue, Mark describes how centralized data storage for his 6 home PCs at Amazon S3 is great for security. Unfortunately, all he is really showing is that there is value in offsite storage. Storing data at Amazon S3 doesn't help much when those 6 systems are part of Calin's botnet in Romania. This is an example of focusing on one aspect of security (availability) while ignoring the other parts (confidentiality and integrity). Don't get me wrong -- I think cloud storage is great and I use a variety of services myself. However, it only helps with one aspect of the security landscape, and if not properly utilized introduces other vulnerabilities and exposures not found in other models.

Next Mark talks about using cloud services for data analysis.

Event logs can provide an incredible amount of forensic information, allowing us to reconstruct an event. The question may be as simple as which user reset a specific account password or as complex as which system process read a user’s token. Today there are, of course, log analysis tools and even a whole category of security tools called Security Event Managers (SEMs), but these don’t even begin to approach the capabilities of supercrunching. Current tools run on standard servers with pretty much standard hardware performing relatively crude analysis...

[T]he power and storage that is now available to us all if we embrace the new connected computing model will let us store vast amounts of security monitoring data for analysis and use the vast amounts of processing power to perform complex analysis. We will then be able to look for patterns and derive meaning from large data sets to predict security events rather than react to them. You read that correctly: we will be able to predict from a certain event the probability of a tertiary event taking place. This will allow us to provide context-sensitive security or make informed decisions about measures to head off trouble.


Does Mark mean that the real problem we've had with detecting and responding to security events is a lack of processing power? Good grief. I hear thoughts like this quite often from people who don't actually detect and respond to security incidents. Even academic security researchers in their ivory towers are probably laughing at Mark's angle. "Oh, you're right -- we've just been waiting for a supercomputer to run our algorithms!"

Mark then talks about using Business Process Management (BPM) software to improve security:

When security BPM software (and a global network to support it) emerges, companies will be able to outsource this step not just to a single company, in the hope that it has the necessary skills to provide the appropriate analysis, but to a global network of analysts. The BPM software will be able to route a task to an analyst who has a track record in a specific obscure technology (the best guy in the world at hacking system X or understanding language Y) or a company that can return an analysis within a specific time period. The analysts may be in a shack on a beach in the Maldives or in an office in London; it’s largely irrelevant, unless working hours and time zones are decision criteria...

This same fundamental change to the business process of security research will likely be extended to the intelligence feeds powering security technology, such as anti-virus engines, intrusion detection systems, and code review scanners. BPM software will be able to facilitate new business models, microchunking business processes to deliver the end solution faster, better, or more cheaply. This is potentially a major paradigm shift in many of the security technologies we have come to accept, decoupling the content from the delivery mechanism. In the future, thanks to BPM software security, analysts will be able to select the best anti-virus engine and the best analysis feed to fuel it — but they will probably not come from the same vendor.


Again, this is so detached from reality, I am curious how anyone could think this is possible. Mark works for Microsoft. Would you ever imagine Microsoft pivoting on a dime to "select the best anti-virus engine and the best analysis feed" -- or would they stick to their own product, because it's their own product? What about your company -- have you witnessed the organizational inertia associated with any IT product or system?

How about trust factors? What if "the best guy in the world at hacking system X or understanding language Y" works in a country with a reputation for industrial espionage? What if that guy was just hired by a competitor, or is working for a competitor now? How long does it take outside help to become familiar with the aspects of your business that eventually determine success? There's a reason why companies are not collections of free agents working independently.

Mark's last section talks about social networking for the security industry, talking about how people should share what they know. There are indeed certain collaborative forums where this works, but you are seldom if ever going to find any serious company telling other companies how their security defenses work, how they fail, and what is lost as a result of that failure. Individual collaboration occurs, but there could be severe consequences for a security staff member who unloads specific technical security information to a social network. The most productive associations that currently exist are found in certain private mailing lists, associations of peer companies that sign mutual nondisclosure agreements, and individual exchanges among peers.

Mark is a smart guy but I think his prognosis for the security industry in his Beautiful Security chapter are largely incomplete and unrealistic.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Thursday, June 25, 2009

SANS Forensics and Incident Response 2009

The agenda for the second SANS WhatWorks Summit in Forensics and Incident Response has been posted. I am really happy to see I am speaking on Tuesday, because I will not be available Wednesday. Day 1 appears mainly technical, and day 2 is mainly legal. Please consider registering for the two-day conference. It's the best incident response event in the US this year!


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Wednesday, June 24, 2009

DoD Creates USCYBERCOM

Today is an historic day for our profession, and for my American readers, our country. As reported in The Washington Post and by several of you, today Secretary Gates ordered the creation of U.S. Cyber Command, a subordinate unified command under U.S. Strategic Command. The NSA Director will be dual-hatted as DIRNSA and CYBERCOM Commander, with Title 10 authority, and will be promoted to a four-star position. Initial Operational Capability for CYBERCOM is October 2009 with Full Operational Capability planned for October 2010. Prior to CYBERCOM achieving FOC, the Joint Task Force - Global Network Operations (JTF-GNO) and the Joint Task Force - Network Warfare (JTF-NW) will be "disestablished."

As one of my friends said: "Step one to your Cyber Service -- what will the uniforms look like?"


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Tuesday, June 23, 2009

Free .pdf Issue of BSD Magazine Available

Karolina at BSD Magazine wanted me to let you know that she has posted a free .pdf issue online. I mentioned this issue last year and its focus is OpenBSD. Check it out, along with Hakin9!


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

The Problem with Automated Defenses

Automation is often cited as a way to "do more with less." The theory is that if you can automate aspects of security, then you can free resources. This is true up to a point. The problem with automation is this:

Automated defenses are the easiest for an intruder to penetrate, because the intruder can repeatedly and reliably test attacks until he determines they will be successfully and potentially undetectable.

I hope no one is shocked by this. In a previous life I worked in a lab that tested intrusion detection products. Our tests were successful when an attack passed by the detection system with as little fuss as possible.

That's not just an indictment of "IDS"; that approach works for any defensive technology you can buy or deploy off-the-shelf, from anti-malware to host IPS to anything that impedes an intruder's progress. Customization and localization helps make automation more effective, but that tends to cost resources. So, automation by itself isn't bad, but mass-produced automation can provide a false sense of security to a certain point.

In tight economic conditions there is a strong managerial preference for the so-called self-defending network, which ends up being a self-defeating network for the reason in bold.

A truly mature incident detection and response operation exists because the enterprise is operating a defensible network architecture, and someone has to detect and respond to the failures that happen because prevention eventually fails. CIRTs are ultimately exception handlers that deal with everything that falls through the cracks. The problem happens when the cracks are the size of the Grand Canyon, so the CIRT deals with intrusions that should have been stopped by good IT and security practices.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

You Know You're Important When...

You know you're an important when someone announces a "Month of Bugs" project for you. July will be the Month of Twitter Bugs, brought to my attention in this story by Robert Westervelt. The current project is led by a participant in the Month of Browser Bugs from three years ago named Avi Raff.

I don't see projects like that as being irresponsible. What would be more irresponsible is selling the vulnerabilities to the underground. Would the critics prefer that? In many cases, "Month of" projects are the result of running into resistance from developers or managers are not taking vulnerabilities seriously. In many cases the vulnerabilities are already being exploited. Sure, packaging all of the vulnerabilities into a "Month of" project gains attention, but isn't that the point?


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Sunday, June 21, 2009

The Centrality of Red Teaming

In my last post I described how a Red Team can improve defense. I wanted to expand on the idea briefly.

First, I believe the modern enterprise is too complex for any individual or group to thoroughly understand how it can be compromised. There are so many links in the chain that even knowing they exist, let alone how they connect, can be impossible.

To flip that on its end, in a complementary way, the modern enterprise is too complex for any individual or group to thoroughly understand how its defenses can fail. The fact that vendors exist to reduce firewall rule sets down to something intelligible by mere mortals is a testament to the apocalyptic fail exhibited by digital defenses.

Furthermore, it is highly likely that hardly anyone cares about attack models until they have been demonstrated. We seen this repeatedly with respect to software vulnerabilities. It can be difficult for someone to take a flaw seriously until a proof of concept is shown to exploit a victim. L0pht's motto "Making the theoretical practical since 1992" is a perfect summarization of this phenomenon.

So why mention Red Teams? They are central to digital defense because Red Teams transform theoretical intrusion scenarios into reality in a controlled and responsible manner. It is much more realistic to use your incident detection and response teams to know what adversaries are actually doing. However, if you want to be more proactive, you should deploy your Red Team to find and connect those links in the chain that result in a digital disaster.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Offense and Defense Inform Each Other

If you've listened to anyone talking about the Top 20 list called the Consensus Audit Guidelines recently, you've probably heard the phrase "offense informing defense." In other words, talk to your Red Team / penetration testers to learn how they can compromise your enterprise in order to better defend yourself from real adversaries.

I think this is a great idea, but there isn't anything revolutionary about it. It's really just one step above the previous pervasive mindset for digital security, namely identifying vulnerabilities. In fact, this neatly maps into my Digital Situational Awareness ranking. However, if you spend most of your time writing policy and legal documents, and not really having to deal with intrusions, this idea probably looks like a bolt of lightning!

And speaking of the Consensus Audit Guidelines: hey CAG! It's the year 2000 and the SANS Top 20 List wants to talk to you!

The SANS/FBI Top Twenty list is valuable because the majority of successful attacks on computer systems via the Internet can be traced to exploitation of security flaws on this list...

In the past, system administrators reported that they had not corrected many of these flaws because they simply did not know which vulnerabilities were most dangerous, and they were too busy to correct them all...

The Top Twenty list is designed to help alleviate that problem by combining the knowledge of dozens of leading security experts from the most security-conscious federal agencies, the leading security software vendors and consulting firms, the top university-based security programs, and CERT/CC and the SANS Institute.


Expect at some point to hear Beltway Bandits talking about how we need to move beyond talking to the Red Team and how we need to see who is actively exploiting us. Guess what -- that's where the detection and response team lives. Perhaps at some point these "thought leaders" will figure out the best way to defend the enterprise is through counterintelligence operations, like the police use against organized crime?

For now, I wanted to depict that while it is indeed important for offense to inform defense, the opposite is just as critical. After all, how is the Red Team supposed to simulate the adversary if it doesn't know how the adversary operates? A good Red Team can exploit a target using methods known to the Red Team. A great Red Team can exploit a target using methods known to the adversary. Therefore, I created an image describing how offense and defense inform each other. This assumes a sufficiently mature, resourced, and capable set of security teams.



This post may sound sarcastic but I'm not really bitter about the situation. If we keep making progress like this, in 3-5 years the mindset of the information security community will have evolved to where it needed to be ten years ago. I'll keep my eye on the Beltway Bandits to let you know how things proceed.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.

Response to the Möbius Defense

One of you asked me to comment on Pete Herzog's "Möbius Defense". I like Lego blocks, but I don't find the presentation to be especially compelling.

  1. Pete seems to believe that NSA developed "defense in depth" (DiD) as a strategy to defend DoD networks after some sort of catastrophic compromise in the 1970s. DiD as a strategy has existed for thousands of years. DiD was applied to military information well before computers existed, and to the computers of the time before the 1970s as well.

  2. Pete says DiD is

    "all about delaying rather than preventing the advance of an attacker... buying time and causing additional casualties by yielding space... DiD relies on an attacker to lose momentum over time or spread out and thin its massive numbers as it needs to traverse a large area... All the while, various units are positioned to harm the attacker and either cause enough losses in resources to force a retreat or capture individual soldiers as a means of thinning their numbers."

    That's certainly one way to look at DiD, but it certainly isn't the only way. Unfortunately, Pete stands up this straw man only to knock it down later.

  3. Pete next says

    "Multiple lines of defense are situated to prevent various threats from penetrating by defeating one line of defense. 'Successive layers of defense will cause an adversary who penetrates or breaks down one barrier to promptly encounter another Defense-In-Depth barrier, and then another, until the attack ends.'"

    It would be nice to know who he is quoting, but I determined it is some NSA document because I found other people quoting it. I don't necessarily agree with this statement, because plenty of attacks succeed. This means I agree with Pete's criticism here.

  4. So what's the deal with Möbius? Pete says:

    "The modern network looks like a Moebius strip. Interactions with the outside happen at the desktop, the server, the laptop, the disks, the applications, and somewhere out there in the CLOUD. So where is the depth? There is none. A modern network throws all its fight out at once."

    I believe the first section is party correct. The modern enterprise does have many interactions that occur outside of the attack model (if any) imagined by the defenders. The second section is wrong. Although there may be little to no depth in some sections (say my Blackberry) there is plenty of depth elsewhere (at the desktop, if properly defended). The third section is partly correct in the sense that any defense that happens generally occurs at Internet speed, at least as far as exploitation goes. Later phases (detection and response) do not happen all at once. That means time is a huge component of enterprise defense; comprehensive defense doesn't happen all at once.

  5. Pete then cites "Guerrilla Warfare and Special Forces Operations" as a new defensive alternative to DiD, but then really doesn't say anything you haven't heard before. He mentions counterintelligence but that isn't new either.


I've talked about DiD in posts like Mesh vs Chain, Lessons from the Military, and Data Leakage Protection Thoughts.

I think it is good for people to consider different approaches to digital security, but I don't find this approach to be all that clever.


Richard Bejtlich is teaching new classes in Las Vegas in 2009. Regular Las Vegas registration ends 1 July.