Larry Suto is back with another report outlining the differences between some of the top web application scanners on the market. Before you get all uptight and start flaming me, I in NO WAY sponsored, encouraged or had anything to do with this test in any way. In fact, I only found out about it a few days ago. Not that I think that’ll stop the flame wars, but just direct your ire appropriately, please! Anyway, he took a different approach this time, and instead of running the scanners against something he had devised up to be used only in his own lab, he turned all the scanners on each other’s public test sites. You might think they should all fair fairly well since they’re all public and there’s nothing stopping them from testing to their heart’s content. But that’s anything but what he found. You can download Larry’s report here.

Some of the more interesting findings were that Burp Suite Pro (an extremely cheap product built by Portswigger - and a damned fine manual testing tool, I might add) fared better than Qualys or WebInspect when trained. I always loved Burp Suite! Larry’s commentary is particularly amusing as you go through it, with choice quotes, like:

Accuntix missed 31% of the vulnerabilities after training and 37% without training. This is a significant cause for concern as they should be aware of the links vulnerabilities on their own site and be able to crawl and attack them.

And…

WebInspect missed 66% of the vulnerabilities, even after being trained to know all of the pages. They missed 42% of the vulnerabilities on their own test site after being trained and 55% before training.

NTOSpider by NT Objectives came out in the lead with the best overall score of the application scanners tested (which included Accunetix, Appscan, Burp Suite Pro, Hailstorm, WebInspect, and NTOSpider). He also measured things like how long the various scanners take to configure, support and so on - all important things for companies about to make the big investment. This isn’t all scanners everywhere (notably WhiteHat is missing as is the newest player to the field, NetSparker, and other free web assessment tools, like Nikto etc…), but it’s a great start to a long future of heavily debated research, I’m sure. Love him, or hate him, Larry’s always got interesting research to share!

by Christian Moldes

In Outliers, Malcom Gladwell analyses how plane crashes are the result of a combination of errors. I found this analysis very interesting because of the similarity with most security breaches. A brief excerpt of his book:

“Plane crashes rarely happen in real life the same way they happen in the movies. Some engine part does not explode in a fiery bang. The rudder doesn’t suddenly snap under the force of takeoff. The captain doesn’t gasp, “Dear God,” as he’s thrown back against his seat. The typical commercial jetliner – at this point in its stage of development – is about as dependable as a toaster. Plane crashes are much more likely to be the result of an accumulation of minor difficulties and seemingly trivial malfunctions.

The typical accident involves seven consecutive human errors. One of the pilots does something wrong that by itself is not a problem. Then one of them makes another error on top of that, which combined with the first error still does not amount to catastrophe. But then they make a third error on top of that, and then another and another and another and another, and it is the combination of all those errors that leads to disaster.”

Security breaches happen exactly like that. They are the result of a combination of minor or seemingly insignificant errors. Let me illustrate this. A few years ago, a merchant suffered a breach, and its case is one of the best examples for this topic. Their e-commerce website was developed in-house but some of the components had been developed by a third party. The application had been thoroughly reviewed for security vulnerabilities and none had been identified as risky. However, one of the components was not reviewed, it was added a few days after the application review had been completed, and since it was not related in any way with payment transactions, it was deemed as non-critical.

The merchant had a network IDS which was maintained and monitored by a MSS (managed security services) vendor. The device had signatures that were able to recognize SQL injection attempts and they were supposedly enabled. One of the vendor’s security analysts disabled rules monitoring attacks on port 80 and 443 for the e-commerce servers. This was probably because they generated many false-positive alerts, and was most likely intended as a temporary action. As a result, none of the attacks and unusual traffic on those ports was detected by the IDS.

The e-commerce site was using a trusted relationship to connect to the database. Credit card numbers had been encrypted in the database a few months ago. During the process as a contingency plan, the DBA exported the tables containing sensitive data before encrypting some of the columns. The backup files had been left on the database server since then.

Hackers found the security vulnerability in the e-commerce website; the third-party component was vulnerable to SQL injection. By exploiting the vulnerability, they were able to create local administrator accounts on the database server and run OS commands with local administrator privileges. Unfortunately, since the IDS was not monitoring traffic on ports 90 and 443, none of the SQL probes was detected by the IDS, nor was any other unusual traffic on those ports. Remote management tools were installed and password hashes were cracked off-line. The hackers reviewed every folder on the web server looking for scripts, source code, and data files. They found the backup files left behind by the DBA.

The merchant was only aware of the intrusion several months after the fact, when they were notified by law enforcement agents that their data was on sale on one of the carders websites.

This case clearly illustrates that even when proper security controls are in place, a breach could happen at any moment. Relying on single controls or single layers of security is never sufficient.

The case also illustrates the need to assess security controls independently of any other surrounding security or other layers of security. QSAs and internal staff in charge of PCI DSS compliance should not consider risk-based discussions until all the security controls have been independently assessed.

by Christian Moldes

In Outliers, Malcom Gladwell analyses how plane crashes are the result of a combination of errors. I found this analysis very interesting because of the similarity with most security breaches. A brief excerpt of his book:

“Plane crashes rarely happen in real life the same way they happen in the movies. Some engine part does not explode in a fiery bang. The rudder doesn’t suddenly snap under the force of takeoff. The captain doesn’t gasp, “Dear God,” as he’s thrown back against his seat. The typical commercial jetliner – at this point in its stage of development – is about as dependable as a toaster. Plane crashes are much more likely to be the result of an accumulation of minor difficulties and seemingly trivial malfunctions.

The typical accident involves seven consecutive human errors. One of the pilots does something wrong that by itself is not a problem. Then one of them makes another error on top of that, which combined with the first error still does not amount to catastrophe. But then they make a third error on top of that, and then another and another and another and another, and it is the combination of all those errors that leads to disaster.”

Security breaches happen exactly like that. They are the result of a combination of minor or seemingly insignificant errors. Let me illustrate this. A few years ago, a merchant suffered a breach, and its case is one of the best examples for this topic. Their e-commerce website was developed in-house but some of the components had been developed by a third party. The application had been thoroughly reviewed for security vulnerabilities and none had been identified as risky. However, one of the components was not reviewed, it was added a few days after the application review had been completed, and since it was not related in any way with payment transactions, it was deemed as non-critical.

The merchant had a network IDS which was maintained and monitored by a MSS (managed security services) vendor. The device had signatures that were able to recognize SQL injection attempts and they were supposedly enabled. One of the vendor’s security analysts disabled rules monitoring attacks on port 80 and 443 for the e-commerce servers. This was probably because they generated many false-positive alerts, and was most likely intended as a temporary action. As a result, none of the attacks and unusual traffic on those ports was detected by the IDS.

The e-commerce site was using a trusted relationship to connect to the database. Credit card numbers had been encrypted in the database a few months ago. During the process as a contingency plan, the DBA exported the tables containing sensitive data before encrypting some of the columns. The backup files had been left on the database server since then.

Hackers found the security vulnerability in the e-commerce website; the third-party component was vulnerable to SQL injection. By exploiting the vulnerability, they were able to create local administrator accounts on the database server and run OS commands with local administrator privileges. Unfortunately, since the IDS was not monitoring traffic on ports 90 and 443, none of the SQL probes was detected by the IDS, nor was any other unusual traffic on those ports. Remote management tools were installed and password hashes were cracked off-line. The hackers reviewed every folder on the web server looking for scripts, source code, and data files. They found the backup files left behind by the DBA.

The merchant was only aware of the intrusion several months after the fact, when they were notified by law enforcement agents that their data was on sale on one of the carders websites.

This case clearly illustrates that even when proper security controls are in place, a breach could happen at any moment. Relying on single controls or single layers of security is never sufficient.

The case also illustrates the need to assess security controls independently of any other surrounding security or other layers of security. QSAs and internal staff in charge of PCI DSS compliance should not consider risk-based discussions until all the security controls have been independently assessed.