In-Depth

The Good and the Bad of MBSA

Microsoft's free vulnerability scanner works well—as long as you don't have to stretch it too far.

• By Joanne Cummings
• 05/01/2005

MBSA does have a lot going for it. In addition to being free, it's a simple vulnerability scanner that's easy to use and configure, most users say. The latest version (1.2.1) checks for configuration errors and security holes not only in Windows 2000, XP and Windows Server 2003, but also key Microsoft applications like Office, IIS, SQL Server and Internet Explorer.

"At first, I used MBSA quite a bit," says Ben Hearn, systems administrator at Cincinnati, Ohio-based financial services firm GAFRI. Hearn is responsible for managing more than 1,200 Windows XP servers. "I've really gotten away from using it at all now because it just proves to be too cumbersome when you're dealing with lots of machines."

Hearn's primary complaint is the lack of flexible reporting capabilities or any sort of standard report formatting. "MBSA can scan an entire domain of 1,200 computers, but then it generates one giant list of results," he says. "There's no good built-in way to see the percentage of my machines that are missing patches."

MBSA scans every computer within an organization and returns a full list of items. Those items designated with a green check are checked out as secure. Others are flagged for remediation. That's about as deep as MBSA's reporting goes, and it's not deep enough for most users. "It just takes too long to try and decipher the list," says Justin Clutter, CIO of Appserve Technologies LLC, a small hosting services provider based in Dallas, Texas. "Most of the time, you'll get the little green check back, but what I really want to see are the critical issues that need fixing."

Clutter says he wishes the MBSA reports were integrated with something like SQL, so he could import the scan results into a database and make it easier for users to run exception reports.

"Integration with SQL would be great," agrees Jeff Hinrichs, technical lead at Dermatological Lab and Supply Co., in Council Bluffs, Iowa. He also agrees that MBSA's reporting is its weakest feature. "What I want it to do is throw flags to show me what's different. Right now, it can't do that for me." Hinrichs has built his own workaround so he can sort through MBSA's XML-based results to better understand the most critical issues. He takes the newest scan results and the results he has saved from the last time he ran an MBSA scan. "I take both XML files and flatten them," he says. "Then, I run a standard DIFF tool on it to find the differences between the two files."

Without this extra step, Hinrichs says it's difficult to see what has changed and what needs his immediate attention. "Maybe 90 percent of my machines are updated for this patch, but that means there are 10 percent that didn't take it and that's what I need to know about."

Questionable Results
Another thing users have noticed is that MBSA's reported vulnerabilities don't always match those reported by other tools, like Windows Update and Windows Software Update Services (WSUS). "When I use MBSA to scan one of my servers, it comes back saying that four critical updates could not be verified or need to be updated," Clutter says. "But when I go to the WSUS site, it says the server is completely up to date."

In most cases, this is because Windows Update focuses on OS updates, whereas MBSA also checks for application-level vulnerabilities like those found in Office and IIS. "They work off different databases at Microsoft, so that's why you get the conflicting results," Clutter explains.

However, some cases aren't quite as clear-cut. Stephen Olson, owner of SJO Computer Services in Millerstown, Pa., says he often receives MBSA scan results that are less than definitive. "I just ran a scan and it told me that it couldn't verify whether I needed a certain update," he says. "It turned out that it was an update for Windows Media Player 9, but we had already upgraded to Windows Media Player 10. MBSA couldn't tell that and so it was flagged as a possible vulnerability."

The problem, Olson says, is that there's no way to configure MBSA so it doesn't flag those types of issues. "It just keeps reporting it every time I do a scan, which can be a pain," he says.

In other cases, MBSA will report that it is unsure whether or not a patch has been installed on a scanned machine, an event that Hinrichs attributes to Microsoft's less-than-linear patch naming policy. "MSBA should be able to look at the version number of a DLL and tell you whether the patch is installed or not," Hinrichs says. "If you install a patch from Microsoft, but Microsoft can't detect that it's installed, well that's a problem."

Although Microsoft says you can use MBSA across a network and multiple domains, most users say its network support is not a strong suit. For example, MBSA can scan Office for vulnerabilities, but you need to do the scans from a local machine, not via a network. "That's really annoying," says GAFRI's Hearn. "I'm not about to physically go to each machine. It's almost a tease."

"[MBSA] is good in security issues, like making sure IE or the IIS server is set properly." Justin Clutter, CIO AppServe Technologies

Similarly, users needing to scan multiple servers across domains can run into password issues. "If you try to run an MBSA scan across two domains where the admin user name and password aren't the same—which technically, they shouldn't be—it doesn't work," Clutter says. "There's no way to designate that the two domains use different passwords, so you end up having to scan them separately."

Smaller Is Better
There is good news for MBSA. Those who use MBSA to scan single computers or smaller environments give the tool high marks for its comprehensive scanning and ease of use.

SJO's Olson uses it to support his clients, which are primarily one-person, small or home office environments. "It's a great tool," he says. "It doesn't do anything that I couldn't do manually, but it's very easy to run and it's nice to have this little report come out."

Olson says he uses the MBSA reports to give his customers peace of mind. "They can look and see that their computer has strong security, according to Microsoft, and it gives them a good feeling."

Because Olson runs MBSA on single computers, the tool's reporting capabilities are more than adequate for his needs. Plus, he says, MBSA is reliable. "The thing has run flawless every time," he says. "It's definitely a comprehensive and easy way to keep your Microsoft computer updated."

Going Beyond the OS
Others say MBSA's biggest asset is its ability to go beyond the OS to ferret out holes in various applications. "It's good in security issues, like making sure IE or the IIS server is set properly," Clutter says. "I use it to make sure that I have everything locked down."

This helps Clutter ensure his servers won't be easily hacked. "If somebody hacks into one of my machines and decides to install the FTP service on my domain controller, I can run this utility and see that right away," he says. "It lets you spot application-level things like that quickly."

Brendan O'Connor agrees. As the network and systems administrator for the William Floyd School District in Mastic Beach, N.Y., he uses MBSA to lock down every machine before it enters the school network. "It's one of the steps we take when we create an image now," he explains. "We put on Windows, all the Service Pack updates and all the Office applications before it goes out the door, but then we run MBSA to make sure we haven't missed anything," he explains. "It's a good baseline tool, and it's free, so you really can't complain too much."