Study Says Anti-Phishing Toolbars Work -- If You Do

CHI 2006's last presentation on security contained a mixture of good and bad news. Rob Campbell (who was at pains to say that he was only presenting the work of his student Min Wu, who couldn't make it) gave us details on a study that was conducted to see if browser toolbars designed to thwart phisher attacks actually worked.

The MIT study's scenario went something like this: thirty people were given the role of a personal assistant who had to shop for certain items, based on instructions and URLs sent by their employer via e-mail. Their job was to keep their boss happy and, while they were at it, not let his personal information get stolen. As in the Harvard study I mentioned, they were warned ahead of time that the people running the study would be trying to fool them.

The good news is that the anti-phishing toolbars used in the study worked perfectly, correctly identifying fake sites. The bad news is that the people in the study often ignored the toolbars' warnings. (The really bad news: 20 of the participants were MIT students.) Why ignore them? It turns out that many of the MIT study's findings corroborated those from the Harvard study: faith in snazzy-looking sites, an inability to properly parse URLs, and so on. But the most interesting reasons were the rationalizations users came up with, especially this nugget: one person felt the toolbar's threat assessment was probably inaccurate because her e-mail's spam filter regularly reports false positives.

My curiosity is now officially piqued. Do you use anti-phishing utilities? If so, do you believe what they tell you? And do you have similar stories?