Demolishing the Bellman Rule
Jinfo Blog

14th September 2008

By Tim Buckley Owen

Item

‘What I tell you three times is true’ was the rule of the Bellman in Lewis Carroll’s nonsense poem The Hunting of the Snark – and it used to be a good principle for researchers to follow. But the speed with which errors can be replicated on the web has changed all that, and nowhere more spectacularly than in the recent case of the search engine and the airline. Basically the facts are these: In 2002 the Chicago Tribune reported correctly that United Airlines had filed for bankruptcy, and its sister paper the South Florida Sun Sentinel also reported the story at the time. But in the small hours of Sunday 7 September 2008 an apparently small amount of traffic to the six year old story became enough to push it onto the Sun Sentinel’s ‘most viewed’ list. A passing Google News spider then picked it up and, according to Google http://digbig.com/4xmjm the only date it could find indicated that the article was current. From there it travelled to Income Securities Advisors Inc, which provides http://www.incomesecurities.com/bloom_index.cfm news of ‘distressed’ companies to Bloomberg. Bloomberg reported http://digbig.com/4xmjn that it then also published its own headline on the basis of its subcontractor’s incorrect information. The erroneous reports were corrected in minutes – but not before the markets had reacted, wiping 76% off the value of UA’s shares. Why did it happen? One accusing finger points at the spider, which didn’t realize the story was old – in fact, the Dow Jones subsidiary MarketWatch http://digbig.com/4xmjp even quoted Ed Keating, vice president of the Software Information Industry Association, as acknowledging that ‘it’s just an algorithm, and the algorithm did not go to journalism school’. We’ve looked at algorithms before on LiveWire, including http://www.vivavip.com/go/e5233 programs that attempt to act on text-based intelligence as well as on figures. However in this instance, after the initial automated error, all the subsequent errors were apparently human ones; could an algorithm have done any better? Chicago Tribune spokesman Gary Weitman told Associated Press http://digbig.com/4xmjq that the discrepancy in the UA share prices – 97 cents in the 2002 story as opposed to the around $12 at which the shares were actually trading when the error occurred – should have been sufficient to arouse a human reader’s suspicions. In theory, a text analysis algorithm should have been even quicker at spotting the discrepancy, pulling the story while it checked its accuracy. So, with hindsight, the warning signs should have been there. But put your hand on your heart and admit as a responsible researcher that, if you’d been told something three times – by the world’s biggest search engine, by a professional analyst and by a top financial news service – wouldn’t you assume that it was true?

« Blog