Computers predict sentiment - yeah, right
Jinfo Blog

By Tim Buckley Owen

Item

If lots of people say it does that make it true? As Twitter moves up another gear, that’s the premise behind one tweet analysis algorithm – but as recent troubles with Google Places demonstrate, crowdsourced intelligence can also be subverted.

Just five years after its first tentative tweet, one hundred million people now use Twitter actively, a recent blog proclaims. Half of them use it every day – although 40% of those active users log in not to tweet but simply to find out what’s going on.

Some commentators believe that Twitter’s current trumpeting of its usage is part of its attempt to ramp up its advertising business (try PC World or Computer Weekly for example) following last year’s launch of its Promoted Tweets (2010 LiveWire coverage encompasses Twitter’s importance for business, use of analytics and scale versus stability). Now the Financial Times reports that, with a reputation for having neglected the development of its business in the early days, Twitter acknowledges that it will always largely be dependent on advertising.

That doesn’t prevent others from trying to make shed-loads of money out of tweets. Earlier this year, Derwent Capital Markets launched what it claimed was Europe’s first social media-based hedge fund, using an algorithm developed by Johan Bollen of Indiana University Bloomington, which analyses the sentiment behind millions of tweets to predict – and bet on – stock market movements.

Explaining the process on the BBC’s Today programme recently, Dr Bollen also cited other studies which showed similar algorithms working to predict flu rates and box office receipts. The possibilities seem endless – so what could go wrong?

Well the Economist is worried about sarcasm, for one thing, which computers still have difficulty recognising – although they are getting better at it. Results from social network analysis are variable at the moment, it seems, although improving all the time as people worry less about sharing their intimate personal details with market researchers.

But another threat could conceivably come from deliberate subversion. Take Google Places, for example, which has recently had to apologise to numerous business owners following the placing of "Permanently Closed" labels against their entries – apparently for nefarious purposes.

Google has been working hastily on a fix. But up to now anyone has been able to stick such a label on the business of (say) a competitor, which presumably stays there until the horrified business owner discovers what’s been going on.

That’s one-off villainy of course, not collective delusion – but social media are by their nature viral, and sentiment that misreads a situation will spread just as quickly as hunches that prove to be correct. In the circumstances, perhaps keeping a critical eye on Derwent’s longer term performance mightn’t be a bad idea.

« Blog