FreePint Report: Monitoring and Mining Social Media and News
Jinfo Blog

By Aileen Marshall

Abstract

The rise of Web 2.0 has opened virtual gold mines of new forms of data that can be mined and analysed for an organisation’s purposes. The days when almost all the sources analysts could rely on were one-way sources, from the creator to the consumer, are over. The articles in this report examine the implications and impacts of this change on organisations, analysts and their work.

Item

The rise of Web 2.0 has opened virtual gold mines of new forms of data that can be mined and analysed for an organisation’s purposes. The days when almost all the sources analysts could rely on were one-way sources, from the creator to the consumer, are over.

Social media in all its different forms has given consumers a voice that is almost instantaneous and unfiltered. What was once defined as news (newspaper, TV, radio) has no monopoly anymore over this domain. The traditional content providers are no longer the only ones in charge - though they attempt to keep control through mechanisms like paywalls. News, once controlled and carefully orchestrated, is now in the hands of consumers. This shift makes is oftentimes unpredictable in which direction(s)  a message will spread.

With so many unpredictable variables, these new news sources will require that the methods or approach to monitoring and mining will have to change. Spotting, collecting and manipulating data from these two different sources are more connected and complementary than it may seem at first. Content from social media, for all intent and purposes, is news, and it usually spreads faster than content from traditional sources. Broadcasts happening within the social community are the real news, whereas newspapers, TV and radio stations provide more of a digest, and sometimes analysis. 

The distinction between news and social media is disappearing. The one-way flow of information versus information that spreads uncontrolled and unpredictably characterise two forms of content that seem to be vastly different. Maybe there will always be a formal boundary between them. However, we cannot – and should not - ignore the evolution that is going on right before our eyes. It is exciting to be a part of this evolution, and we should make every effort to learn from it, and help shape it.

The articles in this report examine the implications and impacts of this change on organisations, analysts and their work.

A big challenge is weeding through different social media and content platforms. Several authors discuss software that aggregates different sources and makes the data stream manageable. In his product review of NewsDesk, Scott Brown points out that this software can track traditional and print as well as social sources. Penny Crossland and Dale Moore introduce similar software such as NewsWhip, Silobreaker Premium and Newsedge.com. Developers of these programs are aware of the change that resources are undergoing.

Nonetheless, Crossland notes that aggregation software may not always provide reliable results, and should still be used with some caution. Tim Buckley Owen illustrates how sentiment analysis is still in its early stages. Misspellings, slang, sarcasm and other characteristics of natural language online make it difficult for software to properly aggregate data. Brown punctuates Buckley Owen’s point by providing a general overview of Opfine and how useful it is for sentiment analysis.

Finally, Buckley Owen offers an approach on how to determine which software is best suited for your needs and the work in which you are engaged. The market for these programs is expanding rapidly, software development is booming, and new vendors and sources constantly appear on the scene. These factors will make the process of choosing the right tool difficult. But this is also part of the evolution of data mining and analysis, and takes us on the exciting journey of our tools and our profession.

« Blog