Google Caffeine stimulates search
Jinfo Blog

11th June 2010

By Penny Crossland

Item

When we search Google, we tend to assume that we are searching the most of the web universe. In fact, as many info pros point out, Google only provides us with a fraction of information available on the world wide web. It actually goes through its own search index, which until the introduction of Google Caffeine was structured in layers, some of which were only updated every two weeks. This obviously led to a significant delay between content being published and appearing on the search engine. With social networking sites such as Twitter and Facebook becoming increasingly important in disseminating information and turning into search engines themselves and bloggers providing near real-time news and commentary, no wonder that Google realized that its indexing system was in need of an upgrade. In addition, different types of media such as videos and images are taking up more and space on the web. Coupled with users’ increased expectations in terms of speed and currency, Google’s old indexing system was not up to scratch. According to Google (http://digbig.com/5bbsjd), the new indexing system, which was due to be launched at the end of last year, analyses the web in smaller chunks and is able to update its search index continuously and faster. New information is added to the index as soon as it is found. With Google Caffeine, the company is promising 50% fresher results than under the previous layered approach to indexing. This is certainly good news for business researchers who rely on up-to-date information. Googles accompanying press release (http://digbig.com/5bbsjf) provides some eye watering statistics : * every second Caffeine processes hundreds of thousands of pages in parallel -if this were a pile of paper it would grow three miles taller every second * Caffeine takes up nearly 100 million gigabytes of storage in one database and adds new information at a rate of hundreds of thousands of gigabytes per day * You would need 625,000 of the largest iPods to store that much information; if these were stacked end-to-end they would go for more than 40 miles Caffeine is up and running, so it will be interesting to see if we can spot the difference.

« Blog