Google announced on its main blog and the Google Webmaster Central blog that it has created and implemented a new search index it likes to call Caffeine. The new index moves away from a layered approach to a faster refreshing index that can add new or newly discovered items into search results faster.
So why did we build a new search indexing system? Content on the web is blossoming. It’s growing not just in size and numbers but with the advent of video, images, news and real-time updates, the average webpage is richer and more complex. In addition, people’s expectations for search are higher than they used to be. Searchers want to find the latest relevant content and publishers expect to be found the instant they publish.
The Search Index is now updated in a continuous fashion with hundreds of thousands of pages being processed simultaneously and “adds new information at a rate of hundreds of thousands of gigabytes per day.”
Wow! Let’s hope that this change brings about better search results and not a more easily gamed or more fruitful source for SEO poison bots and other tricks.
You can read the official Google announcement at http://googlewebmastercentral.blogspot.com/2010/06/our-new-search-index-caffeine.html