Do you believe in the Red Shift theory?

The first time I heard the term Red Shift was from my portfolio company, Greenplum.  Greenplum has used red shift to characterize the nature of the existing database market where exponential data growth driven by network computing and internet applications have outstripped the capacity of existing mainstream vendors.  Hence, a new approach was needed (our database software running on commodity clusters) which would allow companies to load and query terabytes of data at 10-100x performance and scale over traditional vendors.  Ok-enough of the sales pitch.  Moving on, it is clear that red shift data requirements are only a fraction of what’s necessary to meet this exponential growth as it will put tremendous strain on the existing IT infrastructure consuming ever-increasing amounts of CPU cycles, energy, storage, and more.  If you want to read more about this red shift theory, I suggest checking out a great article by Richard Martin in Information Week.  Martin neatly summarizes Red Shift as defined by Sun’s Greg Papadopoulos to be:

• Red Shift refers to companies experiencing exponential growth in demand for raw computing power
• Red-shift companies tend to be Web 2.0 focused like YouTube and MySpace, or big financial, energy, or pharmaceutical companies
• Those companies, Sun CTO Greg Papadopoulos says, will experience similarly high levels of growth in users, revenue, etc., while blue-shift companies will grow relative to GDP
• Along with the cost of powering and cooling in-house data centers, the red shift is driving a surge in utility computing and software as a service

Based on my experience with both consumer Internet and companies selling infrastructure, I can say that this all feels right to me.  It is also no wonder that virtualization which helps IT consolidate servers and increase capacity utilization and utility computing are top of mind again.  Think about Amazon’s S3 and EC2 which I have written about before as utility storage and processing for the masses. I am definitely meeting more and more startups which are starting to offload some of their computing requirements to these services.  And of course, while Greg Papadopoulos is pushing this vision of the red shift, he has put Sun in a great spot to execute on this with new platforms and ways of keeping up with this exponential demand.  The only question as Mark Anderson points out in the article is not if there will be an exponential increase in servers sold but how many of them will be Sun servers running Solaris versus open systems.  Either way, it looks like the stock market has been voting with its feet as Sun has been performing quite well as of late.  And as a VC whether you believe in the red shift or not, we would all like to find companies experiencing hypergrowth where one of the main uses of capital will be for scaling the infrastructure to meet demand.  That is what I call a good problem to have.