Category: Web/Tech

In early 2006, I wrote a post titled "The next generation web, scaling and data mining will matter."  In it, I highlighted some thoughts on the future:

I truly believe the next battleground will be based on scaling the back end and more importantly mining all of that clickstream data to offer a better service to users.  Those that can do it cheaply and effectively will win.  The tools are getting more sophisticated, the data sizes are growing exponentially, and companies don't want to break the bank nor wait for Godot to deliver results.

Ok, clickstream data and data mining sound kind of geeky, but with every one of our calls, clicks, and purchases being tracked and logged it is an important topic.  My friend Scott Yara, co-founder and President of Greenplum (full disclosure-my fund is an investor), wrote an interesting post the other day which is more mainstream calling this ongoing revolution the "Your Data" revolution.  

The point here is that data can be a wonderful thing if used the right way and if controlled by us.  For example, Scott points out;

Your Data can lead you home with turn-by-turn directions on Mapquest. It can find you love by sorting through the profiles of 20 million other lonely hearts on eHarmony. It brings you up-to-the-second stock prices, sports scores, and flight delay alerts. It helps doctors fight diseases and engineers design safer cars. It gives environmentalists the power to track the movements of endangered animals and biologists the tools to map the structure of our genes.

Your Data, in short, is transforming everything.

However, with all of this data comes great responsibility and opportunity.  As Scott points out:

We also need to make sure we can use all the information we're collecting. That means better schools that will turn out kids who are able to cope with the age of Your Data. And we need better, cheaper technologies to enable companies of all sizes, as well as organizations and individuals, to get all the information they want and do something useful with it.

Knowledge is power, and we know more than any previous generation could even conceive. We're moving into a world of infinite information. The challenge we face is turning all that information into insights, conclusions, and revelations — in other words, turning that knowledge into wisdom, without letting it be turned against us. We need to make sure Your Data doesn't oppress us, but serves us. And we need to do that fast, because the revolution is well underway.

From a VC and entrepreneurial perspective, what excites me is that we are just scratching the surface of what to do with all of this data and how to turn it into actionable, meaningful insight.  In order to make data and insight more accessible to everyone we first need the back-end technology that makes data storage and analysis better, faster, cheaper (enter companies like Greenplum-ok, shameless plug 🙂 ).  We then need great entrepreneurs to continue to build new services that help end users seamlessly and implicitly help everyone make better decisions, discover new things, and empower and motivate us to do more.  In addition, we also need to consider cultural factors.  For example while privacy still needs to be at the forefront of the Your Data revolution, we also need the ability and power to choose what we want to share and when with the world.  Little did we know that four years ago, more people than ever would be willing to share their whereabouts through services like Loopt or Foursquare or Twitter and their every thought through Facebook or even their credit card purchasing data through new services like Blippy.  It is clear that the once sacred walls between private and public information are increasingly disintegrating based on these cultural factors. While we clearly have to be careful not to extrapolate too much from early successes like Blippy and Foursquare, we also cannot underestimate the power of these cultural factors as once young start-ups like Facebook and Twitter have exploded in growth.  The question is who will create the next great back-end technologies and new web services that drive a whole new conversation and new way of thinking about what we do with the data that is around everywhere.

I had lunch with a friend last week when we were talking about the days years and years ago where it was cool to have an email address on your business card.  In fact, I remember picking attorneys to work on our venture deals in the mid-90s not only based on cost and experience but also based on how digital they were – no AOL email addresses please and if you use IM, then great.  Now I can honestly say that I can be overburdened at times dealing with my email, IMs, sms messages, phone calls, LinkedIn and Facebook messages.  So I must say it was quite refreshing last week when I received a hand delivered note from Robert Samet who runs Madison Search Partners, a well respected boutique search firm for senior level sales searches in the digital media and software sectors.  He, of course, had sent me a few emails before that and also followed up with an email afterwards.  Robert went old school with snail mail and physical communcations and with that got my attention.  Yes this is an old marketing trick but one that sometimes gets lost in the shuffle of digital communcation.  I, of course, had to take his call and when we spoke I asked him how his campaign went.  His hit rate was quite high and given his creativity, he is definitely a guy I want to use in the future for a search.  So in this day of constant and immediate communication, physical mail and snail mail can sometimes leave a lasting impression.  As for myself, I actually got some personal stationary last year to send note cards to friends and business contacts when I want to make sure that I deliver a more effective message.

I had some interesting meetings yesterday and as I reflected on them this morning, one common theme emerged which is that the next generation of the web will be built on data mining and extracting intelligence from the reams of data web services collect on a daily basis.  This reminds me of a post I made in March of 2006 titled "The Next Generation Web – scaling and data mining will matter" where I mention:

I truly believe the next battleground will be based on scaling the back end and more importantly mining all of that clickstream data to offer a better service to users.  Those that can do it cheaply and effectively will win.  The tools are getting more sophisticated, the data sizes are growing exponentially, and companies don’t want to break the bank nor wait for Godot to deliver results.

My first meeting was with a well known research analyst covering Internet stocks.  While we discussed the usual topics such as how the Internet was taking share from traditional advertising budgets and how the top brand advertisers have not really embraced the web yet, our most lively discussion centered around next generation advertising technology which all centered around increasingly complex forms of data analysis.  To that end, I mentioned one of the fund’s portfolio companies, Peer39, which is using natural language processing and machine learning to create highly precise matching of commercial offers and user generated content.  As you might guess, the secret sauce is the algorithms that the company has created.

Later in the day I had lunch with a friend who we had funded years ago.  What was interesting to hear was how many of the future product lines that we discussed a few years ago were finally starting to emerge as real revenue drivers for the business today.  Years ago the company’s first data center cost around $20mm and the latest one which has orders of magniture more customers cost only $3mm.  Clearly, any data-driven opportunities a few years ago were cost prohibitive in the first place and too early for the customer to understand in the second place.  That was the case because many businesses were just worried about not getting Amazoned and today they are all on the web thinking about how to drive better results.  That is why our discussion led to a massive data warehousing project his company was working on to take all of that data across his huge customer base and to help them better monetize their sites.

What I love about these kinds of opportunities is that algorithms scale, have high gross margins, and are proprietary and defensible.  The next generation web is not about what you click and see but what is happening behind the scenes every time you click on a page and move from site to site.

I am helping a friend who has the most popular books on baby naming bring it all online. Our development on the backend is close to complete, and we are looking for a great web designer who can create a few templates and themes for us. This is a database driven site and the homepage will include some social elements in it as well. If interested please send me a note with some samples of your sites you helped design. Simple sites we like include dictionary.com, urbanbaby.com, apple.com, and deliciousdays.com.

As you know, I am a huge believer in open standards and that open standards (over time) will usually prevail over proprietary, closed networks.  And my one wish from a social networking perspective was to really be able to manage all of my relationships from various networks and my interests from one meta-application.  In June this year I wrote about Linkedin and Facebook on a possible collision course.  And in November I wrote about the promise and potential shortcomings of OpenSocial:

OpenSocial is like Java for social networking apps-the promise of write once, run anywhere.  It goes back to my point I made in an earlier blog post – I am completely inundated now from requests from Facebook, LinkedIn, and now PlaxoPulse.  I am having a hard time keeping track of all of my contacts, messages, and the like.  It would be great if I could have a service that sat on top of these apps and allowed me to manage all of my relationships from one place.  Sure, some contacts may only be a Linked in contact, some may be a Facebook and LinkedIn, etc.  Check here if you want your music to be shared on this network and not the other one, etc.  You get the idea.

Of course, I thought that I was dreaming and that it wouldn’t happen anytime soon until I read the announcement today that Facebook, Google, and Plaxo joined the Data Portability Group (see Read/Write/Web).

The DataPortability Workgroup announced this morning that representatives from both Google and Facebook are joining its ranks. The group is working on a variety of projects to foster an era of Data Portability – where users can take their data from the websites they use to reuse elsewhere and where vendors can leverage safe cross-site data exchange for a whole new level of innovation. Good bye customer lock-in, hello to new privacy challenges. If things go right, today could be a very important day in the history of the internet.

The proof will be in the pudding and in the implemenation as it is with Google’s OpenSocial. That being said, this is a great move by Facebook, stemming the negative tide that was building about who owned their data and also, in my mind, locking them in as a defacto leader for years to come.  Facebook is the current gorilla in the space and gorillas do what they want.  However, rather than take on every other player who campaigned on the "open" platform, Facebook has thrown its hat into the ring, and I am sure will play a major role in helping make the standards as well.  Let’s just hope our data is really portable and only when users can run their social networks and share their data from one or any platform easily will we truly be in an open market.  That being said I agree with Marshall that this could be an important day for the Internet, one where the consumer’s voice truly carries weight and one where openness will prevail.

Tim O’Reilly has a great post on Google’s OpenSocial.  At the end of the day, I couldn’t agree more with Tim’s thoughts that OpenSocial is great for developers but a who cares for users. 

If all OpenSocial does is allow developers to port their applications more easily from one social network to another, that’s a big win for the developer, as they get to shop their application to users of every participating social network. But it provides little incremental value to the user, the real target. We don’t want to have the same application on multiple social networks. We want applications that can use data from multiple social networks.

Would OpenSocial let developers build a personal CRM system, a console where I could manage my social network, exporting friends lists to various social networks? No. Would OpenSocial let developers build a social search application like the one that Mark Cuban was looking for?  No.

I agree Tim.  OpenSocial is like Java for social networking apps-the promise of write once, run anywhere.  It goes back to my point I made in an earlier blog post – I am completely inundated now from requests from Facebook, LinkedIn, and now PlaxoPulse.  I am having a hard time keeping track of all of my contacts, messages, and the like.  It would be great if I could have a service that sat on top of these apps and allowed me to manage all of my relationships from one place.  Sure, some contacts may only be a Linked in contact, some may be a Facebook and LinkedIn, etc.  Check here if you want your music to be shared on this network and not the other one, etc.  You get the idea.  It is not hard to view this data in one place by sucking in RSS feeds from the various services but viewing it in one place vs. managing all of my relationships from one place are two different value propositions.  Jeff Nolan has a recent post about this as well.  Of course the challenge is that the value of these services is their proprietary networks which creates lock-in for the user.  Once users can export and manage that data and without visiting these various platforms then the service begins to lose its lock-in. We see this problem over and over again in many web services – the constant battle between closed and open standards and networks.  If you are the big guy, why bother.  If you are the small guy, it makes sense to join up with many of the other smaller players.  Anyway, enough digression here – I would love to hear your thoughts about how you are spending your time managing your various relationships across different networks and what you would like to see.

I am definitely the first one to understand that there is no free lunch on the web.  At the end of the day, someone has to pay for all of the great services and content out there.  To boot, I am a big believer in the ad-driven model of content.  To this point, there is a battle being fought at every web content company on a daily basis between product manangement, engineering, and ad sales.  What is clear on the web is that every little change can make a huge impact in terms of usability, traffic, and revenue.  The more you err on one side vs. the other can help companies make or break their numbers.  It is this battle between usability (simple and clean) vs. revenue (balance between getting what you want vs. being cluttered) that is constantly fought behind the scenes. 

Take Forbes.com as an example.  I have always liked the content but over the last 6 months I have basically stopped going to their site or any link that someone sends me from Forbes.  Why?  I cannot stand the in-your-face advertising and the clutter.  First, it starts with a big-ass splash page before getting you to the site and once there a Forbes.com video clip starts with a pre-roll ad.  Once I click on another page, I am confronted with another video ad that starts right away.  Once again, I like the writers but honestly this site has become too revenue focused and consequently too cluttered.  As a user, I feel like I am spending more time dealing with turning off video and audio ads and skipping splash screens rather than reading content.  I am sure the Forbes.com business folks have done their analysis between lost unique visitors versus more revenue per page, but in the long run striking the right balance between usability and revenue is key.  And as I sit down with my portfolio companies, it is also this balance that we all seek to achieve because we understand that what we may gain in short-term revenue increases may hurt us in the long run if our audience base declines over time.