Chris Lunt is currently the CEO of Nombray, a startup that helps you to own your name on the web. Simply...Get a domain, collect your content, control your destiny. In the following guest post, he elaborates on some of his experiences at Friendster during it's meteoric growth. Read on for his facinating thoughts around Customer Development and the value of "Test, Rinse and Repeat".
Friendster was the first modern social network to gain a large audience, very quickly growing to 3 million registrations in the summer of 2003. Jonathan Abrams hired me to help with the scaling issues that appeared with such rapid growth. Beyond the scaling issues, there was a broader question of "what is this site for?" Jonathan had conceived of the site in a dating context, but his VCs encouraged him to think more broadly. What followed was both a tragedy and a comedy. And I share the blame and the acclaim for the strange journey Friendster began at that time.
Jonathan had a grand vision of the site as a portal that would frame all of your web interactions, encompassing music, instant messaging, blogging, etc. He laid these ideas out immediately after the 2004 New Year's Day. At the time, the traffic to the site was completely performance bound, and Jeff Winner (the VP of Engineering) called for a complete rewrite of the site. As we began that exercise, the question was raised, how do we prioritize these new features? Which begs the question, what on earth are people doing on the site now?
Friendster had the advantage that most small startups don't have: traffic. Blogs everywhere extoll the virtues of A/B testing. Which ad is better? Just run them both and measure! That's great if you're at Google, or you're running a $10,000 ad campaign, but here's the uncomfortable truth for most of you reading this:
You do not have enough traffic to A/B test
Statistical significance happens when you have scores of people signing up for your product. In the very beginning, you're going to have to rely on something else. Direct interaction with people. You can run web surveys, you can run focus groups. You can do user testing in a lab. At my next startup, WisdomArk, we did exactly that. The only trouble there is:
The plural of anecdote is not data
"One of the users in the focus group said that they'd pay a lot of money for our service!" You'll find that people say they'll do things that they won't follow through on (run a consumer product long enough, and you get a very dim view of the human attention span). And you'll find that the very process of finding people who are willing to give you feedback colors the results you get.
Regardless of the scale of use you have, by all means use the data you have. You have 100 active users? Talk to them directly! Narrow in on a couple of key questions. I'll hazard that this is true:
The seed to success is within your first 100 active users
Rarely do companies get it right on the first try with their product, but the really rewarding problem to solve is probably known by one of your early users, who came to your product in the hope of solving it.
When I tried recruiting people in 2003, engineers would say, "People tried this before, what changed that's making it work now?" It took me a long time to figure that out. I created a visualization of our traffic that made it apparent. I took all the data for a specific day of traffic, and for each page on the service, I drew a circle, sized proportionally to the amount of hits it had. Then I drew arrows between the circles, showing where people went from one page to another. The thickness of the arrow matched the amount of traffic. When I laid it out, it was apparent what people were doing. They came to the site, went to their friends' pages, and then looked at all their pictures. People were coming to the site, to look at pictures of their friends.
At the time, we allowed people to have 3 pictures on the site. I went to the CEO, and said, "make it 6". Immediately, our traffic went up. So we raised it to 12, and then finally made it unlimited. (For those of you who don't understand why we didn't just go directly to unlimited, remember at the time that even eBay didn't let people store photos--the cost of disk storage was just starting to plummet, and it was becoming economically viable for the first time. Myspace didn't even do their own photo storage, they dealt with PhotoBucket.) There's another key lesson here. Figure out a cheap and fast way to figure out if your hypothesis has merit. If it does, keep doubling down:
Find a small test, and if it works, iterate until it doesn't
I then did some digging around, and realized why Friendster (and more broadly, social networking) took off in 2003. 2003 was the first year that digital cameras outsold analog cameras. Social Networking is a phenomenon triggered by a new technology: cheap digital cameras. How much of Facebook's traffic today still centers around photos? (As an aside, this also was an omen for the then coming irrelevance of Friendster: photo pages do not monetize well.)
This was all important because we always felt that the people search was the most important feature we had, and it's the one we spent most of our resources on--it's the feature our brightest engineers worked on. You could search Friendster for "Women, between the ages of 25 and 30, within 2 degrees of me, and within 25 miles of my house". That's great when you're a dating site, but what if people are really just at your site to ogle all the pretty people, and see the silly pictures their friends had put up? Did we blow our opportunity by working on the wrong problem? (I have my own argument about that: Friendster succeeded, but with the wrong audience, but that's a story for another time.)
The whole exercise was a positive example of Friendster working well as a company.
1. Ask the right question. "What feature is really drawing people to the site?"
2. Get the data. "Time to parse the logs."
3. Synthesize the data. "How do I understand this? Can I turn it into a single diagram?"
4. Draw the conclusion. "People really come here to look at photos."
5. Make the change. "Let's offer 6 photos."
6. Measure and repeat. "Hey, that boosted traffic, let's offer 12!"
