The Lean Startup, Video Interview, Eric Ries, Part 1

Interview with Eric Ries Part 1

Interviewer: So, who are you?

Eric: I am Eric Ries. I write the blog Startup Lessons Learned.

Interviewer: Yeah. So, I wanted to meet with you because you’re doing an event and you are getting known, but the way startups get formed is changing, isn’t it?

Eric: Yeah. I think it really is the most exciting time in history to be an entrepreneur. There is this trend in industry after industry where, as software kind of effects what people are doing, it spreads an incredible amount of uncertainty, disruption. People thought they had stable industries with solid barriers to entry and all that stuff you read in economics textbooks. That made sense because there were physical constraints in what could be done, and software has this unbelievable quality that it’s, it’s like the direct translation of imagination into tangible form and so it has this really weird non-linear effect. When I meet with people who are in larger companies, that’s terrifying but for entrepreneurs, it’s just opportunity, opportunity, opportunity. So that’s very exciting.

Interviewer: First of all, tell me what the Lean Startup is? Then we’ll talk about why it resonates with so many people.

Eric: Yeah. Thanks for saying that. I was an entrepreneur for about ten years and what I saw in company after company, starting with the dot com bubble but then even beyond in Silicon Valley especially, where people building companies with way too much money, way too many resources and a really premature launch, so that if the product was slightly off or more likely, if the product was something that basically people didn’t want, the company didn’t find out until it was way too late. So I have had friends, the most talented people I know, spending years of their lives building something that, fundamentally, nobody wanted.

You think about that in context of entrepreneurship. We are supposed to be taking big risks and we are supposed to be taking on incredible challenges and yet, when we have companies fail, not because the technology didn’t work but as Steve Blank likes to say, because there were no customers for it at the end, that’s a preventable condition. That’s an inappropriate reason for companies to fail and so when I started to work with companies, after I left my last company, started to write this blog, I was looking for how do we create a theory that explains why some companies succeed, why some companies don’t and, more importantly, can help entrepreneurs make good decisions about how to test ideas and how to prevent that kind of premature mortality of their company.

That’s what eventually led me to this thing called the Lean Startup, which is an application of lean manufacturing principles to the process of innovation itself. The basic ideas are to test ideas early, cheaply and with customers way in advance of having built some new technology and then to get into a learning feedback loop that’s company wide and kind of is continuously testing new ideas so that the idea failure doesn’t have to equal a company failure.

Interviewer: Interesting. As an entrepreneur, if you had the impulse to start a company right now, what would be the first thing you would do to set yourself up for success and set yourself up for a Lean Startup?

Eric: We usually start thinking about a company because we are thinking about a particular new product or new technology. We have a vision for how that could change the world. The Lean Startup doesn’t change that. Vision is still the most important thing for an entrepreneur to hold but what we are going to do in a Lean Startup, as opposed to the more traditional models, instead of immediately raising money and hiring some engineers and starting to build, we are going to begin with testing elements of that vision with customers as soon as possible.

What we do is we just split the company into two. There’s one thing, what we used to call sales, marketing and business, we now call the problem team, which is the part of the company dedicated to figuring out what problem are we really trying to solve for customers because we wrote a business plan, we think we know what problems customers have but if you look at the stories of how companies actually evolve, they do this thing we call the pivot. They wind up solving the wrong problem or having the wrong solution to that problem and they wind up changing course in this kind of zig zaggy path that ultimately leads them to some kind of success.

We got one team that is really focused on talking to customers. We have beginning, just like who are they? Do they have a problem? Who are the early adopters? What is the business model? Those kind of questions, in parallel with what we used to call engineering, ops and QA. We now call the solution team. These are setup people trying to build this thing we call the minimum viable product. Not how many features can we cram into a certain amount of time but what is the absolute minimum amount of work needed to be done to test out the first element of the product’s vision.

Interviewer: I think the best practitioner of this was Evan Williams at Twitter.

Eric: It’s amazing.

Interviewer: He not only constrained the product, it had no features. The thing that was cool about it was it had less features than what had come off. You can only write 140. It was a really a buy-in tool that you could only write 140 characters into it, right?

Eric: Yeah and I am sure now, of course, they knew all along that that limit was actually going to be a really, really good idea but at that time, at least as I understand it, that was a constraint of the technology. They said well let’s just accept this constrain for now while we work on what else we can do with this SMS sharing thing. They were surprised at how powerful that constraint turned out to be, which is one of the really cool things about minimum viable product. You see this, you see it in Twitter, in Facebook, in Google. Sometimes the minimum version of a product is actually better than the maximum one. There’s fewer features are often more powerful than bigger features. It’s not just that. Visionaries, my experience is they are really afraid of exposing to the world a tiny piece of their vision because they are worried that it will be rejected prematurely but if we have a bit of humility and a little courage, what we can find as entrepreneurs is sometimes we discover new opportunities that come out of the kind of simplicity and extreme constraints of actually having customers interact with a minimum version of our product.

Interviewer: When I met with the engineering team at Twitter early on, they admitted that they didn’t engineer it for very big scale, right? In other words, they didn’t overbuild it.

Eric: That’s right.

Interviewer: Talk about how you would engineer products, and FriendFeed talks about this too, that they would engineer for the next 90 days. They wouldn’t try to engineer for three years from now.

Eric: That’s very, very important.

Interviewer: Because they could only see 90 days with usage anyways, so are you going to prepare for 90 million users that are never going to show up and never did in their case?

Eric: Yeah that’s right. Oh god, that’s the worst.

Interviewer: If they had spent all that time engineering for 90 million users, they would have wasted all that effort because they got bought by Facebook and now they are not even allowed to work at that product.

Eric: I am not going to comment on their situation. I don’t know anything about that, but we call it the curse of prevention, that when you anticipate a problem too much and you try to prematurely solve it, not only do you waste a lot of time solving a problem that might not arrive, scalability is the worse, where if your mental model of what customer behavior is going to be is just a little bit off, after a year of building heavyweight architecture, and I have made this mistake personally on several occasions, it doesn’t even actually scale. The scalability is something that you really can only figure out when you are in contact with actual customer behavior, which is actually a metaphor for almost everything else that startups have to encounter. They say that no plan survives contact with customers. IMVU, the last company that I founded, we practice this thing called just in time scalability, where we explicitly engineer our software infrastructure for agility rather than for scalability, so it wasn’t very scalable.

We started off with a very, very crappy, very simple architecture, but we could change it really rapidly. At IMVU, we were called the Disneyland calendar because we got a lot of kids using our products. When they were in school, they weren’t on IMVU, so nights, weekends, holidays, those were our peak times. So we would come in on a Monday morning. We would look at our server logs for the last weekend. We are on this crazy growth curve, we are growing at 20, 30s and that’s 50% a week. We’d say all right, what parts of our infrastructure, not just the physical parts but the software, everything, every system, every subsystem, which parts almost fell over this past weekend. We would set a goal. All right, for each of those systems, we are going to refactor them and make them more scalable by Friday. So we would kind of make a decision about how much time we are going to invest in scalability on Monday, by Wednesday or Thursday, we would be deploying the new horizontally partitioned database scheme or whatever it was. Then we have it up and running on Friday.

It had to be in order to work in the next weekend and the reason we were able to do that is not because we were scalability geniuses. Actually, most of the small fixes were relatively simple. It was that we had good analytics to know what the problem was and we practiced this thing called continuous deployment that allowed us to put software into production in 20 minutes. So between the time we would write a piece of software and the time we live in production would be no more than 20 minutes. That’s why IMVU is famous for being able to deploy software 50 times a day.

We talk a lot about how that’s really helpful for running experiments and testing adult customers and it’s great for that. It’s also awesome for scalability because we could make scalability changes with no fear. When you are refactoring a core database subsystem, most engineers and most systems are terrified. If you make the tiniest mistake, take the whole site down but because of this thing, continuous deployment, one of the elements of it is this thing called the cluster immune system. So our servers had introspection during the process of deploying your software. They could tell if you were making a catastrophic mistake and could autonomously revert that software and send it back to you.

Interviewer: Wait a second. That sounds like a lot of engineering? How did you guys engineer that ahead of time?

Eric: We didn’t. That’s the most amazing thing. Not only did we do just in time scalability, we built that system also in a just in time way.

Interviewer: To most people, I mean at Microsoft, when I worked at Microsoft, they shipped software every three years.

Eric: I know, I know. Well I said, that’s why you have software with a date and a name.

Interviewer: Yeah, exactly. You’re talking about shifting software, how many times a day?

Eric: 50 times a day on average, sometimes more, sometimes less.

Interviewer: How does that happen and is there any chance for bringing a company like Microsoft into that world, or do you really need to start over with a new team, a new methodology and there is no way to bring Microsoft Office into that world?

Eric: I can’t speak to any specific product and I worked very briefly at Microsoft. Boy, like 15 years ago, so I don’t know. I don’t know what it is like now. I can’t speak to that.

Interviewer: It’s even slower.

Eric: Yeah, that is the pattern in organizations that practice what we call large batch development. The size of a batch is how large is a piece of software you kind of deploy the customers all at once. In the case of something like Microsoft Office, when you burn it on a CD and mail it to people or put in a store shelf, it’s basically the largest batch of you can imagine. It’s millions of lines to code, whereas at IMVU, we would deploy five lines of code as a batch sometimes, no problem. It didn’t matter and the insides from the total production system and the manufacturing is that when you reduce the batch size and work, you get faster feedback and problems are instantly localized. So, if you deploy three million lines of code all at once and there is problem, you spend a lot of time debugging. If everything is running fine, you deploy 20 lines of code and then you have a problem, high confidence it was the 20 lines that caused it. yYu revert. You figure out the one that interact badly. I want to answer your question but having taught this now for a while, the hardest thing about continuous deployment is it sounds complicated and like magic and that can cause people to not want to get started and it’s actually exactly wrong.

We don’t take a year off and say we’ll build a great deployment system and then start. Instead, you let the problems that you are actually having with your whatever it is that you are doing teach you where you need to invest in prevention. It’s one of those Toyota sayings that drives people crazy. Every defect is a treasure. Nice to hear that.

Interviewer: They have a lot of defect systems.

Eric: Sounds like they have got a little bit away from the philosophy. I hope that they find a way to fix that. I believe that anybody can move towards continuous deployment right away and the reason is there’s a great 80-20 rule. 80% of the fire fighting and wasted time that most organizations spend is being caused by 20% of their system. It’s just you don’t know which 20% it is. So I don’t want to get in too much detail unless you are interested but we practice this thing called five whys, which is a way for doing root cause analysis when there is a problem.

Eric: You ask the question why five times.

Interviewer: Give me an example.

Eric: I will give you an example. So imagine a server goes down. Okay, why did that happen? Well, we deployed some new piece of code and it caused the load average to spike. Why did we do that? Well, someone, in that piece of code, they used an obscure API that has a side effect of taking the server down. Why would they do that? Were they being malicious? No, they didn’t mean to take the server down. They thought it was going to work but they’re a new engineer and they didn’t realize that this API has that effect. Why didn’t they know? It’s because their manager doesn’t believe in training new engineers. So, you are like whoa, hold on. I thought we had a technical problem of a server being down and you’re telling me we actually have a managerial problem of a manager who doesn’t believe in training? The answer is yes. Behind every technical problem, so called technical problem, is actually a human problem that needs to be fixed. So technical five whys. First part of it is to realize that even though the symptom is acute, the problem is at a distance.

Interviewer: This is deep because at Rackspace, an employee may…

Eric: That’s all right.

Interviewer: I can only go 14 minutes. That file system, Subtler, it ruins our lives. Anyway, we were talking about at Rackspace, we had an employee make a mistake and the chairman said that it’s not his fault. It was the team’s fault.

Eric: Yeah. We used to get new engineers when we to hire at IMVU and we still do this, we would have them deploy code to production on their first day. Maybe their second day at work but if it took to the third day, we would be really upset. We were bring in some veterans who have been in large organizations for a long time. They would be really afraid. They would go wait a minute, what if I accidentally take the site down? We would say, if you can take the site down on your first day of work, shame on us for making it so easy. Our whole idea was to make this system resilient against human errors because humans are imperfect, they make mistakes.

If you think about that five whys analysis, we have a server problem, an API that’s too hard to use, an engineer whose not trained and a manager doesn’t believe in training. The second key insight is we want to make a proportional investment in prevention at each of the five layers of the hierarchy, so we’ll obviously bring that server back up, revert that piece of code, maybe we’ll fix that API, so it’s not easy to shoot yourself in the foot. We’ll talk to that engineer, make sure they understand why what they did was a mistake and we will talk to their managers about the importance of training.

The work proportional is really important because, like what happens is, engineers sometimes will say oh, we had a minor problem. Okay, well I’ve got to take six weeks off and invest in a whole new… It’s like no, no, no, no. If it’s a minor problem, we will do a minor solution. So my classic thing I use this in workshops all the time. I actually had this argument. Go to the manager and say, “Manager, it’s really important to start doing training for your engineers.” Manager will say, “I totally agree. You are exactly right. In fact, I have an eight week program I’m ready to do to create a new world class training program. So if you can find someone to do everything I was planning to do for the next eight weeks, I’d be happy to do it.” That’s manager speak for go to hell, right?

Interviewer: Yeah.

Eric: This gives us away out of that dilemma, because we would say what was a minor problem so we’ve decided we are going to spend one hour in prevention at each of the five levels. Could you please do the first hour of that eight week plan? The manager might say, “What’s the point of doing the first hour?” You say, “Well that’s okay. Think of before you could do a hundred hours, you had to do something first so do that first.” Okay, well maybe I could create a Wiki page where we say things that we should train new engineers about. I say great, let’s start there. That doesn’t sound very powerful but it really is, because if training is a recurring issue in the organization, and you do five whys regularly, this is going to keep coming up and every time it comes up, we will do the next hour and the next hour and the next hour, until we stop having problems caused by training.

I use that example a lot because if you had come to me at IMVU, I was the VP of engineering, the CTO. It was my job to make these kinds of decisions and you said to me, “You need to have a world class training program.” I would have said, “Come on, give me a break. We are a startup. Training is for big companies. We don’t have that problem. We hire the smartest people.” I would have given you a hundred reasons why that’s stupid. Yet, after doing five whys for a couple of years, we wound up with a training program so good that we could get people productive on their very first day.

We never had to make that decision like we should really have a really important training program. Instead, naturally, organically, incrementally, as a result of the problems that were actually happening to us, we wound up building this really powerful system. When people see it now, they’re like wow, that’s so advanced. You must have really spend a lot of time on it but not really. What we did is we borrowed it from the budget of time we were spending on fire fighting. That was actually wasted time that we could reclaim for a better purpose. That’s the essence of Lean Startup, is to use the mistakes and the failures as opportunities to learn something new and to get better at what with practice.

Interviewer: That’s pretty brilliant and it works not just with software either. I sat in on Tech Crunches in one of their first tech TV meetings, and Arrington has this bias for action, like just go do it. Go do it and fail. Go do it, make it happen. Stop giving excuses about why you are not doing it already. Get out there and do it. He forced his team to get out the meeting and go shoot a show. That’s sort of what you are talking about. It’s this bias for action. Get something done and then figure out what didn’t work about it but at least we are further along. We had something done.

Eric: You are always better off having taken action than talking about taking action. What’s interesting is, and this is where I think the Lean Startup differs from what I call the just do it school of entrepreneurship, which is more on the kind of 37 Signals approach, which I think has a lot to recommend it or like the release early, release often school that comes out of open source. Those are really good system for getting started but the problem is okay, you just did it. You put something out there. As soon as you have three customers, you already have 50 opinions about what your product should do, right? Now what?

What happens is people sometimes confuse like forward motion with progress and first of all, talking is not progress. So getting people into motion and doing things is very important but then we need some kind of framework for understanding how do we make sense of the information that we’re getting back because entrepreneurship is classic overwhelming situation. You are always being told, some journalist think you should do this, just customers say you should do this. Your most valuable customer says you should do that. Your early adopters think you should do this. I think it’s really important to be focused on kind of having a structure for thinking about what are the underlying assumptions that my vision is based on and how do I prove or disprove my vision is correct rather than like taking in feedback and doing what people say.

It’s very interesting. Lean Startup is very customer-centric and yet, you don’t get a gold star for listening to customers. In fact, listening to customers is not in any way important except in so far it gives you really important information about them, and that’s really hard. As human beings, we are not good at holding these two ideas in the head at the same time. My vision is brilliant and will change the world and it’s probably fatally flawed in some way. That can give you a headache because that’s cognitive dissonance right there but by getting feedback, by testing the vision against what people will or won’t do, by measuring their behavior rather than just listening to what they say, we are going to figure out which elements of the vision are right and then double down in those areas.

Interviewer: This is really important and hard to do. One of my bosses used to say, “Don’t listen to customers too much because if you were an engineer at Porsche and you ask your customers what they want, they would tell you more and more head rooms, smoother ride. Well, then just design a Volvo.”

Eric: That’s right.

Interviewer: What you are saying is design the Porsche and then test if they like the Porsche.

Eric: Exactly right and you don’t get a free ride either way. You can’t just be like well I did what customers said, so I am good. It’s like too bad but you also can’t say well I am the visionary, it’s my job to tell customers what they want. No. Your job is to invent the future and then double check to make sure that you are right. It’s that process.

Interviewer: Even Steve Jobs does this, by the way.

Eric: I think he does it incredibly well.

Interviewer: A lot of people think he is a dictator and he is but he also tests. I know people who saw the iPad a year ago and were in focus groups and he tested out the idea to see if people would pick this thing up and like using it and stuff like that.

Eric: Yeah. I love that example because entrepreneurs get really confused by what they are reading in the press and, frankly, most of the stories you hear about how companies are run in the press are wildly inaccurate.

Interviewer: Sorry about that.

Eric: People respond to the incentives that they have and it’s a good story to tell about how Steve Jobs is this mythic hero who never makes any mistakes and doesn’t need any feedback. It’s just the evidence is overwhelming that that’s not the case. People who are on the inside know that. Part of what I am trying to do with Lean Startup, with this conference in April, is to, as an industry, get us to start putting the practice of entrepreneurship on a more rigorous footing. So we can start to think of it as a profession, as a kind of management that requires actual thinking about how to do it well and we don’t just assume that the best practices that work for so and so in such and such context work for everybody and that I think that’s a challenge. It certainly has rubbed some people the wrong way and I think that is actually a pretty good sign.

Interviewer: I also loved that instead building an entire training program, you just said do the first hour. That sounds a lot like David Allen’s getting things done, where he ask you what’s the next action.

Eric: That’s right.

Interviewer: You need to plan a trip to Europe. What’s the next action that will get you on that path? What’s the first thing you need to do? I think that’s really brilliant.

Eric: Thanks.

Interviewer: Give me some more ideas like that. What else? What other kinds of ways do you find that you get teams moving or get people shifting code 50 times a day instead of making excuses about how they have to wait three years?

Eric: What’s interesting is that most of the techniques and the getting thing done analogy is actually really good because most of these techniques have this really interesting kind of long term, short term pairing right? Long term vision of I want to go to this specific place but very short term tactical, what’s the first step I have to take or what’s the first assumption I have to test? For example ,at IMVU, we are really proud of ourselves that we managed to ship our very first beta in only six months. In our previous company, we had spent like almost five years to get our first product out so we were like wow, six months that’s really fast. We killed ourselves to build this first version and basically, it sucked. I mean it was really buggy and we were really embarrassed by it and we thought people are going to hate it. They are going to think we stand, we are a bad company all that stuff but then we discovered that nobody used it. It’s like wait a minute, hold on. We couldn’t even get people to download the product, let alone discover how buggy it was.

That was one of those moments when I said wait, how can this be? If our goal of these six months was to learn what customers want and don’t want, why do we have to wipe all the software to do it? If we just put up a landing page that says, please download the software, and nothing happened if you click the button, it would have been the same, because nobody clicked the button. Right? We were completely wrong in terms of what customers wanted and so that was the first step towards reorienting my sense that progress in entrepreneurship has to be denominated in learning, not code. I think that’s a breakthrough idea, especially because if you read the Agile Manifesto, which is a document I think is profoundly important and has really influenced me a lot. It says right there in Agile Manifesto that a line of working code is kind of how we are going to measure progress and that’s better than measuring progress in the older way, which is through milestones and schedules and documents and planning. In situations of high uncertainty, which software is the root cause of so much uncertainty in this world, code is not progress, learning is progress. It’s very freaky.

Interviewer: That’s actually really deep and I think it’s why Rackspace funds me to do this, to go around the world and learn. That’s one-third in my mission, is just to go out and learn and find great ideas.

Eric: It’s surprisingly important and it’s very hard to be an expert now in any job anywhere, because the rate of change that we are coping with is so high, but you ask for specific techniques, so I’ll give you another one that helps team reorient from code to learning as an inner progress. You probably seen that traditional Agile thing with story cards, where, for those who don’t know, what you do is you put up a big whiteboard with three sections on it and each card represents a relatively small task, one to two days. They call them story cards in Agile. What you do is the three sections of the whiteboard are backlog, that is things you might want to do in the future, in progress and done. So, it’s a visual way, a form of visual control for seeing what needs to be done, what’s being done and what has been done and then one of the rules in most Agile systems like Scrum or XP is once a card moves into the in progress bar, we will never cancel it. So if you are an engineer, you are working on something, you won’t be interrupted to go work on something else.

In exchange, the team agrees that they will always pull the top card off the backlog so the kind of product owner or product manager whoever is in charge of prioritizing things, they can be reordering the backlog everyday, all day, because it doesn’t bother anybody. No one is anticipating what we might need to do. Everyone is focused on what needs to get done. There is a technique, also from Toyota production system called kanban, where we will constrain the number of cards that can be in each bucket and it’s a way of preventing us from taking on too many things at once, which is bad for feedback.

For example, let’s say you have five people on your team. You might say we are only going to have five cards in progress at any one time. So, it forces each person to basically work on one thing at a time and that way we kind of get more learning. It’s not as efficient from a pure code production point of view but it’s better because you get faster feedback and when a task gets completed, then you pull the next one, and that’s it. Then, the people who are managing backlog realize, oh, we only need to have five cards at most in the backlog at any one time. We don’t need to be keeping a list of 25,000 future features because six through 25,000 are irrelevant. Only the top five can be acted at any one time.