I recently re-read Eric Ries' book "The Lean Startup" in preparation for an upcoming talk at his conference in December. It got me thinking about things that have changed about Shutterstock since its early days and how we've managed to scale what is a very lean organization. Here's a few things we are most proud of followed by some challenges as well.
We've implemented a number of systems in the organization that keep all of us close to the customer. As we've grown, we now have a great qualitative research team dedicated to customer development. There are 5-10 customers in our office (or remote) per week for developers, product owners, and marketers to speak to and validate learning. The important aspect of scaling customer development is to build it into the process and make it incredibly easy to put your idea in front of a customer. Skip the focus groups; they don't work and take too long to set up. Create a steady stream of customer input that anyone can dip into. Another way we test assumptions is with world-class tools.
Testing Culture & Tools
The concept that your "big idea" is nothing but a hypothesis until we test it is a part of the Shutterstock culture. We learned some hard lessons through A/B testing, but realized quickly that the pace at which we could perform tests and the systems for reading tests had to be excellent and easily accessible by many different groups in the organization. Because of this and because we believe our data is a competitive advantage, we chose to build many tools ourselves. At this point we have our own open-sourced click tracking "lil brother," an internal data visualization tool built on Rickshaw as well as an AB testing platform called Absinthe. We pride ourselves on the speed of testing hypotheses and reading those tests. Shutterstock is in a competitive market, but we have the most traffic and the most usage, meaning that we can run more tests (achieve significance sooner) and learn faster than our competitors.
Speed matters if you hope to build, measure and learn faster than your competition. As Shutterstock has grown, there are a few key elements to our continued development speed.
1. Small, autonomous teams are helpful, the more a team can do on their own, the faster they go. The lengthy hand-offs between teams are (mostly) eliminated and it creates a good startup vibe as well.
2. Continuous deployment has kept us lean - we don't have release trains and code goes live to the site every day of the week.
3. Continuously improve/refine process
Depending on which team you are talking to at Shutterstock, we may be agile or Kanban or some other flavor depending on the type of problem that team is designed to solve. Be flexible enough to change your process across teams as you scale. Different types of work will call for different systems. Scalability projects dealing with large infrastructures might benefit from longer release cycles than product teams running experiments on a pricing page.
As with all businesses, there are challenges to staying lean, these are some of ours.
Focus on people over process
Jon Oringer and I talk a lot about process - and fighting unnecessary process where we see it at Shutterstock. I think we've instilled a culture where team members question every bit of process. "Is that meeting necessary?" "Perhaps there's another way to solve this problem without a meeting." One quote I repeat often is "Process is easy to put in place and very difficult to remove, so be careful when you add any." Too much process is how once lean organizations calcify into something slower, little bits of process creep in and soon entire departments are created to handle those processes. Often, more people in an organization requires more coordination and more alignment, but that doesn't have to mean more meetings. I like the idea of an adaptive organization where the company adjusts it's process and performance, but I can't say we've mastered it yet either. Striking a balance between getting everyone's opinion but staying efficient is important to us and something we are actively working on.
Vanity Metrics. The challenge of removing vanity metrics is something we think about often. Everyone wants to be more accountable and measure their progress (a great problem to have), but its difficult to keep the metrics meaningful. Product team velocity is a great example. It's a helpful metric when trying to figure out the efficiency of a team, but at the end of the day, it has little bearing on whether what the team is building is effective or not. You can have a team with great velocity building something that customers don't want or vice versa. Another Vanity metric I've seen is visits - a stat that I've rarely seen anyone take action on. Usage of a new feature wasn't very effective until we changed it to repeat usage. Lastly, we used to track code deploys to production because we were so proud of our continuous deployment practice, but past a certain threshold (we are somewhere north of 200 deploys a month) this too is a vanity metric.
Technical Debt is something every team focuses attention on regularly. As with most evolving systems that are over 10 years old, ours has increased complexity over the years. There are some amazing efforts going on internally to simplify our systems and modernize our stack but the effort required to do so is something new to our organization. Oftentimes there are decisions made without understanding of the technical ramifications not because someone is vindictive, but because they have a pre-conceived notion of how complex something is. Working to keep systems simple is an important part of our product strategy, we regularly delete lesser used features in order to keep a simple scalable system. Still, we have a ways to go.