During my time at Instagram, I shipped a major redesign to one of the most highly-used surfaces in our app, Camera.
Redesigns are admittedly some of the most painful product efforts in "impact driven" companies – the benefits are hypothetical, hard to measure and long-term (i.e. let's simplify the app so we have space to add more features); while the costs are measurable and immediate (i.e. sharp drop to engagement).
Along the way, my team developed a playbook for them - how to ship on schedule, with a high degree of craft and minimal regressions. Sharing some of what we learned hoping it might be useful for other folks, too.
Background on Viewmaster
Viewmaster aimed to make Instagram Camera “more expressive and fun”, by making AR Effects and Formats easier to discover. After 9 months of iteration and bouncing through several PMs, the project had no line of sight to shipping: regressions to business metrics were too high and key leads believed it no longer met our bar for Simplicity and Craft.
Ryan O’Rourke and I were asked to step in and help. We were able to ship it on schedule with a high degree of craft and minimal regressions to business metrics and performance. Together with the launch of AR Platform, Viewmaster made AR Effects become the most popular capture creative tool (surpassing even Boomerang!) and helped our pillar double our Camera Sharing goal (traditionally really tough to move and wasn’t even my team’s goal!).
A few best practices for redesigns
Before you start...
Align on goals and tradeoffs, write them down
This seemed a bit silly to include, but redesign projects often struggle with it. Beyond the visual refresh, folks often aren't aligned on what the redesign should achieve. When that's not clear, it's very hard to make trade-off decisions (which you inevitably have to, because you moved things around in a surface people use every day).
This was the case with this project as well. The success criteria was somewhat implicit (Increasing discoverability of Effects, Scaling Camera Formats, Unblocking Create Mode). We wrote these down and stack ranked how important they were to us.
Leads were reluctant to give us a "budget" of how much we could regress important company metrics (like sharing from Camera). This frustrated us, but in hindsight, I can understand why: we'd stop trying to mitigate losses upon reaching this number instead of trying our best.
Set realistic timelines to avoid stress and protect craft
Redesigns have a lot more complexity than an average "incremental" feature launch, but teams seldom account for this when planning. In my experience, while you can get an initial test out fairly quickly, accounting for all the edge cases and regressions can take way longer than anticipated. This stresses teams out; and craft work often gets deprioritized to meet deadlines.
In this project, I engaged my eng partners closely on their estimates and tried to provide a "devil's advocate" perspective on what could happen, which adjusted our estimates. (Or you could skip all this and just take the eng estimate and double it, which has also worked for me in the past 🙃).
This might be a bit controversial and culture dependent. I personally prefer keepings sustainable for my team and prioritizing craft even if that might mean we move a little more slowly.
Recruit the right engineers
Some engineers enjoy growth-style iteration, some enjoy thinking through scalable backends, some enjoy the intricacies of design details. If you have the luxury to do so, make sure you recruit engineers that belong to the latter group. You'll be able to move a lot faster, since these engineers will notice things proactively (as opposed to you waiting for them to be done and testing things) and their initial build will have a higher degree of craft.
As you design and write code...
Anticipate and solve problems before they happen
Iteration cycles on mobile are VERY long: coding, QA, shipping a test, collecting data, interpreting results... This makes it very costly to test and learn sequentially, which redesign efforts often need to do as user behavior changes in unpredictable ways.
To solve for this, we "pre-mortemed" potential issues that could happen, or reasons why we'd see regressions to behaviors we cared about. This helped us preemptively come up with solutions for them. We coded these and put them in the build as experiment parameters.
At this point, some folks in your team might tempted to test these parameters "just to see what happens". Resist this temptation, it will slow you down. Have a strong opinion about what you're testing and only after getting concrete data that something may not be working, test another hypothesis.
By "pre-morteming", we were able to launch new tests without waiting for a mobile release and cut down our iteration cycle from 6 weeks to 2 weeks. This contributed to being able to ship on time.
As we did this, we also realized we had a huge long tail of lightly-used features that add complexity and risk. We unshipped them independently of the redesign, which simplified our test setup.
After you get test results...
When interpreting results, compare what's in production, not mocks
This might be a bit Instagram specific since we have so many teams working on so many features and it's often hard to know exactly what the app experience is like. Teams sometimes struggle interpreting results as they might be looking at just the "test" experience or mocks of what it was supposed to be. What I recommend instead: frequently screenshot production test/control experiences, paste them into a document, compare them side-by-side. We had a few occasions unrelated work interfered with our test, but had we only looked at mocks, we wouldn't have known.
Mitigate "analysis paralysis" with this handy table
When I took over this project, I was handed a 23 page analysis of what we were seeing in the test. It was a great piece of data science work, but overwhelming and hard to action on.
Here's a format I find helpful to analyze experiment results in redesigns. I throw this into a table, which helps create a bird's eye view about what's going on and what to do next. I often find it helpful to pick one core metric to compare everything "apples to apples".
-
Hypothesis ("People get distracted playing with effects instead of sharing in the moment")
-
Supporting evidence ("People try more effects in this design. The more effects they try, the less they share").
-
An estimate of how much of the regression is caused by this. These are hard to do scientifically, so you might need to push your DS partner to be comfortable making assumptions and guesstimating.
-
Potential mitigation ("We'll cap the Effect Tray to 25 items")
-
An estimate of how much of the regression you can recover.
As you iterate...
NAME your variants!
Another silly thing, but helps a lot. Likely you'll iterate on the design many times to find acceptable trade-offs and it will become super unwieldy to refer to them ("v4 but format picker was a bit bigger and we moved library to the left"). We started giving nicknames to each of these variants that simplified communication within our team and with leads.
Set the bar for craft amongst your leads group
While PM, EM and PD can come to the table with different viewpoints and negotiate (as we should) , we need to find balance in a direction everyone feels good about; ideally moving quickly AND ship with a high degree of craft. This requires aligning on we consider blocking for team/company dogfood, small/large public test and launch.
In our case, we aligned on a prioritization scale for issues and maintained burn-down lists for these issues that cropped up. This ensured we had line of sight to craft issues and design was bought in to when things were going to resolve. This was a simple, flat list we looked at in each meeting, rather than a complex spreadsheet or something in a Tasks/Jira type tool.
It also really helps that all leads actively test the feature and find issues, not just design. Seeing PM, EM and other leaders flag issues signal to the team that craft is actually important, not a platitude.
When trying to make a decision...
Be comfortable decoupling some goals so you can ship
Our preferred variant had regressions we couldn't recover, so we had to let go of the goal to “scale formats” and kept Boomerang & co in the footer. This initially disappointed me, I'd rather have taken a metric hit and ship the vision we had. However, the following half, we found a different design that met this outcome and were able to ship it without regressions.
Decoupling helped us actually launch and take a breather; and provided time and space to find a better solution. I also have a hunch people have an easier time adjusting if you don't change everything at once.
Parting thoughts
Redesigns can be hard and thankless, but I loved the challenge because they require every function to collaborate very closely and be on their A-Game. This project was no exception. Thanks to Eric Dorman, Ryan O’Rourke, Kevin Huang, Leah Acosta, Brandon Kieft, Jonathan Lee, Jose Zuniga and Kevin Hiscott and others I may have missed on a great partnership.
I have a lot of tips for "how to turn things around when a redesign goes south", but didn't include them this time for brevity. If that sounds interesting to you, please let me know and I'm happy to share.