What have you learned from your users? – VC Cafe


The deadline for YCombinator applications was this week and Paul Graham, the founder of YC tweeted that if he had to distill the most important advice he has for startups it would be the title of this post: “Explain what you have learned from your users” It may sound trivial, but many products/startups that no longer exist have ignored this advice at their peril.

In his own words:

“Even if you only thought of the idea yesterday, you can still talk to users. In fact, that’s the first thing I’d do after working out the first version of most ideas: run it by some potential users to see how they reacted. Incidentally, it won’t carry much weight if you merely say “Potential users love the idea!” You have to say what you *learned* from users. I.e. in what ways your initial model of them was mistaken. If you can’t answer that, it’s probably not because your model was perfect.”

Paul Graham, Ycombinator

As startup founders ask themselves this question, some of the answers might be specific to their product: i.e. we should improve our onboarding, change our freemium model, etc. But many of the answers are true universally, and have to do with how startups collect, digest and act on user feedback.

So I took a short poll of some of our portfolio CEOs at Remagine Ventures and got some insightful quotes I’d like to share with you:

Everything we build is a hypothesis until the users are using the feature (hands-on) and we have a tangible way to measure its impact. Throughout our journey, we had a few ‘solid’ theories and concepts that we were confident would work and we invested many resources into developing them. Now, we design every feature as lean and fast as possible to get early validation and only after the initial validation we build more robust capabilities.

Or Briga, Co-founder and CEO of Novos.gg

We co-develop our product with our community. Roblox is all about the community, and they are highly involved in everything we do. We never tell them what they need, we listen to what they want and implement it in our development cycles and our LiveOps on a weekly basis. If we have a new feature or game mechanic we’ll usually invite them to play test with us, get their feedback and make the changes they asked for.

Anat Shperling, founder and CEO of Toya, a Roblox gaming studio

Listen, not speak-is probably the most important thing you could do prior to PMF, and as you move forward developing your business. Many startups founders look to solve a problem that may affect them personally, but the founders are often not the main persona that they are building for. So early conversations (prior to building) becomes even more essential in your product strategy. Features should not be the topic of any discussion with users, but rather, truly listening to the issues/problems that brought them to your product in the first place, and their expectations against their experience-that should be the essence of these conversations

Yoav Oren and Matan Guttman, Co-founders of Zoog (a B2C mobile app)

Your product’s “Time to Hello World” should be minimal! Users should be able to set up an account or complete the installation as fast as possible and see the product in action! This will allow they to experience the product’s value right away.

Alon Grinshpoon, Co-founder and CEO of Echo3D

You can have all the advanced features you want, if you don’t reach feature parity [with competitors] on some features, it’s an uphill adoption fight. Once you have achieved that balance, there is no one like you.

David Stiff, Co-founder and CEO of Vault AI

1. The Mom Test

It turns out that people have been lying to you. When you ask your mom “what do you think about my startup idea?” she might say it’s great, because she loves you and doesn’t want to hurt your feelings. The Mom Test is a book by the wonderful Rob Fitzpatrick focusing on how to talk to customers, ask open ended questions and get honest user feedback.

For example, below are good questions to dig into feature requests:

  • “Why do you want that?”
  • “What would that let you do?”
  • “How are you coping without it?”
  • “Do you think we should push back the launch add that feature, or is it
  • something we could add later?”
  • How would that fit into your day?
The Mom test in short (source)

2. Create a Sandbox

A sandbox is an isolated environment where you can test and experiment before making changes to production.

The process of building products is one big experiment, broken down into many small experiments. As Or Briga pointed out in his quote, every feature is an hypothesis that was tested and validated. But if we constantly change the product for all users, it can backfire and break the trust: imagine you went to Google and everyday it looked different? Even though Google also runs hundreds of experiments in a given month, it’s done subtly using A/B testing and the concept of Sandbox.

Get in the habit of testing, and always keep a control group. The best companies learn do to this at scale and it’s true for both B2B and B2C businesses. When assumptions have been tested and properly validated, they can be rolled out to all users gradually. There are now many SaaS tools to manage A/B testing (and even some that will create product optimisation ideas using AI) – give them a try!

A/B testing tools

3. Eat your own dog food

Put yourself in the shoes of your user. Would you rave about your product? how can you make it more accessible or easier to use?

Google was very disciplined at testing products internally before they went out to real users. It doesn’t meant that they didn’t make mistakes (there are plenty to choose from), but I’m sure this strategy help them catch more bugs and get critical user feedback before taking it to the public.

Developers, designers, product managers in your team should use your product every day.

Dofooding by Dilbert

4. Pay close attention to your user feedback

I love this quote by Brian Chesky, co-founder of AirBnb:

“It’s really hard to get even 10 people to love anything, but it’s not hard if you spend a ton of time with them. If I want to make something amazing, I just spend time with you…. Early on, Joe Gebbia and I literally commuted to New York from Mountain View [to visit our Airbnb hosts in person]. We literally would knock on the doors of all of our hosts. We had their addresses and we say, “Knock knock. Hello. Hey, this is Brian, Joe, we’re founders and we just want to meet you… And then you literally start designing touchpoint by touchpoint. The creation of the peer review system, customer support, all these things came from — we didn’t just meet our users, we lived with them. And I used to joke that when you bought an iPhone, Steve Jobs didn’t come sleep on your couch. But I did”

Brian Chesky, AirBnb co-founder and CEO

Not only user feedback will help you design a better product, but you can also start leveraging the feedback as validation for other users. We often have a herd mentality when it comes to purchases. When faced with the challenges of choice, we look for external validation: reviews, testimonials, referrals… can all make the difference between a conversion or a churn.

Collect user feedback actively and pay close attention to the results

5. Get serious about UX testing

To start with a simple definition: UI (User Interface) is how things look. At the end of the day, it’s a deliverable, a set of mocks/design assets. UX (User experience) is a process, that defines how things work/flow and it should be iterated regularly.

In startups, we often conflate these terms as resources are limited, but it would be a mistake to skip usability testing and focus only on aesthetics.

Did you know that Google offers a free UX design certificate course? The program follows the design process from beginning to end: empathising with users, defining their pain points, coming up with ideas for design solutions, creating wireframes and prototypes, and testing your designs with users to get feedback. By the end of the program, you’ll have a professional UX portfolio with three design projects.

Never stop experimenting. User feedback should be a guiding voice in the product decisions you make and testing time should be baked in to your product roadmap. In addition to running experiments, it’s good practice to build a community of users using Discord, Slack or any other group that enables you have a dialogue and shorten the feedback loop.

* Image credit: Stable Diffusion

Eze is managing partner of Remagine Ventures, a seed fund investing in ambitious founders at the intersection of tech, entertainment, gaming and commerce with a spotlight on Israel.

I’m a former general partner at google ventures, head of Google for Entrepreneurs in Europe and founding head of Campus London, Google’s first physical hub for startups.

I’m also the founder of Techbikers, a non-profit bringing together the startup ecosystem on cycling challenges in support of Room to Read. Since inception in 2012 we’ve built 11 schools and 50 libraries in the developing world.

Eze Vidra
Latest posts by Eze Vidra (see all)

Go to Publisher: VC Cafe
Author: Eze Vidra