Skip to content

How we collect, organize, and implement customer feedback

How we collect, organize, and implement customer feedback

For any startup, customer feedback is your north star, guiding your product to meet the needs and wants of your customers. In order to promote and sustain long-term growth, it is essential to effectively collect, organize, and implement high-quality customer feedback.

Feedback comes in many shapes and forms. It’s all over the place – in your NPS survey responses, support tickets, emails, tweets, and more. It’s often very subtle too and doesn’t look like feedback. This can leave you feeling overwhelmed. Without a disciplined process in place, you won’t be able to see trends or draw insights from the feedback you get.

We take customer feedback very seriously here at OpenPhone and have been asked many times about how we handle it. So I thought I’d share what works for us. I hope you enjoy, and please let me know what you think – and yes, I’m asking for feedback in an article about feedback 😉

Our customer feedback philosophy

Unlike quantitative feedback, qualitative feedback tends to be messy and raw. So we need to follow certain principles to make sense of it. Here are the 3 principles we follow to gather and implement high-value feedback.

  1. Focus on customers who “have their hair on fire”

In the earliest stages of a company, it’s tempting to get your product into the hands of as many different types of users as possible. While that can be helpful for some types of products and accelerate your learning, it can leave you overwhelmed. I know we were in the early days of OpenPhone.

Focusing on users desperately in need of your solution is the best way to get to product-market fit. These users will give you the right feedback that effectively helps craft the roadmap to build a product users radically love and happily promote.

Before reaching product-market fit, focusing elsewhere is a distraction and will lead to building a product that is just “OK”.

  1. Don’t seek validation for your existing beliefs

You already know what you believe about your product. In order to get valuable feedback from your users, it’s important to not lead your users on or prompt them to give the type of feedback that you want to hear.

When we ask for feedback, we keep our questions as neutral as possible: “how can we make OpenPhone better for you?” This open-ended question forces our users to think critically about what they want out of a phone solution and how we can improve.

  1. “…they would have said faster horses”

When reviewing feedback, it’s important to remember that users may not always know exactly what they want. Users can only articulate the problem they’re experiencing and it’s up to you to solve it.

For example, we often hear from users who want to forward their OpenPhone number to an international number. They say that they are looking for international calling (ps – it’s coming soon!). But in reality, they can get calls directly via the OpenPhone app without call forwarding. So this is usually an opportunity to educate the users on our product and improve how we describe our features.


Our customer feedback process

Feature requests

Your early adopters will enthusiastically tell you about all the features they need. At least that’s been our experience.

We receive the majority of our feature requests over our OpenPhone text support number and Intercom. We also receive feature requests over email, phone interviews, and surveys. Most of the time, our customers proactively tell us how to make OpenPhone better for them.

Some feature requests become very common and you hear about them all the time. That’s a good sign that users have certain expectations for your product and the problems it solves. Other times, you get one-off feature requests that are only applicable to a single user. Here I’ll share how we track them both.

How we prompt users for feature requests

At the end of nearly every conversation, we encourage our users to give feedback, stating: “Please let us know if you ever have any questions, feedback, or feature requests.”

We also automatically send a text message to our power users asking specifically how we can make our product better for them:

Asking for feedback through texting in OpenPhone

How we log feature requests

We log all feature requests in Productboard. Productboard helps us organize and quantify customer feedback.

Logging a customer feature request using Productboard's browser extension

Its browser extensions are compatible with most of the tools we use (including OpenPhone and social media), allowing us to never even leave the page we’re on and manually tag feature requests from different sources.

Our support team also tags feature requests in specific Zendesk tickets that automatically push to Productboard.

How we implement feature requests

When creating our product roadmap, we use the logged features requests to ensure we’re creating the best product for our users. Every two weeks, we have a product sprint planning meeting. In the meeting, we’ll review the top requests over the previous two week period as well as the top requests overall. Depending on the difficulty of implementation, we’ll typically design our product roadmap based on the magnitude of specific requests.

Net promoter score (NPS)

Net promoter score (NPS) is a metric used to quantify your user’s likelihood to recommend the product to a friend on a scale of. Using a scale of 1-10, respondents are grouped as Detractors, Passives, or Promoters based on their responses. Then, an index from 0 – 100 is created to track your NPS over time.

How we collect NPS
We use Survicate to send NPS surveys. We send a survey after a user has been on OpenPhone for 25 days. Since our setup is very quick and we offer a 7-day free trial, we found this the optimal time to ask for feedback. This also allows us to get feedback from folks who need some help setting up OpenPhone.

Here are the questions we ask:

  1. How likely are you to recommend OpenPhone to a friend or colleague? 
    (This determines the NPS score)
  2. Why did you leave this score?
    (If someone left a great score, this usually helps us understand why they love our product and what’s so great about it. If someone left a low score, this helps us understand how we can improve)
  3. What do you primarily use OpenPhone for? 
    (We can understand the use case and segment NPS score by use case)
  4. What devices do you use OpenPhone on? 
    (To segment NPS by platform and help us act on feedback faster)

How we use NPS survey data

Survicate not only calculates our NPS score but also allows us to segment groups of users based on their responses. This lets us dive deeper into our users’ experiences, informing us of what we can do to improve.

These insights go to a dedicated #customerinsight Slack channel for the whole team to see.

Survey response automatically pushed into a Slack channel

These responses also automatically push into Productboard and are categorized by our product team to capture insights around specific existing features and potential additions our customers would like to see.

When we make large product changes, we track and monitor how they affect our NPS score (we also segment this data by different user types). While not hard science, NPS serves as a great starting point in helping us track our user’s satisfaction with our product over time.

Customer satisfaction (CSAT)

Customer satisfaction (CSAT) is a metric used to quantify the degree to which customers are satisfied with your service. CSAT is a 1 – 5 scale, from Very Unsatisfied to Very Satisfied, sent to users after interacting with your support team. CSAT scores are measured as a percentage from 0 – 100. Only a score of “Satisfied” or “Very Satisfied” results in a positive CSAT score.

How we collect and measure customer satisfaction

We use Intercom for managing our support email and live chat. Once a conversation is closed, Intercom has a built in tool that prompts users to answer how satisfied they with were our support. Intercom’s reporting tools allow us to visualize this feedback over time.

How we use this data to improve our customer satisfaction

When customers leave any review that is less than “Satisfied”, our customer support team looks into these communications and use it as an example of how we can improve our customer experience. While you can’t always please everyone, as a team, we can help provide thoughts and ideas regarding how we can improve our support interactions in the future.

Personally, when I first started at OpenPhone, there were a few times that my conversations with customers left something to be desired. Syncing with Daryna, we were able to come up with strategies to better display empathy and understanding. Now, we consistently have a +95% CSAT 😉

Churn feedback

In order to reduce churn, it’s important to know why your previous customers left the service. The best time to discover why a user has decided to stop using your product or service is immediately after they’ve canceled. Collecting churn feedback is the best way to turn something negative (i.e. losing a customer) into something positive and actionable.

How we collect churn feedback

When a customer cancels their OpenPhone subscription, we automatically follow up with a cancellation email confirming their cancellation and asking them directly if they can clarify their reasons for ending their subscription.

In this email, we embed a Survicate survey to capture additional data. Users can leave us feedback just a single click.

Customer feedback survey sent if someone cancels their account.

How we use churn feedback

All of the survey data is stored in Survicate for later viewing. As for the responses to our cancellation emails, we use Productboard to track any issues, bugs, or features that influenced the cancellation.

We also use churn feedback to recover any lost users – when a feature they’ve requested is implemented or a bug they’ve experienced is resolved, we reach back out and ask them to give us another chance.

Product-market fit survey

Product-market fit surveys help you identify those users that desperately need your product. They’re not only a great way to directly ask your customers for feedback but also to gather valuable insights into the types of customers that you’re serving. They force your customers to really think about your product – both on what they love and what can be improved upon.

How we collect product-market fit surveys

After users have been using OpenPhone for 85 days, we send them our Product-Market Fit Survey survey via an automated email from Customer.io. When creating this survey, we were inspired by Superhuman’s process to measure product-market fit.

For how we find out who among our users desperately needs our product, please see question 6 👇 (if they respond they’d be “very disappointed” if they could no longer use OpenPhone).

Customer survey question in a form: How would you feel if you could no longer use OpenPhone?

Check out our customer service automation guide for other specific interactions we automate.

How we use product-market fit surveys

We further segment the users that desperately need OpenPhone to gather additional insights into the different types of users we serve. This helps reveal which customer profiles or markets are closer to product-market fit.

We then prioritize their feedback in our product roadmap – doubling down on the areas they love and improving the areas they feel we’ve missed the mark. Building our product for these users has created a growing community of dedicated fans who gladly promote us to friends and colleagues – the best form of marketing 😊

Phone interviews

Surveys are great, but can only tell you a part of the story. Nothing replaces actually talking to your customers and hearing what problems they have in real-time. This is one of the core things we’ve learned at Y Combinator.

How we conduct phone interviews

While we’d love to be able to talk to all customers, we have thousands of users and don’t have the bandwidth to personally interview all of them. So we proactively reach out to users with interesting use cases and who need more from our product. If that sounds like you, send us a note.

Naturally, we use OpenPhone for interviews. Using OpenPhone’s call recording feature, we’re able to record and all conversations in-app for later listening. Additionally, we’ll take detailed notes on the calls and store customer profiles in Notion. This helps our growing team stay connected to the problems we solve for our customers

Here are some of the questions we ask:

  1. What problem is OpenPhone solving for you?
    (Helps understand the pain points we’re solving for our many use cases)
  2. Why did you pick OpenPhone instead of other options?
    (Helps with competitive positioning)
  3. What do you like the most about OpenPhone?
    (Highlights our benefits from the customer’s perspective)
  4. What do you like the least about OpenPhone?
    (Shows us areas for improvement)
  5. What do you wish you could do with OpenPhone?
    (Open-ended question to help understand our customer’s other problems)
  6. What would you use if OpenPhone was no longer available? 
    (One of the most useful questions IMO – helps understand your competitive alternatives. Spoiler: it’s not always what you think!)

Bugs and UX issues

While not the most enjoyable to receive, product issues are a necessary (and frequent) type of feedback for any software company. If users experience an issue with using your product, it’s important to document and track this feedback. These issues could be a bug within the product itself, where the software is either not responding or not performing the expected behavior. They also could experience a user experience (UX) issue, where something within your product is not immediately clear. One good thing about this type of feedback is it’s almost always received unprompted. If users are having issues with your product, they’ll usually let you know!

How we collect and act on bugs and UX issues

We primarily receive our reports of bugs or confusion regarding UX design through Zendesk or our OpenPhone text number. Just like feature requests, we initially track these reports in Productboard, where we correctly tag bugs and UX enhancement requests respectively. For bug reports, we also capture all of the relevant information and submit it to our engineering team via Linear. Linear is our project management tool that helps us streamline software projects and track bugs.

For UX issues, we typically don’t submit a Linear ticket until there’s a trend of identical or related issues. For example, if one user has an issue finding your product settings, it’s probably user error. If 30 users can’t find it, you need to update your user interface immediately!

Just as we do with feature requests, when a bug or a UX issue is resolved, we notify every single person who has reported it to close the feedback loop.


Wrapping up

Collecting and implementing customer feedback will help you build a product that users radically love. While the philosophy and methodology detailed above works for us, there is no one-size-fits-all solution. Regardless if you collect feedback in Productboard, Airtable, or a heavily-improvised Google sheet, it’s essential to have a clearly defined process in place to manage and implement feedback.

I hope you’ve enjoyed this post. I’d love to hear what tools or processes you use to promote, collect, organize, and implement customer feedback. Please share them in the comments. If you end up implementing some of our processes, let me know how it goes!


If you’re hearing about OpenPhone for the first time here and curious about what we do, let me share more.

We’re building a modern and delightful business phone for small businesses and startups. It works right on top of your existing devices (via an app).

You can use OpenPhone to get customer feedback via text messages and to conduct your user interviews (and record them too!)

OpenPhone mobile and desktop apps
Preview of the OpenPhone mobile and web apps

We’ve got a free trial so check it out for yourself and let me know what you think!

4.9/5 - (14 votes)