If you’ve read my first three blog posts, you likely perceive me as a guy who doesn’t know his a** from his hand when it comes to startups. If that’s the case, then mission accomplished.
Let’s change up the pace a little bit, because not knowing my a** from my hand is a little harsh. Like I mentioned in my first post, I’m not here to put myself on a pedestal or say how great I am; that’s for networking events and job interviews. I’m doing this mostly to force myself to think back about what mistakes I’ve made, why I made them, and also what I’ve done right, and why. So today I’m going to talk more about something I have a bit more success with, consumer research interviews.
Consumer research interviews should be an integral part of the marketing and product management process of every company regardless of industry, size, or location. This applies not to just ‘consumers’, but whoever your potential customer is. Under different names, I’ve seen this tool appear in many ‘theories’ about startups and product development, namely in Lean Startup during customer development and idea validation, and in human-centered design. For those unfamiliar with those fancy-schmancy terms, it would suffice for you to know that consumer interviews are extremely helpful in determining if you have a product worth building, identifying a target consumer for that product, and doing continual product evaluation.
Research interviews are what put the nail in the coffin for Wedkey. During those interviews we learned that our users loved the concept of a deal site specifically for weddings, they loved that the deals were up on our site for 30 days at a time, they loved the design, but there weren’t enough vendors for them to choose from, and most importantly the deals weren’t good enough. We learned that we had made several key assumptions early on that were totally annihilated with info from these interviews. I’ve also used interviews at ServiceAlley a fair bit in the last year to identify our target consumer and to shape the development of our core site features.
If you’ve never done this before, you’re probably saying to yourself, ‘when am I supposed to do this?’, and ‘I wouldn’t even know where to start’. That’s cause you’re dumb.
No, of course you’re not; I’m just kidding. This won’t be easy for all of you to do. I completely understand that it’s unsettling to put yourself out there, have conversations with complete strangers, try to understand what they are saying, and to know the right follow-up questions to ask. Lucky for me, if you put me in a room with a bunch of strangers, I can talk to them for hours and actually enjoy it. You may think that’s a bit weird, but hey, I’m a type A entrepreneur, it’s in my blood.
Anyway, let’s jump into this and I’ll do my best to convey my process for these interviews. If you have questions or other ideas you want to share, please comment on the post, and I will respond to them all.
1. You have an idea
Let’s assume that you have a general idea for a product. In a flash of brilliance, you’ve come up with this amazingly brilliant idea after identifying what you think is a consumer need.
Now you want to figure out if the idea is worth building and how you should shape the product to best meet the consumer’s need you’re trying to fulfill.
A side note here for people early in the process of developing an idea: If you’re starting from scratch (no company, no product yet), don’t try and name your idea. You become inherently more connected to an idea when you name it, and it then becomes more difficult to properly interpret comments from an interviewee.
I’m going to use a recent round of interviews we did at ServiceAlley as an example.
During this round of interviews, my goal was to better understand how people approached identifying home service providers to evaluate so I could develop a new feature for the site that they would find valuable.
2. Assumptions & Hypotheses
I start my process by laying out my assumptions about the product and the consumer. Assumptions are your preconceived notions about your general idea, your product, and your consumer. Writing them all down isn’t easy to do, and I find it helpful to talk through them with someone, like a friend, a colleague, my wife or my dog Breck. Although Breck isn’t very helpful… every time I ask him a question, he just tilts his head to the side and wags his tail.
In truth, talking through assumptions with other people can often help identify other assumptions you’re subconsciously making.
Some of my recent assumptions about people for ServiceAlley:
- Use the same providers they have in the past
- Like to look for home service providers online
- Value online ratings and reviews
- Don’t care about provider located as long as the provider will come to their home
- People prefer active search vs. passively having a website tell them who to use
Having a list of assumptions will be a good starting point for writing interview questions later on in the process. If you wanted to take a more scientific approach, you could use your assumptions to develop proper hypotheses. A proper hypothesis is structured as an “if-blank-then-blank” statement. For example, “if I don’t take my dog for a walk twice a day, then he will poop in my house”. I don’t usually develop full hypotheses for research interviews but if you care to, may the Schwartz be wit you (I love Mel Brooks).
It’s hard to say one thing in particular is the most critical part of successful research interviews, but if I had to, I’d pick writing the questions. That’s because writing just any old questions won’t do you any good. This is going to sound like a Dr. Seuss book, but the questions need to
- Be general yet specific
- Trigger an emotional response but not be too personal
- Flow from one to the next but be flexible enough to be asked at any point in the conversation
- Most importantly, NOT be leading questions
Having a good set of interview questions can help you gain valuable insight, but having bad questions can lead you down a dark alley full of bad ideas, wasted time, and broken dreams.
Try to keep most of the questions broad and more general. These types of questions enable your interviewee to talk freely and make comments that you can then probe into further. That is where the real learning happens.
‘How do you think about the quality of a home service provider?’
‘Tell me about the last time you had to have work done in, or on, your home.’
Leading questions, on the other hand, are poison in a research interview. If you’ve never written questions before, try and ask for some help your first time. If you want to give it a go solo, you can check to see if you’ve written a leading interview question by reading it back to yourself. At the simplest level, if you can answer the question with a yes or no, it may be a leading question. Furthermore, the real key to not asking leading questions is to ensure there’re no values (positive/negative) implied in the question (i.e. the question itself is neutral). Leading questions usually contain information that forces a mental predisposition towards an answer. For example, if your goal was to find out if a person believes the skier Lindsay Vonn has taken steroids, you might ask: ‘Do you think four-time overall World Cup skiing champion Lindsay Vonn takes steroids?’. This question unnecessarily makes multiple willful connections that make it leading; it connects “Lindsay Vonn” to both “four-time champion” and to “steroids”. A less leading way to ask the question is ‘Are you aware of any professional skiers that have used steroids?’. In this second example you can obtain the same information from your interviewee without your own associational logic affecting the answer.
Here is a helpful article about seven common survey question writing mistakes that apply here as well.
For a 30 minute interview, I tend to write a list of 15-20 questions, and order the questions based on what I think an ideal conversation would sound like. Then, I toss away any hope of having an ideal conversation and start to plan for how I’ll handle the inevitability of each conversation having a haphazard order of topic discussion.
I like to group my questions based on general topics with big headlines on one sheet of paper. This way, when a topic comes up, I can easily find the relevant set of questions and reference them as a group. In essence, I write the questions for the ideal, but plan for the randomness of normal conversations.
Below are some of my questions from the recent ServiceAlley interviews. Note that I begin with very broad questions that are easy for the person to answer. I want to make the interviewee feel as comfortable as possible right at the beginning before we get to the more detailed questions.
- How long have you owned your home?
- Do you own a house, a townhome, an apartment in a condo?
- Do you have a set of companies that you go to for work in and around your home? (like plumbing, hanging a tv, house cleaning, etc)
- How did you build this list of companies?
- How/where do you store this list?
- When was the last time you had to hire someone to fix something in/around your home? What type of company was it?
- What was the job?
Reviews & Finding Providers
- Do you look at online review sites for anything (restaurants, products, google, yelp, consumer reports, etc)? If yes, which one do you like best and why? If no, why do you not use them?
- How do you think about the quality of a provider?
- Where did you find the company you mentioned previously?
- Probe further about asking friends, family or neighbors
- Want to know how they go about it: technology used, evaluation approach
- Willing to make your need publicly known in order to get a recommendation for a provider? Who are you comfortable telling you need help?
4. Prepping for the conversations
Comfort is key to having successful research interviews, and it starts with you. If you are uncomfortable giving the interview, the person you’re talking to will also be uncomfortable. They will sense your lack of confidence, and you’ll end up with bad interview results. Knowing your questions, being confident in your ability to ask them, and ensuring the conversation remains smooth and fluid will increase the level of comfort of the person you’re interviewing. This stuff isn’t easy, and that’s why there are professionals that get paid a lot of money to do it for you. BUT, I’m cheap, usually working with limited resources, and think there is real value in doing it yourself… so here we are.
Being a confident, comfortable interviewer begins with proper preparation. To some people this may seem like a waste of time. You think to yourself, ‘I’m a personable guy/girl. I know how to have a conversation’. Well you may know how to shoot the sh*t with your buddies, but getting the most out of a research interview is a whole different ballgame.
For me, preparation means practice. I’m an active learner so I like to do some practice interviews with friends and family before I talk to my first unknown interview subject. Practicing with friends and family does a few helpful things for me. It helps me: learn the questions, ferret out leading questions, test different follow-up questions, and practice moving seamlessly between topics. This practice reinforces my idea of planning for ideal and preparing for randomness.
My final piece of pre-interview preparation is a scripted introduction. Your opening to the conversation sets the tone for the rest of the interview. While I could get the conversation going on the fly every time, I find that having a script ensures I set the tone I want consistently and comfortably across the interviews. My openings always include:
- Something about who I am
- My goal for talking to them
- Something a little funny to lighten the mood.
Here is a recent intro I used at ServiceAlley:
‘First, thank you very much for talking the time to chat with me today. I’m Danny Beck and I work here at ServiceAlley. My goal for this conversation is really to better understand how you approach getting work done in and around your home. I may ask you some questions about ServiceAlley, but most will be about other random stuff. One thing I’ll ask of you is to please be as brutally honest as possible; I promise it won’t hurt my feelings. J In all seriousness, a lot of times hearing the negative feedback about our site or your general experiences is more helpful than hearing that something is wonderful. So let’s jump in.’
5. Finding research subjects
Finding people to talk to can be tricky. If you have a company that’s up and running and have an email list then you’re in good shape. If you’re trying to validate a new company or product idea, then you’re going to have to do a little more leg-work.
Here are a few methods you can try if you don’t have an email list you can recruit from:
- Intercept people on the street
- Talk to salespeople in retail stores
- Post an ad on craigslist
- Ask a friend with a company and an email list to send out an interview- recruitment email for you
- Cold-call people in your perceived target market (i.e. if you have an idea for new graphic design software, cold call some local digital agencies).
If you’re not comfortable trying any of these tactics, talking to friends and family is a decent place to start.
With ServiceAlley, since I had an email list to recruit from, I started by deciding what type of person I wanted to talk to. Did I want a stranger (someone unfamiliar with ServiceAlley), a user (someone who had used the site before but has not signed up), a member (someone who had signed up for our emails but hadn’t purchased anything or ‘converted’), or a customer (someone who had purchased something or otherwise ‘converted’)?
I decided for this I wanted a user or a member who was in my primary target market: 25-40 year old first and second time homeowners split between male and female. We have a large email list and I only wanted to talk to 20 people, so I used a combination of incentives and demographic screening to land on my 20 candidates.
I took a random sample of 5,000 people from our email list that had opened an email in the last 8 weeks and then drafted a clear and concise email inviting them to talk to me. I offered a small incentive for each conversation (I find the incentive that works best across almost any demographic is an Amazon gift card). But, before sending out my email, I created a simple Google Form that I asked each interested person to fill out. Having gender, age ranges and home ownership status as three of the fields in this simple form allowed me to easily track each person interested and gave me the demographic info I needed to select interviewees from the list that were in my primary target market.
Here is the email invitation I used:
I’m Danny, and I run ServiceAlley’s consumer research and product development team. As a valued member of the ServiceAlley community, I’d like to hear your thoughts about ServiceAlley and how you think about getting work done on your home.
If you could spare 15-20 minutes for a phone conversation with me next week, I’ll gladly send you a $20 Amazon gift card for your time.
There are only 20 spots available. If you’re interested, please fill in your name and contact info on this form. If you are selected to participate, I will follow up with you in a separate email to schedule a time to chat.
ServiceAlley Research Conversation Sign Up [this was a hyperlink to my Google form]
I sincerely look forward to speaking with you.
I’ve tried many different types of subject lines for these interviews. Something short, simple and vague like “Quick question” tends to get high open rates but not great response rates. The subject line I went with for this round of interviews was “A $20 Amazon gift card for you”. Using this subject line I got a 45.6% open rate and a 9.2% click through rate. Not bad for an email like this.
Once you have your list of potential interview candidates, you can export your Google form to Excel, sort by your demographic criteria, and then use a random number generator to select your participants (just use the row numbers in Excel as your reference point for the numbers the generator spits out).
To keep yourself organized, try using the Appointment Slots feature in Google calendar. You can easily set blocks of time on your calendar from which your interviewees can choose from and sign up for. The appointments show up on your calendar and the interviewee will get a reminder. One caveat: both you and the interviewee must have a Gmail email address in order to use this feature.
6. Conducting the interview
With all of your preparation and practice behind you, conducting the interviews themselves should be a fun and enlightening experience.
Instead of calling someone on the phone directly, try using a free service like FreeConferenceCall.com, Speek.com, or Google Hangout. Services like these offer a combination of free calling, video chat and call recording. Oh yea, and don’t forget to take notes!!!
Now that you’re fully on your way to conducting kick a** research interviews, remember to keep an open mind. It can be very hard to stay objective when your doing research about something you’re passionate about. To help me remain objective, I like to ask a random colleague (someone who is not as close to whatever product/project I’m working on) to join me to listen in on each interview.
I often get more people interested in talking to me than I have time for. I hate tempting interview candidates with an incentive only for the majority of them to never get a chance to earn it. To make up for this, when creating my list of interview questions, I set aside ~5 that are more survey-friendly, objective questions with straightforward answers. I then send this survey to all the people I was unable to talk to and offer a high value (maybe $50) gift card to one random person who completes the survey. This typically gets a very high participation rate and prevents me from alienating my audience.
Consumer research interviews should be an integral part of the marketing and product management process of every company regardless of industry, size, or location. They have gained more mainstream popularity in recent years through Lean Startup and Human-Centered Design, and for the average entrepreneur they can be extremely helpful in determining if you have a product worth building, identifying a target consumer for that product, and doing continual product evaluation.
Consumer research interviews can be time consuming, but if you take the time to do them on your own, they can be very cost effective. I promise that the information you learn will be much more valuable than whatever incentive you offer the interviewee for their time.
The interviews I recently completed at ServiceAlley were enlightening for sure. Many people told us they valued a recommendation from a friend or neighbor over any amount of online reviews from strangers. However, they still felt general online reviews were necessary in their provider evaluation process after receiving a recommendation from someone they trusted. This information combined with a consistent discussion with interviewees about community list-servs led us to develop a new social based provider search on ServiceAlley and integrate an easy way for a user to get a recommendation for a provider from their Facebook friends. You can check out the new Friendorsements functionality on servicealley.com or take a look at this great walk through of how it works on our company blog.
Do you conduct research interviews at your company or your startup? How does your process differ from mine?