The temptation is to use a on-line survey tool to save your time, but I think for your early customers a questionnaire may only give you the answers that you are looking for, not the information that you need.
Conversation Works Best With Early Customers
One on one conversation works best in my experience.
It’s important early on to ask open ended questions and to consider your product more of a hypothesis (See Steve Blank’s “Four Steps to the Epiphany” for more on this framework) than an accomplished fact. Even though it’s been debugged and ready for rollout it doesn’t mean you understand the benefits that customers (much less prospects) perceive that it offers.
You should also consider instrumenting your product if it’s SaaS (or adding a “flight recorder” if it’s on-premises software or delivered as an appliance) that with the user’s permission can “phone home” some usage patterns. In particular you want to be able to assess how much use (and what commands, command options, service areas, etc.. are being accessed) they are making. It’s not uncommon to start removing commands, options that are little used.
You should pay as much attention to your “dropouts” as much as your “frequent flyers.” With the kind of customer counts you are talking about you should be trying to e-mail/IM/Skype/call as much as construct a survey. Even up to a 100 or so early users you want to be as open ended in your data collection as possible.
Don’t Wish For Smarter Customers Or React When They Call “Your Baby” Ugly
It’s easy to become frustrated or wish for “smarter users” when your customers look at the value of your offering differently than you do, or don’t adopt certain features or commands that you thought would be compelling. Sometimes it can help to have a third party interview customers and non-customers as they will have less of a “you are calling my baby ugly” reaction.
One thing to focus on as you scale up and add more prospects is how your existing customers invite new folks to evaluate your offering. What is the value they promise if someone new adopts: this “language of referral” is extremely important. You should probe for it in your conversations and incorporate it into your messaging. It can help you to identify distinct types or segments of users who get different kinds of value from your offering.
Maximize Learning by Being Efficient With Customer’s Time
The temptation as engineers is to look for a technology solution that’s efficient with your time, but surveys and the like to channel answers along pre-determined paths. This can cause you to overlook real benefits, and real problems, with your product–especially on the part of your early customers.