In 2005, we designed a study to explore and address a common concern in the marketplace at the time: Namely, that online community members would become brand fans as a result of interacting with each other and sponsoring companies over time, and that, as brand fans, their feedback would be overly-positive, systematically skewed, and untrustworthy.

Building off our 2005 methodology, we analyzed over 100 “standard” concept test survey questions (e.g., intent to purchase, appeal, believability, etc.), grouping responses into tenure groups. We tested for significant differences across groups and found nothing at all in over 99% of the cases. For the one and only significant difference we did find, the longest tenured community members were also the most critical.  In other words, community members with the shortest tenure were more likely to rate concepts positively than were members with the longest tenure.

What follows is a write-up of the original 2005 study. We invite you to read it with the understanding that now, six years later, we see the same dynamic at work in our over 400 private market research online communities that we saw then: Highly engaged, long-term dialogue with a brand doesn’t create risk, it mitigates it, by enabling an unparalleled level of specificity and candor in member feedback.