At the Community 2.0 conference, I noticed there were many attendees who were in the very preliminary stages of community development – some were sent to investigate potential applications for the company, some were in the middle of their planning process, and in our case, our first community, MyCancerHub, was launching the following week on May 19, 2008. For this audience, there was a wealth of information and presentations, but Shannon and I were just as interested in learning how to actually measure the success of our community once it was launched.
Just as publishing was the “1.0″ predecessor to participation in “2.0″, statistics like page views and unique visitors in communities should take the backseat to measurements like engagement, sentiment (which I am covering next week), and social technographics. To understand social technographics, I highly recommended reading Charlene Li’s post on “Forrester’s new Social Technographics report.”
User Engagement in Online Communities
Although it’s somewhat open to interpretation, I understand engagement within the context of a community as “the level of interest in an idea, person(s), or brand that depends on the content and environment within.”
Within a community, social network, or a participation-enabled website, tracking engagement is much more effective than page views and most standard metric data in many cases. The latter can easily be skewed by the site activity of community managers, moderators, and developers. Even putting high value on the number of discussion posts, members, groups, etc. can be a mistake if the quality of the content isn’t there. This is where engagement and sentiment measurements prevail: they take quality, depth, and feelings of member content into account to measure success based on the core goal of any community: interaction.
The Real Question: How Do I Measure Engagement?
Francois Gossieaux’s (President of Corante) pre-conference workshop, “How to Measure Progress and Success in Business Communities”, provided an overview of measuring success, but it didn’t really get into the “how-to”.
As it turns out, it’s not easy. Engagement can be measured as lightly or as heavily as you wish. It can be based simply on loyalty and return visits, which is only a quantitative approach that doesn’t account for the quality of content. A much more comprehensive calculation is covered in this post by Eric T. Peterson on WebAnalyticsDemystified. Eric’s measurement compiles the following indexes (to get more details on each, I’d recommend reading his post):
- Click-Depth
- Recency
- Duration
- Brand Index
- Feedback
- Interaction
- Loyalty
- Subscription
Even from my fairly inexperienced point-of-view, this seems like a perfect way to measure overall member engagement within a community. However, I know that within our Online Communities team (which is just Shannon and I at this point), we don’t have the resources/time to put together reports based on these indexes.
This is where new “Measurement 2.0″ companies come into play. We met with Networked Insights at the Community 2.0 conference, and we learned that their all-in-one community product compiles a fast-paced, discussion-based community platform with their Insight Platform measurement product, which is its real competitive advantage. Their algorithm, which I’m sure is somewhat similar to Eric T. Peterson created, scans the community content in real-time and provides highly-customized, on-the-fly reports. The disadvantage is that their Insight Platform isn’t able to “plug in” to any other community platforms but their own.
Networked Insights’ Interface
I’d like to see Networked Insights work with the larger white-label community vendors to enhance the existing reporting features. This would allow NW to expand the business model to fit more community types – not just market research – and add value to platforms that need expanded reporting features. We love Clearspace Community platform, but to be honest, the reports and metrics it provides are extremely lacking.
Where’s the Value in Measuring Engagement?
This is a harder question. I think it’s easier to answer for community types like market research and customer service. For market research, there’s already cost savings from less face and phone time from focus groups and studies. Measuring and aiming for a higher level of engagement means deeper insight for research, and therefore added-value for the community, maximizing the ROI. For customer service communities, a higher level of engagement means cost savings from a high level of engagement that might normally be on the phone….again, maximizing the ROI.
For entertainment and support communities, it’s more complicated. I’m still forming an opinion, but this brought up a heated debate during the “Customer Engagement: The New Metric and What it Means to Your Business” presentation from Networked Insights founder, Daniel Neely. The audience seemed very doubtful about the value of engagement, which really surprised me. I would think that social media enthusiasts would be embracing this measurement with open arms, but maybe the concept is still ahead of its time.
Tags: Community 2.0, engagement, metrics, Networked Insights, social technographics, value
