I’ve heard lots of talk recently around metrics and ROI on social business tools. Many of the Intranet, IT, HR and Marketing people I speak to are looking for metric packages that will provide some of the traditional measurements around online sites.
In terms of ‘measurement’ here are some of the criteria I use to generally benchmark knowledge and collaboration communities and programmes. There are some basic metrics to measure participation and activity within the communities.
• Total number of users
• Number of new users per week
• Number of new posts, threads (plus response), ideas, blog and other content
• What are members doing in the community?
• What are the popular trends in posts? Blogs? Forums?
• What resources are being used?
However, these basic metrics won’t provide a true understanding of the value around the collaboration that a platform could provide to individuals and the community as a whole. For the measurement of value around collaborative communities I would generally look at analytics that articulate:
• Attraction
• Attention
• Adoption
• Social Knowledge – this can be defined in many ways such as assets being shared around a community (and beyond) and related practices emerge.
• Relationship development – the ability to create new relationships and networks that previously didn’t exists
Some of the questions I ask to evaluate these items would include:
• % of members / users which make a contribution
• Members active within the past 30 days
• Contributions per active member and the value of these contributions related to the purpose of the community
• Content popularity
• Number of relationships created by individuals – look at followers and participation in threads
• Discovery of communities – have members joined communities outside their ‘physical’ or existing network
The various social business tool reports (i.e. Community Manager Reports in Jive) will not provide this type of information and much of it will be antidotal evidence. Social analytics are poor within most social tools (it will be a major revenue stream for a vendor that can start to provide some of the softer metrics that articulate quality and not just quantity).
Over the years of working within communities I’ve compiled a list from various sources that are useful in articulating some measure of value or return on the community. Top 30 are:
1. What % of newcomers remain members for more than a month.
2. Speed of replies to discussions. How quickly are discussions receiving a reply? The faster the responses, the higher the level of social presence within the community and the greater the level of participation.
3. The % of newcomers which initiate a discussion. This highlights whether newcomers may be unmotivated or intimidated to start discussions.
4. Language and tone of voice. What language do members adopt when they address each other? Is it formal and polite? Is there friendly banter? Is there a sense of familiarity? This will let you know what stage the community is in.
5. Do members’ interactions have continuity and depth? (Are members engaged in productive, on-going, interactions?)
6. What collaborative activities are emerging?
7. What documents, tools, resources, or other artefacts are created and utilized. (How are these useful to the members?)
8. Is the community providing value for its sponsors
9. Is participants’ involvement in the community affecting their professional practices and learning.
10. What are the on-going practices and processes that contribute to the “life” of the community and keep members engaged?
11. How is knowledge being shared within the community? Beyond the community?
12. Are leaders or roles emerging in the community? In what ways? How are they being cultivated?
13. How are members being supported in the community?
14. How are members contributing? Posting? Replying? (When? How often?)
15. What are the prevalent patterns of interactions?
16. What is the proportion of new topics that get 5+ replies? The percentage of new blogs that get 5+ comments? The percentage of (video, audio, lesson plan, etc.) uploads that get 50+ downloads or 5+ comments?
17. What proportions of new topics or new blog posts are un-responded to or uncommented on (an important measure of the responsiveness of the community, which in turn affects key factors like trust)?
18. What is the average new (topics, replies, blog posts) created per member?
19. What are the emerging benefits of the community for members?
20. What is the average number of “followers” that community members list or have collected in/on their member profile pages?
21. What are the proportions of topics or replies that specifically relate to your practice?
22. The proportion of replies where links to potentially helpful resources or other referrals are provided
23. The proportion of replies to a post in which helpful or constructive advice is directly provided
24. The proportion of replies that build on previous posts (as opposed to just responding to the original poster)
25. The proportion of replies that contain offers of collaboration or introductions to potential collaborators
26. The proportion of replies that contain creative, novel, or innovative ideas
27. The proportions of replies that summarize, distil, or synthesize prior posts/replies
28. The proportion of posts in which community members show or express vulnerability, such as a lack of domain knowledge
29. The proportion of posts in which community members share personal stories
30. The proportion of posts in which community members are (emotionally) supportive or helpful to other members
