Tracking discussions: using near-real-time participation data to go beyond ‘getting enough posts’

The real-time demands during an intensive online discussion are multifaceted – the project team needs to be well organised and prepared to respond to the unpredictable aspects inherent to the dependency on user-generated content.

Introducing the ‘participation report’

Although we stress that discussion events can never be fairly compared as ‘like-for-like’, its not surprising that project teams frequently look to participation levels as an indicator of success.

In the first few hours of a discussion event, particularly, project teams are most concerned that contributions start to flow in sufficient quantity.

So without fail, the first agenda item on the event day one project team conference call is ‘do we have enough posts?’

We tackle this question by sharing a ‘participation report’ with them, which we produce immediately before the call. The real value of this report is not to provide a Yes / No answer to the burning question, but to shift the focus away from quantity and onto quality.

Ultimately, our aim is to use near-real-time participation data to improve what happens next during the discussion, regardless of what has come before.

How data can drive discussion facilitation activities

There are two kinds of activities we use data to inform: generating more posts; and generating the right kinds (in terms of diversity / quality) of posts. As project teams get more confident and / or exceed their expectations of quantity measures they shift the balance of their attention to the latter.

Most first-time project teams don’t have particularly ambitious requirements, and as such reports we prepare for them generally include just the following:

  • Raw data on visits to the discussion group by members
  • Raw data on activity (including posts) within the group by members
  • Summary presentations of ‘top’ contributors
  • Identification of ‘lurkers’ – visiting members who have not posted

When reviewing the data we always invite the project team to focus on what the data reveals about individuals. Because it is common that events take place over many timezones and last just a few days, at any given point there is a strong likelihood that some participants will be offline (asleep, at home, in transit, in meetings, at lunch etc).

So we prompt project teams to connect the data they see with their aspirations for participation in the immediate future i.e. the next 3-4 hours. Questions are: Who is likely to be around for the next period? What could they say about the questions we want to focus on during this time? Who is best placed to send them a prompting message? How do we make it easy for them?

Lurkers are an obvious target for outreach. The assumption is that if a person has visited the discussion space a number of times they must be interested and available enough but may need a little support to jump into the conversation. Commonly, a quick prompt from the project team makes the difference.

Making quick sense of the data

This core set of information needs to be clearly presented to the project team in ways that make sense to them. However, it also needs to be quick and easy to gather, so data can be provided within a matter of minutes.

Below is an extract of a simple tabular presentation that is commonly used, cheap to produce and accessible.

In this example data is extracted directly from the application used to host the discussion and compiled into an Excel document.

Within this an ‘overall summary’ the following raw data is brought together using a series of pivot tables and manual data entry. These are:

  1. A list of group members
  2. The quantity of visits to the group by each member
  3. The ‘project team’ status of each member
  4. The number of contributions made by each member (split by members who have visited the group and those who haven’t)
  5. Additional demographic data – in this case country of residence
  6. A list of key ‘lurkers’ – used to make recommendations to the project team for outreach activities.

This presentation is simple but effective. It is flexible and accessible and has proven useful for many project teams. More than anything else, it reminds us to take the challenge of bringing together a defined set of people around a chosen agenda in very practical terms.

Even when discussions seem to flow easily and far exceed the aspirations of their convenors and participants, significant ‘on the ground’ efforts play a big part. The key to doing this well is good data, and participation reports are essential in augmenting the project team’s knowledge of the situation so they can make effective decisions to improve the quantity and quality of contributions to online discussions.

Leave a Reply

Your email address will not be published. Required fields are marked *