How do we want to measure success #63
Description
Lorraine and I checked in on this today. Previously we've discussed:
- number of issues
- number of comments
and the ratio of these suggests something how the levels of collaboration etc etc and then there is a bit of a black box for "we'll feel when it's working".
Today @lorrainechu3n noted the feedback form as a key place to consider where we measure success, so that makes sense. Follow-up action here are:
- decide what makes sense to find out at that point
- suggest wording for questions
We both have paid close attention to the stated goals in designed the site:
- Generate fresh ideas, rapid prototypes, and an understanding of key challenges for new projects by Monday COB.
- Energize existing projects with momentum, enthusiasm, and new contributors to face their biggest challenge.
- Build on the ideas of OpenCon collaborate and seed it with an active community before, during and after the event to enable the Do-a-thon to have a lasting impact.
and our success should be measured vs this obviously. Breaking this down a bit leads to the following as things we'd want to measure:
- fresh ideas
- prototypes
- understanding of challenges
- new contributors
- more momentum
- more enthusiasm
- community activity pre / post & during
Following that chain of thought... you can start to consider ways to measure each:
- fresh ideas
First question here is "for who?". For the people having the ideas or for "us" or "the community". I can say it was intended as the later - but that isn't to say the former has no value. Perhaps this is a point where encourage challenge --> project progression #62 comes into effect. This, to a rough approximation, may be a way to measure it unless people have a new idea for a project elsewhere and immediately put it up as a project. In this case, we may just need to go through by hand afterwards which I expect would be manageable as a one-off exercise. - prototypes
We could simply look to count this... and then the question would be how to make that efficient. - understanding of challenges
Potentially measurable as encourage challenge --> project progression #62 again, or through the outputs generated per challenge? - new contributors
if we presume that most people at OpenCon are the sole representatives from a project (this is almost always the case) then a purely numerical approach here could work. E.g % of challenge issues with a comment, or something like that. Of course, this neglects to take into account if they're lasting, but this could be covered in a survey after? - more momentum
Needless to say... this is hard to measure. Could be asked in the main feedback survey or in a follow up for people who took the lead on projects/challenges. We have been considering ways to encourage others to show their appreciation for people as part of this, and perhaps that's a way to handle it. - more enthusiasm
As above. - community activity pre / post & during
Although a bit reductive, the number of issues/comments here seems like a perfectly fine measure of activity.
- Finally, this all relies on making sure people bring things back to issues. I think we've been careful about encouraging this... but I do want to note it's intended that:
- throughout the event (and before / after) people update their issues
- before breaks people document what they've been doing
- after breaks people check in with what's happened
and perhaps that's not clear enough yet!
This is closeable once we've decided a path forwards with this, but should be followed up with smaller issues.