You're probably familiar with the work that Clifford Cobb and Craig Rixford did while over at Redefining Progress. Ten years ago, they published Lessons Learned from the History of Social Indicators (PDF) that did a great job of summarizing what we knew. The lessons are still highly relevant, so I was pleased when I caught a nice summary of their work at Rashid's Blog. I encourage you to read the article (it's only 40 pages), but if you're pressed for time here's the summary.
- Obtaining a figure does not equate to establishing a good indicator. This is because quantities should reveal qualities, but qualities are “ always ambiguous and that any statements about them are provisional rather than final.”
- Effective indicators require a clear conceptual basis. Ideally, concepts should be defined before data is collected, but in practice that is not easy. On the other hand, “although measurement can help clarify a concept, the concept itself will not simply emerge from data.”
- No indicators are free from values, because “ all serious indicators work is political”. Value judgements prevail from selecting indicators to the formulation of survey questions.
- Comprehensiveness may tamper with effectiveness, because historically the most effective indicators tended to focus on one single issue, by guiding people to consider deeper questions. In additions, interpreting indicators is more important than simply describing them. Indicators covering a small area for specific audiences tend to be more effective.
- The symbolic value of an indicator may outweigh its value as a literal measure, especially the fact that indicators serve as metaphors, not statistics.
- Indicators must not be confused with reality, because “even the best indicator is only a fractional measurement of the underlying reality”. Multiple indicators measuring the same social phenomenon may overcome this problem.
- Democratic establishment of indicators requires more than good public participation processes, the result of emphasis on procedural justice. This practice will effectively suffocate changes to status quo, and perhaps substantive justice, such as equal opportunities, should be emphasised instead.
- Measurement does not necessary include appropriate action.
- Better information lead to better decisions and improved outcomes, but this is not easy. This is because indicators only have indirect effects to policy making, and behaviour plays a larger part.
- Innovative thinking on causes of a particular social problem is often required to resolve this problem, because indicators have a function of enlightenment, which are supposed to lead people to reconsider the common understanding of the problem.
- Indicators that reveal causes, not symptoms, of a particular social problem should be searched in order to take action. Mere description of indicators without providing insight to trends is more unlikely to lead to remedial action.
- Indicators provide the basis for setting outcomes as long as one has control over resources, that is if developers of indicators “ have a connection to those with the power to make substantive changes.” This brings us to determine who actually has the power to take action.
Rashid also provides more analysis and summary information, so check out his blog as well.
