Measuring immeasurable learning

In a comment worth reading in its entirety on my previous post: Solving test scores problems, Virginia asked me:

So what do you think would be an effective measure of student learning that could be used as they move from state to state?

I came to the conclusion a few months ago that authentic learning is immeasurable. When we isolate a small portion of learning ecologies to measure learning, we have excluded the diversity that makes the learning effective. We've merely captured the useless, showy display of "pseudo learning". We presume the measured learning will predict the subsequent applications of the learning in other contexts, yet 90% of measured learning never gets used. Instead it is quickly forgotten after the test or final exam.

Here's a different set of questions to be asked in order to support learners getting ready for results-only work environments and P2P collaborations:

1. What differences can you make?
2. What results can you produce?
3. What problems can you solve?
4. What outcomes can you realize?
5. What symptoms can you alleviate?
6. What changes can you facilitate?
7. What strategies can you revise?
8. What designs can you create?
9. What critiques can you offer?
10. What future vision can you provide?

The answers to these questions depend on prior learning, but that learning is immeasurably complex. The answers depend on the kinds and quantities of experiences with striving for these achievements. They reflect how much of the learning was hands-on, correlated with immediate feedback and fine turned by making mistakes. They indicate how much intrinsic motivation was engaged by the very selective use of extrinsic rewards. The answers also depend on the amount and quality of reflective practice to integrate those valuable experiences and to make them accessible in other situations. The answers also depend on the social context of interactions among mentors with insights about common mistakes, exemplars worth imitating and cohorts facing similar challenges.

These questions seek answers of quantity. Another level of complexity gets added by considering the quality of the responses:

• How well can you deliver?
• How effective is your impact?
• How enduring are your interventions?
• How valuable are your contributions?
• How supportive are your interactions?

With so much complexity opposed to simplified measures of learning, it seems to me that it's wise to let the results speak for themselves:

• Show us what you can do when immersed in these opportunities
• Demonstrate your ability to handle those challenges
• Prove you have the capability to deliver these outcomes

We have left the realm of testing, and entered the realm of tryouts, obstacle courses, and performances. We've abandoned the false premise that authentic learning is measurable. We let the immeasurably complex learning get revealed by the results.