When is Social and Emotional Data Just Vanity Metrics?

Is it just hype to claim that social and emotional data can shift school culture?

Edutopia put out an article that promotes “warm data” as a valuable addition to the traditional “cool data” that is often collected in schools. 

The article entitled “Using Social and Emotional Data to Promote a Positive School Culture” by Victoria Curry and Mike Setaro suggests that

Quote “The visual representation of this information and analysis can cause positive shifts in school culture. 

Highlighting warm data to support both relationships and social and emotional learning creates infinite possibilities and promise for our students.” End Quote

Those are some pretty bold claims.

Actionable SDT-validated Tool: Classroom Climate Formative Assessment

Social and Emotional Climate

I was recently at the ASCD conference in Chicago and was asking around about climate measures which would be the more general term for the kinds of “warm data” promoted in the article. 

One principal was able to pull up the Panorama app and show me her dashboard, the array of measures that they compiled about her school. 

It looked similar to the array of data that Curry and Setaro shared with their article. 

I noticed out loud to this principal that some of the measures seemed to be on the undesirable side. 

So, my next question was, “What do you do about the ones that you want to change?” 

She didn’t have a clue.

There are two ideas that I want to share from the entrepreneurial world that I think are pertinent to understanding how experiential climate data can fit into school management. 

First, the Gartner Hype-cycle. 

Second, the difference between vanity and actionable metrics. 

Social and Emotional Climate Claims

Let’s start by focusing on the bold claims that Curry and Setaro made. 

Specifically, notice the first claim that visually presenting data with some analysis can “CAUSE” a shift in culture. 

If merely visualizing some information for analysis could transform cultures we would be able to change cultures quickly and easily. 

Anyone with any experience of organizational culture change or the literature about it knows that it is a long hard slog with no assurance of success. 

And merely analyzing some visual information, no matter what it is about, could not possibly affect culture in any meaningful way, at least on its own. 

Should we write their claim off as mere hyperbole, since, at face value it is absurd?  

I am inclined to take their claim as mere hype for a new technology, but I also want to look a little deeper to extract a potentially useful lesson. 

Their claim is evidence that measures of school and classroom climate might be headed towards a “peak of inflated expectations.” 

Applying that phrase “peak of inflated expectations” means that we are putting climate measures into the Gartner Hype Cycle. 

The Gartner Hype Cycle is a model of how new technologies are presented to the public over time. 

The key insight is that our media environment, with the help of the self-serving folks who have a vested interest in the new technology, have a strong tendency to over-inflate the initial expectations of what can be achieved. 

That leads to “the peak of inflated expectations” which gives way to “the trough of disillusionment” as the most unreasonable promises made by the early promoters fail to be realized. 

The disillusionment leads to a period in which expectations are unreasonably low. 

With lowered expectations the applications of the technology gets refined in the areas where it is capable of making a difference more reliably. 

That honing-in process for the application of the technology is called “the slope of enlightenment.” 

The end of the cycle is a “plateau of productivity” in which the technology has found its niche. 

Once it can reliably serve some function it fades into the background as a regular feature of the environment. 

Social and Emotional Hype

Curry and Setaro are obviously making an overinflated claim about what their favored data can accomplish. 

If we take school and classroom climate data collection to be a new technology then they should be regarded as purveyors of hype about it. 

What wise principals, superintendents, and policy-makers need to recognize is that experiential climate data IS a new technology that will have some potentially transformative effect on their schools. 

But, they should, independent of wherever the public perception of its value is within the Hype Cycle, aim to achieve the plateau of productivity as soon as possible. 

In other words, their leadership around the adoption of the technology needs to be attuned to reality, not the hype. 

I want to help get climate measures onto the plateau of productivity. 

In order to achieve that we need to get clear about how to make the data useful, not merely a fashionable addition to the datascape of schooling. 

That brings me to the distinction between vanity metrics and actionable ones. 

According to ProductPlan.com, a site that supports entrepreneurs, 

Quote “Vanity metrics are statistics that look spectacular on the surface but don’t necessarily translate to any meaningful … results. 

Examples include the number of social media followers or the number of views on a promotional video. 

While the data might seem superficially impressive, these metrics do not accurately reflect an organization’s key drivers (e.g., active users, engagement, the cost of acquiring new customers, etc.) and provide very little insight into how a product or initiative relates to broader … objectives.” End Quote 

Translating this into the standard grammar of schooling, consider the usefulness of a letter grade for a student. 

When a student is given a letter grade it is often a vanity metric that evokes some feelings, good or bad, but does not provide any guidance about how to to improve a poor performance nor how to maintain good performance. 

According to FirstRound.com, another entrepreneurial support site, 

Quote “Grades can become a tool for teachers to contrast students or for school boards to evaluate schools, rather than for students to improve.” End Quote 

The principal who shared the Panorama data visualizations of her school climate did not have any idea what to do about the data she had been given, therefore, they are vanity metrics.

The opposite of a vanity metric is an actionable metric. 

One way that students can be given actionable data is through rubrics that include the evaluation criteria for various levels of performance.

Those descriptions make it clear to the student what it would take to improve or maintain their level of performance. 

In my new book Schooling for Holistic Equity: How to Manage the Hidden Curriculum for K-12 I  am proposing that actionable climate data requires the use of instruments validated by Self-Determination Theory researchers.

Specifically, SDT-based measures of psychological need satisfactions, patterns of motivation, and degrees of engagement would be actionable because a recent paper has listed a clear set of behavioral Do’s and Don’ts that are scientifically known to affect those measures of climate. 

The same measures can be used by principals with regards to teachers because the same set of factors apply universally to all humans regardless of age. 

Social and Emotional Plateau of Productivity

The main point I want to make is that we can skip the peak and the trough and go straight to the plateau in this case because there is decades of scientific work that has already established a precise understanding of the psychology of need satisfaction, motivation, and engagement. 

That same literature has also reinforced how centrally important those same factors are to deeper learning. 

Curry and Setaro over-hyped the role of visualized social and emotional data in their article and it appears to me that Panorama is over-whelming principals with vanity metrics. 

What is clearly needed in the realm of climate data is that honing-in on how to make the data truly useful. 

That is where Self-Determination Theory can help. 

If they get clear about the causal model for learning that is implied by SDT and that I explain in my videos online and in my new book, then they can get right to the plateau of productivity. 

For more information about my new book or how to provide better school leadership visit HolisticEquity.org.

Thanks for watching!

This article was printed from HolisticEquity.com

Print Article