Global Fiat Currency Image by LeaderAmp, All Rights Reserved
By Dr. Matt Barney
View original publication on LinkedIn
For nearly everything in business, leaders want to know if investments pay off. After all, who wants to waste money? A very recent study by The Conference Board, EY and Development Dimensions International shows that when measured at the firm level, leader development seems to pay off, consistent with longstanding research. But the new insight this year is that corporations that extend their development of leaders far below senior levels are 4.2 times more likely to outperform in financial measures like EBITDA, operating margin, and return on equity. They also tend to have better diversity results.
And surprisingly, the money spent on developing leaders is less potent than the hours invested to support their development. The DDI study showed that hours spent had a 68% larger relationship with business outcomes and financial outcomes, where the absolute amount of money spent had no such link. However – isn’t this a bit of a conundrum? The more busy leaders practice, the better they get – but leaders are often extremely busy and forget to practice. And how can you afford to scale this to the masses, without breaking the bank? The good news is that some new technology addresses these gaps, but before that, it was tough to scale executive coaching throughout the enterprise.
Is ROI a Good Framework?
Before I review a few new ways to improve the pay off of coaching, what evidence is there that traditional coaching paysoff sufficiently compared with alternatives? Almost 100 years ago in 1920, Du Pont created the idea of Return on Investment (ROI) – a financial calculation of whether the investment’s returns from one asset was more or less valuable than another. Particularly in people-oriented investments, the idea of ROI has been attractive, and especially promoted by practitioners such as Jack Phillips, who built upon Don Kirkpatrick’s framework for evaluating training, and whether they paid off. The USD$2B coaching industry – including my company – has an interest in trying to show that our solutions are worth the money we charge our customers, so we’d like to show an ROI if possible.
Unlike the DDI, Conference Board and EY study, too often coaching is evaluated at the individual rather than team, or organization levels for impact. Four comprehensive and systematic reviews of coaching’s effectiveness were done in 2009, 2013, 2015 and 2016 consistently show that coaching really works to shift cognitions, affect, skill and behavior. One meta-analysis showed across 591 coaching studies, that other people noticed a moderate-to-large amount of improvement in the leader’s skill gain and/or performance improvement. And in that same study, the leaders being coached in 173 additional studies reported even larger impacts than what observers had noticed. But these outcomes traditionally come at a premium cost: a six-month coaching engagement costs $15,000 to $75,000 per person, and time off the job for highly compensated executives. Because of the benefit and the expense, it’s easy to see why people would look to ROI estimates as a way of trying to evaluate coaching, training and alternative investment options. At the same time, many experts who focus on the micro-level of analysis are skeptical about whether it is sensible to use an ROI framework to evaluate coaching’s impact.
At the individual level, some coaching experts worry is that there are often many flaws in the way ROI is computed to trust it all of the time. Too often, many factors other than that leader are driving movement in operational, customer or financial measures, making it hard to tease out the impact from their improved performance caused by coaching. For example, imagine a coach trying to help a CFO cope with the great recession of 2008 – a successful engagement might have meant less fiscal loss for the organization than others in the industry, or lower amounts of loss than if the coach hadn’t helped the CFO improve his or her performance. Another risk with ROI calculations is that the estimates are done retrospectively and subjectively. Past studies using different methods range from a low ROI of 221% to a high of 700%, but these are done by looking back on the past and asking people to estimate the financial gains which may not be accurate or precise, and are likely to be subject to numerous cognitive biases. To avoid these flaws, scientists always prefer experiments. The gold standard for research – the double blind, controlled experiment, with pre-assessments, control groups and random assignment to conditions – is rarely done in ways to make strong causal claims about coaching causing financial benefits.
So if ROI is flawed, what’s the best way to evaluate coaching? Perhaps both the detailed and big-picture approaches need to both be used.
Cross-Level and Time Series Evaluation
At the individual level, a better and unbiased method is examining the degree to which the coaching objective was achieved. There are lead indicators for this – like clients practicing, and perceiving their own baby-steps of progress. And there are lag indicators, such as 360s showing that others noticed improvements. Ideally, we’d like to show how coaching drives sustained behavioral improvement over multiple 360 re-measurements, to confirm that all gains were permanent. Last year Joel DiGirolamo of the International Coach Federation, and I created and calibrated another lag indicator of coaching outcomes. It looked at the degree to which the coaching goals were achieved, and the impression of the leader as to whether the effort to achieve the goals were worth it – six months after the engagement stopped. These efforts don’t require perilous financial assumptions, or crude estimates of how much a leaders improvement contributed to financial measures.
As the new DDI study has shown, if we really want to look at financial impact, the aggregate across many individuals is a more pragmatic way to examine likely monetary impact, because when coaching is the primary way an organization invests in multiple leaders, it is easier to do experiments at the team, department or organization levels and control for possible confounding effects. These results show further that when firms ignore top practices grounded in science, they’re likely to waste about $1.6M, and 1,900 person-days. This suggests that there is a negative return when firms ignore evidence-based approaches to coaching.
For most of my career, I struggled to scale development like coaching globally. Fortunately, new cloud/mobile technology solves the problem of supporting clients attempts to practice on their jobs. By using new forms of Artificial Intelligence, clients can schedule the days and times they wish to practice in-between sessions with a coach. The coach can then see how they journal about the lessons they learn from their attempts to practice without spamming, calling, emailing or otherwise bothering them to ask how things are going. By reading and responding privately to a clients journal, coaches can make their relationship embedded into each client’s daily life. And crucially, from a financial perspective, the AI allows for less expensive coaching models, so a single coach can support 4-5 times more clients than they could with traditional models. When these new, more affordable coaching models are plugged into the same financial models as the 2018 DDI study, coaching can be an even bigger pay off at every level of the corporation.