From time to time in technology transfer I hear the quip “you can’t manage what you can’t/don’t measure.” The general drift of the quip is that something has to be counted or measured to determine whether a university IP program is any good. The idea is that by means of metrics people will be “held accountable” or “given incentives” and this focus on the “numbers” translates into more efficient, if not more effective work. Unfortunately, the quip is wrong on a number of levels. It reflects a bogus desire to create data to that makes a good impression, except in the bozonet, where it is more like a mating call.
Let’s look at this quip and underlying desire:
1 The quip misrepresents Deming’s advice
2 The data universities report publicly is useless for innovation management
3 Reported data obscure practices and outcomes
1 First off, the quip comes from Deming, and he means something rather different by it. The quip is a distortion of Deming’s interest in the use of data in management. Here’s a quick discussion. Deming argues that there are many things that one must manage that can’t be measured. Quippers of course don’t get it that they have got the quip wrong. That’s a bad sign. They want a justification to measure things. Why? Perhaps because they do not have any idea what they are doing, and numbers provide an appearance of orderliness and control.
2 I’ve been through a lot of technology transfer annual reports, I’ve sat through meetings discussing incentive programs for technology transfer managers, and I’ve spent more time than I’d like with the AUTM Licensing Survey and with academic papers that try to make sense of the “data” there. This sort of data is not used to manage innovation or make innovation decisions. No, it is used by management to deal with itself. Data of this sort is an after-effect, chosen for spin not substance. One works at the numbers to make the numbers look good, not to manage innovation. Again, another hallmark sign of a bozonet–of management folks more concerned with how they appear than how they work.
The general idea of university metrics is, up is better. More disclosures is good, more patents filed and issued is good, more licenses, more money. Some tech transfer offices even try to stage payments so revenue is always increasing, year to year. That way, they don’t have to explain to senior administration why there was a “drop”. They don’t have to reveal that patent licensing is a step function, that money comes in chunks until there’s a big royalty-bearing deal, and then it comes in as a function of sales activity, until the patent expires, when there will be a huge step down, and it has nothing to do with the performance of the office. But who in senior administration cares?
3 The AUTM Licensing Survey “data” obscures rather than reveals the nature of the activities being reported. Offices do not report their expenditures on salaries and operations, they do not report inventions made with government funds, they do not report dates of first commercial sale or use, and they do not report help to small businesses, collaborations with industry, or inventions developed outside the ownership claims of the institution. They do not report the ratio of claimed inventions to released inventions, nor the number of unlicensed inventions to licensed inventions, nor licensed inventions with commercial product for sale to licensed inventions without such sales.
When it comes to job creation, in particular, technology transfer offices have no clue. The AUTM approach to jobs is to take a typical royalty rate, like 5%, and multiply the reported licensing income by the reciprocal (this, purportedly to get the total income to a company), and then dividing this number by an average industry salary to “estimate” the number of jobs “created” by the license.
None of this holds up. For established companies, a licensed patent might merely add to an existing product line. An effect of the license then is to move an existing income stream into a royalty calculation, so that there is now less profit than otherwise. The licensed patent did not create the revenue stream, just appropriated it for royalty calculations.
Licensed patents may provide a basis for eliminating jobs, such as by creating production. A new technology may eliminate jobs in a company’s value chain of suppliers or distributors, may eliminate jobs in competing companies, and may suppress the creation of jobs by allowing a company to hold monopoly rights and not fully develop the underlying invention. Worse for regional economic development is licensing a patent to an out of region competitor, who then proceeds to destroy the business of an in-region company. Yet universities do not report how many licenses are made to in-region companies. What do they not want folks to know?
The problem of using licensing income to estimate job creation is further compounded because many university technology transfer offices require up front payments and annual license “maintenance” fees regardless of sale of product. For these companies, payments under a license may mean spending investment capital on license payments that otherwise would go to creating jobs, and such payments may not reflect new money obtained by the company. Upfront payments, legal cost recovery, milestone payments–these do not reflect actual sales under a royalty rate and should be tossed.
Finally, we must look at profits. Since only a very, very few university patent deals result in substantial income to the *licensee*–well under 1%–these deals tend to dominate all accounts of income. Yet where there are billions in income, even after recovering costs of development, only a portion of that profit gets plowed back into company growth. Some goes to shareholders, some goes to budgets that transition from reliance on other products’ income to the new source. One has to know the reinvestment of income on the royalty base to determine how many new jobs were created by the patent license. It is, we can expect, a whole lot less than the numbers put forward by university technology transfer offices.
Job creation–even an estimate–is not available by means of treating licensing income as an earned royalty, constructing an assumed “base” and then assuming that base supports “jobs”. One might say it is naive. But it is not. It is intentional and aiming to be sophisticated, at that. The idea is to find a big number (what companies “make” based on patent licenses–meaning, actually, an iffy claim regarding the royalty “base”), and use it to claim the importance of university technology transfer for policy makers and senior university administrators. Essentially, this is cooking the books. At best it’s irresponsible. At worst, it’s a public fraud.
It appears, however, at least some policy folks *want* to be snookered by these sorts of numbers. It is to their advantage for the books to be cooked. If enough folks chime in, especially academics who write papers with footnotes citing each other, then the fix is in. The quippers who want metrics cannot tell the difference between spin, fraud, and incompetence. They don’t care. It’s a good life, bullshitting everyone and collecting a decent paycheck. As long as it looks official, who has the courage to call an end to it? University technology transfer offices could just pull out, not supply AUTM with “data”. That would be a good start.
The reality is, next to nothing that matters for management of innovation is reported publicly by university technology transfer offices, and next to nothing that would indicate productivity. Beyond the utter obviousness of keeping track of money, there’s little to suggest that the “metrics” reported by university technology transfer offices have anything to do with management of innovation. We can go further and question whether the thing that innovation needs most, early on, is “management”, and if so, is institutional management the right form of “management”, and if so, then should that management impose itself on innovation activities or should it be invited to participate?
My sense is: things work a whole lot better when management is invited, and arrangements are worked out for the nature of that involvement. So that’s my number one metric for university technology transfer: how many inventions were presented to the university for management, and how many were compelled to be given over to university control? That would say a lot about whether a university is open to innovation, or is trying to play the dragon corporation, claiming everything it can, terrorizing the population, doing nothing with treasure but hoarding it.