A personal update and musing on certain failures of statistics in medicine

Just a note here about my health and the direction for this blog. First, it’s great to see that people are still coming to read articles about technology transfer and cockroach living. Maybe there’s hope out there, or at least curiosity about how things have come to be in the strange, often confused world of research IP. Now about the cockroach living. Against the odds, I am still!

Last March, one Sunday morning, I went full cardiac arrest–cut a three mile walk short for what I thought was indigestion, came home, sat down with a glass of water, and blinked out. No warning, no pain, no light in a tunnel or floating above anyone–just gone. My wife happened to be in the room and heard me collapse–I sent the water glass and tray table to the floor, I’m told. She did all the right things–called 911, unlocked the front door, started chest compressions. The EMTs arrived and defibbed me twice. Second time was a charm, and got me to the local hospital for an emergency stent. I was in an induced coma for about a day, with some uncertainty about what my situation would be, if I even would have a situation (40% chance said the EMTs). Of course I wasn’t uncertain because, well, I wasn’t aware. They brought me out of the coma when things stablized, and hey, I seem to be okay, considering. I did a few months of cardio rehab–which mainly meant pushing all their gym equipment to its limits for an hour twice a week with a bunch of sensors on. I had basically no risk factors–no family history of heart disease, not overweight, not smoking or using street drugs, working out daily plus five to ten miles of walking (fast)/jogging six days a week, cholesterol was not “elevated,” no diabetes. Diet was mostly meatless, fish, low salt,

When the rehab folks did the diet assessment (after the extended depression/suicide screening, which was in its way mildly depressing to consider because, hey, that isn’t me), they went all glum and said that no one scored as high on their good diet measures. So it wasn’t easy to find a lifestyle excuse. As the cardiologist who added four more stents a few days after the initial one said, “It was bad luck, not your fault.” I think he was trying to reassure me, but it sounded more like “we really don’t care why you of all people got a clot in a coronary artery. It happens, and well, we are interventional cardiologists and our market is dealing with these bad luck things not finding ways to prevent them in the first place.” I’m sure that’s not what he meant to impress me with–he was very nice, very skilled. But still.

So I went on a deep dive into the literature. Read all the clinical trials for all the various drugs they had me on. It became clear that most of the trials were useless. I have 400 level engineering statistics, and apparently doctors don’t get that far. As one example–statins. The clinical trials are reported as reducing the risk of another cardiac event by 30% or so by blocking the body’s ability to make cholesterol, because, well, cholesterol is found in the coronary artery walls where there are plaques, and co-occurrence is asserted to be a sign of causation. But that 30% figure is meant to mislead–really, it is intentional. Dig into the numbers and the definitions and what the trials were finding was that in the control group of about 4,000 people, all with a range of risk factors and lots with serious heart problems (crappy LV ejection fractions, diabetes, smoking, and on and on), 3 out of 100 people during the study period had a trip to the hospital for a cardiac-related event–could be angina, or another heart attack, or whatever. In the study group–dosed with a statin–only 2 out of 100 people had another trip to the hospital with a cardiac issue. So, 3 reduced to 2 is that 30% reduction in risk. But the “risk” is already rather low–and this is for folks with some pretty tough conditions to start with. We are talking that dosing with a statin a group of 100 bad off cardiac patients saves one of them a trip to the hospital for a cardiac event in the next three to five years.

I had questions. First one was: why did the cardiologist think that I was the one patient out of 100 that would be saved that return trip to the hospital? The cardiologist I was assigned to (not the one I was referred to–we are talking institutional bureaucracies here, I guess) had no answer and went into a mini-tirade about how he could be sued for malpractice if he didn’t insist on statins, apparently for all his cardiac patients, even though the benefit would be realized (if the clinical trials were any indication) by only one out of 100. Why me? He had no idea how to handle the question, no understanding of the math reasoning required by the clinical trial design.

The clinical trial was set up to justify (in a twisted way) the approval of a statin drug for treating cardiac issues. That’s the purpose of a clinical trial–to get approval to sell a drug. Perhaps folks could be forgiven for focusing their design on that. The clinical trial design argument is a population argument. It is population statistical math. A basic premise of statistics is that any particular statistic is chosen, weighted as it were, for a purpose. An average, for instance, throws out information about variation and is useful when one already knows the structure of the things to count or measure is relatively uniform. An average is crap when the data follow a power law, for  instance. The “randomized” part of the design also throws away otherwise helpful clinical information that would be specific to individual patient situations. The randomization is only partial (and actually rather deliberately dumb), since what’s meant by “random” is that first, one collects a cohort rather selectively of a bunch of different patient situations with one common theme–such as one or more heart conditions. The other happy morbidities can be what they are.

The “clinical trial” is designed to find effects that work across a population while not showing how the benefit to the group translates into a clinical decision for any given doctor based on the particulars of an individual patient. The idea is that the doctor is not supposed to have to care about those particulars–dose them all. That’s the sweet spot for sales, of course. Dose 100 to help 1 or 5 or 10. In the case of statins, it’s a 30% relative risk reduction, after all, for the population. And, amazingly, that is exactly how this cardiologist of mine approached it. He did not care to look at my daily blood pressure readings, or at my rehab reports, or my diet or lifestyle, or even my current drug regime, barely examined me at all (yeah, had someone take a blood pressure reading).

When I mentioned to my cardiologist some studies I had read that seemed relevant to my situation, he said, “There is nothing you can show me in current published research that will change my care.”  He wouldn’t change his practice, he said, until there was overwhelming evidence that he had to change. Or, another way, he would’t change until some major organization changed its guidance, or the massive healthcare organization he worked for changed its guidance. I suppose changing would mean he admitted he had been wrong in his past practice. He couldn’t do that. He couldn’t afford to care. He didn’t work for me. He worked for Them. And They did not care about me. They cared about Them. There really was no point in having me come in to see him. He could prescribe in the abstract. See you in six months. I walked out and found another cardiologist, who was taking new patients, and would see me in another three months. So I went nearly six months after my cardiac arrest without a cardiologist, and had to make decisions about my situation mostly on my own. American healthcare.

I happen to be working with a start up using AI pattern recognition to review medical records and even medical conversations to construct backing support for diagnoses and possible interventions. The startup’s approach is already being used in some major settings, where the doctors say it has saved lives by checking observations in real time where real time matters. My thought was–AI would surely be better than this cardiologist! Doesn’t he see that he is making himself obsolete by working for Them? (well, not quite, because he was also an interventional cardiologist, and he did come in on his off-day Sunday morning to place the first stent–so I’m grateful, though I was stunned he was so unsuited as a rehabilitation cardiologist).

For most clinical trials, the population statistic reported as clinically significant is a population level measure of apparent effect–an effect arguably arising across a given population. The very “randomness” of the population works against making a clinical connection between the population as a whole and any given individual in that population. If all the individuals in the population were very similar, then at the individual level, the doctor’s purpose would be to ascertain if the patient he is currently considering is similar to that population. And the clinical trial ought to show a benefit for *most* of that similar population. If it doesn’t, then what ought to be learned is that the conditions and measures thought to be similar are not the right conditions and measures–that is, the population may be similar in those chosen ways, but not in a clinically meaningful way, only in a way that allows a drug company to, say, snooker a review committee with stupid math into granting approval to sell a drug. Most clinical published results are wrong, and when not wrong mostly not clinically useful, as John Ioannidis has argued. Yeah. Looks right.

The clinical trial designed not to be “random” turns out to carry more clinically relevant information than a “random” clinical trial. I wanted a clinical trial–or even “anecdotal” reports of how people in conditions like mine–without a bunch (or any) co-morbidities–respond to a given intervention. That “data” doesn’t exist, and no doubt would be disparaged, dismissed, as if it is produced by the “gold standard” for research progress, as the double-blinded randomized clinical trial is claimed to be.

Now, here’s another thing. The clinical trials I read did not report the trips to the hospital for anything other than cardiac events. Other issues were treated as “side effects” and suppressed in various ways to highlight the population-level claim for “clinical significance” of the candidate drug (“significance” itself is a weirdly arbitrary quantitative claim about how likely a given effect is “not random” (whatever that might mean)). What’s clear is that a “randomized” clinical trial does not hold everything “constant” but for the introduction of a candidate compound; rather, it ignores (or suppresses or mixes together) most everything else and assumes that all that everything else in the test group and the control group doesn’t operate, or cancels itself out, or can be “subtracted” as a baseline that is assumed to be uniform (because mixed together) and relative to the candidate compound, inert. It makes no sense, other than as a way to game a bureaucratic system set up by a government to (mis)use population statistics in place of a doctor’s judgment in each particular instance.

While there’s a claim made about the benefit to a population for everyone taking a candidate drug even if for most of them it shows no benefit (as is the case with statins), there’s not the same attention to be paid for other problems that may be caused by that same candidate drug, beyond apparent “side effects.” For instance, for statins, there are reports that statins induce diabetes in 1 dosed patient out of every 100. That’s an absolute “risk” of 1%. I had a bit of a Spinal Tap moment (but this amp goes to 11) with my second cardiologist. So, I said, if I take the statin, I may be the special one person in 100 that saves a trip to the hospital with another cardiac issue, but I also may be the one special person in 100 that develops diabetes as a direct result of taking the statin. Cardiologist looks at me, and says, but it’s only a 1% chance of diabetes and a 30% reduction in risk of another heart problem. Sigh.

The population stat for a “random” population does not map to the individual (“why me? why out of 100 am I the one to benefit, or to get diabetes?). The “control” is not a control at all, just another melange. There’s nothing to show that breaking a melange of morbidities into two groups (“randomly”) will result in both groups showing the same distribution of responses to the targeted condition or to the candidate intervention. The absolute incidence (1%) looks less than the relative reduction (30%), but really these two things are about equal. And the overwhelming thing is that the benefit is really, really underwhelming, if there really is a benefit at all. And even then, there’s no way from the clinical trial reporting to compare the benefit (if any) with the adverse effects (if any) for any individual person. It’s more like a quantitative superstition turned into a formal institutional ritual performed at the patient interface by an expensive doctor. More like an institutional scheme to avoid liability, which makes sense to have given how dumb (but sophisticatedly, expensively dumb) the approach is.

Beyond all this, there’s the weird way that statins are thought to “work.” They block a step in a multi-step chemical cascade by which mevalonate, a precursor molecule to a number of useful molecules (including cholesterol and coenzyme Q10), is made. So “lipid” panels return a lower number for “cholesterol” but also cholesterol is needed for cell membranes, for the production of hormones, and for bile acids. It’s like carpet bombing to kill a feral pig digging up one’s garden. Or, you won’t get toenail fungus if we destroy all your toenails. That sort of thing. Star Trek’s Dr. McCoy’s comment from the (fictional) future about the brutish nature of 20th century medicine seems less like a funny fiction than commentary.

We aren’t done, given that there are articles arguing that cholesterol isn’t the problem, even if it is associated with the problem. Cholesterol produced by the liver gets assembled into particles of a few thousand molecules. These particles also include triglycerides and other molecules, all held together by five-fingered proteins–apolipoproteins (there are a number of these, just as there a number of varieties of particles that carry cholesterol and other molecules). These particles circulate in the blood to deliver their payload to cells. Some get depleted and turn from roughly spheroid to a denser torus–small, dense LDL particles. And some of these sdLDL particles may get compromised, and so are not recognized by the liver’s LDL receptors that would otherwise sweep these particles out of the blood. So compromised sdLDL particles circulate for four days rather than 24 hours, and have to be removed by macrophages, which engulf the sdLDL particles and in doing so are transformed into foam cells. The argument goes that sdLDL particles are small enough to become trapped in the vasculature system that supports the walls of arteries, including coronary arteries, and when macrophages find them there and engulf them, the resulting foam cells accumlate, forming plaques that contain, yes, cholesterol among other things. And over time, these plaques can cause the artery wall to swell and “remodel” to keep blood flowing, and some of these plaques can form a necrotic core with a thin “cap”–thin-capped fibroatheromas. If the cap breaks, then the necrotic fluid is exposed to the blood, causing a big clotting response, and well, that can be very bad in small arteries the heart relies on.

People differ on their responses to these articles, but institutional choices (that become guidelines which are in effect demands or even threats about liability) make cholesterol to be the cause and dismiss compromised sdLDL in arterial walls, even though the cholesterol argument makes no physiological sense, just as the statin intervention shows little if any benefit–just enough “clinical significance” to justify allow its sale. Perhaps it is complicated enough for folks without statistics training to give up and rely on “trust.” Maybe that’s the point. My sense is that the medical establishment cannot tell cause from co-occurance, and has been conditioned not to care, as if it is for the researchers and government committees to decide. And woe to us if those researchers and committees are compromised. Just a little slack, a little overconcern for reputation or getting the next grant by flattering some funding organization’s reviewers, and well it’s like a building inspector looking the other way. Bad things happen, but everyone has their liability screen so hey it’s a sweet time at the funding honey pot.

The fundamental question for a clinical trial that appears to benefit 1 out of 100, or even 6 out of 100 is “why me?” It is simply quantitatively dishonest for doctors to tell every one of their patients that each will benefit from a drug when the clinical trial data says arguably only, say, 6 of 100 benefits. Or with statins, more like 1 in 100. The move from population statistics used to justify sale of a drug to the medical insight to determine that a given intervention will  benefit a specific individual is essential, and the guidance for such a move is strangely, distressingly lacking in the scholarly medical literature.

I end up with questions. Why has coronary artery disease emerged as a thing only since about the 1930s? Why especially the coronary arteries? Why do LDL particles get compromised? When? Why more in men? Why aren’t doctors curious about edge cases?  Why don’t they document and share with each other individual interventions and outcomes? With pattern recognition AI (not the hallucinating text generating algorithms), we don’t need randomized clinical trials (if we ever did)–we can now assemble thousands to millions of anecdotes with all their variations in observations and related and unrelated conditions, tracked over years, even decades that people visit doctors or even self-report to their own medical records (which they cannot do now). Now it’s possible, even much simpler and less expensive than any big clinical trial. Maybe things are changing that will put the “randomized” clinical trial experts out of business. We can hope, at least.

Anyways, I appear to be doing fine. I’m in better shape now than before. Off of most of the meds waiting for the stents to resorb. As for this blog, I have been writing articles but not posting them. I should,  but well, there have been distractions as I realize I have lots of unfinished projects in other areas and maybe I should tend those gardens, too. I am back working on Piers Plowman–a medieval English poem that has contributed to my thinking about tech transfer since the poem is about money and truth, or reward and truth. “Teach me to know the false” applies as well to spiritual well being as it does to looking at clinical trial “data.” It is weird thinking that a sermon from the 14th century on the problems parish churches have with friars over money can have any bearing on research enterprise, but the connection is way more direct that one might think, though I doubt most folks will commit to learning Middle English to see it through. No matter.

I also have worked on a project involving the connection of craft to character, to ethics, and to the growth of technology in the context of need, luck, ingenuity, and living beyond mere survival. More on that later, if there’s interest. Could be a novel, or a video game, or a set of essays, or just lots of notes to get dumped down the road. Do you know how to field dress a deer? Find sources of iron? Make a kiln? Glass? How does the supporting technology differ between baked salmon and lasagne?

As for this Research Enterprise thing. I’ve spent a good two million words mostly aimed at how truly bad the Bayh-Dole Act is as an innovation promoter, or commercialization promoter, or anything other than source of bureaucratic bloat, and how truly bad universities and other nonprofits have been about refusing to comply with the Bayh-Dole Act and with the whole-hearted collaboration of NIST attempting to recast the Bayh-Dole Act in the form that it would have never got through Congress, and into a form that makes an utter disaster of university and nonprofit research findings. It really is a discovery battlefield littered with the patented (and unpatented but institutionally claimed) bodies of tens of thousands of university research findings. It’s horrible. The success stories put out by universities are mostly frauds and meant to deceive, which they have done rather successfully. But there it is.

I’m just someone that was embedded in the tech transfer “industry” for over a decade, ran tech transfer operations, talked to a bunch of people, consulted lots of places, gave talks. Committed to helping researchers where I could and getting out of the way (and getting the institution out of the way as much as possible) when it was clear I had nothing to offer. One can only do so much, and insight–even when proven out in practice–doesn’t count for much in the tech transfer crowd. And that’s something of the problem, when a practice like tech transfer attracts a disproportionate number of people who really don’t care to change, are afraid to change, are threatened by others if they consider changing.

My thinking is, I should be working more on the problems of research run by institutions and where discovery and better things might emerges outside of institutional thinking and controls, discovery beyond the grasp of Moloch, discovery that does not arise under the control of the infrastructure. Rough beasts, but not headed toward any known town. Out there, there’s mystery, excitement, possibility, and a kind of freedom that no competitive grant proposal with a pre-planned budget acceptable to a university and a funding agency with half the research already covertly done can get at.

Maybe then you’ll see some changes in emphasis here, even if I also add more articles on the problems of Bayh-Dole and “technology transfer.” (I have a few hundred draft articles left from the years of being at this, in addition to the few drafted in the past ten months.)

That’s the update. Thanks for reading, and even if I don’t hear from you, I do see that 50 to 100 to even more visit each day. I hope you find somewhat of what you are looking for.

 

 

This entry was posted in Bad Science and tagged , , , , . Bookmark the permalink.