“Statistical significance is in the eye of the beholder.”
- Larry E
Erm, not according to the definition of statistics.
*bangs head on desk*
{July 2, 2009} Quote of the day
-
Recent Comments
-
et cetera
- "girl" theme by Stacey Leung
- Get a free blog at WordPress.com
Thanks for the link even though I hardly think the quote is being treated fairly.
I could defend the comment by noting that, for example, in psychology, a p of less than .05 is the usual standard for judging significance but someone working in one of the “hard” sciences might regard that as laughably loose. But leave that aside.
I could also note that the actual quote was “’statistical significance’ is often in the eye of the beholder,” where the quotation marks and the word you omitted should have been enough to indicate I was making a political, not a scientific, observation. But leave that aside, too.
I do think that if folks go and read the linked post and comments, my point will become clear: You persisted in using a measure of statistical significance as an argument for inaction in a case of government regulation.
But standing alone, that is an invalid basis for such a judgment, as the rest of my quoted comment made clear (or at least was intended to):
“Those killed on 9/11 amounted to a trace over 0.001% of the population. How would you have reacted if someone called that death toll ’statistically insignificant?’”
Yes, I know I was using the term incorrectly technically. I was being flip. The point was and is that saying “statistical significance” (without even saying what value of p you are using) is not, standing alone, a basis for determining policy.
Footnote: Speaking of significance, is there any to that little smiley face on the left edge of the background?
I need to throw in a quick mea culpa, in that I cited a percentage of the population killed on 9/11 in that thread that was off by two orders of magnitude. In checking LarryE’s figure, I realized my error. D’oh!
LarryE – the ommision of “often” wasn’t intentional. To be honest, either way, it still sounded pretty daft to me. As for your clarification, I pickin’ up what you’re puttin’ down, and I did not interpret that to be your meaning (as I said, it sounded daft).
“You persisted in using a measure of statistical significance as an argument for inaction in a case of government regulation.”
Yes – for the most part. The pinning point of my argument was that I don’t think the numbers do the work of showing that there is a significant problem to begin with. The reason I chose the normal distribution and statistic parameters I did, it’s because the population under consideration, being at least 1 million, is large enough to justify treating it that way. As for whether that supports an argument for inaction, I’m not sure I’m entirely justified in saying that it does (support a case for inaction), but I don’t think that means taking action was thereby a necessity.
Okay, I think we’re nailing things down here.
If I get you correctly now, your argument is that the numbers do not demonstrate – at least not adequately – a connection between high doses of acetominephen and liver failure. Is that right?
If so, I won’t dispute that because I don’t feel qualified to judge. It’s not my field and I don’t know what level is considered significant. But I will say I think it appeared to several – it did to me – that you were actually arguing that the number of cases was insignificant, i.e., not worthy of attention, especially when you compared the number of acetominephen-related ER admissions to the population of the US. Which provoked the unsurprising reaction “56,000 ER admissions and nearly 500 deaths are insignificant?”
Not quite, but getting warmer…
“If I get you correctly now, your argument is that the numbers do not demonstrate – at least not adequately – a connection between high doses of acetominephen and liver failure. Is that right?”
erm, no. There is most definately a connection between liver failure excessive usage of acetominephen. I’m not arguing that there isn’t a connection there.
“you were actually arguing that the number of cases was insignificant, i.e., not worthy of attention, especially when you compared the number of acetominephen-related ER admissions to the population of the US.”
I believe I compared the number of ER admissions to an estimate of the number of people using acetominephen. I don’t have the exact number of users (this I called the “population”, using that term in the statistal manner to describe the population being sampled, so I can see how you might’ve thought I meant U.S. population. Or maybe I initially started talking about the U.S. population…in any case, I think it is appropriate to compare the ER admissions and deaths under discussion to the number of people taking acetominephen. I’ll go on…), so I’m conservatively basing this number on the consumer purchases of acetominephen products, which are reported as over 1 billion (there’s a linked paper somewhere in one of T’s comments). Permit me to get a little hand-wavey here, I promise, I’ll be conservative in my estimate, but taking the starting point as consumers bought 1 billion in tylenol (I say I’m being conservative in my estimate because I’m not including other sales of acetominephen products, which would push the population comprised of acetominephen users even higher). Since I don’t know anything about the distribution of purchases, as in whether they were little inexpensive bottles or supersized more expensive bottles, then in sticking with making conservative estimates, let’s say each purchase was a 50.00 expenditure on acetominephen (so, I’m saying I want the number of users that my calculation will turn out to be on the minimum edge, rather than the maximum.) 1 billion dollars over 50.00 gives me an estimate of 20 million people who buy – and I think it’s fair to say use at least once – acetominephen annually.
56000 ER visits over 20 million usages amounts to 0.28%.
It seemed the argument veered towards “how can I call 56,000 ER admissions and 500 deaths “insignificant?”
I continued in my protest of this description of my view because it is an abstraction; what I am arguing is that 1) it doesn’t make sense to look at it in this abstract manner because we’re considering a population that is in the millions, and when we are considering large populations, a statistical treatment is justified; 2) in taking the statistical treatment into consideration, it is *my opinion* that 0.28% is not enough to support an argument that there is a significant problem with the system.
I’m going to stand by my opinion that I don’t think the data I’ve seen demonstrates a significant problem. I will, at this point, say a few things about my original post:
1. my “I should run the FDA” antic is kinda, well, an antic. I’m not entirely serious. One of my main goals in life is to save enough money to move off of the grid and farm my own food/live the simple life so that I don’t have to be around all the assholes who irritate me (a sentimate mostly experienced every time I drive on the highway or go to the grocery store and see “cheese” being spelled “cheez” etc.)
2. My calling the course of action by the FDA “stupid” was kinda glib, in retrospect. I said a while back in the extra-long comments thread that on second thought it’s not a total atrocity (I do however think the banning of ephedra amounts to an atrocity, and I’ll make a brief note on that at the bottom), but only time will tell if it is something that will have an effect. I revise my initial remark of “stupid” and say that I definately think the action is premature, and that I think more information is in order. It warrents concern, at most, because it wouldn’t be fair to say 56,000 ER visits and 500 deaths are irrellevant (something that I’ve never claimed). But I would say “keep your eyes on it and wait.”
T, you and I are still in disagreement over ephedra. You recently said it hasn’t proven effectiveness as a good weight-loss treatment. I won’t argue with that, but what I will – and did – argue is that I think it’s worthwhile to research it as a bronchiodilator to treat metabolic syndrome. Bronchiodilation improves insulin sensitivity in subjects with metabolic syndrome, which attenuates some of the other harmful symptoms that result from being insenstive to insulin, such as glucose regulation. A normal person experiences bronchiodilation when they exercise. But some, such as the morbidly obese, are not able to achieve adequate bronchiodilation by exercise. A lot of other metabolic syndrome treatments only act on consequences subsequent to poor insulin sensitivity, essentially treating the symptoms not the cause. This helps, but it does not fix. If we are able to improve insulin sensitivity – exercise is one of the main ways this is done – then a lot the subsequent problems associated with metabolic syndrome clear up. In lieu of the necessity for non-surgical, affordable treatments to a growing problem in the U.S., I think ephedra deserves further examination on it’s potential to be succesfully used as a prescribed medical treatment for metabolic syndrome* (I’m fascinated by the insulin sensing pathways and how they respond to different treatments in the long and short term, and how they are damaged by physiochemical conditions and how they might be repaired…. and, man, sometimes I wonder why I left that field…oh, yeah…I thought it needed more analytical chemical measurement tools…) Just wanted to clarify my angle.
*It was never treated as a presciption medication, but I believe it could be safely administered as such.
***Ok, for some reason, wordpress is confused and is telling me LarryE’s most recent comment was posted (I knew I hit ‘approve’ instead of ‘delete’!) while it remains obviously absent. I salvaged it from my inbox, here it is, with my response after the endquote…***
.
.
.
LarryE -
First, you complained that I misquoted you by the accidental ommission of a single word, and also that I wasn’t treating your statement fairly. Now, you’ve made a blatant abstraction of something I said in my previous response so that it looks like I’m trying to get away with some sort of chicanery:
“I can see how you might’ve thought I meant U.S. population. Or maybe I initially started talking about the U.S. population
Um, yeah, you did….”
The *full* quote, which you cut off mid-phrase like some sort of tabloid artist, was “Or maybe I initially started talking about the U.S. population…in any case, I think it is appropriate to compare the ER admissions and deaths under discussion to the number of people taking acetominephen.”
I acknowleged that I shouldn’t have those figures in my earlier comparison and clarified that I, in recognition of my mistake, revised my point using the more appropriate (IMO) statistics of ER admissions compared to the estimated number of people taking acetominephen…and you chop my sentence in half and imply that I’ve mislead you, sarcastically saying “um, yeah, you did,” as though I’ve tried to pull some kind of a bait-and-switch on you and then rudely denied it?
How is that fair?
How is that fair?
Aw, now, c’mon, this is getting silly. The “*full* quote” was right there, in the comment immediately above mine, one it was entirely reasonable to think people had read: I do assume people read comments in the order in which they are posted. It wasn’t a misquote from a comment thread on a different blog. (NB: I fully accept the misquote was accidental but unless someone had followed the link and found the original they would have no way to know that lacking my pointing it out.)
And after all, it’s hardly unusual for people only to quote the part of a comment which they are addressing. Here, the issue was the source of the confusion that had arisen over to what “population” you were referring and so mentioning that, yes, you had initially compared the death toll to the total US population was relevant because that was something else that people reading these comments would not know unless they followed the link and read the comment thread. (Even more than that, in fact, it was to remind you in your own consideration of the source of that confusion that you had done it.)
How from that you got that I was implying that you deliberately misled me with some “bait-and-switch” scheme mystifies me, especially since the very next paragraph directly addressed issues of population size in a way that reflected a clear awareness of your estimate of the relevant population.
I’m sorry, but I really think your reaction is misguided and way over the top.
Ok, time to pronounce the horse as dead.