Post Reply 
Rating Thread Options
Science, scientific models and evidence-based medicine
Author Message
Post: #1 Science, scientific models and evidence-based medicine , 04-04-2013 01:34 PM


Advanced Low-Carber
***

Posts: 50
Joined: Jun 2012
Rhett Allain over at Wired has written a series of short and interesting posts on how the scientific community usually presents and describes its work.

Well its just a theory
http://www.wired.com/wiredscience/2012/0...-a-theory/
Three science words we should stop using
http://www.wired.com/wiredscience/2013/0...top-using/
What's wrong with the scientific method
http://www.wired.com/wiredscience/2013/0...ic-method/

In them he discusses how the general public is often confused by terms used to describe the scientific process and scientific discoveries. For example the words “theory” or “law” have different meanings for scientists than they do for the general public.

In his posts he suggests that there might be a better way to describe the scientific process. Science he says, is really just about building models that describe reality.

“Really, that is what we do in science. We try to make equations or conceptual ideas or computer programs that can agree with real life and predict future events in real life. That is science.”

Not only does looking at science this way make it easier to understand, he also argues it makes it a lot more fun. He says the process of creating a scientific model isn't a whole lot different than the work we did as children to construct model cars or doll houses.

Our current medical system of course strives to be science based. Treatment decisions are made based on models of how the human body works or how a disease develops and progresses. For example, here is a model of how statin drugs work to reduce cholesterol in the body and of the cholesterol transport system in general:

http://www.wikipathways.org//wpi/wpi.php...ldid=59209

The Framingham Risk Score is a model of risk factors that have been found to influence when and how quickly cardiovascular disease will develop and progress.

http://en.wikipedia.org/wiki/Framingham_Risk_Score

However the medical system doesn't accept just any model. It tries to only recognizes those supported by strong evidence. In other words someone needs to show that a given model accurately predicts some type of future event.

http://en.wikipedia.org/wiki/Evidence-based_medicine

So how is that evidence evaluated by the medical community? Steven Novella discusses this question in the following post:

Evidence Thresholds
http://www.sciencebasedmedicine.org/inde...hresholds/

In this post he outlines a list of four criteria that have to be simultaneously met before most doctors will begin moving off of a currently accepted model and onto another.

1- Methodologically rigorous, properly blinded, and sufficiently powered studies that adequately define and control for all relevant variables (confirmed by surviving peer-review and post-publication analysis).
2- Positive results that are statistically significant.
3- A reasonable signal to noise ratio (clinically significant for medical studies, or generally well within our ability to confidently detect).
4- Independently reproducible. No matter who repeats the experiment, the effect is reliably detected.

To see how this criteria is applied, you may want to take a look at reports like the following. Both these reports, which are written to help doctors decide how to evaluate and treat patients, begin by discussing how evidence was selected and evaluated.

“Adult Treatment Panel III (ATP III)”
(Background and Introduction, Development of an evidence-based report, page I-1)
http://www.nhlbi.nih.gov/guidelines/chol...p3full.pdf

or the

“ADA Clinical Practice Recommendations”
(Introduction, Grading of scientific evidence, page S-1).
http://care.diabetesjournals.org/content...pplement_1

Here is just the Introduction.
http://care.diabetesjournals.org/content...l.pdf+html

If you have ever found your doctor less than receptive to an alternative treatment you learned about from a book or article, Dr. Novella's list should give you a better idea of what it will take to convince your doctor otherwise. This is an admittedly high bar to meet. But as the Doctor notes:

“Rigorous science anchors us to reality. Without it our beliefs will drift off into a fantasy world that caters to our emotions and desires but has little connection to reality.”
Quote this message in a reply
Post: #2 RE: Science, scientific models and evidence-based medicine , 04-04-2013 06:43 PM


Unregistered

 
There is some very good science being done in health and nutrition, and a lot of very bad science.

As a science-trained engineer, I'm not impressed with what medicine has been doing overall, or their overall understanding of "science based."

Those pople talk and write a lot, but they just are not that good.

Usually when I hear someone trying to argue that a point is "science based," my first reaction is to switch them off because it is probably weak and wrong.
Quote this message in a reply
Post: #3 RE: Science, scientific models and evidence-based medicine , 04-05-2013 12:00 AM


Expert Low-Carber
*****

Posts: 914
Joined: Jun 2011
From what I've read, (and I'm certain Pete will correct me if I'm wrong), there's a public hearing process to determine the ATP guidelines, similar to the process involved in adopting the USDA "My Plate" guidelines. The drug companies are given a seat at the table. So when all is said and done, whatever rigorous science is applied, it comes down to a political lobbying process in the end with the monied stakeholders having a big say.
Quote this message in a reply
Post: #4 RE: Science, scientific models and evidence-based medicine , 04-05-2013 01:03 PM


Unregistered

 
(04-05-2013 12:00 AM)Janknitz Wrote:  From what I've read, (and I'm certain Pete will correct me if I'm wrong), there's a public hearing process to determine the ATP guidelines, similar to the process involved in adopting the USDA "My Plate" guidelines. The drug companies are given a seat at the table. So when all is said and done, whatever rigorous science is applied, it comes down to a political lobbying process in the end with the monied stakeholders having a big say.

I agree there is plenty wrong in the political nature of how science for the masses is selected for the public to "believe."

It's not the real problem with the "science based" fallacy. That is a separate problem, but they do interrelate.

The "Science Based Fallacy" in my opinion is the logic that unless a statement meets the double-blind, placebo controlled rigor of lab testing, it can not be true. The flip side of that fallacy is that anything that has been taken as fact in the past, remains fact until there is a double-blind, placebo controlled lab study to prove it is not fact.

There is a site called Science Based Medicine and it's terrible for this reason. I also used to hang out on a "rational skepticism" forum that worked that way, and I left it because they tried to apply this logic to diet and health and it was too painful to read.
Quote this message in a reply
Post: #5 RE: Science, scientific models and evidence-based medicine , 04-05-2013 01:20 PM


Senior Low-Carber
****

Posts: 377
Joined: Feb 2012
"Science" is often applied in a biased manner. Uffe Ravnskov points out very clearly in his books (and he's just one author who does that--there are many). Studies that are used to support many "conventional wisdom" things (i.e., CICO, the lipid hypothesis) can be total crap and riddled with problems, but they are still used to support a faulty and/or unproven hypothesis. However, it is often the case that when good, solid studies are done that disprove such conventional wisdom, they are negated or ignored because "there isn't enough evidence." It's quite an interesting double standard. The title of one of Ravnskov's books is "Ignore the Awkward" and that is a good description of what goes on in so much of nutritional and medical science.

GCBC is another example of a book that delves deep into the research, looking at the actual studies and noting where the conclusions are not supported by the findings and where studies that don't even support a hypothesis are cited as supporting.

Then the bad is compounded when the next researchers only read the conclusions of a prior study, and then build on faulty conclusions while never looking at the actual findings. Pushing statins is a great example of this type of bad science.
Quote this message in a reply
Post: #6 RE: Science, scientific models and evidence-based medicine , 04-06-2013 08:54 PM


Unregistered

 
Quote this message in a reply
Post: #7 RE: Science, scientific models and evidence-based medicine , 04-23-2013 02:13 PM
(This post was last modified: 05-01-2013 10:25 PM by Janknitz.)


Expert Low-Carber
*****

Posts: 914
Joined: Jun 2011
Statin Nation http://youtu.be/iZctVYxiW2w toward the very end, discusses how politics affect the science that goes into the ATP guidelines.
Quote this message in a reply
Post: #8 RE: Science, scientific models and evidence-based medicine , 04-29-2013 12:41 AM


Advanced Low-Carber
***

Posts: 50
Joined: Jun 2012
(04-05-2013 12:00 AM)Janknitz Wrote:  From what I've read, (and I'm certain Pete will correct me if I'm wrong), there's a public hearing process to determine the ATP guidelines, similar to the process involved in adopting the USDA "My Plate" guidelines. The drug companies are given a seat at the table. So when all is said and done, whatever rigorous science is applied, it comes down to a political lobbying process in the end with the monied stakeholders having a big say.

The process to prepare the ATP 4 guidelines is listed here:

http://www.nhlbi.nih.gov/guidelines/cholesterol/atp4/

The (mostly short) articles discuss how the process was funded, the expert panel was selected and potential conflicts of interest are handled.

The expert panel members are listed here:

http://www.nhlbi.nih.gov/guidelines/chol...embers.htm

From what I can tell most seem to be academic researchers. If you want to know who these people are and where they stand, most appear to have rather extensive public bios.

For example, here is the bio for the panel's chair, Dr. Neil Stone:

http://fsmweb.northwestern.edu/faculty/f...?xid=17759

Dr. Jennifer Robinson, vice chair, has her bio here:

http://cph.uiowa.edu/faculty-staff/facul...on_bio.pdf

Personally I'm not seeing any evidence of the pharmaceutical companies being extensively involved in this process. Perhaps you are able to find something I am missing.
Quote this message in a reply
Post: #9 RE: Science, scientific models and evidence-based medicine , 04-30-2013 05:14 AM


Expert Low-Carber
*****

Posts: 914
Joined: Jun 2011
Just to quote Statin Nation, the documentary (at about 47 minute mark) points out that eight out of nine members of the panel creating the ATP III guidelines had financial ties to the companies that make statins. The ATP III guidelines lowered the threshold for treatment with statins from a TC of 250 to 200, which effectively included millions more people in the group to receive statin treatment. The method of reporting risk in clinical trials on statins (relative risk vs. actual risk) was characterized as "propaganda, misrepresentation, and even fraud". These studies formed the basis of the guidelines forATP III.
Quote this message in a reply
Post: #10 RE: Science, scientific models and evidence-based medicine , 05-01-2013 04:46 PM


Advanced Low-Carber
***

Posts: 50
Joined: Jun 2012
(04-30-2013 05:14 AM)Janknitz Wrote:  Just to quote Statin Nation, the documentary (at about 47 minute mark) points out that eight out of nine members of the panel creating the ATP III guidelines had financial ties to the companies that make statins. The ATP III guidelines lowered the threshold for treatment with statins from a TC of 250 to 200, which effectively included millions more people in the group to receive statin treatment. The method of reporting risk in clinical trials on statins (relative risk vs. actual risk) was characterized as "propaganda, misrepresentation, and even fraud". These studies formed the basis of the guidelines forATP III.

Regarding Statin Nation, I have already posted my opinion of this video here:

http://www.livinlowcarbdiscussion.com/sh...#pid168726

I'm sorry, but I just don't view it as a credible source of information.

One of the problems I have with these types of videos is that they have a tendency to quote documents out of context for dramatic effect. A good example is the video's assertion that “eight out of nine members of the panel creating the ATP III guidelines had financial ties to the companies that make statins” (at 47:50). The video of course implies that the filmmakers were “exposing” this information and that there was some type of conspiracy by the pharmaceutical companies to influence the writing of the ATP III guidelines.

What the video didn't say was that this information was publicly disclosed by the panel members.

http://www.nhlbi.nih.gov/guidelines/chol...sclose.htm

In other words the panel members were trying to be as upfront as they could about any possible bias. Also not discussed by the video was the fact that the initial drafts of the panels documents were extensively reviewed by at least 90 other professionals before the guidelines were released. The reviewers included:

“35 representatives of leading medical, public health, voluntary, community, and citizen organizations and Federal agencies, and then by the scientific and steering committees of the American Heart Association and the American College of Cardiology”.

In addition there is a huge difference between someone receiving payment for giving a lecture (a common practice throughout the academic world) and accepting a large bribe in return for altering documents. How did the pharmaceutical companies supposedly persuade the panel members to lower the threshold for statin treatment? If it was back room arm twisting rather than honest scientific debate, where is the evidence? If the conspiracy was so obvious, why is it that all the filmmakers can offer is a public financial disclosure form from the panel members themselves?

Even if the pharmaceutical companies somehow were able to coerce the panel members, is it reasonable to assume that the entire medical community would suddenly be duped into going along? Do you really think the insurance companies are going to willingly open their check books to pay for expensive patented drugs without at least some evidence that the drugs are safe and effective? Do you really think most doctors never read, discuss or debate the ATP guidelines and just blindly accept them?

I known scientists and doctors can make mistakes, but I find the likelihood of a large scale conspiracy by the pharmaceutical companies just a little too far fetched. I agree that the medical system is perhaps too aggressive in prescribing these drugs. Ideally I would like to see doctors do a better job of encouraging people to make lifestyle changes before they reach for their pad.

But given the number of heart patients I know who continue to puff on cigarettes while eating their supersized fast food lunch, I can understand why doctors just roll their eyes when people tell them they plan to change. For every person who is willing to change, I would argue there are at least 10 people who won't even entertain the possibility. Is it really that surprising statins are so popular?

BTW, your link to the video doesn't work. You need to remove the comma at the end that you inadvertantly made a part of the url.
Quote this message in a reply
Post: #11 RE: Science, scientific models and evidence-based medicine , 05-01-2013 05:59 PM


Expert Low-Carber
*****

Posts: 529
Joined: Aug 2011
And yet, relative risk and absolute risk are still completely different things.
Quote this message in a reply
Post: #12 RE: Science, scientific models and evidence-based medicine , 05-01-2013 06:30 PM


Senior Low-Carber
****

Posts: 377
Joined: Feb 2012
It doesn't have to be a conspiracy...official or otherwise....to be something that is good for a particular business but bad for the people at the end (patients). The relative vs. absolute risk data massaging alone is reason to doubt what is being publicly said. And you may not think that a researcher being paid an honoriarium is anything to worry about, but it is--it is just the tip of the iceberg. In academia, the need to keep getting grants is paramount for departmental and individual survival. Researchers will make sure that the conclusions are what the funding agency or company want to hear. If conclusions are contradictory to what is wanted, the results are buried or data is eliminated and massaged into "proper" shape. If the researcher persists in trying to publish the results, that researcher's academic career will most likely be over; at the very least he/she will be discredited or ostracized.

You're not going to change anyone's mind here, petekl, why do you keep trying? For every link and study you put up, there are many more that contradict them. You can easily find them yourself if you're curious. The fact that you don't think it's a problem that researchers being paid by the companies tells me everything I need to know about where you're coming from and that you're not open to any contradictory evidence.
Quote this message in a reply
Post: #13 RE: Science, scientific models and evidence-based medicine , 05-01-2013 11:02 PM
(This post was last modified: 05-01-2013 11:26 PM by weeji.)


Expert Low-Carber
*****

Posts: 529
Joined: Aug 2011
Thank you LLL. I contemplated posting something similar, but then thought to myself "do I really want to start yet another debate here?". I'm kind of over the debating. I believe that RCTs are the usually best way to answer diet-related questions, short of a feeding trial, which is expensive, and almost never happens. So what do I believe that RCTs show unanimously? Saturated fat is good or at the very least harmless. Grains, sugar, and oils unnaturally extracted from things that shouldn't produce oils are bad, and will kill you faster and more often, even if they lower cholesterol, LDL or otherwise. Statins produce an ever so tiny reduction in cardiac events (when looking at what matters: absolute risk), but produce a larger exacerbation of many other problematic symptoms (such as Alzheimer's). Do I think everyone agrees with me? No. Do I care? Not after the first few weeks of people not listening to me. I got over it, and now I just live my life the way I think it should be lived, based on what I believe to be the best information available to me. Do I want to help people that ask for it? Absolutely, and that's why I frequent this forum.

Every time Dean Ornish or T. Colin Campbell, or the news media trumpet a rat study, or observational study, or teleoanalysis of the 2, I cringe that people believe it, since there have likely already been 5+ RCTs with actual human beings performed that show the opposite result. At the very least, this should cause just a little bit of pause.

I just wish that anything that contradicts the "official story" wasn't regarded as a conspiracy theory, and therefore, considered by the public to be 99% likely to be false.
Quote this message in a reply
Post: #14 RE: Science, scientific models and evidence-based medicine , 05-01-2013 11:20 PM


Expert Low-Carber
*****

Posts: 914
Joined: Jun 2011
Thanks, Pete, I fixed the link. I'm not much of a conspiracy theorist myself, but I do think that money talks loud and clear in the process of developing such policies and when the studies supporting the policies are conceived in the first place.

Statin use grows exponentially, but the problems they supposedly treat/cure are getting worse at the same time.

Doctors continue to give the same old advice, "eat less, move more." If you think there's one single obese person in this universe who has NOT tried to "eat less and move more" more than once then show me that person. People know they are fat, they know that it's detrimental to their health, and they DO try. But the advice is flawed in the first place. It does not work long term for a large percentage of people because eating less means fewer nutrients, more hunger, slower metabolism, hormonal disruption, and ultimate failure. Of course there are exceptions, but those exceptions give lie to the theory that "eat less, move more" could work for all and if it doesn't work, then the individual isn't trying hard enough.

So doctors continue to give advice that doesn't work for most, patients continue to try it until they give up in frustration, and doctors conclude there's something wrong with the patient (most doctors conclude it's laziness and stupidity). Once a person has had a few unsuccessful attempts (usually resulting in MORE weight and more problems), then their eyes glaze over when the doctor starts making the recommendations, and BOTH know it's futile. See this very good dramatization of exactly that: https://www.youtube.com/watch?v=LVX4_s5IP3g It says so much!

This is why I so admire doctors like Vernon, Westman, Enefeldt, Davis, Schwartzbein, Fox, Atkins, Eades, etc. who STOPPED blaming their patients and took a careful and critical look a their own advice. They studied the research and thought outside the box. And applied their findings to their patients and saw good results, including patients who were again motivated for their health because the advice was actually WORKING. They are out there in the clinical settings, and they learn to TRUST their patients and their own instincts to great effect.

I was fortunate to once have a doctor like that who pointed me to low carb in the first place and supported me to be successful with it when "eat less, move more" failed. The results were astounding. Prior to that, and after, I shut out a lot of other doctors who had nothing to offer me but the conventional advice--we BOTH knew the lifestyle advice they were parroting was a waste of their breath and my consideration. I am fortunate to have a doctor now who is in a very regimented HMO system, but has permitted me to guide my own path with the promise that he would tell me when I was going off course. I trust him a thousand times more than most of the doctors in this system who disregard me totally and continue to spout CW because I don't fit into the HMO's neat little boxes.

Pete, you obviously have a lot of trust in the scientific process as it relates to statins and the guidelines that arise from it, and I admire your stance even if I don't entirely agree with it. I like to think that you're an "outside the box thinker" or you wouldn't be here, and that your patients get the benefit of your careful consideration of more than just the conventional wisdom.
Quote this message in a reply
Post: #15 RE: Science, scientific models and evidence-based medicine , 05-02-2013 12:48 AM


Advanced Low-Carber
***

Posts: 50
Joined: Jun 2012
(05-01-2013 05:59 PM)weeji Wrote:  And yet, relative risk and absolute risk are still completely different things.

Absolutely. But this doesn't mean that one is better or more meaningful than the other.

You might find the following article helpful. In addition to explaining the difference between absolute and relative risk, there is a nice discussion on how an individual can use absolute and relative risk numbers to help decide whether or not to take a treatment.

Absolute Risk and Relative Risk
http://www.patient.co.uk/health/Risks-of...lative.htm

Keep in mind that the audience for research and scientific papers is other researchers and scientists. Like any technical document, authors assume the reader will have a certain base level of knowledge. You don't need to have a PhD to read these documents, but you do need to know certain terms and concepts to understand how to discuss them.

In the case of medical research, authors assume the reader has at least a rudimentary understanding of statistics. In particular authors assume the reader understands the difference between absolute and relative risk and why one is used over the other.

Unfortunately many journalists don't have the basic knowledge needed to read medical research. This of course results in a number of poorly written news articles and a lot of confusion on what these studies mean.
Quote this message in a reply
Post: #16 RE: Science, scientific models and evidence-based medicine , 05-02-2013 12:26 PM


Expert Low-Carber
*****

Posts: 529
Joined: Aug 2011
That's what we're trying to say. If there's a study with 3,000 people, and in one group, 3 people die of heart attack, but in the other, 6 people die of heart attack, I call that a wash. When a study paper, purposefully or otherwise, leaves out absolute risk, in favor of relative risk, it is a very good indication that the absolute risk change was very unimpressive. With those stats I gave, a newspaper could say "100% more likely to die of a heart attack!!!", but it wouldn't really be an accurate portrayal of what happened.
Quote this message in a reply
Post: #17 RE: Science, scientific models and evidence-based medicine , 05-02-2013 04:31 PM


Advanced Low-Carber
***

Posts: 50
Joined: Jun 2012
(05-02-2013 12:26 PM)weeji Wrote:  That's what we're trying to say. If there's a study with 3,000 people, and in one group, 3 people die of heart attack, but in the other, 6 people die of heart attack, I call that a wash. When a study paper, purposefully or otherwise, leaves out absolute risk, in favor of relative risk, it is a very good indication that the absolute risk change was very unimpressive. With those stats I gave, a newspaper could say "100% more likely to die of a heart attack!!!", but it wouldn't really be an accurate portrayal of what happened.

From the patient.co.uk link above:

"Treatments for medical conditions are often quoted in the press along the lines ... "New treatment reduces your risk of X disease by 25%". However, although this sounds good, it usually refers to the relative risk. BUT, THE BENEFIT REALLY DEPENDS ON HOW COMMON OR RARE THE DISEASE IS." (emphasis mine)


Heart disease is common and serious. In the US, a persons absolute lifetime risk of developing heart disease is over 50% (more than 1 in 2). In other words, of the 300+ million people living in the US today, at least half will likely develop heart disease at some point in their lives (assuming we continue to live like we do today).

Study Shows More Than Half of All Americans Will Get Heart Disease
http://healthland.time.com/2012/11/07/st...t-disease/

A treatment to prevent heart disease that results in a relative reduction on the order you are discussing (50%), projected over the lifetime of the entire US population, would potentially prevent 75 million people from developing heart disease (assuming there are no serious side effects from the treatment). Personally I would call that something other than a "wash".
Quote this message in a reply
Post: #18 RE: Science, scientific models and evidence-based medicine , 05-02-2013 05:47 PM
(This post was last modified: 05-02-2013 05:47 PM by weeji.)


Expert Low-Carber
*****

Posts: 529
Joined: Aug 2011
Well, I did say that I was tired of debating, and now you're giving theoretical stats based on my example of how relative risk can be abused, so I guess this is my exit from this topic.
Quote this message in a reply
Post: #19 RE: Science, scientific models and evidence-based medicine , 05-16-2013 12:33 PM


Expert Low-Carber
*****

Posts: 914
Joined: Jun 2011
Quote this message in a reply
Post: #20 RE: Science, scientific models and evidence-based medicine , 05-16-2013 02:25 PM


Senior Low-Carber
****

Posts: 377
Joined: Feb 2012
And if anyone thinks that "peer reviewed" journals means that science is paramount above all else have no idea of what really goes on with those things.

Peer reviewed really means that a few people become gatekeepers of what is published--whether it is true, good science, or simply ego-based data massage. You'd be amazed at how much ego-based data massage gets through because of who is "reviewing" the proposed article...and how much influence advertisers in a journal have over the process of what gets published. The threat of pulling advertising looms large for many journals.
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread:
1 Guest(s)


Archive Current time: 04-20-2014, 09:28 PM