Once upon a time, when I was a fresh-faced new grad, a client brought in her new puppy to see me. She had adopted the puppy from a breeder a couple weeks prior, and things were going splendidly at home. The puppy was happy and bouncing around, as puppies are wont to do. After a complete examination, I told the new owner that her puppy looked great and was completely healthy. She had received her first vaccine from the breeder’s vet just prior to being adopted, so it was a little too early for us to give her the next vaccination. I told the owner to come back in about a week for her booster.
A few hours later, I received a frantic phone call from the owner. Her new puppy had had a seizure. This had never happened before, and the owner was understandably upset. Just two hours prior, there was no indication that there was any problem with this little girl, and now all of a sudden, she was having a seizure.
Now what do you think that new pet owner would have thought had I vaccinated the puppy during our visit? The conclusion would have been inevitable: my vaccine obviously would have been the cause of her puppy’s seizure. There is no doubt in my mind that if I hadn’t decided to postpone the next booster, that there would be no way in the world for me to convince my new client that the vaccine I had given was the cause of this new problem. She had owned the pet for two weeks already with no history of any prior seizures. I gave her pet some evil needle, and now two hours later, her pet was having a seizure. The timing was just too perfect. I just may have believed it myself.
We should all be so wise.
Such is the danger when attempting to draw conclusions about cause and effect. The above example (if I had actually given the puppy a vaccine), would have been antectodal evidence that vaccines can cause seizures. Obviously, this conclusion would have been wrong, but we only know that in hindsight. Had the situation been different, we would be left wondering. The owner, since it happened to her, would likely have believed it with all her heart.
This is why we need science, and more specifically, double-blinded placebo controlled studies. Without good scientific evidence we have no way of weeding out the coincidence from the fact. In my last blog posting, I rambled on about placebos, and how they can screw up our perception of which treatments are actually effective. If you haven’t read it yet and are skipping ahead, then calm yourself. Go back and read it by clicking the photo below. This is a two-parter, people. No cheating allowed.
Calm yourselves, people.
A good quality study examining the effictiveness, or efficacy, of a medication or treatment, has several important ingredients. Wanna know what they are? I thought you’d never ask…
1. The study must have a control group. In order to know if a drug truly is effective, we need to know if it helps more patients than we would expect if it were just a placebo. For this reason, we need two groups of patients to compare. One group gets the real drug being tested and one group gets a placebo.
No peeking.
2. The study must be blinded (masked). Neither the people performing the study (the investigators) nor the patients (nor their owners if dealing with pets) can know if they are getting the real drug or the placebo. If people know that they are getting the drug, then they may be more likely to notice any slight improvements and conclude that the drug is helping them. Conversely, if they know they are on placebo, then they may be more likely to poo-poo any slight changes for the better.
3. The study must be randomized. Patients must be allocated to one group or the other based on chance and chance alone. Handpicking of patients to get drug vs placebo is a big no-no which could easily interfere with results and make them untrustworthy.
A good start.
4. The number of patients, or sample size, must be large enough. Fancy statistics exist which tell scientists how many patients they need for the results of their studies to have meaning. You’d be surprised at the number of crappy studies out there that have only looked at very small groups of patients. If only a handful of patients are in the control group and the drug group, then variations due to chance alone are much more likely to occur. If we’re dealing with large numbers of patients, then any differences we see between the two groups at the end of the study are much more likely to be meaningful.
5. The results of the studies need to be analyzed statistically to see if any real differences occur. People love to hate statistics. How many times have you heard “Statistics can be used to prove anything!”? Well, this is not entirely true. Statistics is just a tool that enables us to determine how much we should trust any differences we see between the two groups. It’s how the study is set up that can skew the results and lead to false conclusions… Were the sample sizes large enough? Was the study appropriately randomized? Was there a flaw in the study design? These are the things that will make or break a study. Which leads us to our last point..
6. Studies must be published in peer-reviewed journals. Studies must be given the opportunity to stand up to scrutiny from other scientists working in the same field. That’s how the good studies are weeded out from the bad ones. Scientists can be a nasty bunch — if they think that something’s not kosher about a study, then they’ll let everyone know it. And hopefully, they will do their own studies to see if they agree with the conclusions. As more and more well designed studies reach the same positive conclusions, we can be more confident that our treatment is actually effective.
So there you go. The double-blind, placebo-controlled study. It is the gold standard of proof in medicine. Antecdotal evidence, like in our vaccine-causing-seizures example above, can often lead us down the path to quackery. Does that mean that everything we do in mainstream medicine has good studies like this to back it up? Not by a long shot. Good studies are expensive, and some diseases are just too uncommon for us to get good sample sizes, especially in veterinary medicine. But it is something that we strive for, and must always be mindful of when deciding how to treat our patients.
Or we could just give them some mystical herbs, wave our hands in the air to unclog their magical blocked-up energy channels, or ask the gods to take away their demon-illnesses. Ultimately, the choice is up to you.
Quack, quack.
