Replication Fall Offs

The replication of experimental results is a fundamental part of the scientific process. If you get some interesting results, someone else should try it too, to make sure what you got wasn't just a fluke. The Truth Wears Off: Is there something wrong with the scientific method? in the New Yorker documents many examples of scientific research in which there was an initial report of significance, followed by a sequence of replication attempts which yielded poor confirmation. Many of these cases were in medicine and psychology, fields in which experimental trials are extremely difficult and expensive.
This really isn't surprising at all. Scientific research is now a major industry. Thousands upon thousands of people are out there racking their brains to come up with new results. Each individual may try and discard dozens or hundreds of ideas before stumbling across something interesting and apparently significant. Now let's imagine a million people around the world are flipping coins every day. On some days, some of the flippers will get a very skewed outcome. A tiny number of people might even get a skewed result several days in a row. But if they keep trying to "replicate" their results, they will eventually find that the significance falls off due to a regression toward the mean. Alas, despite all the training and talent that goes into science, most of the novel ideas researchers try out will not end up being significant. But when thousands of insignificant ideas are tested, a small percentage will end up looking good - initially. Researchers are paid to get results, they will keep trying new ideas until they get a skewed result. But was that result truly significant, or was it just a statistical fluke? There are billions of circuit elements following Ohm's law in the computer I'm typing this with - that's not a statistical fluke. Some scientific results have been tested and replicated an astronomical number of times, especially when they prove useful in technology. But it shouldn't be surprising if difficult experiments - especially involving complicated living organisms - often prove to be disappointing despite some initial promise.
The Journal of Irreproducible Results collects science humor. Some of the examples in the New Yorker article aren't too funny however, especially in the medical field.