When you compile data from scientific publications you should check the quality of the database for a metric called publication bias. The term publication bias describes the often observed behavior of researchers only to publish a study when the results are positive. Here positive means not that the outcome of the study was good, but merely that the analytes were found.
Imagine you are compiling data about cruise ships and the occurrence of norovirus infections on board. If you didn’t have any information about the many cruises that go over without major vomiting outbreaks, you might think people don’t go on cruises to relax but to catch some good norovirus infection.
In environmental research, it is not unusual that studies are exploratory. Researchers test a particular area for a bunch of parameters hoping to find something interesting. They may have a hunch but no clear hypothesis. But what if they don’t find much of interest?
While it is easy to show that something is there, by detecting it, one cannot conclude that something isn’t there because they cannot find it. The method may not be the right one; maybe samples were processed incorrectly. Often there is doubt. But worst of all, there is no story to tell.
Without numbers, there are not graphs to draw, no statistics to perform, no problems to solve. It’s much more difficult to look good in the eyes of editors. When you think you don’t have enough data, there is the danger of changing your study design midway or of using the same data to generate and test a hypothesis. You are no longer objective, you are biased.
But usually, information about what is not there is as valuable as knowing what is there.
If you ever checked for monsters underneath your bed as a kid, you know what I mean.
A way to make sure studies don’t disappear because they don’t have ‘sexy’ results is to require pre-registration for funding. Consider it a batch of honor for doing research the way it should be done. Bias-free.