Research Ethics & Honesty

Your supervisor shows you a chart from an old Bluewater Basin project. A scientist tested nitrate levels against fourteen different possible causes and only reported the one that came back significant. This is called p-hacking (testing many things in secret and only showing the one result that looks interesting), she says, and it is one of the fastest ways to publish something that is not actually true.

She hands you the station's four-item integrity checklist that every researcher here signs before starting a project: decide what you are testing before you see the results, report everything you tried including what did not work, keep your data and methods available for someone else to check, and never adjust a result to match what you expected to find.

It is tempting to try many analyses and report only the interesting one. The problem is that with enough attempts, something will look significant purely by chance, not because it is real. Deciding your test in advance, and reporting everything you tried, is what separates a genuine finding from a statistical accident.

Reproducible (able to be repeated by someone else and get the same result) means someone else, or you in six months, could take your data and your documented steps and get the same answer. Every mission from here on will ask you to keep enough of a record that your own future self could reproduce today's work.

It is like flipping a coin fourteen times in secret and only telling your friend about the one flip that landed on heads five times in a row. It sounds impressive, but you hid all the boring flips that would have told the real story. Real researchers show every flip, not just the exciting one.

Pre-registration (publicly committing to a hypothesis and analysis plan before seeing results) and open data sharing are now standard practice at most research institutions, precisely because p-hacking and irreproducible results turned out to be widespread problems in real published science over the past two decades.

Explain to a friend why deciding your hypothesis before looking at the data matters, using an example that is not about Bluewater Basin.