Personal Science Week - 240620 ChatGPT Cheats

Professional scientists who cheat, plus more links

Personal scientists know not to put blind trust in scientists or their peer-reviewed publications. Now that ChatGPT makes it easy for anyone to write like a professional scientist, our cheat detection skills need to be on high alert.

This week we’ll show examples of cheating in peer-reviewed science, plus more reminders of our need to be skeptical.

Early users of ChatGPT chuckled when we first saw this example of obvious cheating, no doubt by a student too lazy to bother reading the auto-generated text he submitted for his essay.

I mean, if you’re going to let ChatGPT do your homework, shouldn’t you at least try to appear like it’s your work?

It turns out that even professionals are tempted to do the same thing. What’s worse, the peer-reviewed journals that publish them are horrible at catching the cheaters.

Here's one tiny example, since retracted, published in a relatively prestigious peer-reviewed journal for physical scientists:

or this one (posted on X):

You’ll find zillions of other examples if you look look at every paper published recently that contain the obvious ChatGPT giveaway terms "certainly, here is" -chatgpt -llm.

The problem isn’t just on the submission side. I spoke recently with an editor of a prestigious journal who was asked to read an appeal of a paper that had been rejected. Upon closer examination they found that the peer reviewer hadn’t bothered to read the paper itself, but had instead relied on a crude ChatGPT analysis — which turned out to be completely wrong.

Bottom line: the new generation of LLMs are enabling a whole new way for scientists to prove that they are as human as the rest of us.

Dangerous Over-the-Counter Drugs

Speaking of expert decisions that sometimes don’t go as well as we think, it’s important to remember that just because something is widely available and sold over the counter, that doesn’t mean it’s safe for you.

I was reminded of this when an elderly relative saw his kidney function (as measured by eGFR) suddenly plummet to dangerously low levels. It turned out that, under advice of his doctor, he had been taking over-the-counter omeprozole (aka Prilosec), a common treatment for gastroesophageal reflux (heartburn). We noticed the change in kidney function only because (1) he happened to have regular blood tests and (2) the plunge coincided with taking the new medication.

Why wasn’t this mentioned on the medication box as a possible risk? A quick literature search made clear that kidney failure is a known side effect of these drugs, but almost always in elderly people. And when I checked the studies that the manufacturer used as the basis of its FDA approval, guess what: the large sample didn’t include people over age 65!

Trevor Klee points out another scary fact about one common over-the-counter drug:

Another interesting grandfathered drug I came across recently was Afrin, the nasal decongestant. Afrin can be bought over the counter, and yet it becomes addictive in just 3 days of use, in the sense that you will become physically dependent on it to open up your sinuses. People who are predisposed to addiction can also become easily addicted to it in the more classical sense, suffering anxiety that can only be alleviated by more Afrin.

These examples should be a warning to anyone before you take any medication, over-the-counter or not.

Links worth your time

We’re fans of the SleepSpace app (see PS Week 230209 for details of my successful experiment). Now they want to test OverAchieve, a reishi mushroom-based drink that claims to improve sleep. Get a free 1-year subscription code ($250 value) to SleepSpace by participating in the Overachieve Sleep Study and receive 6 nights of the drink for free.

“Mediocre” developer Maggie Appleton proposes that LLMs will enable new Home-cooked software, made by normal people to solve everyday problems, like a simple glucose monitoring app

Personal Health

• The Aurorean summarizes the latest meeting of the American Society of Clinical Oncology : very promising results in colo-rectal cancer, lung cancer and more. Some new drugs appear to drive remission at outstanding rates (>50%) versus standard treatments.

• Mass General Brigham hospital used deep brain stimulation (DBS) to improve Parkinson Disease symptoms n most of the 200+ patients tested. Requires electrodes inserted in the head, so it’s not quite ready for personal scientists, but a little Googling and you’ll find clinics in your area that offer services.

• Speaking of brain stimulation, a new study on 36 volunteers with “love trauma” concludes that a few minutes a day can ease heartbreak.

• There doesn’t seem to be an easy way to find your chr21q22 gene status on 23andme, but if you know that sequence, you might be on your way to identify your susceptibility to inflammatory bowel disease (IBD). A new study printed in Nature gives the details, including links to the source code so you can do your own calculations.

About Personal Science

Personal scientists think for themselves, relying on our own experiments as much as possible while approaching every result skeptically. We’re also open-minded, happy to entertain ideas that the professionals will often dismiss.

We publish this newsletter each Thursday for free, but paid subscribers can also receive our Unpopular Science series, in which we discuss topics that are too dangerous for wider circulation, such as our most recent post on evolution.

If you have other topics you’d like to discuss, please email us, start a chat, or leave a comment: