As AI-generated text continues to evolve, distinguishing it from human-authored content has become increasingly difficult. This study examined whether non-expert readers could reliably differentiate between AI-generated poems and those written by well-known human poets. We conducted two experiments with non-expert poetry readers and found that participants performed below chance levels in identifying AI-generated poems (46.6% accuracy, χ2(1, N = 16,340) = 75.13, p < 0.0001). Notably, participants were more likely to judge AI-generated poems as human-authored than actual human-authored poems (χ2(2, N = 16,340) = 247.04, p < 0.0001). We found that AI-generated poems were rated more favorably in qualities such as rhythm and beauty, and that this contributed to their mistaken identification as human-authored. Our findings suggest that participants employed shared yet flawed heuristics to differentiate AI from human poetry: the simplicity of AI-generated poems may be easier for non-experts to understand, leading them to prefer AI-generated poetry and misinterpret the complexity of human poems as incoherence generated by AI.
In 1962 Phillip K Dick put out a book called “Man in the High Castle.” In it there was a scene that stuck out to me, and seems more and more relevant as this AI wave continues.
In it a man has two identical lighters. Each made in the same year by the same manufacturer. But one was priceless and one was worthless.
The priceless one was owned by Abraham Lincoln and was in his pocket on the night he was assassinated. He had a letter of certification as such, and could trace the ownership all the way back to that night.
And he takes them both and mixes them up and asks which is the one with value. If you can no longer discern the one with “historicity,” then where is it’s value?
And every time I see an article like this I can’t help but think about that. If I tell you about the life and hardship of an artist, and then present you two poems, one that he wrote and one that was spit out by an LLM, and you cannot determine which has the true hardship and emotion tied to it, then which has value? What if I killed the artist before he could reveal which one was the “true” poem? How do you know which is a powerful expression of the artist’s oppression, and which is worthless, randomly generated swill?
There’s no contradiction here.
With high value art you definitionally buy a story not the content. Without a certificate of authenticity or a story that goes with it there is no story and no value to it.
With K Dick’s example the two lighters would become of different but equivalent value, perhaps the new value is in the story of how two identical copies and yet different came to be.
You could 3d scan the statue of David and reproduce it down to its tiniest detail. And yet the copy is only worth as much as the cost to make it or even less, while the original is invaluable.
You can see the Mona Lisa on your phone any time you want and yet millions will take the trip to the Louvre to see what is most likely not even the original.
The story and the history of an object is what you purchase when buying art or antiques of high value.
Art, like the value of Lincoln’s lighter, is in the eye of the beholder.
Often, people find art in completely natural occurrences. Or even human designs seen in certain ways, like how two or more separate buildings might come together in unintended ways.
So, even if it’s not strictly intentional human art, it’s still valid to appreciate it.