s [r] blog

They cannot evoke a poet’s self, but they can sometimes come up with useful lines.

Like many writers and teachers, I feel the gathering threat of machine-written literature. In the old days of computers, there used to be a saying, “Garbage in, garbage out.” Today, into the maw of large language models, some garbage no doubt goes. But these large language models are also fed, as I understand it, all the text that exists on the internet and that includes the great works of world literature (though maybe not very recent works). It is becoming more common to allow these models or chatbots to write essays, analyses, business plans, and the like. As a teacher of poetry writing, I wanted to see how these bots might operate in the creative-writing classroom. I wanted to find out if robots made good poets.

To attempt to answer this, I created an exercise for my Summer 2023 Drexel University poetry writing class to see how and how well ChatGPT generated poems. The exercise comprised several steps. Students first had to engineer a prompt that included a topic, a mood, and some key words. That took some thinking and attention to self. Next, students told their chatbots to produce a poem based on their engineered prompts. The bots produced the poems in fifteen seconds. Following this, students shared the bot poems out loud in class and posted them on our teaching platform (Blackboard), and we workshopped the robot poems. Finally, the students had to harvest from their robot poems any usable lines and make them part of their own follow-up poem. As a means of attribution, the students were required to underline the robot-written lines they added to their self-written follow-up poems. (It does feel weird to use the term self-written).

Occasionally the students found some strong and usable lines in the robot poems. More often than not, however, the students condemned the robot poems as soulless and rote imitations of verse. Not only were the students vehement and harsh critics of the robot poems, they were enthusiastic about each other’s real voices, praising each other’s follow-up poems for their sincerity and heart. I was very encouraged by the students’ overall negative attitude to the chatbot poems. As much as I hate to say this, I was a little less harsh than the students about some of the AI poems.

The students in this class were sophomores on up and came from a wide variety of majors. No one was required to participate in this exercise. Three students did opt out of the exercise: a philosophy major, a computer science major, and an English major.

Of course, the robots were good at spelling, mechanics, and standard English grammar. This was a benefit to all, especially the English language learners in the class. I appreciated the clean copy as I did not have to pinpoint glitches in proofreading. Almost always when the tool saw the term “poem” in the prompt, it generated a rhymed poem even when the student’s prompt specified an unrhymed poem. Once in a while, the bots committed the classic error of sacrificing meaning or correct usage in the service of rhyme. Sometimes I could detect the influence of English and American poets from the tradition in the AI poems even when the students did not request a poem “in the style of.” 

I present here a selection of student-engineered prompts, the raw robot poems, students’ own poems connected with the exercise, workshop comments about the robot poems, and a few other observations. Finally, I include some of my students’ overall comments on the exercise and some of my own opinions about the future of AI-generated literature. All the students quoted and named here gave me permission to use their work and their names.

The precursor prompt

Before I began the more creative part of the exercise, I had all the participating students ask ChatGPT to write an unrhymed ten-line poem from a prompt I engineered. The bot was to write a poem about a man named Sam who encounters a wolf while hiking in the woods, incorporating the words “gray,” “the smell of green weeds,” “teeth,” “gun,” “afraid,” “run,” “yell,” “twilight,” “safe.” All the robot poems were different!

Samples from the multi-step exercise

English major Diamond Warney offered ChatGPT this prompt: 

No rhyme scheme. Ominous and foreboding mood – “Something is coming. Use the verbs ‘devour,’ ‘drown,’ ‘mourn,’ ‘sleep’ and ‘drive.’  Use the adjectives ‘godly,’ ‘delicious,’ ‘heavenly,’ and ‘pungent.’ Use elements of different religions and faiths. Use the words ‘purple,’ ‘garlic,’ and ‘hospital.’ Use the phrase, ‘Richie, don’t be late for dinner.’” Diamond underlined the phrases in the raw ChatGPT poem that she included in her original poem, and she underlined those raw phrases (or their revisions) in her follow-up poem.

Here is the raw ChatGPT poem generated from Diamond’s prompt:

untitled

In shadows cast by gods of old and new,
An ominous veil, foreboding, weaves its view,
As whispers 'cross diverse faiths, they fly,
The harbinger of change, impending nigh.

Devouring stars, a cosmic tempest looms,
Religions' ancient scripts in heavenly tombs,
They drown in sacred waters, souls in mourn,
While deities in slumber, await their cosmic turn.

From east to west, the chariots of fate do drive,
A pungent scent of garlic, offerings alive,
Yet Richie, don't be late for dinner, pray,
For in this mystic hour, worlds may fall away.

Purple shrouds the horizon, darkness to distill,
A hospital for hope, a reckoning, and will.

Of this poem, Diamond remarked, “The first and last lines are interesting and could be reworked and added to a better work. I also like the line, ‘Richie, don’t be late for dinner, pray.’ I didn’t think about praying before meals when I wrote the prompts. I think ChatGPT could be used to connect ideas you had but didn’t realize were related. The bot didn’t pick up on tone and literally used the words in the poem, but there are some ominous images in here like the mass of purple on the horizon. The bot also connected some of the words that I used in the prompt, like garlic and pungent.”

A few observations from me as instructor:

My take here is that the AI knew many stock phrases about spirituality and the cosmos and churned them into a rhymed sonnet even though Diamond didn’t ask for a sonnet. The program mostly used Diamond’s key words in the order she listed them in her prompt. It did not use the word “godly,” but did use the word “god.” 

Below is the follow-up poem Diamond created incorporating some AI-generated phrases. These are underlined.

Kitchen Sink 

By Diamond Warney

Babies left behind 
by gods of yesterday and today
at fire stations, on doorsteps, in pews
reaching out plump hands 
trying to catch the stars.
The only sign the gods were ever there.

My mother left me
in a gas station. I drank warm fuel from the pumps
and got high on the smell of Black & Mild’s.

In death, our atoms spread 
to become tiny pieces of everything.
We are beautiful and cosmic. 
We are ourselves again. 

You said you knew I was trouble
from the way I drive, with my head out the window
devouring stars and flies. 
No one taught me to stay inside the lines

So drown me in the sacred waters 
your mother bathed you in
before you grew out of the kitchen sink. 
Maybe then, I’ll know how to be loved. 
Maybe then, I will be cleansed.

A few observations from me as instructor:

Diamond’s new poem cries out in sympathy for discarded and abandoned children whose only solace, sadly, comes from a mysterious connection to the universe. Then she shifts to a literary speaker who is an outsider, a foundling left at a gas station and who knows the world from gas station odors, such as the smell of tobacco and gasoline. The voice then speaks as an adult who still sees herself as an outsider evoked in part by the image of her dangerously driving with her head out of a car window.

Aliyadeth Griffiths, an English major, did a turn-about. First, she wrote her own poem on the theme of striving, critically assessing the self in the mirror, and incorporating the sentences, “Be more. Do more.” 

Are You Satisfied Yet?

By Aliyadeth Griffiths

As I look at myself in the mirror,
Reflections of all I could be watch me intently,
Their eyes gazing with avid curiosity.

They haunt the bathroom mirror,
With a sinister smile plastered across their face. 
Like lurking shadows, they taunt me. 
“Be more. Do more.” They breathe against my neck.
In return, I stare at my decaying appearance.
My skin, fractured and pale, crumbling under my touch,

The dark circles underneath my eyes rise as I try to smile,
My hands trembling from fatigue.
“Be more. Do more.” They whisper again, their cries louder this time, 
The voices of the expectations echoing on the reflection,
Reminding me that I am meant to be nothing but perfection. 

The bones in my body creak loudly as I stand,
Aching and frail, I stare at all the work that haunts me in my sleep.

I can always do more. 
I can always be more.
I can always work harder than I ever have before.

The voices in the mirror seem pleased by my response,
As I turn away from my reflection,
My body collapsing on the floor.

Aliyadeth’s own poem speaks through graphic and even ghoulish imagery that invokes feelings of self-criticism and self-condemnation, and it includes a strong charge to try even harder to achieve better things. There emerges an interesting dialogue with the self in the mirror.

After composing her own poem, Aliyadeth provided this prompt to ChatGPT. “Write a horror 22-line poem about being a burnout high achiever, including the phrase, ‘Be more. Do more.'” She also asked the bot to create a title. Here is what the robot wrote. 

Whispers of the Abyss: The Burnout's Lament

In shadows cast by fractured dreams, I dwell, 
A burnout high achiever's cursed spell, 
Once driven by the whispers of success's lore, 
Now trapped within a labyrinth of endless chore.
Be more. Do more. The mantra drilled in bone, 
A relentless pursuit, a heart turned stone, 
I climbed the pinnacles of acclaim and might, 
Only to find the abyss in that blinding light.
Frenzied steps echoed, accolades in tow, 
Yet the void within grew with every glow, 
A phantom grin masked the wearied soul, 
As I danced on embers, an unholy toll.
Be more. Do more. The voices never cease, 
A crescendo of torment, a maddening peace, 
Shadows stretch, merging with fractured mind, 
In this twisted symphony, where hopes unwind.
Reflections leer, distorted and askew, 
A burnout's tale, a nightmare woven true, 
Haunted by echoes of an insatiable roar, 
Be more. Do more. Until you're no more.

Here are peer comments on Aliyadeth’s two poems.

From Maura Mealy, a psychology major: 

As always, your work is stellar! As much as I hate to say, I enjoyed an aspect of the bot poem, the repetition in the “Be more. Do More.” The lines were surprising from ChatGPT, but it was part of your original prompt! Your actual poem blew the AI out of the water. The actual narrative and story felt realistic and relatable. The repetition in this felt much more natural and flowed better in the poem, and it was just, in general, more of an enjoyable read. The AI gets bogged down in trying to use fanciful words while you actually utilize descriptors to move the poem along. Great job, as always!!

From Zakee Aleem, a finance major:

The poem drafted by AI was surpringly good even tho still void of so much emotion or even a setting. I think it somehow works because the prompt lends itself to a cold poem void of love and affection. Your poem is much more emotional and I can really place myself in this poem much easier. Great work!

A few observations from me as instructor:

In the bot-generated version, I heard echoes of Byron, Shelley, and Poe. The AI poem had a drumming rhythm and remorseful tone that suited the subject and the injunction to “Be more. Do more.” 

Every so often, the bot would come up with a very sophisticated line. Malachi Solomon, a general studies major, told the bot to write an unrhymed sad poem with these terms: “kick, punch, stupid, good, honest, helpful, caring, fragrant, bang, uncle.” Although the AI poem was filled with the stock observations and persistent rhyme, the robot did come up with one terrific example of anthimeria (use of one part of speech as if it were another): “A kick to the heart, a punch in the think.” “A punch in the think.” What a great phrase. It evokes a harsh and shocking assault to the mind. While Malachi did not decide to use that phrase in his follow-up poem, I have to give AI some props for generating it. Makes me wonder what was going on in the think of the program. 

How beneficial is ChatGPT to you as a poet?

I posed this question to the class, and here are some of the students’ responses.

From Anna Bokarev, and English and writing major:

To me personally, I don’t think ChatGPT will be all that beneficial. In fact, I found it really difficult to workshop the poem that ChatGPT spit out for me. It didn’t really have a tone or voice, and workshopping that into the poem required me to tear it apart and delete most of it. I also wasn’t the biggest fan of the phrases it used. Some of them were pretty interesting, but they were all so surface level that I couldn’t really incorporate them into a narrative-styled poem. ChatGPT was doing a lot of telling but not a lot of showing. 

I also just think reading the work and finding inspiration from real writers is a far better way to do it than consulting ChatGPT. There’s voice and passion in the work of human writers, which I believe is what ultimately inspires a writer. For now, I’d rather avoid the tool at all costs. It seemed to make my writing process more difficult and dull.

From Grace Dhankhar, a computer science major. Grace was one of the students who opted out of the ChatGPT experiment:

Personally, I am very against the use of AI tools like ChatGPT, and I have very strong feelings about it, but I understand that people are going to use it regardless of my opinion of it. With that being said, if people are going to use it, I think that it would be a cool tool if someone wanted to find a new way to say something or a new phrase, since some of the phrases the AI came up with were sort of eloquent, but I would absolutely never rely on it to create anything actually creative. With all of the examples shown in class, the human poem or revision of the poem is so much better and less nonsensical since the AI couldn’t really put together coherent thoughts together.

From Aliyadeth Griffiths, English major: 

For me, ChatGPT has always been more hindering than helpful, especially when it comes to the writing process. I find that reading the bot’s writing before coming up with my own often confuses me, because of its excessive eloquence, along with feeling as if I can’t use any idea the bot might’ve come up with because of the questionable ethics. I also don’t particularly enjoy using even one line from it, as I feel like I’m stealing from the authors the bot has been fed. 

While I am not completely against ever using AI (I feel like there is no point in being so against it—this is the way technology moving forward, and there’s no point in trying to fight it; especially when it is being used so often, by so many people and organizations), but I don’t particularly enjoy it. It feels wrong, as we can’t make sure which of these authors would actually be okay with their work being used this way. While I do think it’s crazy how far technology has come (for better or for worse!), it feels wrong to use it, especially with the writers’ strike going on as writers are not being paid enough to make a living. 

[Note: The Writers Guild of America strike, which began on May 2, 2023, was still going as of the writing of this post in late August 2023.]

From Maura Mealy, psychology major:

I think it’s essential to make the distinction that ChatGPT/AI models like it should be used as tools and not solutions. We talked in class about how these models are fed information and language from great literary sources, like Poe or Wordsworth or whomever, but anyone can feed data into these models. You can tell the AI 2+2=100 enough times, and it’ll believe it. When using it to generate ideas or references, it’s super critical to fact-check them, especially when using it in a creative sense. We saw some fascinating lines come out of the AI poems from the class exercise, some of which didn’t totally suck. While the chance the AI-generated those lines entirely on its own is high, I still feel the need to do a quick Google search to check if these “good lines” were plagiarized material/lines someone else had fed into the machine. 

In the context of writing poetry, this is not a useless tool. It can come up with pretty words/fancy phrases if you are looking for some old-school inspiration. The narratives it generates are straightforward and basic, with surface-level meaning. I feel like most poets seek a deeper meaning behind the pretty imagery used, which the bot just couldn’t create. 

From a majority of students in the class:

The class suggested that I try this exercise next year because they said that the bots will change and people’s perceptions about large language models and other forms of AI will also change. 

Some perspectives on the future of writing and AI:

The students’ pushback against the robots and their observations on the dangers of AI gave me faith in the value of human sensitivity and creativity. But the exercise also made me fearful. ChatGPT did occasionally come up with good lines. The large language model was built on the DNA of the literary tradition, and thus was more learned that the students or me or anyone. It suffered no gaps in education or memory. True self-expression grows out of subjectivity and that oddness that puts the stamp of individuality on a writer’s voice. Can a machine ever duplicate that individuality, or if not duplicate it create a new individuality altogether?

I am concerned, as many others are, that facile AI creations might dominate the arts or “popular culture” (whatever that is). Will the bot scribes, in time, coax us into accepting their recombinations and mutations of preexisting literature, no matter how elegant and respectful of the human condition that preexisting literature is? Will AI-generated literature become so widespread that we accept its verbiage as literature?  Or, more concerningly, might AI create new believable individualities that produce real poetry? 

This is not the first time that human ingenuity has unleashed great powers for human benefit or harm. Consider the printing press, the radio, the telephone, the internet. Think of the atomic bomb, gain-of-function viruses, human-caused climate change. Then think of increased freedom of expression, nuclear nonproliferation treaties, laws (heeded or not) against chemical and biological warfare, medical progress, the fight against global warming. Reflect on how effective AI already is for speeding along propaganda and untruth. In the midst of doomsday threats, the prospect of AI-created poetry seems barely worth the worry. Depending on how seriously AI dominates, poets and writers can resort to samizdat. It would not be the first time writers had to do that.

You can read Lynn Levin’s poem, “Sam Shipper and the Rock: Fiction Writing 101” in Superstition Review‘s Issue 6. Additionally, her guest post, “Beloved, Open Your Door” can be read on our blog.