Should you use ChatGPT to automate your email program?
To ChatGPT or not to ChatGPT? That is the latest question sweeping the net. And whether we choose individually to embrace generative AI, or leave it the heck alone, we can’t put this cat back in the bag – but we can harness and automate it. We’ll show you how in this post.
PUBLISHED ON
ChatGPT, do we love it? Do we fear it? Do we need it? When researching for this post we polled our team internally for their reactions to ChatGPT, and the responses were a mixed bag. As an email service provider, we see the benefits of generative AI – and that’s what we’ll explore in this post – but we also want to talk about us, the humans, and how AI is reshaping our internet and our interactions.
Don’t worry, we’re pretty sure this isn’t the end of the world.
Table of contents
What ChatGPT is not
Reinforcement learning from human feedback
How using ChatGPT could affect your deliverability
AI and the inbox
What is ChatGPT
Sometimes, it’s better to hear it straight from the horse’s mouth. To kick off this post, we asked ChatGPT to define itself.
Mailgun: “What are you? ” ChatGPT: “I am an AI language model developed by OpenAI. I’ve been trained on a diverse range of internet text and can respond to a wide range of questions and generate text on various topics. I am designed to provide information and respond to user queries to the best of my ability, but I do not have consciousness or personal experiences.”
Not bad, but let us dive in with a bit more detail. ChatGPT (Generative Pre-trained Transformer) is a chatbot developed by OpenAI launched in November 2022. It is built from OpenAI's GPT-3 family of large language models. ChatGPT is unique because of the way it uses reinforcement learning from human feedback. This allows it to be more conversational and more “human” in its language sequence patterns, making it possible to influence tone and style more effectively:
Mailgun: “Rewrite in a more conversational style and add a relevant joke.” ChatGPT: “Hey there! I’m ChatGPT, a super smart AI language model. My job is to help you out with whatever questions or tasks you need assistance with. Think of me like your own personal Siri, Alexa, or Google Assistant! I was created by OpenAI using an advanced machine learning algorithm that taught me how to process and understand natural language. So don’t worry about being too technical with me, I’m pretty much fluent in human-speak. Now, here’s a joke for you: Why was the computer cold? Because it left its Windows open! I know, I know, I’ll stick to answering your questions instead of cracking jokes. What can I help you with today?
Before we jump into concepts like reinforcement learning, cement in your mind the idea that ChatGPT is a language learning model. AI is the umbrella term describing all generative and predictive machine learning tools. Language models specifically are trained on statistics and use machine learning algorithms, probabilities, and sequences to predict words and responses.
This, plus the internal knowledge base it was trained on, is how ChatGPT can provide believable answers to questions without connecting to an internet search engine.
What ChatGPT is not
For our last bit of housekeeping (and then we promise we’ll start dissecting) here are a few of ChatGPT’s limitations. The most important takeaway? ChatGPT is a predictive language model, not an infallible knowledge source.
ChatGPT is not currently connected to the internet in terms of retrieving information. In fact, all the information it was trained on is from 2021 and prior.
ChatGPT is not always accurate.
ChatGPT is not always objective and may produce output with a topic bias.
How does ChatGPT work?
ChatGPT works by predicting responses based on the statistical structure of language. It is the latest language model from OpenAI and represents a significant improvement over its predecessor GPT-3.
In this section on the nuts and bolts of how ChatGPT works, our source information comes from the Training language models to follow instructions with human feedback report that was created for InstructGPT – developed between GPT-3 and ChatGPT – which OpenAI touts as the sibling model to ChatGPT. This report is the most comprehensive document currently available from OpenAI that applies to ChatGPT.
Reinforcement learning from human feedback
Reinforcement learning from human feedback (RLHF) uses humans, referred to as “labelers”, to align the language learning model to user intent. Adding in a human control element generates a more conversational language structure and helps create comparison data to train the language model to better handle these common pitfalls:
Language models are often not helpful.
Because they predict text, these models may “make up” facts because the predicted words fit with their understanding of the language structure.
Responses may be unable to be interpreted.
Language models may generate output with topic bias based on their datasets.
“Our aim was to select a group of labelers who were sensitive to the preferences of different demographic groups, and who were good at identifying outputs that were potentially harmful.”
Open AI 2022, Training language models to follow instructions with human feedback <https://arxiv.org/pdf/2203.02155.pdf>
Here’s how it works:
Humans are hired to be labelers, usually freelancers or subcontractors from sites like Upwork, and they collect human preference data based on OpenAI’s user alignment criteria. The goal is to hire labelers that are empathetic towards different cultures and demographics. Think of human labelers as a jury of peers that evaluates ChatGPT’s output. Labelers interact with the language model at various stages throughout its training and development.
Phase 1: A pre-trained language model is supplied with a controlled subset of data to learn a supervised policy, the SFT model, which generates output from a set list of prompts.
Phase 2: The language model mimics human preference based on review performed by the human labelers. The labelers vote on the best output. What results is a new dataset of comparison data called the reward model.
Phase 3: The reward model is used to improve the original SFT model, and that model is called the policy model.
In practice this looks like this:
OpenAI 2022, Training language models to follow instructions with human feedback - Figure 2
Three AI laws to know before using ChatGPT with your email program
AI isn’t a good thing or a bad thing, it’s just a thing. We’re gearing up to talk about using ChatGPT with your email program. Here’s the knowledge you need to keep top of mind:
AI is not a replacement for human creativity. In fact, AI often makes mistakes and is capable of generating content with a bias.
AI generates responses only when directed. To integrate with AI you have to create a flow that prompts the tool (ChatGPT) to create a response.
AI behaves based on data it is trained on. For the moment, ChatGPT can regenerate responses based on your feedback within its current knowledge bank. It may learn universally from data you input, but only if OpenAI implements that data on the backend.
AI can be used to automate tasks. Automations can trigger AI tools like ChatGPT to create text, but it doesn’t perform autonomously.
How can you use ChatGPT to automate your email program
We asked ChatGPT to write a paragraph telling us how we could use it with Mailgun to automate email sending. This is what it said:
ChatGPT can be integrated with a tool like Mailgun to automate the process of sending emails. With Mailgun, developers can use APIs to send, receive and track email and ChatGPT can be used to generate personalized and dynamic email content based on user interactions or other data inputs. This can be used to send automated emails such as personalized newsletters, purchase receipts, or automated follow-up messages. Additionally, ChatGPT can be used to generate the subject line and recipient list for the email, providing a more personalized and efficient email automation experience. This integration can significantly improve the efficiency and personalization of email communication for businesses and organizations. To integrate ChatGPT with the Mailgun API, you will need to follow these steps: 1. Create an account with Mailgun and obtain an API key. 2. Use the Mailgun API key to send an email via the Mailgun API. This can be done using a programming language such as Python, and the requests library to make a POST request to the Mailgun API endpoint for sending email. 3. Use ChatGPT to generate the content for the email by providing it with the necessary prompts and context. 4. Use the output from ChatGPT as the body of the email when sending it via the Mailgun API. You may also need to set up webhooks in order to receive any incoming email and trigger the chatGPT to respond accordingly. 5. Finally, you will need to test the integration to make sure that it is working as expected. Note: This is a high-level overview and the actual implementation will depend on the specific requirements of your use case and the programming languages and libraries you are using.
We’re not going to lie, this sounds legit. Not only did it give us the general process for sending an email via API, it told us where and how to use ChatGPT to generate the copy for the emails, and, when prompted, creates email copy like this:
Mailgun: “Write copy for a transactional email triggered when users request a password reset.” ChatGPT: “Subject: Reset your password Hi [Username], We received a request to reset the password associated with your account. No worries, we’ve got you covered! To reset your password, click the link below: [Password Reset Link] If you didn’t request this password reset, please ignore this email and contact us immediately at [Contact Email]. We take the security of your account very seriously, and we want to ensure that only you can access your account. Once you reset your password, please log in and check that everything looks good. And don’t forget to create a strong and unique password to keep your account secure. If you have questions or concerns, please don’t hesitate to contact us at [Contact Email]. Best regards, [your company] Team
But Step 4 is where it starts to get vague: “You may also need to set up webhooks in order to receive any incoming email and trigger the chatGPT to respond accordingly.”
There are a couple possible ways to integrate ChatGPT with a tool like Mailgun (or any cloud-based tool for that matter). Notably, Zapier has built out a basic Zap for ChatGPT integrations that you can use to connect ChatGPT to other tools and platforms. You can also use webhooks to integrate.
So, we asked ChatGPT to take a stab at writing a webhook to integrate itself with Mailgun:
from flask import Flask, request import requests import jsonapp = Flask(__name__)@app.route('/webhook', methods=['POST']) def webhook(): # Get the email data from the request email_data = json.loads(request.data) # Use ChatGPT to generate a response to the email response = generate_response_with_chatgpt(email_data) # Send the response email using the Mailgun API send_response_email(response, email_data) return 'Response sent'def generate_response_with_chatgpt(email_data): # Use ChatGPT to generate a response to the email response = "Generated response using ChatGPT" return responsedef send_response_email(response, email_data): # Set up the data for the API request api_key = 'key-your_api_key' recipient = email_data['from'] subject = 'Re: ' + email_data['subject'] body = response # Send the email using the Mailgun API requests.post( "https://api.mailgun.net/v3/your_domain.com/messages", auth=("api", api_key), data={"from": "Mailgun Sandbox <postmaster@your_domain.com>", "to": recipient, "subject": subject, "text": body})if __name__ == '__main__': app.run()
This code isn’t ready to go right out of the box, though. When we asked ChatGPT to generate the code, we didn’t specify language, so it appears its default is Python – at least in this instance. However, Python isn’t everyone’s go to when writing webhooks.
Our first recommendation when using ChatGPT is to be specific. ChatGPT is capable of generating code and solving mathematical equations, but defining specific parameters for ChatGPT is ridiculously important.
The next callout is to be prepared to take formatting offline. ChatGPT has a copy/paste feature for code but it either isn’t completely accurate, or it doesn’t carry over the formatting to all external readers.
We had to update the code to be bootable:
You’d need to run this generated webapp on either an actual public server or use a service that allows you to expose your application to the internet through a local host tunnel. You’d also likely need to make a few tweaks to the code depending on your tools and environment.
Ok, so where did we land? We love automating things and making processes efficient. Transactional emails that are triggered by user action (password resets, abandoned carts, etc.) are phenomenal for your email program. ChatGPT can help you build that part of your email program from copy to code if you use it as a draft generator and ditch any expectations you have of perfection on the first iteration. We are not at the point yet where AI tools can absorb the complete creation of these processes.
Should you use ChatGPT to automate your email program?
If this were just a matter of saving time, it would be very easy to recommend integrating a tool like ChatGPT into your email program. But, in the sage wisdom of Vanilla Ice, we can only recommend you “stop, collaborate, and listen” because this brand-new invention isn’t perfection.
Before you fully embrace automation with ChatGPT, make sure you understand how your email program works, and the points where implementing an integration like this makes the most sense.
How using ChatGPT could affect your deliverability
We asked ChatGPT if it could explain our deliverability services – just for fun, in the style of George Carlin. Here’s what it said:
Email addresses verifications, expert support to understand what can land you in the spam folder, and analytics are all part of deliverability, but a lot is missing here (including a likeness to George Carlin). This generated pitch proves our point that deliverability is not only nuanced but varies from sender to sender.
“It’s the sender that understands who their audience is and is able to connect with that audience, and audience can vary greatly across different verticals. When we look at getting to the inbox, deliverability problems will vary from sender to sender, so a thoughtful approach is needed when we look at improving deliverability.”
Theresa Kinney, Technical Account Manager at Mailgun
ChatGPT uses RLHF to generate its output and human labelers are tasked with being objective in terms of cultural biases, but the labelers are not tasked (as far as we can tell) with training ChatGPT in depth on different industries, demographics, or communities.
With deliverability, step one is knowing your audience and working to relate to them specifically – even something as simple as the length of the email. If your demographic is mostly Gen Z, then four or five paragraphs of text won’t engage them, which is what ChatGPT will usually produce if you don’t give it a parameter for length.
The same goes for writing code, or error troubleshooting. As our deliverability team tested ChatGPT for various use cases, they found that a lot of review is necessary. This is very obvious in code generation. If we implement scripts from ChatGPT and then try to troubleshoot the subsequent errors, it goes a little something like this:
In general, calling ChatGPT out on errors doesn’t elicit a corrected response, it just reframes the response to restate the error with the information you provided. The takeaway? ChatGPT is a generative response tool, but it doesn’t quite have the ability to troubleshoot, correct, or validate its own responses.
AI and the inbox
As tools like ChatGPT hit the market, and more and more companies use them to generate content like email copy, is there a risk that ISPs will start to crack down on recognizing AI content as spam? Is this even possible? Instead of asking for ChatGPT’s thoughts on this, we talked to Mailgun’s Product Manager, Chris Farmer.
Most mailbox providers rely on user input to identify what is or is not spam content. Users interact with what’s relevant to them, based on what they’ve asked for. So, if ChatGPT is used to generate content for a marketing campaign, and that content contains spammy keywords, content that other users have marked as spam, or content that is not relevant to users based on what they subscribed to receive, then the AI content could easily be marked spam.
“ChatGPT is a tool to add to your toolbox, just like any other tool, and it has the possibility of making our lives more efficient, but it surely won’t replace us.”
Chris Farmer, Product Manager at Sinch Mailgun
In terms of using ChatGPT with your email program, we can see use cases where you could create copy and integrations like:
Generating email copy for transactional emails or campaigns
Writing a program to send email through Mailgun
Writing a program to automatically reply to emails based on an existing integration with your application, such as importing calendar schedules and resource availability to respond to incoming inquiries
But ChatGPT is not a set-it-and-forget-it tool. Anything it creates will need to be edited and adapted.
Final thoughts
Even while writing this post, there have been changes in the world of chat AI tools. Notably Microsoft, who is integrating their Bing search engine and Edge browser with ChatGPT features, and Google who is rapidly developing their own solution called Bard. The results on both fronts are intriguing but definitely not sentient.
On the email front, we’re watching intently with a fresh bowl of popcorn (and one hand on the power cord... just in case), but we aren’t surrendering all our tasks to AI just yet.
We hope you enjoyed this post, but it was just a warm-up. Check out our upcoming webinars and choose your fighter. Marketers vs. Bots, and Email Geeks vs. ChatGPT.
