In this post, I’m diving into an experiment I conducted as part of a university assignment. The course focused on surveys, and I decided to explore whether people could detect AI-generated content compared to human-written copy. The results were intriguing, so let’s jump into my findings and see which AI tool comes out on top for me: Microsoft Copilot, Google Gemini, or OpenAI ChatGPT. (Just for fun!)

My survey was titled “Perception of AI-Generated Marketing Content” and included a range of questions designed to measure how well people could distinguish between content written by humans and content written by AI. Let’s take a look at the questions I asked and what I found.

Respondents:

To start, I asked some introductory questions to better understand the demographic makeup of my survey participants. I collected data on their age, gender, education level, and occupation. For a quick overview, check out the pie charts below, which show the breakdown of the audience I surveyed.

Four pie charts displaying demographic data. Age: Largest segment is 40.6% for ages 35–44, followed by 20.3% for 45–54, 11.6% for 25–34, 11.6% for 55–64, and 10.1% for 18–24. Gender: 49.3% Woman/girl, 46.4% Man/boy, with small segments for other identities and preferences. Occupation: 69.6% employed full-time, 11.6% student, with smaller segments for part-time, retired, unemployed, and other roles. Education Level: 30.4% Bachelor’s degree, 21.7% A-Levels, 18.8% GCSEs, 13% Master’s degree, and smaller percentages for other qualifications.

AI Awareness and Usage

Next, I wanted to gauge the respondents’ familiarity with AI. It’s important to note that this self-assessment was subjective, so respondents were able to rate their familiarity based on their own understanding. I asked: “How familiar are you with artificial intelligence (AI)?” and the most popular response (46.4%) was a mid-range answer (level 4 out of 6).

I then asked participants to identify the AI-powered products and services they had used. The top five responses were:

  • Search engines
  • Social media platforms
  • Virtual assistants
  • Recommendation systems
  • Chatbots

Lastly, I inquired about their usage of generative AI tools like ChatGPT, Gemini, and Copilot. Here are the results in the pie chart below:

Pie chart showing frequency of an activity. Segments include: 31.9% Weekly (red), 26.1% Monthly (orange), 15.9% Daily (blue), 13% Quarterly (green), and 13% Never (purple).

Perception of AI-Generated marketing content

For the next part of the survey, I presented participants with various pieces of content and asked them to determine whether each was AI-generated or human-written. Here’s a breakdown of the content samples and how participants responded.

  1. Sample 1: “Discover the joy of sustainable living with our eco-friendly products. Crafted with care and passion, our products are made from natural materials and designed to minimize their environmental impact.”
    • 68.1% of participants thought this was human-generated, but it was actually AI-generated by Google Gemini.
  2. Sample 2: “Unleash the potential of your business with our cutting-edge solutions, tailored to meet your every need. From start to finish, we’re here to support your journey toward success. Let’s grow together.”
    • 63.8% of participants identified this as AI-generated, which was correct, as it was created by Microsoft Copilot.
  3. Sample 3: “A perfect blend of sweet and spice makes everything nice. Indulge in our in-house freeze-dried sweets, crafted in a whimsical, Charlie and the Chocolate Factory-inspired space.”
    • 53.6% thought this was AI-generated, though it was actually written by a colleague of mine.
  4. Sample 4: “Drive off in style and find the vehicle that’s just right for you. Visit our dealership today to explore a wide selection of models and discover your perfect match!”
    • 58% of participants identified this as human-generated, which was correct—it was written by a friend.
  5. Sample 5: “Get schooled in success with our university apprenticeship. The future is waiting—are you?”
    • 55.1% of participants thought this was human-generated, but it was actually written by OpenAI’s ChatGPT.

 

I followed up with some open-ended questions to understand why participants thought certain pieces of content were AI-generated or human-generated.

Here’s a summary of some of their responses:

  • Reasons for thinking content was AI-generated:
    • Overuse of buzzwords and clichés
    • Unusual word choices or “over-the-top” adjectives
    • Lack of creativity or personality
    • Unnatural flow and rhythm
    • Generic tone
  • Reasons for thinking content was human-generated:
    • Creative and flowing style
    • Personalisation and relatability
    • Use of rhetorical questions and conversational tone
    • Slang or colloquial language

Microsoft Copilot vs. Google Gemini vs. OpenAI ChatGPT: Which is best?

Graphic comparing three AI platforms with their logos: Microsoft Copilot logo on the left in a multicolour gradient, Google Gemini logo in the centre with a star icon, and OpenAI logo on the right in black. The word “Vs” appears between each logo.

Logos from chatgpt.com, gemini.google.com/app and copilot.microsoft.com

Now, let’s dive into the comparison between Microsoft Copilot, Google Gemini, and OpenAI ChatGPT (my personal opinions!):

  • OpenAI ChatGPT: In my opinion, ChatGPT excels at creative text generation, particularly when it comes to clever wordplay, puns, and out-of-the-box thinking. If you need a bit of wit or humour in your content, ChatGPT is the go-to after a bit of prompting.
  • Google Gemini: Gemini stands out for its human-like feel, particularly when assisting with tasks like writing blogs. It feels more natural and “polished” compared to some other AI tools, making it ideal for research and solving search queries. This is my least-used of all AI Assistants.
  • Microsoft Copilot: Copilot is incredibly effective when it learns from you. It offers great support for a variety of tasks once it gets to know your preferences. It’s particularly useful for business-related tasks and generating marketing copy that’s tailored to your needs, with lots of human input (Which is probably why i like it. Less robot, more human).

Caution and considerations

A quick word of caution: Free versions of ChatGPT and Gemini use the data you input to train their algorithms, which means that any sensitive or proprietary information you share with them could be used for further training. Always be mindful of the data you share with these tools, especially when dealing with NDA-protected content. If you’re uncertain, Microsoft Copilot is a safer choice for private or sensitive tasks, as it stays within your domain (Make sure to log in!)

Which is the best AI tool for me?

Each AI tool has its strengths and weaknesses, and the best choice depends on your needs. For creativity, ChatGPT takes the crown. Image gen? Gemini. For a more human-like touch in writing and problem-solving, Gemini is a great option again. And for a reliable, personalised experience that learns from your preferences, Microsoft Copilot is hard to beat.

Would you like to explore my research on public opinion about AI and its impact on marketing? Feel free to reach out!

Tags: , , , , ,

« Previous Post
Next Post »