Unraveling the Quandary: The Problem with AI-Generated Content Detectors

Elizabeth Seigle | December 27, 2023 |
Reading Time: 6 minutes

While conducting our review of generative artificial intelligence (AI) options at IDS, we tested and scrutinized various tools and adopted a conservative approach to AI, keeping a primary focus on human writing.

Amid our testing process, two things became apparent:

  1. AI is going to be a big deal.
  2. AI detector/checker tools are going to be something we will run into quite often.

Businesses can maintain the safety and integrity of their online platforms by using a well-rounded approach that utilizes content detectors in tandem with AI-generated content.

These tools use advanced algorithms to sift through large amounts of information, identify language patterns, and flag content that meets programmed parameters indicating the use of AI. Although the intention behind these systems is good, they’re not perfect and can create more problems than solutions, often leading to negative consequences.

The Rising Prevalence of AI-Generated Marketing Content

While the latest iterations of AI might seem like cutting-edge, modern-day marvels, generative AI research actually began in the 1960s. It wasn’t until 2006, however, that generative AI began to develop into the predecessor of its current form, and up until 2014, there were few innovations in the field.

ai search engine marketing

In the following years, developments allowed generative AI to advance to where it is today. However, most users weren’t aware of generative AI until the past couple of years, and once the technology was released to the general public, awareness skyrocketed.

Consumers were quick to test generative AI technology like text-to-image model services, image-to-image translations, and large language model-based chatbots capable of delivering conversational dialog responses.

As more users utilize generative AI tools, there’s been a rising prevalence of AI-generated content. Here are some interesting industry stats:

  • ChatGPT was adopted by 1M users within the first five days of its release.
  • The global AI market is projected to reach $1.345B by 2030, experiencing significant growth from its estimated revenue of $86.9B in 2022.
  • The AI market is expected to see an annual growth rate of 37.3% between 2023 and 2030.
  • Some 64% of business owners think AI will help increase their overall productivity.
  • The top ways consumers are using AI include responding to messages, answering financial questions, planning travel itineraries, preparing for job interviews, crafting social media posts, and breaking down complex or lengthy information.

As generative AI technology continues to advance and is increasingly infused into every program, more industries are expected to adopt these tools into their daily operations. Innovations are already driving advancements in the journalism, marketing, entertainment, design, automotive, healthcare, retail, finance, and manufacturing industries.

Why Would Someone Use AI Detector Tools?

As AI use increases, so does the amount of machine-generated information in circulation. Most users seem open to the idea of more content from generative AI systems. Still, some individuals and organizations have taken a firm stance against publishing AI-generated content without making it clear that the content was generated via machine.

Users may take it upon themselves to employ AI detector tools to verify that their content is, in fact, generated by a human. This is becoming especially prevalent in the following fields:

  • In education, as teachers and professors review submitted student work for plagiarism or cheating
  • In journalism and marketing fields, like agencies, as clients review their contracted content
  • In other professional settings where allegations can impact an individual’s credibility and career

And it’s not without its merits - with more humans adopting and accepting AI, it’s important to ensure the benefits of automation are balanced with ethical and societal considerations, transparency, and responsible practices.

Many concerns, limitations, risks, and issues still surround AI and have yet to be firmly established. Building trust in AI requires a multidisciplinary approach involving technology, ethics, regulation, and education.

ai in digital marketing

How Do AI Content Detectors Work?

There are many detection tools available that use different methods to determine if content is produced by a human or a machine. Just as generative AI tools are powered by data and pattern recognition, so are AI content detectors.

Using machine learning algorithms and natural language processing techniques, content detectors analyze a variety of features and patterns for known markers of AI generation. Two of the overarching categories for detection methods are linguistic analysis and comparison with known AI-generated text.

Linguistic analysis involves using classifiers and embeddings to sort data. Classifiers analyze the language patterns to recognize text generated by a specific AI model, and embeddings are representations of identified clustered data points.

With classifiers and embeddings, content detectors compare the text against characteristics commonly associated with AI writing and identify patterns. This includes how fluent the text is, how frequently words are repeated, how unique words are, and other markers of AI generation, as well as if the text is similar to previously identified AI-generated content. By identifying known AI patterns in text, content detection tools can thus determine if the sample was generated by a machine or a human.

Are AI Detectors Accurate?

Now that you have a brief overview of how AI detectors work, it’s time for the million-dollar question: Are they accurate?

. . . . No! 

OpenAI, the company that developed ChatGPT, is very transparent regarding the reliability (or, rather, lack thereof) of its AI detection tools. When its AI Classifier detection tool was introduced in January 2023, it was quietly decommissioned six months later because it was not reliable.

Given their experience with complex large language models, OpenAI has found that AI detectors DO NOT WORK. Here are three main points they provide to illustrate their findings:

  • ChatGPT does not possess the knowledge to determine if content is AI-generated. The tool will sometimes make up random responses that have no basis in fact when prompted with a question like, “Could this have been written by AI?
  • Detectors sometimes suggest human-generated content was produced by AI. OpenAI tested well-known, human-written texts (Shakespeare's literature and the Declaration of Independence) in their content detector and found it was labeled as AI-generated.
  • Content detectors can also negatively impact users who have learned English as a second language or those whose writing is especially formulaic or concise.

The company also noted that even if these detectors could accurately identify AI-generated content, there are two considerations:

  1. The Federal Trade Commission warns against companies making claims that do not accurately reflect their tools’ abilities and limitations, citing these inaccuracies as violations of consumer protection laws.
  2. It’s possible to evade detection by making small edits.

franchise marketing strategist using AI tool

Potential Pitfalls of Relying on AI for Content Detection

Content detectors seem promising in a world that’s evolving to adopt more AI-generated media. However, this incipient technology has also introduced a set of potential pitfalls that are bound to increase as its use expands. Becoming aware of these issues can help people make a more informed decision about using detection tools.

Here’s a roundup of the problems with AI-generated content detectors we’ve already touched on:

  • Limitations: Detectors aren’t yet capable of identifying AI-generated content.
  • False positives: They aren’t always accurate and can suggest human-written content is AI-generated.
  • Bias: They may present bias toward non-natural language speakers and formulaic writing.

So, what are some other problems with AI-generated content detectors? Let’s take a look.

  • False negatives: AI detectors can fail to identify sophisticated AI-generated content - a problem likely to increase due to advancing algorithms.
  • Lack of nuance: Machines operate based on predefined rules, patterns, and metrics and can’t be programmed to understand context, intent, idiomatic expressions, or cultural references.
  • Evolving AI models: AI can only generate content from what it’s trained on and may lack the data necessary to understand newer models.
  • Negative consequences: False accusations of AI-generated content can result in academic penalties, damage to one’s professional reputation, or legal action.

Distinguishing Between Authentic and AI-Generated Content

The need for accurate and effective AI-generated content detectors grows as algorithms become more sophisticated and detection avoidance strategies evolve. In the meantime, it’s important to develop and employ strategies that can help you combat misinformation when using AI in digital marketing.

With generative AI tools increasing in efficacy, it’s getting harder to distinguish between human-written and machine-generated text. Earlier, we talked about how content detectors look for AI through two main categories - linguistic analysis and comparison - and the two main detection methods these categories use.

Linguistic analysis uses classifiers and embeddings to determine if the text has AI characteristics. In addition to the linguistic analysis classifiers and embeddings, there are other metrics factored into content detectors that are worth considering.

Perplexity refers to the predictability of text. This can be measured by whether the text is diverse and harder to predict, as human-written text has a higher complexity due to nuance and intent.

Burstiness is a measurement of variation within sentences. As generative AI tools only contain the information set they’re programmed with, this can lead to an overuse of certain words and a lack of variation.

Navigating the Evolution of Content Detectors

There are many benefits of AI in digital marketing, but content detectors have a ways to go before they are truly useful, effective, and reliable. These tools should help users manage and filter vast amounts of information in addition to providing other benefits like increased efficiency, accuracy, and scalability.

Developers are teaching AI detectors techniques like classifiers and embeddings to more effectively examine content, and they’re good practices to consider when reviewing content for authenticity yourself.

The continuing evolution of AI and its detectors is an ongoing opportunity to reshape how we communicate, express ideas, automate processes, and consume information in the digital world.

To learn more about how IDS can help your business with a specialized digital marketing campaign, get in touch with a member of our team today!

About the Author

Elizabeth Seigle
Elizabeth Seigle is a senior content writer at Integrated Digital Strategies, specializing in creating SEO-optimized content for a variety of clients. She stays up-to-date with the latest digital marketing trends and SEO updates to help her clients boost their online presence and stay competitive. Elizabeth is an experienced writer, holding a Bachelor of Arts in Literature from Stockton University and is HubSpot certified in Inbound Marketing and Content Marketing.

Elevate Your Franchise Marketing Efforts

Subscribe to our newsletter today!

Contact IDS!

Are you ready to work with IDS on your new website development project? Contact IDS today to get started

footer wave seperator image
© 2024 Integrated Digital Strategies Powered by Ignite Visibility | Privacy Policy | Sitemap | All Rights Reserved
linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram