Concerned About Risks in AI-Generated Content? Start with Caution in Your Prompts

Experienced AI users know that poorly crafted prompts can lead to content that requires extensive editing to become usable. This is why many courses focus on designing effective prompts to yield impressive results. A new field, “prompt engineering,” has even emerged, with numerous pre-designed prompts available online from experts.

However, while these “spells” can generate high-quality content, they often overlook the varying degrees of associated risk. For example, if AI-generated content resembles another’s work too closely or depicts recognizable celebrities, it may infringe on others’ rights.

Image from the original Medium article.
Image from the original Medium article.

The left image is a scene from the movie The Dark Knight, while the right image was generated by Midjourney using the prompt “The Dark Knight Movie Batman and Joker screencap.” Do you think such AI-generated content infringes on others’ rights? This comparison was shared by visual artist Red Southen on his X (formerly Twitter) . Southen, who has contributed to several major superhero films such as those from Marvel and DC, frequently voices his concerns on social media about AI tools imitating human-created work. Red Southen on X

AI is Just a Tool; Responsibility Lies with the User

Artificial intelligence services are essentially neutral tools. If the generated content infringes upon someone’s rights, the responsibility falls on the user, not on the tool itself or its creators — much like how Microsoft isn’t liable if a user writes a plagiarized document in Word.

Thus, users must assume full responsibility for the content they generate and not use AI as an excuse to infringe on others’ rights. If public release of such content results in harm to a third party, consequences can range from public criticism to lawsuits. Since AI functions as a “black box” and its inner workings are opaque, users seeking to avoid risk should start by being meticulous with their input. Below are three common prompt-related mistakes that users should avoid.

Mistake 1: Intentionally Replicating Others’ Work

The most common risk with AI-generated content is copyright infringement. Technically, AI models are trained on a large variety of data, so they generally won’t produce content closely resembling any single work — unless it’s prompted to do so explicitly.

Image from the original Medium article.
Image from the original Medium article.

The author tested Microsoft Copilot by attempting to generate an image of Hello Kitty without mentioning the character’s name. Even with this approach, the resulting image differed significantly from the target character. (Prompt: A cartoon character with a white cat, no mouth, a yellow nose, and a red bow on one ear.)

In tests with Microsoft’s Copilot, for example, even when attempting to generate images of “Hello Kitty” without naming the character, the output still significantly differed from the original character. However, if the prompt includes specific details or even the character’s name, the output might become uncomfortably close to the copyrighted work.

Image from the original Medium article.
Image from the original Medium article.

If “Hello Kitty” is used directly as a prompt, the results might indeed be quite “satisfying.” However, generating such an image is clearly inappropriate. This image was similarly created by the author using Copilot. (Prompt: Generate a cartoon image with Hello Kitty at the center.)

When prompts are overly specific, AI is more likely to generate highly similar content, increasing the risk of copyright infringement. This reinforces the idea that users must bear ultimate responsibility, as infringement risks often stem from the prompt itself.

Even more serious infringements can occur when users upload existing articles, images, or videos to be mimicked or modified. Many AI services now include features to detect and block these types of prompts to prevent intentional copyright violations.

Mistake 2: Mimicking an Artist’s Style

AI can easily replicate an artist’s unique style, be it visual art, vocal timbre, or other personal characteristics, by learning from large datasets. This falls into a legal gray area rather than outright plagiarism, but it has still become a moneymaking strategy for some users.

Although personal style isn’t always legally protected, using it still carries risks, particularly for businesses. Before ChatGPT became widely known, a Japanese company launched an AI tool called Mimic, which could generate art in an artist’s unique style by analyzing their previous work.

Image from the original Medium article.
Image from the original Medium article.

Mimic can directly generate works in a similar style to existing artworks (top left), producing new images in the artist’s style (bottom). This has sparked significant controversy. Image source: Mimic official X post.

Mimic was initially designed as a tool for artists to use AI to assist in creating their own work. However, due to its inability to prevent malicious misuse by others, the service was ultimately discontinued amid overwhelming criticism. This demonstrates that society often places a higher moral responsibility on companies than legal regulations, meaning that even in legally ambiguous areas, certain boundaries should not be crossed.

If companies want to generate content in another’s style, they should seek permission from the original artist. Mimic’s latest version includes more stringent account verification and transparency, as well as a watermark to identify AI-generated images, helping to prevent misuse.

Mistake 3: Using Identifiable Individuals in Prompts

In addition to artistic imitation, users should avoid intentionally generating images with famous faces or other recognizable brand elements. For example, in January 2024, an American podcaster generated a video featuring the likeness of the late comedian George Carlin and posted it on YouTube, causing viewers to mistake it for an actual Carlin performance. This led to a lawsuit.

Another case involved OpenAI’s release of GPT-4o in May 2024. OpenAI’s choice to promote it with a voice that resembled Scarlett Johansson’s character from the movie Her created a public outcry, even though there was no evidence of actual voice sampling. OpenAI quickly removed the voice to avoid further PR issues.

Conclusion: A Foundation in Legal Compliance

In all these examples, the core issue remains the same: even without AI, these actions would still constitute infringement. Often, such cases of infringement are intentional, and a responsible user will typically recognize inappropriate use and make necessary adjustments. Users should also be mindful of situations involving the sharing of names, addresses, phone numbers, or other private information, as well as the spread of false information.

Image from the original Medium article.
Image from the original Medium article.

In April 2023, images circulated online showing former U.S. President Donald Trump being arrested by police; however, these were actually AI-generated fake photos. Source: Mehn Updates (X post).

For companies and individual users alike, actions that would be off-limits without AI remain unacceptable when using AI. From a management perspective, this is beneficial because it means that a few well-designed training sessions on copyright, Portrait rights, and privacy can foster responsible AI use without overly complex policies. This approach empowers employees to use AI tools sensibly and reduces business risks.