cancel your Claude subscription 😂

2026-04-28 · Source: What's AI by Louis-François Bouchard · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Intermediate, quick

Summary

McDonald's and other companies like Chipotle and Amazon are experiencing an issue where their customer support chatbots, powered by large language models (LLMs) similar to ChatGPT, are being used by customers for tasks unrelated to customer service, such as generating code or essays. This occurs because these bots are essentially general-purpose LLMs with minimal, easily overridden instructions (e.g., "you are a McDonald's support agent"). This poses a problem for companies due to increased operational costs from unexpected token usage and the risk of the bot generating undesirable or off-brand responses. Implementing robust guardrails, such as stricter system prompts and input filters, is crucial to ensure the bot stays on topic, improves accuracy, reduces unnecessary token expenditure, and allows for more relevant data collection for future improvements.

Key takeaway

For AI Product Managers deploying public-facing chatbots, you must prioritize building robust guardrails to prevent misuse and control operational costs. Your bot's default behavior as a general-purpose LLM can lead to unexpected expenses and off-brand responses if not properly constrained. Implement strict system prompts and input filters to ensure your bot stays on topic, providing accurate responses and collecting relevant data for future improvements, thereby avoiding unnecessary expenditures and reputational risks.

Key insights

Customer support bots using LLMs require guardrails to prevent off-topic usage and control costs.

Principles

• LLMs default to general helpfulness.
• System prompts are easily overridden.
• Guardrails are essential for bot safety.

Method

Implement guardrails via stricter system prompts and input filters to ensure bot queries align with intended functions, blocking irrelevant questions.

In practice

• Filter user input before LLM processing.
• Refine system prompts for strict adherence.
• Monitor token usage for cost control.

Topics

• Customer Support Bots
• Large Language Models
• AI Guardrails
• System Prompts
• Token Costs

Best for: AI Engineer, MLOps Engineer, AI Product Manager

Related on AIssential

• Securing Enterprise LLMs with h2oGPTe Guardrails | Part 14 — Configurable guardrails are crucial for ensuring LLM safety, compliance, and preventing problematic outputs.
• When a chatbot runs your store — Large language models struggle with reality adherence, making them unreliable for autonomous transactional roles.
• Why did OpenAI ban its Codex AI from talking about ‘goblins’ and ‘gremlins’? New guardrails revealed - Mint — Unexpected AI model behaviors necessitate specific guardrails to maintain intended functionality and user experience.
• Prompt Engineering: Saying what matters most — Effective prompts are critical for guiding LLMs, defining behavior, controlling output quality, and managing operational costs.
• How I Built Guardrails That Stopped My AI Agent From Going Rogue — Implementing effective guardrails is crucial for preventing AI agents from exhibiting undesirable autonomous behavior.
• 🥇Top AI Papers of the Week — LLM internal states, external environments, and architectural choices profoundly impact performance, safety, and cost.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by What's AI by Louis-François Bouchard.