How to Test a Chatbot Before Going Live (Small Business Guide)
A chatbot can be your best employee or your worst. The difference usually comes down to what happens in the week before you flip the switch.
It's a familiar failure: a bot that answers "Do you ship to Canada?" with a recipe for banana bread. It's also avoidable — a week of deliberate testing catches most of it before a paying customer sees the mess.
This guide walks you through how to test a chatbot before going live. No technical background needed. We'll cover what to test, how to test it, and which tools (Tidio, Crisp, and Botpress) make the process easier.
One note on sourcing: prices and limits below come from the vendors' public pricing pages and documentation, not from paid testing.
Start with a written script of your most common questions
Before you touch the testing tools, grab a pen. Open your last month of customer emails, DMs, and chat logs. Write down the 20 questions you answer most often.
That list is your test script, and it's often a surprise. A store can spend weeks polishing generic FAQ answers, then find that most of its real questions are about one narrow thing — custom orders, pickup windows, a single product line. Without the list, you launch a bot that misses the point.
Once you have your list, sort it into three buckets:
- **Simple facts** — hours, address, return policy
- **Order-specific questions** — "Where is my package?"
- **Judgment calls** — complaints, refund requests, custom quotes
Your bot should handle the first bucket perfectly. It should handle the second only if it connects to your order system. It should hand the third to a human every time.
Test the bot like a confused customer would
Most people test their chatbot by asking clean, polite questions. That's a mistake. Real customers type in lowercase, make typos, send three messages in a row, and ask the same thing five different ways.
Here is the routine that catches the most:
1. Ask the same question five different ways. "What are your hours?" "r u open saturday" "hours??" "when do you close" "open tomorrow?" 2. Misspell things on purpose. "shippin cost" and "retrun policy" should still get real answers. 3. Send a message with no context. Just "help" or "hello?" See what happens. 4. Try a question your bot can't answer. Does it admit that, or does it make something up?
That last one matters most. AI chatbots trained on your content will sometimes invent an answer rather than say "I don't know." This is called a hallucination, and it's the number one cause of chatbot embarrassment.
Tidio's Lyro AI is built to answer from your FAQ content, which keeps its replies tied to what you published. Botpress gives you more control over fallback behavior, but only if you actually configure it. Crisp's chatbot is simpler and more scripted, which means fewer surprises but less flexibility.
Run a full conversation, not just single questions
A customer rarely asks one question and leaves. They ask about shipping, then ask about returns, then ask to speak to a person. Your bot needs to survive that whole journey.
Sit down and role-play a full conversation. Pretend you're a customer with a problem. Start with a simple question, then follow up with something related, then ask for a human.
Watch for these failures:
- **Memory loss** — the bot forgets what you just said
- **Looping** — it asks "Can I help with anything else?" forever
- **Dead ends** — it gives an answer and then goes silent with no next step
- **Handoff failure** — you ask for a human and nothing happens
That handoff step is where a lot of small businesses get burned. A bot that can't cleanly pass a customer to a real person is worse than no bot at all.
Check the handoff to a human
Test this specifically, and test it more than once. Ask for a human in three different ways: directly ("I want to talk to someone"), indirectly ("this isn't helping"), and angrily ("let me speak to a manager").
Then check what the human sees on their end. Does the chat appear in the inbox with the full conversation history? Does the customer get told someone is coming? How long does it actually take?
Crisp does this well because its shared inbox is the core of the product — chat, email, and chatbot conversations all land in one place. Tidio also handles handoff smoothly, especially on the Shopify side where order details come through with the chat. Botpress can do it, but you'll need to build the workflow yourself, and that's where non-technical owners usually get stuck.
Test on a staging page first, not your live site
Most chatbot tools let you install the widget on a hidden test page or preview it before publishing. Use that. Don't learn about bugs in front of paying customers.
Here's a simple staging setup:
- Create a hidden page on your site with the chatbot installed
- Run your full test script there
- Invite two or three real customers or friends to try it cold
- Fix what breaks, then repeat
The cold testers matter. You know your business too well to test honestly. Someone who's never seen your site will ask questions you didn't think of.
For a free option, Tidio's free plan includes 50 conversations per month, which is plenty for testing. Crisp's free plan gives you two seats with no AI credits, so you can test the inbox and chatbot flow but not the AI features. Botpress offers 100 conversations per month on its free tier plus a $5 monthly AI credit — enough for a serious test run if you're comfortable with a steeper learning curve.
Test the boring stuff: speed, mobile, and after hours
Three things get skipped in almost every chatbot test, and all three matter.
**Speed.** Time how long the bot takes to respond. Anything over three or four seconds feels broken to a customer.
**Mobile.** More than half your visitors are on a phone. Open the widget on your own phone and check that it doesn't cover your menu, block your checkout button, or shrink to unreadable text.
**After hours.** Set your business hours and test the bot at 11pm. Does it still answer? Does it tell people when you'll be back? Does it collect an email so you can follow up?
A bot can work perfectly at 2pm and go completely silent at 9pm, because nobody tested the schedule settings. A customer who messages at night and gets nothing assumes you're closed forever.
Keep testing after launch
Launch day isn't the finish line. Set a reminder to review chatbot transcripts weekly for the first month.
Look for conversations where the bot failed, gave a wrong answer, or handed off unnecessarily. Each one is a fix. Add the missing question to your FAQ. Tighten the fallback message. Adjust the handoff trigger.
Tidio and Crisp both let you read full transcripts in the inbox. Botpress gives you deeper logs, but you'll need to know where to look. On Tidio's lower plans, advanced analytics are locked to higher tiers, so plan accordingly if reporting matters to you.
The businesses that get real value from chatbots aren't the ones with the fanciest setup. They're the ones that keep tuning for a few weeks after launch.
Frequently asked questions
How long should I test a chatbot before going live?
Plan for at least one full week of active testing, and longer if your bot handles orders or payments. Run your full question script, do the cold-tester round, and test on mobile and after hours. If you can, keep the bot on a hidden page for a second week while you fix issues.
What's the most common mistake when testing a chatbot?
Testing with clean, polite questions. Real customers type messy, short, misspelled messages and ask the same thing multiple ways. If your only test is "What are your business hours?", you haven't tested anything yet.
Do I need a technical team to test a chatbot properly?
No. Tidio and Crisp are both set up for non-technical owners, with visual builders and preview modes. Botpress is a different story — it's a developer-grade tool with a real learning curve, so only pick it if you have someone comfortable building workflows.
What should I do if my chatbot gives a wrong answer during testing?
Write down the exact question and add a corrected answer to your FAQ or knowledge base, then re-test. If the bot invents answers to questions it can't handle, adjust the fallback message so it says "I don't know, let me connect you to a person" instead of guessing. Repeat this after launch — wrong answers will keep showing up in real conversations.
Ready to pick one?
Marcus Reed — Marcus Reed is the senior editor at ChatbotRadar. He tracks live chat and AI chatbot tools for small teams, verifying pricing on vendor sites and trialling tools on their free plans. How he tests · Report an error
Disclosure: this article contains affiliate links. If you buy through them we may earn a commission at no extra cost to you. Pricing and features are re-checked regularly but can change — always confirm on the vendor's site.