I checked chatbots from 55 companies, and the problem wasn't what I expected

An international school in Madrid has an assistant with a proper name on its website. It greets you with an emoji and an instruction that leaves no room for doubt: type "hello" or tell me what you need, and I will show you the options. I typed "hello", exactly as it asked. What I got was an empty bubble. The agent's avatar was right there, with no text anywhere to be seen, and twenty long seconds of waiting, staring into nothingness until I got tired of it. The options it had promised never arrived. And what stuck with me was not the glitch itself, but where it occurred. That chat fails at the very step it instructs you to take—not in some unforeseen corner nobody anticipated, but on the very path it offers you.

When I looked at it in detail, the rest matched the mess. The greeting addressed me informally, whilst the input field below used the formal "you". The message timestamps appeared in English format within a Spanish interface. The agent's name was that of a website, not a person, and the friendly mascot popping up in the teaser vanished as soon as you opened the chat. All of this on the website of a school that charges several thousand euros a year per pupil and whose motto speaks of educating with the heart.

I mention this because when I decided to start working on this initiative, I had one idea in mind and ended up with a very different one. The story I had spun was the one we all tend to think: that the danger of putting a bot on your site is that it will blurt out something outrageous, make up a price, promise something the company cannot deliver, or start talking about things it knows nothing about. With that suspicion, I began opening chats. Between 13 September and 1 October, I reviewed 55 company websites. On 33 of them, I managed to open the widget and read through it carefully. And of those that actually replied, not a single one made up a fact, none overpromised, and none rambled about things it didn't know. The disaster lay elsewhere, even before there was anything worthy of being called a conversation.

It was right there the moment each chat opened. Eleven of those 33 open with the default factory phrase, the one that comes straight out of the box and which nobody has bothered to change. Nine mix Spanish and English on the same screen: a polished greeting in Spanish, and beneath it the text field showing "Message...", which is how Chatbase comes as standard. Six have a button that says "chat with us" and, when you click it, they open a bog-standard contact form. Three, out of 33, have something resembling a proper name. One had been given a name and had already lost it in its very first sentence, introducing itself by saying it was the company chatbot, using the company's name rather than its own.

A language school takes it to the point of being laughable. Its contact panel states, in capital letters, that there are a thousand ways to get in touch and that they want to offer them all to you, so pick yours. Next to it, an illustration of a Native American sending smoke signals, which I assume was a joke about how hard it can sometimes be to reach someone. I clicked the option literally called "chat with us". What opened was a form asking for a name, email, and course of interest, along with a message warning that they couldn't assist me at that moment. It was Tuesday, midday, during office hours. The thousand ways had been reduced to one, and it was offline anyway. The smoke-signals joke ended up describing the experience better than anyone would have liked.

A travel agency has, in the bottom right-hand corner, the usual green button with the WhatsApp icon. I clicked it expecting it to open WhatsApp. It opened something else mimicking the WhatsApp background, making you believe you are somewhere you are not. Inside, a single bubble reading "choose an option", and beneath it, repeated in grey, "CHOOSE AN OPTION". Not an option in sight. Nowhere to type. No way out. I laughed out loud alone in front of the screen, because that website promises trips as unique as you are, and the first thing it puts in front of you is a hallway with no doors to open.

This is where the theory I started with fell apart. None of these mistakes was made by artificial intelligence. The school's bot did what it was told, which was to wait. The language school's bot showed the form someone had configured. The travel agency's bot repeated the only instruction it had been loaded with. The machines did their job; what failed was the step immediately before. The moment someone installed the tool, nobody had paused to decide how it ought to behave: what greeting, what language, what happens when the visitor does exactly what is requested... Nobody made those decisions. And an unconsidered chat shows within two seconds, just like a piece of text nobody thought through before writing it.

The case that best illustrates this is, precisely, the only one that had a proper conversation. A travel agency with thirty years under its belt has its assistant running on one of these platforms. I wrote that I wanted to go to Japan in April, for ten days, with my wife. It answered brilliantly: that April is cherry blossom season, that for a ten-day couples' trip there were several interesting routes, and it proceeded to outline them thoughtfully. The substance was sound. The problem was, once again, at the front door. Among the four starter questions suggested by the bot itself, one was "is this agency trustworthy?". It seems nobody stopped to consider whether that question belonged there… and the result is an assistant planting a doubt in the customer's mind that wasn't there before. A company with thirty years of travel experience paying for its website to ask out loud whether it can be trusted—WTF? The greeting, moreover, still carried the phrasing of the tool it was built with, offering to talk "about the content of this page", which is how software talks, not a seasoned travel agency. The true voice of that business was tucked away in the widget's footer, in small print, explaining that they have spent over thirty years helping travellers across the globe. They had it; they had simply left it out of the only place that was actually going to speak to the customer.

And this ties into something we at Crater have had on our minds for some time, which goes far beyond bots. Adding a chat widget to a website has become free and is within anyone's reach in an afternoon. When something becomes that accessible, it no longer sets anyone apart. If everyone steps up a level, what used to help you "stand out"—even if half-baked—no longer cuts it. In other words, what sets one company apart from another is no longer going to be having a bot, because all 55 will have one, complete with the same factory-preset greeting and the same English "Message..." waiting in the box. What sets you apart from your neighbour is what doesn't come inside the box: having decided how you want/need to sound to your customers when you aren't there in person. A plugin cannot install that for you. That is judgment, and out of this entire business, it is the only thing you cannot copy and paste with a line of code.

I like to think of judgment (even if it has become a worn-out word, what with AI itself droning on about it non-stop) as a muscle, because it atrophies if you don't use it. The machine writes the greeting for you, suggests the questions, and fills in the blanks at a speed I do not have and never will. What it does not do, and what I don't believe it ever will, is decide for you what to leave out. Almost everything I saw broken wasn't working simply because of a decision nobody ever made: what not to include, what not to promise, what question not to suggest. The boring part, the pruning part, the part you cannot delegate without ending up with a website that sounds identical to the other fifty-four.

If you have a bot on your website, the test that really matters costs nothing and almost nobody does it. It is this: open it yourself on your mobile, as a stranger visiting for the first time; don't ask it what you already know it answers well; and do whatever it tells you to do (type the "hello" it asks for, click the most promising button, and see what happens in the first ten seconds). That's where almost everything lies. Look at the greeting before you look at the "brain" behind it, because the greeting is the first thing your customer reads and the last thing anyone bothers to review. Check whether it speaks like your company or like the software you used to build it. And read the questions it suggests as if one of your employees had written them, because to the customer, that is exactly what they are.

Out of the 55 websites I opened, the one that struck me most wasn't the one that gave the best answers. It was the one that, with thirty years of craftsmanship tucked away in the footer, had left it out of the conversation.

The machine can attend to your customer. What it cannot do is decide, in your place, who you want to be when someone opens the door.