A hand draws a squiggle on paper, and the AI starts guessing what it is. A few more lines, and it calls out a duck. Then a cup game: a ball goes under one cup, the cups get shuffled, and the AI follows the ball. It chats along the whole time, like a friend sitting across the table.
That was Google’s launch video for Gemini in December 2023. It looked like the future. A few days later, we learned it wasn’t quite what it looked like.
This post tells that story and four more. Some of the biggest companies in the world are in it, and so is a startup that was valued at well over a billion dollars. Each story follows the same shape: what was shown, what was reported later, and one rule you can use the next time someone shows you an AI tool.
I’ve already written a longer checklist on how to tell if an AI product is real or just a demo. This post is the other half: the real cases behind those questions. One thing before we start. None of this means AI is fake. I build AI products, and I use AI every day. The point is narrower and more useful: a demo is marketing, and you should treat it like marketing.
Every fact below comes from published reporting or official statements, listed at the end. Where a company disputed the reporting, I say so.
1. Google Gemini: the video that looked live
What was shown
On 6 December 2023, Google launched Gemini, its most capable AI model at the time. Alongside it came a “Hands-on” video, about six minutes long. A person draws, plays with toys, shows hand gestures, does a cup-and-ball trick, and Gemini responds out loud, apparently in real time, apparently by watching.
In one moment, the person makes hand shapes without saying anything, and Gemini says, in effect, “I know what you’re doing, you’re playing rock, paper, scissors.”
It felt like talking to someone who could see.
What was reported
The next day, Parmy Olson at Bloomberg reported how the video was actually made. Google told her it wasn’t a live interaction. The team had filmed the footage, then prompted Gemini with still image frames taken from it, plus text prompts. The voice you hear was not a live conversation. A note on the video itself said latency had been reduced and Gemini’s outputs shortened for brevity.
Google also published a developer blog post the same day showing the real prompts. TechCrunch compared the two. For the rock-paper-scissors moment, the video showed one quick, unprompted guess. The real prompt showed Gemini all three gestures together and asked: “What do you think I’m doing? Hint: It’s a game.”
Google’s position was that the video showed real outputs from Gemini. A Google DeepMind research leader, Oriol Vinyals, said the video illustrated what experiences built with Gemini could look like, and that it was made “to inspire developers.”
That’s fair, as far as it goes. The answers were real. But the thing that made the video feel like magic, a smooth, live, seeing-and-talking conversation, was the editing.
The rule
A demo video is an advert. Ask to see it live, on your own example.
A video shows the product on its best day, with the best takes kept and the slow parts cut. That’s what videos are for. It just isn’t evidence of how the product behaves on your messy inbox at 7 a.m.
2. Tesla Optimus: the robots at the party
What was shown
On 10 October 2024, Tesla held its “We, Robot” event near Los Angeles, mostly to show its robotaxi plans. Then the Optimus humanoid robots came out. Videos from guests show them walking among the crowd, serving drinks, dancing and chatting with people.
On stage, Elon Musk talked about a future where the robot could be a teacher, a babysitter or a bartender, something “anyone could own.”
What was reported
A few days later, Bloomberg reported, citing people familiar with the matter, that Tesla had used humans to remotely control some of the robots’ capabilities at the event. Employees stationed elsewhere oversaw many of the interactions with guests.
Guests had already been posting hints. In one widely shared clip, a robot bartender says it is being “assisted by a human.” According to Bloomberg, this wasn’t mentioned during Musk’s remarks on the webcast. A Morgan Stanley analyst wrote afterwards that the robots “relied on tele-ops,” meaning human remote operation.
To be fair to Tesla, Bloomberg’s sources also said the robots could walk on their own, using AI. They weren’t puppets. Tesla didn’t respond to requests for comment in the reports I read.
The rule
Ask which part is AI, and which part is people.
The most impressive part of a demo is often the part with the most human help. Here, walking was the robot. Chatting and serving had people behind it. Both are true at once, and only one of them was obvious to the crowd.
3. Amazon Just Walk Out: the store with no checkout
What was shown
Amazon’s Just Walk Out stores were one of the best-known ideas in retail AI. You walk in, take what you want off the shelf, and walk out. Cameras and sensors work out what you took and charge your account. No queue, no cashier.
It was in Amazon Go stores, Amazon Fresh grocery stores, and third-party places like stadiums and airports.
What was reported
In early April 2024, The Information reported that Amazon was pulling Just Walk Out from its US Fresh grocery stores and moving to Dash Carts, smart shopping carts that track items as you put them in.
Coverage of the story, including Business Insider’s, also repeated a striking figure from The Information’s reporting. According to a person who had worked on the technology, in 2022 about 700 of every 1,000 Just Walk Out sales had to be reviewed by Amazon’s team in India. Amazon’s internal goal was reportedly around 50 per 1,000. Reports said the team numbered more than 1,000 people.
Amazon disputed this. A spokesperson said the team in India mainly helps train the model, and that associates “may also validate a small minority of shopping visits” where the computer vision can’t tell with confidence what someone bought. Amazon also said the reviewers weren’t watching shoppers live. Just Walk Out stayed available to third-party partners.
So the honest version is: a reported figure, from one unnamed source, that Amazon says is wrong, plus Amazon’s own confirmation that humans review some visits.
The rule
Ask who checks the AI, and where they are.
Humans checking AI is not a scandal. Often it’s the responsible thing to do. But you should know about it, because it changes three things you care about: the cost (people are expensive, so prices can rise), the speed (a review queue can take hours), and who sees your data (your customers’ details may be read by a team you’ve never heard of).
4. Humane AI Pin: the gadget that would free you from your phone
What was shown
Humane was founded by Imran Chaudhri and Bethany Bongiorno, both former Apple employees. In November 2023 they unveiled the AI Pin: a small square device you clip to your shirt. No screen. You talk to it, and it can project text onto your palm with a laser. It cost $699, plus a $24 monthly subscription.
The pitch was a world beyond staring at a smartphone. Plenty of coverage described it as a device meant to replace your phone.
What was reported
The Pin started shipping in April 2024. The reviews were brutal. The Verge’s review said, in short, that it didn’t work well enough. The New York Times found it occasionally impressive but wrong or slow often enough to send the reviewer back to his phone. Common complaints were slowness, wrong answers, and the device running warm.
In August 2024, The Verge reported, based on internal sales data, that from May to August more AI Pins were being returned than bought. It reported lifetime sales of a little over $9 million. Humane said the report contained “inaccuracies” but didn’t say which.
On 18 February 2025, HP announced it would pay $116 million for Humane’s technology, patents and most of its staff. That’s far below the more than $200 million Humane had reportedly raised. Humane said the AI Pin was discontinued, and on 28 February 2025 the Pins stopped connecting to Humane’s servers. Calls, messages and the AI features stopped working.
The rule
Judge it on a normal Tuesday, not on launch day.
The launch happened on a stage, with perfect lighting and a rehearsed script. The real test happened in people’s pockets, on ordinary days, with bad Wi-Fi and real questions. A product that only works on stage isn’t finished.
5. Builder.ai: the “AI” app builder
What was shown
Builder.ai (called Engineer.ai in its early years) promised that building custom software could be almost as easy as ordering a pizza. Its AI assistant, Natasha, would help plan your app, and the platform would assemble much of it. The company raised more than $500 million from backers that included Microsoft and Qatar’s sovereign wealth fund, and was valued at about $1.5 billion after a 2023 funding round.
What was reported
As early as August 2019, The Wall Street Journal reported that the company relied far more on human engineers, many in India, than its marketing about AI suggested. The company responded at the time that it never claimed to fully automate app development, describing its approach as a partnership between people and machines.
Then, in May 2025, it came apart. According to the Financial Times and Bloomberg, lenders had been told to expect around $220 million in 2024 sales. A review cut that to roughly $55 million, and 2023 sales were restated from about $180 million to about $45 million. Lenders declared a default and took most of the cash in the company’s accounts. Builder.ai told staff it was entering insolvency, and it later filed for bankruptcy in the US. Reports also said US prosecutors had requested financial records from the company.
The collapse was mainly about money and governance, not just about the AI. But it started with the same gap as the other four stories: a pitch that sounded more automatic than the reality.
The rule
“AI” on the label isn’t proof. Results are.
The word AI in a sales pitch tells you what the company wants you to feel. What matters is what the product does for you, at the price you pay, after the demo ends.
The pattern
Put the five side by side:
| Demo | What it looked like | What was reported |
|---|---|---|
| Google Gemini video | A live conversation with an AI that could see | Still frames and text prompts, edited together |
| Tesla Optimus | Robots serving and chatting on their own | Humans remotely helping with some of it |
| Amazon Just Walk Out | Fully automatic checkout | A review team in India (scale disputed by Amazon) |
| Humane AI Pin | A phone-free future | Returns outpacing sales, shut down within a year |
| Builder.ai | AI that builds your app | More human engineers than promised, inflated sales |
And now the important part. Gemini is a real, useful AI. Humanoid robots are getting better every year. Cameras do work well in plenty of stores. Not one of these stories proves that AI doesn’t work.
What they prove is simpler: the demo is the best day. Your business needs the normal day.
Why demos get polished (and why that’s not always lying)
To be fair to the companies: every launch is a show. Cooking shows don’t film the burnt attempts. A shop window doesn’t show the stockroom. Polishing a demo isn’t automatically dishonest.
The problem starts when the show is the only thing you get to see. Big companies polish demos to impress investors and the press. Small AI tools do the same thing on a smaller stage, to win your monthly subscription. The pressure is identical.
So don’t get angry at demos. Just don’t buy from one.
There’s also a cost the demo never shows: setup time, checking the AI’s work, the subscription creeping up. I wrote about those in the hidden costs of AI tools nobody shows in the demo.
Five questions to ask before you buy
The five rules above turn into five questions. They work on a salesperson, a website or a video.
- Can I try it live, on my own example? Not their example. Yours. (Gemini)
- Which part is AI, and which part is people? (Optimus)
- Who checks the AI’s work, and what happens when it’s wrong? Who fixes it, and how fast? (Just Walk Out)
- Can I use it on normal days before I commit? A free trial, or monthly billing you can cancel. (Humane)
- Can I talk to a business like mine that uses it every day? Results, not labels. (Builder.ai)
A good tool says yes to all five, happily. If you get dodging, or “trust us, it’s AI,” you’re watching a demo.
If you want to go deeper, the real-or-demo checklist adds the questions these stories don’t cover: where your data goes, whether it connects to the tools you already use, and whether it still works the hundredth time. And if a salesperson tells you their product is an “AI agent,” it helps to know what an AI agent actually is before you nod along.
What a real tool looks like
Here’s the good news. The tools worth paying for want to be tested. They give you a trial. They show you what happens when they’re wrong. They tell you plainly where people are involved.
Here’s an example. Mai is fictional, a salon owner I use in these posts, but her rule is a good one: she never buys anything she hasn’t tried on her own messages for a week.
Say she tests an AI tool that drafts replies to booking messages. On day one, she gives it her real questions, the ones with typos and half-English, not the clean examples on its website. On day three, she asks what happens when it gets something wrong, and the company shows her exactly where she can correct a draft before it goes out. By the end of the week she knows two things: roughly how much time it saves her each day, and exactly which messages she still answers herself.
That’s what a real tool looks like after a week. A bit less magical than the launch video. A lot more useful.
It’s also the standard I try to hold my own work to. With Santara, the morning briefing links every item back to the booking or message it came from, and says “I’m not sure” when it isn’t. That isn’t a flashy demo. It’s what lets a host check the work on an ordinary Tuesday.
The takeaway
Five famous demos, five rules:
- A demo video is an advert. Ask to see it live, on your own example.
- Ask which part is AI and which part is people.
- Ask who checks the AI, and where they are.
- Judge it on a normal Tuesday, not on launch day.
- “AI” on the label isn’t proof. Results are.
The demo is the best day. Test it on your normal day. Good tools are glad you asked.
Sources: Google Gemini: TechCrunch, “Google’s best Gemini demo was faked” (7 Dec 2023, techcrunch.com/2023/12/07/googles-best-gemini-demo-was-faked/), summarising Parmy Olson’s Bloomberg column; Google for Developers blog, “How it’s Made: Interacting with Gemini through multimodal prompting” (6 Dec 2023, developers.googleblog.com). Tesla Optimus: Bloomberg, “Tesla’s Optimus robots were remotely operated at Cybercab event” (14 Oct 2024); TechCrunch (14 Oct 2024). Amazon Just Walk Out: The Information (April 2024) as reported by Business Insider and Gizmodo, including Amazon’s statements. Humane: The Verge, “Humane’s daily returns are outpacing sales” (Aug 2024), via 9to5Mac; The Verge and The New York Times reviews (Apr 2024); Fortune on the HP deal (19 Feb 2025). Builder.ai: The Wall Street Journal on Engineer.ai (Aug 2019), as summarised by Inc42; Financial Times and Bloomberg (May 2025), via Silicon UK. Checked October 2026.
Related: How to Tell If an AI Product Is Real or Just a Demo, The Hidden Costs of AI Tools Nobody Shows in the Demo, and Stop Getting Generic Answers From AI: The 5-Part Prompt Recipe.
