Gemini in Google Home Keeps Mistaking My Dog for a Cat

A cat jumped up on my couch. Wait a minute. I don’t have a cat.

The alert about the leaping feline is something my Google Home app sent me when I was out at a party. Turns out it was my dog. This notification came through a day after I turned on Google’s Gemini for Home capability in the Google Home app. It brings the power of large language models to the smart home ecosystem, and one of the most useful features is more descriptive alerts from my Nest security cameras. So, instead of “Person seen,” it can tell me FedEx came by and dropped off two packages.

In the two weeks since I allowed Gemini to power my Google Home, I’ve enjoyed its ability to detect delivery drivers the most. At the end of the day, I can ask in the Google Home app, “How many packages came today” and get an accurate answer. It’s nice to know that it’s FedEx at the door, per my Nest Doorbell, and not a salesperson offering to replace my windows. Yet for all its smarts, Gemini refuses to understand that I do not have a cat in my house.

Person Seen

ScreenshotGoogle Home via Julian Chokkattu

Google isn’t the only company souping up its smart-home ecosystem with AI. Amazon recently announced a feature on its Ring cameras called Search Party that will use a neighborhood’s worth of outdoor Ring cameras to help someone find their lost dog. (I don’t need to stretch to imagine something like this being used for nefarious purposes.)

In early October, Google updated the voice assistant on its smart-home devices—some of which have been around for a decade—by replacing Google Assistant with Gemini. For the most part, the assistant is better. It can understand multiple commands in a spoken sentence or two, and you can very easily ask it to automate something in your home without fussing with the Routines tab in the Google Home app. And when I ask it a simple question, it generally gives me some kind of a reliable answer without punting me to a Google Search page.

Smarter camera alerts are indeed more helpful at a glance. Most of the time, I dismissed Person Seen notifications because they’re often just people walking by my house. Now the alerts actually say “Person walks by,” which gives me greater confidence to dismiss those. Some alerts accurately say “Two people opened the gate,” though sometimes it will hallucinate: “Person walks up stairs,” when no one actually did. (They just walked on the sidewalk.) It has fairly accurately noted when UPS, FedEx, or USPS are at the door, which is nice to know when I’m busy or out and about, so I can make sure to check for a package when I get home—no need to hunt through alerts.

But with my indoor security cameras, Gemini routinely says I have a cat wandering the house. It’s my dog. Even in my Home Brief—recaps at the end of the day from Gemini about what happened around the home—Gemini says, “In the early morning, a white cat was active, walking into the living room and sitting on the couch.” It’s amusing, especially considering my dog hates cats.

CatDog

ScreenshotGoogle Home via Julian Chokkattu

You would think then that I would be able to just tell this smarter assistant, “Hey, I don’t have a cat. I have a dog,” and it would adjust its models and fix the error. Well, I did exactly that. In the Ask Home feature, you can talk to Gemini and ask it anything about the home. This is where you can ask it to set up automations, for example. I asked it to turn on the living room lights when the cameras detect my wife or I arriving home, and it understood the action. It even guessed that I wanted the lights to come on only when arriving at night, despite me forgetting to mention that.

When I told Gemini that the security camera keeps identifying a cat but it’s a dog, and that I don’t have a cat, it acknowledged this and accepted that I have a dog. Yet the mistakes continued. A Google spokesperson tells me that Home Brief and Ask Home are in an early-access phase, and Google is encouraging users to submit feedback so it can improve the experience.

“We are investing heavily in improving accurate identification, including for pets,” the spokesperson says in an email. “This includes incorporating user-provided corrections (like telling Ask Home that your ‘cat’ is actually a dog) to generate more accurate AI descriptions. Since all Gemini for Home features (Ask Home, Home Brief, and AI Captions) rely on our underlying Familiar Faces identification, improving this accuracy also means improving the quality of Familiar Faces. This is an active area of investment and we expect these features to keep improving over time.”

Google’s Familiar Faces system lets you add names for people the cameras sees often around your home. When it works, it’s great. Especially when I had a holiday party and it announced my friends by name when they got to my front door, which caused everyone inside to erupt into raucous applause. But too often my wife and I are sitting at home, only for the Nest Doorbell to ring and announce that I’m standing outside ringing the bell. (Insert surprise Pikachu face.) Familiar Faces doesn’t work with pets at the moment, so the eventual addition of that capability will likely help Gemini understand that my dog isn’t a cat, even if he acts like one.

It may be early days for these features, but the saga showcases that AI, no matter how intelligent these companies make them out to be, can still fail to distinguish simple details about our lives, even if you tell it the right answer to its face.