The idea of installing a software update on your refrigerator already seems a bit strange, let alone one focused on improving its AI capabilities. But that’s exactly what’s happening to Samsung’s line of custom-made refrigerators this week, and to my surprise, this patch makes great strides in providing truly useful machine learning in a modern cooler.
To quickly recap, Samsung has been offering AI-based features like automatic food recognition and meal planning on its custom refrigerators for a few years now. However, as I discovered after reviewing its flagship model late last year, the company’s AI capabilities are still a work in progress. Previously, the refrigerator could recognize around 60 different types of fresh food (like fruits and vegetables) as well as around 50 other packaged items like yogurt or popcorn. It seemed like a good start, but considering the large number of different items you can find in a typical grocery store, it was far from over. Plus, you often had to enter additional data like the number of items or when something was first added, which made the idea of AI-assisted grocery tracking more tedious than I’d like. I don’t know about you, but I generally don’t want to have to tap on my refrigerator and I’m pretty sure Samsung’s engineers agree, which is probably what prompted this major update to its bespoke refrigerator software.
So what’s new?
The big change is that Samsung is adding support for Google Gemini, which has several important implications. By combining existing object recognition on Samsung’s device with Google’s cloud-based models, the total number of identifiable foods increases from just over 100 items to over 2,000. That means you’ll need to connect the fridge to Wi-Fi, but considering how many other smart features it supports, such as calendar integration and video playback, that’s not a big ask.
Another update is that Samsung is using Gemini to expand voice commands, allowing users to ask the refrigerator to change things like appliance settings, check details like when the water filter was last replaced, or even help troubleshoot problems. And depending on the situation, the refrigerator may even play a tutorial on how to fix the problem.
Alternatively, for more complex or harder-to-fix issues, Samsung is introducing what it calls Reliability AI, designed to monitor refrigerator components or help identify faults before they become too serious. Or, in cases where the device needs to be repaired, AI can provide more detailed information to agents while potentially allowing them to repair certain things remotely.
For example, a Samsung representative told me that if a customer calls and says the ice cubes from the ice maker are coming out in clumps and stuck together, Reliability AI could allow agents to reduce the amount of water added to the ice bin, all without ever needing to physically visit your home. Critically, Samsung says that while the refrigerator will monitor and track the device’s health parameters, owners will need to provide express consent for repair personnel to access this data. Meanwhile, if an issue requires in-person intervention, Samsung says that by sharing this data with repair technicians, it allows users to identify and resolve issues more quickly instead of having to arrive without context and diagnose issues from scratch.
How it works in the real world
My review unit hasn’t had any mechanical issues in the eight months I’ve used it, so I haven’t been able to evaluate Samsung’s repairability AI. That said, after checking it out for the first time at Samsung HQ, I’ve had a chance to use an early version of the Bespoke line’s new software over the past couple of weeks – including its improved support for cloud-based object recognition – and the improvement is profound. Even after using it for a while, I’m still surprised by how many different foods it recognizes. In the back of my refrigerator I have a can of Bull Head Shallot Sauce, a rather specialized ingredient from Taiwan used almost exclusively in Asian dishes. However, the AI had no trouble recognizing it, marking it automatically and notably when it was first added to the refrigerator’s AI Food Manager.
On top of that, the system now recognizes brands much better and counts the number of specific ingredients in order to create more detailed lists. He can distinguish between a Diet Coke and a Coke Zero while accurately noting that there were multiples of each item. And although the fridge often had to ping a cloud-based server somewhere to recognize various items, the results appeared fairly quickly, often in less than a few seconds. And for some foods like avocados, I was pleased that the fridge recorded how long ago you ate them and showed a notification that they might be about to expire. Granted, it’s not always correct, but all I really need is a reminder to check things and that does just that.
Sam Rutherford for Woozad
I also noticed that the refrigerator now remembers when you frequently take out a specific food item and then asks you if you want to add that item to your shopping list. It’s a good reminder to restock the staples you use regularly and it’s frictionless, so it doesn’t get annoying. From there, you can simply check your phone when you’re at the store instead of having to manually make a list every week. Plus, because the refrigerator recognizes and tracks what’s in it much better, it can provide better suggestions on what recipes you can cook with the ingredients you already have.
That said, like many current models, AI does not always master every detail. For example, I was initially impressed when it automatically labeled a jar of fake cream cheese as “Philadelphia plant-based,” until I realized that the label was incomplete and that the AI was simply reading what was written on the lid and lacked the intelligence to complete the description accurately. Don’t get me wrong, it provides enough information to help me understand what’s in the fridge when I glance at the Food Manager. That’s just not entirely accurate.
Outlook and things that still need work
The only problem with this update is that, like many AI services today, Samsung’s new software can be a little overconfident or prone to hallucinations. One day, while my wife was putting something back in the refrigerator, the algorithm took a photo of a brightly colored bandage on her finger and labeled it a vegetable, which it absolutely is not. Other times it seems like it’s just a guess. But I would say that going from around 100 identifiable items to over 2,000 is a very welcome improvement, even with the limitations.
The other strange thing is that even though Samsung leverages Google’s AI models for many of the fridge’s new features, you won’t see any obvious callouts to Gemini inside the device itself. This is a bit disappointing because Bixby remains the only digital assistant you can use and talk to directly.
I’ve said before that Samsung’s AI food recognition is a work in progress and I think that still holds true. With this latest update, the company has moved much closer to the promise of a refrigerator with truly useful AI-powered features. What once seemed more like a promising technology demonstration has quickly become a handy tool for tracking your runs, even with a few hiccups here and there.