A developer had the brilliant idea of using the OpenAI bot as a home automation assistant. The result is stunning, and should serve as a lesson to the assistants of Apple, Amazon or Google…
It’s just tinkering, and yet it may be a first look at the future of personal assistants. Mate Marschalko, a developer probably tired of the answers Siri gave him, had a brilliant idea: test ChatGPT to control his connected home by voice.
In a video posted on Reddit and Youtube, he shows the -absolutely stunning- result of his experiment, which sends Google, Apple and Amazon assistants back to the ropes.
If you regularly use Siri, Google Assistant or Alexa to control your bulbs, radiators and other connected objects, you must have realized that you had to adapt to them. In other words, perform very basic commands: “turn on the light in the living room”, “increase the heating”, “turn off the TV”. Impossible to make them understand more complex requests, so quickly do they lose control.
Marschalko’s tool, built in an hour by combining the power of ChatGPT and Apple’s “Shortcuts” app, is a game-changer. And correctly interprets requests that are eminently more complicated than all GAFA assistants. An example ? This request, which would have clearly put traditional voice assistants in the cabbage: “ My wife is coming in 15 minutes. Turn on the lights for her outside when she goes to park. The program understands her without the slightest worry, and the assistant answers proudly: The lights should come on the moment your guest arrives”. Or this one, which also benefits from the knowledge of ChatGPT: “ adjust the heat in the bedroom to a temperature you think would help me sleep better “. And the assistant to retort: The bedroom thermostat was set at 19 degrees. Enjoy your sleep ! “. Ultimate refinement, the wizard responds differently each time, almost as if it had come to life.
A procedure… not so complicated
To connect his dozen lamps, thermostats, ventilation system and cameras to ChatGPT, he proceeded in two stages. First, we had to “educate” ChatGPT. His long query (see below) looks like a small computer program… written in natural language.
Marschalko thus demanded that ChatGPT respond to each request in the form of JSON, a famous data structuring format easily understood by the Shortcuts application of iOS. He then described the types of requests needed, defined the structure of the JSON that ChatGPT had to generate and finally provided a precise description of his house and the connected objects that were in each room. That is just about everything. Ah yes, he also asked the bot to impersonate “ the brain of the house, an intelligent AI, without revealing its true identity. »
Next, he set up a new shortcut on iOS, which looks like a supercharged version of the one we described to you a few days ago. Long and rather complex, this sequence of commands allows in summary to make the interface, thanks to the data in JSON format, with its connected objects, via the Apple HomeKit platform.
It’s not (yet) for now
Ingenious and effective, Marschalko’s hack is not free from flaws. First, there is the price. This solution indeed requires access to ChatGPT through its API, and is therefore… paying, even if OpenAI provides a (small) free credit. But it is quickly exhausted here, because the request is long. It takes about $1 every 70 requests, according to the programmer. Then there is… the slowness. As such, the video can be confusing, as it appears ChatGPT responds quickly. However, it has been mounted: there is an incompressible delay of a few seconds between the voice command and the robot’s response.
However, we can hope that Apple, Google or Amazon are also thinking about taking advantage of generative AI to improve the relevance of their voice assistants. Who come, with ChatGPT, to take quite a blow of old.
Mate Marshalko’s Blog
