Sony’s Xperia Agent Preview – A house robot to talk to

The Agent is a prototype of Sony’s vision of a physical digital assistant for your home. In the design, Sony has clearly tried to give the Agent something human. Not that the design resembles a small person, far from it, but you can clearly distinguish some sort of torso topped with a round head. The hull is equipped with a screen, so that you can see, for example, who is calling, what music is playing, what the weather will be for that day or what your agenda looks like.

There is a camera on the head, so that the Agent can see who he is communicating with, and also two vertical stripes, which act as eyes. The fact that the ‘head’ can move and the ‘eyes’ are animated – they even blink – gives the human touch. It’s like a character from a Pixar animated movie has come to life.

The intention is that the Agent will be given a central place in the living room, after which you as a user can communicate with it using your voice. The operation is therefore comparable to Amazon’s Alexa and how Google Home should work. This did not fully work in the demo setup at the IFA and the Agent’s head had to be pressed to start a voice command. Ultimately, the system should work with a ‘hot word’ system, where the Agent wakes up as soon as you say a certain word.

By connecting the Agent to as many other devices as possible, you can let it act as a central hub. Because it is connected to the internet, it can look up information for you and, thanks to a Bluetooth connection to your phone, you can use it to make calls. In Sony’s vision it should also be possible to connect to many more devices via Bluetooth.

To make that idea more concrete, the manufacturer had made a setup with an Xperia Agent and a remote-controlled coffee machine. By talking to the Agent it was possible to make coffee and indicate how strong the coffee should be. Although it was clearly a tightly orchestrated demo, it showed well how such a product could work in the future and had a futuristic touch.

Sony had also set up a living room setup, with a television and audio system and the Agent on the coffee table. In the demo that was performed by a Sony employee – unfortunately we were not allowed to talk to it ourselves – a movie trailer was started via the Agent, photos were displayed on the screen and the music system could be operated. Again everything seemed to work very conveniently and intuitively, but it was clearly a limited demo, repeated over and over by the same Sony employee.

As a result, after the demonstrations we were left with a lot of questions. The Xperia Agent is an interesting proof of concept , but we haven’t learned much about how it works. For example, how will Sony get the “intelligence” needed to make the Agent do its job? Will it be developed from the ground up or will it tap into Google services? The latter is a suspicion that is reinforced by the sound the Agent makes, which is the same as what you hear when you start to say a Google Now command on an Android phone.

Furthermore, we have no idea how the Agent will communicate with all those other devices that Google has in mind. With these types of complex products, it is often not so much the hardware that causes difficulties, but the software implementation and compatibility. After all, the Agent has to work with a range of different devices, so APIs are needed for that. And here too the question is: does Sony want to develop it itself or does it want to piggyback on something that already exists? In addition, Sony must convince manufacturers of all that equipment that the Agent is worth supporting and you only have a strong story if the user base is large enough.