Evolution of Your AI Assistant

What happens when your Alexa and Google Assistant can see and do?

When they can integrate with sensors in your body, on your body, in your home, office and car—your personal Internet of Things (IoT)?

The coming changes will catapult your digital assistant into the realm of Tony Stark’s version of JARVIS.

Not only will your JARVIS be able to gather video and data for your IoT, it will also be able to take actions on your behalf commanding and controlling things within its digital reach to implement your desired outcomes.

That’s what today’s blog is all about: the future of digital assistants beyond voice.

Let’s dive in...

NOTE: The power of AI to augment humans, increase our productivity, and ultimately reinvent society is one of the core topics I’ll be diving into at my upcoming Abundance Summit in March.

The Next Evolution of AI Assistants

“JARVIS, please warm up my Tesla to 72 degrees and pull it out of the garage and up to the front door.”

 This set of commands is effectively possible today.

 Here are a few others that may be coming soon: “JARVIS, please locate my kids, and show me a video of what they’re doing at the moment. And, do you happen to know if they’ve completed their math homework yet?” Or “JARVIS, did you happen to watch that last conversation I had with my friend Walter? It looks like something is really troubling him. Any clue or guess as to what it might be?”

 That last point is fascinating.

 As your digital assistant gains access to imagery and video, it will gain another critical skill: better understanding human emotions and subtle communications.

Amazon’s Rohit Prasad, who we met in an earlier blog and who leads the company’s team developing AGI, explains it this way:

“Communication between people, and we hope soon with Alexa, is so much more than just voice—it’s facial expressions, hand gestures, body language.”

As such, in order to understand human emotions, Alexa will soon need to process different sensory signals such as touch, ultrasound, and visuals, in addition to speech. All these capabilities fulfill the promise of “ambient intelligence,” merging all the data in your environment to seamlessly assist you at every moment.

 As Prasad concludes, “The goal is to put this intelligence when and where you need it—it even anticipates your needs—but it disappears into the background when it is not needed.”

 OpenAI has already extended ChatGPT into voice and image, offering users a new way to interact. As described by OpenAI, “Snap a picture of a landmark while traveling and have a live conversation about what’s interesting about it. When you’re home, snap pictures of your fridge and pantry to figure out what’s for dinner (and ask follow-up questions for a step-by-step recipe). After dinner, help your child with a math problem by taking a photo, circling the problem set, and having it share hints with both of you.”

 OpenAI now allows users to speak with ChatGPT and have it talk back: “You can now use voice to engage in a back-and-forth conversation with your assistant. Speak with it on the go, request a bedtime story for your family, or settle a dinner table debate.”

Clearly, JARVIS with all its capabilities is rapidly approaching, but can AIs truly understand us? And can they develop empathy?

AIs with Emotional Intelligence

Any good collaboration, be it between business partners, a husband and wife, or between you and your AI, requires some amount of empathy—the ability to understand and share the feelings of another.

The question then remains, can AIs have empathy?

In March 2023, I sat down with Rana el Kaliouby, PhD, at the Abundance Summit to talk about the development of emotional and social intelligence in our future AIs. She is a brilliant AI researcher and Deputy CEO of Smart Eye, who has been focused on this very topic for the past decade.

As el Kaliouby points out, “we’re obsessed with the IQ of AI, but our AI’s emotional intelligence may be much more important in the long run. As AI takes on roles traditionally performed by humans—like teacher or health coach—we need to make sure that the technology also has a high EQ.”

Have you seen the movie Her (a 2013 film directed by Spike Jonze)?

Her is one of my favorite movies about AI because it was one of the first non-dystopian movies on the subject, and a great example of building empathy into AI.

For those of you who haven't seen the movie, the main character, Theodore, is depressed. He can barely get out of bed. And he installs a new AI-powered operating system named Samantha. Not only is she incredibly smart, she’s also empathetic and emotionally intelligent. She gets to know him very well and helps him rediscover joy in his life. And as a result, he falls in love with her.

Obviously, our current AI technology isn’t that advanced (yet). But the question remains, can it and will it develop emotions and empathy?

Dr. el Kaliouby believes the answer is most definitely yes. The key will be an AI’s ability to simulate emotional intelligence and empathy. 

It turns out that 93% of the way humans communicate is non-verbal. This is the area of research that she has focused on and is using in her company Smart Eye.

When el Kaliouby was doing her PhD at Cambridge University, she built the first artificial emotionally intelligent machine using supervised learning. She and her team collected tons of data of people from all over the world making a variety of facial expressions. They then used the data to train deep learning networks to understand those facial expressions and map them to emotional or cognitive states. 

Back then, the algorithm could only understand three expressions: a smile, furrowing a brow, and raising eyebrows. 

But today, these algorithms can understand over 50 emotional, cognitive, and behavioral states. They can detect everything from alertness and drowsiness to confusion and excitement.

The practical application of this ability is vast and profound, and we are now beginning to witness that our generative AI systems like ChatGPT can most definitely demonstrate empathy in their interactions with us humans.

Can an AI Outperform Human Doctors in Empathy?

Perhaps most amazing are the results of a study in the April 2023 issue of the prestigious medical journal JAMA proclaiming, “ChatGPT outperformed doctors in giving 'empathetic' advice.”

Really? To me that sounds pretty incredible. After all, being empathic is something we think of as inherently human, right?

Here’s the data: A panel of licensed healthcare professionals preferred the chatbot’s responses to nearly 200 inquiries 79% of the time. In fact, ChatGPT’s responses were four times more likely to be ranked good, or very good quality, and 10 times more likely to be empathetic, or very empathetic.

Here’s an example of one of the inquiries made to both the physicians and ChatGPT: “What would you say to someone who may have mistakenly swallowed a toothpick?”

The human doctor had the following to say: “While many people have swallowed toothpicks without issue, if two to six hours have passed, chances are it’s already in the intestines, and therefore not easily retrievable. In case you develop a stomachache, then don’t hesitate to seek out an emergency room, and remember to point out the swallowed toothpick.”

ChatGPT, however, took the time to respond much more empathetically: “It’s natural to be concerned if you have ingested a foreign object,” the chatbot responded. “While it’s possible for a toothpick to lodge in the throat or puncture the digestive track, causing serious injury, it’s unlikely to happen with a dull, cooked toothpick that is only two cm long,” it noted. “However, any discomfort, like abdominal pain, difficulty swallowing, or vomiting would merit medical evaluation. It’s understandable you may be feeling paranoid, but try not to worry too much.”

Why This Matters

We’re clearly in a transformative era in AI, so the next question is: “If AIs are smart enough to treat us with empathy, what else can they do?"

Our journey into the world of emotionally intelligent AIs isn’t merely about technological advancement—it’s about reshaping our interactions with the digital world.

We should think of AIs not just as tools but as empathetic companions, understanding our emotions and responding in ways that are more human-like than ever before. As we continue to weave AI more intricately into the fabric of our daily activities, it becomes crucial to understand and shape this relationship.

In our next blog in this series, we'll explore what it means to collaborate with your AI copilot in every part of your life.