Could biometrics and gesture recognition technology make hybrid meetings run smoother?
Hybrid work meetings can be tough to manage — especially to process typical meeting etiquette like social cues and body language when some people are in the office and the rest dial in remotely.
But developing workplace technology could help read and analyze people’s responses in meetings and feed it back to participants, to help these hybrid conversations run more smoothly.
Biometrics and gesture recognition technology uses a camera that pans to whoever is speaking in a conference room, or from a screen. Biometrics are body measurements or physical characteristics — like fingerprint mapping and facial recognition — that can be used to identify individuals.
Although still in its early days, some companies are using this form of artificial intelligence tech to measure how many people are physically in the room, how often people are talking, and what kind of meeting it is — whether an all-hands or a more intimate one-to-one.
While the tech isn’t at the point where it can identify individuals by name, it does identify the room and people’s locations within it. A camera will adjust itself to frame whoever is speaking, and if anyone leaves the meeting the camera will take account of it and automatically reframe on the remaining people so those on screens aren’t left staring at empty chairs.
For example, 360 degrees cameras, like Owl Labs, are creating streams using multiple cameras, including one that monitors the entire room that determines people’s locations to crop speakers’ images in real time.
The tech increased engagement and collaboration, according to The Bainey Group, a 30-person company commercial construction company, and partnered with Owl Labs.
Audio company Bose Professional is working on a different toolset and has worked with Google, Microsoft and Zoom to develop the tech within video conferencing tools.
“When everyone was forced to go remote because of the pandemic, we adjusted to seeing colleagues in their own square box where their faces were seen clearly and their audio was fairly clear,” said director of emerging business at audio company Bose Professional, Martin Bodley. “It can be better than when you have the challenge of a 30 by 40 room with 15 people in it and you’re trying to figure out who is saying what or you can’t hear what’s being said.”
Bose has a video bar for different meeting space sizes that includes a 4K ultra-HD camera and either four or six microphones that create four audio channels for a conference room depending on the product. It can also mark zones of where you don’t want noise to be picked up from in the instance where there might be a loud fan or a noisy doorway.
“What we really want nowadays is every space that you might use to gather a few people, you want to also have a video capability so you can tie in that remote person or two that are going to be out there that you want to tie in,” said Bodley.
The capabilities are necessary: 58% of Americans reported having the opportunity to work from home at least one day a week and 35% reported having the option to work from home five days a week, according to a recent McKinsey and Company study.
At the same time, each company has a different workflow that they should take into account, said Bodley. Some companies might need whiteboards and a camera devoted to them, while others might mostly be having real-time collaboration sessions. Without that consideration, companies could create meeting inequity between remote and in-person employees, which most companies are actively working to avoid.
How gesture recognition is possible
AI-based tech companies are investing in fixing and automating modern workplace challenges. Tech startup Datagen, which provides enterprises with a self-serve synthetic data generation platform to create smart offices is among them. The Datagen AI platform has created datasets that show people engaged or paying attention in a meeting, or not, which engineers then use to analyze general interactions during meetings and create technology based on it.
“Adding different features, virtual backgrounds, and more are all enabled thanks to computer vision systems that are running behind the scenes and are helping the vision or video data that we are broadcasting to each other to be interpreted in the right way,” said Ofir Zuk, CEO of Datagen.
Datagen has built digital conference rooms, including everything from typical office objects like whiteboards and post-it notes to the in-home office, gaze detection and gestures while speaking.
“What we are adding here is the ability to put tens of thousands of different people into every specific gesture so you are able to get tons of identities, tons of people, performing these different actions so you have a very diverse data set that you can train your model on and then understand things from there,” said Zuk.
The data is given to a computer vision team to digest the data and train their machine learning models with their own development. Potential customers could be Zoom, Microsoft Teams, Google Meet and other technology companies that are developing their own smart conferencing tools and are hoping to increase their gesture recognition capabilities, or companies that are developing services for big conferencing tools.
Another way biometrics is being used is by separating each person in the conference room into their own individual audio and visual streams so they appear as if they had their own Zoom or Microsoft Teams. Something like this might benefit the remote employee because they’re able to see each individual closer, whether they’re speaking, which is different from intelligent framing in conference rooms that only zooms in on who is speaking. At the same time, the people who are in the room will probably be turning and looking at each other in real time and not at the camera, which might leave remote viewers with a sort of side view.
“That kind of a thing is at the bleeding edge right now,” said Bodley. “It’s not widely adopted, but it is being played with by people and some products are starting to come out with those kinds of features. If we want to look way out into the future, I don’t think you can ever get to the realism of an in-person meeting. We can try and get closer to it, but I don’t think we’ll ever get there per se.”