Does real-time translation really work with Zoom?
Photo by Gabriel Benois
Zoom is one of the most widely used video conferencing platforms, enabling online meetings, chat, AI-based transcription, real-time captioning, and caption translation, among other features.
How does Zoom’s real-time translation work?
Real-time subtitles facilitate communication between participants, especially during meetings with deaf or hard-of-hearing participants or participants who do not understand any of the languages spoken at the meeting. Although live subtitles can sometimes improve communication, they also have numerous shortcomings, which we identified during our testing of this technology.
Analysis of Zoom’s simultaneous translation tool
As professional conference interpreters, we have identified clear weaknesses in this tool, particularly with regard to:
Constant lack of accuracy in content
AI-generated subtitles (automatic interpretation) often have difficulty identifying different accents, specific terms and proper names, leading to numerous misinterpretations. For example, during one of our tests, the name Niels was pronounced in the original speech and Zoom displayed it inconsistently as “Nels”, “Niza” and even “nice”. In another test, the original English said “the Dome” (i.e., the O2 Arena building in London), and the Spanish subtitles showed “la celda” (the cell), later “la cúpular celular” (the cellular dome) and “la célula S” (the S cell). This example is particularly interesting, as it was a debate between British architects. No one who did not understand English and had read those subtitles would have been able to know which building was being discussed, which was also the main topic of the debate.
Excessive latency
The time between the original speech being delivered and the translation appearing on screen is excessive. With AI-based tools, the delay can be up to 32 seconds, while the latency of a human interpreter is around 5 seconds in most language combinations. In addition, Zoom subtitles are generated in short phrases divided into fragments, which are reformulated as the algorithm processes the data. This constant change in the text on the screen makes it very difficult for the human eye to read, keep up with and understand the original speech.
Hallucinations
Also known as delusions or confabulations, these are errors that AI technologies often make when their algorithm fails to process something or cannot cope with stress factors such as fast speakers, unusual accents, proper names or uncommon terms, etc. Obviously, machines do not think, so they cannot understand or correct themselves. In another of our tests, the original French expression “…et choper quelques nouveaux clients” (meaning “and get some new customers”) was translated into Spanish without any meaning as “…and kill the masses”.
Privacy
Since these subtitles are displayed in real time, confidential information shared during meetings could be exposed in an unintended manner, which could violate confidentiality. It should be noted that third-party servers do not guarantee data security, nor do portable devices that operate on radio frequencies. The best course of action if the information is confidential is to ask the team of interpreters to sign a confidentiality agreement and to work on an intranet with portable interpreting booths and digital receivers for the audience.
Are you really looking for a professional conference interpreting service?
Although real-time subtitling with Zoom is an interesting technology for inclusion and efficiency, it remains an imperfect tool that cannot do the job of a human interpreter, especially when it comes to understanding cultural differences, different accents, local information, ensuring confidentiality and, in particular, the accuracy of the content.
If you need professional conference interpreting services or sound equipment for your conference…