Artificial intelligence to artificial imposter

Words by Daniel Hunt

Text-based ‘phishing’ scams have become notorious over the past few years. Scammers repeatedly pose as delivery companies, welfare agencies, or even the NHS in an attempt to steal people’s personal details.

These scams are mostly unsophisticated; often all it takes to spot a fake is an unknown number or a dodgy URL, which is probably why young people today don’t think of themselves as at risk. After all, in 2022, only three percent of 25- to 34-year-olds who received a phishing message opened a link, compared to 11% of those 75 and older.

Unfortunately, as technology evolves, so does the potential for malice. Text-generating chatbots have dominated the headlines this year, from the plagiarism posed by ChatGPT to Snapchat’s messaging AI. However, there is a similar technology that isn’t so often talked about: text-to-speech generators.

The many voices of AI

Voice AIs are very much a technology in the making, and there are many different players. Plenty of corporations, like Google, keep their models under secret development. Some start-ups have released their programs to the public, such as ElevenLabs, which offers voices in 28 languages.

I signed up to ElevenLabs, which offers several, quite convincing, sample voices. They flowed through tongue-twisters I fed them and, the program seemed pretty lifelike for text-to-speech. They did, however, struggle with accents – the supposed Irish and Essex models sounded more American or Australian to me.

For just $1 a month, I decided to buy myself a subscription, which allowed me to create a clone of my voice from a five-minute sample. My robot impersonator worked quite well for formal speech but faltered whenever I tried to generate casual or longer sentences. Admittedly, the sample I provided, reading aloud an article, was probably not how I would sound in natural conversation.

That said, I managed to fool a friend and my housemate with short audio messages, which they replied to without question. My boyfriend and my sister saw through the ruse and straight away questioned why I sounded weird. For a technology in its infancy, a 50 percent success rate is not bad.

Old tricks with new technology

Despite its shortfalls, don’t rush to dismiss the danger of text-to-speech scams. The obvious point is that AI models will only improve. Though it can’t always continue, ‘Moore’s Law’ says that computing power doubles about every two years.

Moreover, scammers don’t operate in a logical arena but rely on manipulating your emotions to suppress your better judgement. Time pressures, such as demanding money before a deadline, or social pressures, such as posing as a superior at work, cause enough stress to rush people into making decisions they might usually scoff at.

One such example comes from a Canadian woman, Ruth Card, who earlier this year received a phone call from her grandson in prison who was in need of cash for bail. Panicking, Ruth made a run to two banks to withdraw the daily maximum of 3000 Canadian dollars. Fortunately, staff at the second bank spotted the scam before Ruth could be victimised.

As early as 2019, a managing director of a British energy company was tricked into wiring 220,000 euros to Hungary. A voice on the phone apparently mimicked his boss’ German accent, intonation, and manner of speech. Fortunately for the company, whose name has been kept secret, their insurance covered the money; most members of the public would not be so lucky.

Deepfakes

AI-generated deepfakes are perhaps better discussed because of their dramatic implication in politics and democracy. Within weeks of the Russian invasion of 2022, a video circulated online of Ukrainian President Zelensky calling on his troops to surrender. Though this particular video was widely seen for what it was – it’s said that Zelensky appeared motionless and sounded strange – imagine the calamity a better fake could have caused amongst an unsettled and unsure public.

Risk vs reward

The adaptability of artificial intelligence is unprecedented. There is no doubt that it presents an opportunity to turbocharge the development of knowledge and make human lives more comfortable. However, high reward comes with high risk. It would be a mistake to let the potential of AI be spoiled by the new and unnerving dangers it presents. To avoid this, AI developers and regulators should tread carefully.