Article snapshot taken from Wikipedia with creative commons attribution-sharealike license.
Give it a read and then ask your questions in the chat.
We can research this topic together.
The deep learning model used by the application is nondeterministic. The application supports English phonetic transcriptions (such as ARPABET) to correct mispronunciations or to account for heteronyms—words that are spelled the same but are pronounced differently (such as the word read, which can be pronounced as either /ˈrɛd/ or /ˈriːd/ depending on its tense). The app also supports altering the emotion of a generated line using emotional contextualizers (a term coined by this project), a sentence or phrase that conveys the emotion of the take that serves as a guide for the model during inference.
In 2016, with the proposal of DeepMind'sWaveNet, deep-learning-based models for speech synthesis began to gain popularity as a method of modeling waveforms and generating human-like speech. Tacotron2, a neural network architecture for speech synthesis developed by Google AI, was published in 2018 and required tens of hours of audio data to train. For years, reducing the amount of data required to train a realistic high-quality text-to-speech model has been a primary goal of scientific researchers in the field of deep learning speech synthesis.
A landmark case between Google and the Authors Guild in 2013 ruled that Google Books—a service that searches the full text of printed copyrighted books—met all requirements for fair use. This case set an important legal precedent for the field of deep learning and artificial intelligence: using copyrighted material to train a discriminative model or a non-commercialgenerative model was deemed legal.
Development
15.ai was designed and created by an anonymous research scientist affiliated with the Massachusetts Institute of Technology known by the alias 15, ostensibly in reference to the minimum amount of data required to convincingly clone a voice. The project began development while the developer was an undergraduate. Although the application costs several thousands of dollars a month to keep up and maintain, the developer has stated that they are capable of paying the high cost of running the site out of pocket.
The developer has also worked closely with the Pony Preservation Project from /mlp/, the My Little Ponyboard of 4chan. The Pony Preservation Project is a "collaborative effort by /mlp/ to build and curate pony datasets" with the aim of creating applications in artificial intelligence. According to the developer, the collective efforts and constructive criticism from the Pony Preservation Project has been integral to the development of 15.ai. During months-long hiatuses of the public release of the site in late 2020 and early 2021, a number of test sites were put up for exclusive use by /mlp/ and the Pony Preservation Project for the purposes of testing and generating content.
I've been informed that the aforementioned NFT vocal synthesis is actively attempting to appropriate my work for their own benefit.
After digging through the log files, I have evidence that some of the voices that they are taking credit for were indeed generated from my own site.
January 14, 2022
Avatar of Voiceverse Origins
Voiceverse Origins @VoiceverseNFT
Hey @fifteenai we are extremely sorry about this. The voice was indeed taken from your platform, which our marketing team used without giving proper credit. Chubbiverse team has no knowledge of this. We will make sure this never happens again.
Over 3 million lines were generated within the first two weeks of the September 2021 release of the application. As of February 2022, 15.ai's Patreon was raising over $1,100 per month.
Fandom content creation
15.ai has been frequently used for content creation in various fandoms, including the My Little Pony: Friendship Is Magic fandom, the Team Fortress 2 fandom, the Portal fandom, and the SpongeBob SquarePants fandom. Numerous videos made with 15.ai have gone viral on YouTube, Twitter, and TikTok.
However, numerous videos/projects posted on the internet have also been made and gone viral without having properly credited the source(s) of their AI-synthesized voice clips. As a consequence, many videos that have been made with other speech synthesis software have been mistaken as being made with 15.ai, and vice versa. Due to this absence of proper credit, 15.ai's terms of service has a rule that forbids having 15.ai-and-non-15.ai-synthesized speech in the same video/project.
The My Little Pony: Friendship Is Magic fandom has seen a resurgence in video and musical content creation as a direct result; moreover, the project has been utilized as a creative tool in pornography. For instance, the Pony Zone videos is a series of eroticmusical videos that heavily samples 15.ai as the vocals—the creators of such videos make frequent use of salacious emotional contextualizers and punctuation/ARPABET tricks to induce the models to grunt, sigh, and moan convincingly.
Troy Baker NFT scandal
On January 14, 2022, it was discovered that Voiceverse NFT, a company that video game voice actor Troy Baker announced his partnership with, had stolen voice lines generated from 15.ai as part of their marketing campaign without permission. Log files showed that Voiceverse had generated audio of Twilight Sparkle and Rainbow Dash from the show My Little Pony: Friendship Is Magic using 15.ai, pitched them up to make them sound unrecognizable from the original voices, and appropriated them without proper credit to falsely market their own platform—a violation of the terms of service.
The initial partnership between Troy Baker and Voiceverse was met with severe backlash and universally negative reception. Critics highlighted the potential environmental impact of and potential for exit scams associated with NFT sales. Two weeks later, on January 31, Baker announced that he would discontinue his partnership with Voiceverse.
