1. General
1.1 What is this?
This Pony Does Not Exist is a website featuring AI-drawn ponies. It was made using Nvidia's StyleGAN2 architecture.
The training dataset consisted of ~104k SFW images from Derpibooru, cropped and aligned to faces using a custom YOLOv3 network. The cropping data is archived in this GitHub repository.
The model used transfer learning to fine tune the final model from This Fursona Does Not Exist on the pony dataset for an additional 13 days (1 million iterations) on a TPUv3-32 pod at 512x512 resolution. It was then scaled up to 1024x1024 resolution using model surgery, and trained for an additional 200k iterations to produce the final 1024x1024 model.
1.2 Why?
Why not? It's a learning experience for me, and a cool demonstration of the current capabilites of GANs for others. These projects will hopefully encourage more people to learn about AI and possibly seek carreers in machine learning.
1.3 I'm pretty sure these ponies already exist.
The name "This Pony Does Not Exist" is a reference to earlier GAN-based websites showcasing various other types of AI-generated content:
- This Person Does Not Exist
- This Rental Does Not Exist
- These Cats Do Not Exist
- This Waifu Does Not Exist
Specifically, while the individual characters portrayed in the generated images may resemble images from the training data, the specific images generated by the GAN are unique.
1.4 How are Derpibooru and e621 involved in these projects?
Derpibooru and e621 are not affiliated with TPDNE or TFDNE in any way. Don't blame them.
2. Copyright
2.1 Did you get permission to use the art that the model was trained with?
No. In a perfect world, I would have been able to get permission from all the artists, but realistically it isn't feasible to do so. When I started working on this, I had zero social media presence (in fact, my first tweet was showing some early results, after I'd already created the dataset), so it's unlikely many artists would have responded to some random person asking to use their art for a machine learning project. It also would have been logistically impossible to send out messages to tens of thousands of artists across several websites (Derpibooru, e621, Twitter, DeviantArt, FurAffinity, Tumblr, Inkbunny, etc.) and coordinate the responses, especially considering many of the artists have multiple pseudonyms, are anonymous, or have since left the community. Furthermore, I didn't even know which images would be usable as training data until I had already downloaded them and run them through my face detection/cropping network.
The more data the network is trained with, the better it is able to generalize, and therefore the less likely it is to try to replicate the style or content of any one particular artist or character. So paradoxically, if I had asked for permission from artists, and only a small number of them responded agreeing to let me use their art, the network would only be able to learn from the styles of those artists and I probably would have had more complaints that it was producing output that is too similar to the training data.
From a legal standpoint, while I'm not sure this has been tested in court before, I believe there are strong arguments that using data to train a neural network constitutes fair use. This document from OpenAI addressing the US Patent and Trademark Office makes a very strong case for this:
Furthermore, such sampling is reasonable—and indeed necessary—to the highly transformative purpose for which it is used. AI systems perform best when they are trained on larger amounts of data. Increasing the amount of training data available to the system increases the output system’s accuracy and therefore utility. Thus, non-expressive use of entire works during training is reasonably necessary to the transformative purpose of creating AI systems.
2.2 Isn't the AI just copying and pasting together parts of existing art?
No. Although several people argued that the images on TFDNE resemble art done by other artists, no one was able to provide an example of a generated image copying an existing image.
Again, the OpenAI document provides some clarity here:
AI systems go well beyond preserving the content of individual works by learning patterns in their whole training corpus and then using those patterns to generate entirely novel media. Well-constructed AI systems generally do not regenerate, in any nontrivial portion, unaltered data from any particular work in their training corpus. Indeed, the entire utility of such systems is dependent on the fact that, by learning patterns from its training corpus, an AI system can eventually generate media that shares some commonalities with works in the corpus (in the same way that English sentences share some commonalities with each other by sharing a common grammar and vocabulary) but cannot be found in it. Furthermore, since such patterns only emerge after consuming an enormous number of works, each single work consumed in the training process contributes very little to the overall AI system.
The interpolation videos show that the GAN is able to smoothly interpolate between a wide variety of characters and poses. If the AI were simply replicating already existing art, it would not be able to do this.
2.3 I'm an artist, and I don't feel good about this.
Understandable. We're still in the early days of artificial intelligence, and it can be alarming seeing how quickly AI is progressing on being able to produce high-quality art.
It may be helpful to remember that the GAN isn't simply copying parts of existing artwork and replicating or modifying them. Rather, it is looking at hundreds of thousands of distinct pieces of art and "learning" an internal representation of what constitutes a pony, similar to how a human artist looks at hundreds of thousands of images while learning to draw.
For what it's worth, it might also help to remember that in the grand scheme of things, this project probably isn't a threat to anyone's livelihood. As with This Fursona Does Not Exist, people will likely talk about it for a few weeks and then forget about it.
2.4 Is it possible to opt out of having my art used to train the network?
While I certainly would like to respect the wishes of artists here, there are several factors that make this difficult:
- It is difficult to manually track all of the artists who have asked to opt out, especially when their usernames on Twitter are different than their usernames on other sites, or when their art isn't properly credited.
- Many people have raised concerns about their art being posted on e621 and Derpibooru without their knowledge or permission. In these cases, I would recommend they follow e621/Derpibooru's takedown procedure.
- Once a neural network has been trained, it is virtually impossible to have it "forget" individual images it was trained on, similar to how it is basically impossible to have a real human forget things they've seen. Doing so would require retraining the entire network from scratch with a new dataset, which isn't really feasible every time someone wishes to opt out. Training the network is extremely expensive -- 13+ days of training, costing over $10,000 worth of Google Cloud credits (which were, thankfully, provided for free via Google's TFRC program), in addition to the time cost of having to recreate the dataset, re-train the model, and regenerate all of the sample images on the website after the model has been retrained.
Therefore, it unfortunately isn't possible to allow artists to opt out once the network has been trained.
Also, note that TPDNE was trained on art from nearly 17,000 unique artists, and even the most prolific artists' images make up less than 0.4% of the dataset (less than 400 images out of 104k), so it is unlikely that the network will produce images that copy any one particular person's unique art style.
2.5 Who owns the generated images?
I claim no legal ownership or rights to any of the images generated by this AI. All characters depicted are property of their respective owners.