![]() However, this method would not work well with a mix of landscape and portrait-formatted images as it would tend towards the center cut format. This “center rectangle” format seems to be a good compromise between the center cut and squeezed formats, especially since the original landscape paintings are in, well, landscape format. ![]() The result of this process will be synthetically generated images that are free of distortion. When the GAN produces square output images, I scale them back out to be 1.27:1 to match the original format. I then squeeze the cropped images horizontally into a square format with a resolution of 1024 x 1024 pixels and use the resulting images for my training dataset. Next, I crop each image into a 1.27:1 aspect ratio, pulling out a center rectangle. My hybrid solution requires first determining the average aspect ratio of all the original paintings, which for the dataset is 1.27:1. And each painting will be squeezed by a different amount, depending on the original aspect ratio.įor this project, I came up with a hybrid technique that seems to be a good compromise.ġ.27:1 Rectangle Cut and 1.27:1 Squeezed Formats, Image by Degas, Formatted by the Author For example, in this image, the windmill just got a lot thinner. The issue with the squeezed format is that the objects in the painting are distorted. The third image is squeezed horizontally to be square. The issue with the center cut format is the significant loss of imagery on the left and the right. The second image is in the “center cut” format, which is cropped to keep just the image's square center. The issue with the letterbox format is that the entire image is effectively resized down, losing resolution, and the black parts are “wasted” in the GAN training. The “letterbox” format in the first image above keeps the image uncut and unsqueezed, but black bars are added above and below to make the image have a square shape. ![]() Letterbox, Center Cut, and Squeezed Formats, Image by Degas, Formatted by the Author As the final step, I post-process the final images to adust the aspect ratios.īe sure to check out the image gallery in the appendix at the end of the article to see more results. And I use the generator again to create an image with a mix of style and form selected by the user. I use CLIP again to filter the output images based on a user-supplied text query. Before the real and the generated images are fed into the discriminator, they are modified slightly with the Adaptive Discriminator Augmentation (ADA) module that creates visual diversity in the pictures. The generator creates new images starting with random “latent” vectors for form and style and tries to fool the discriminator into thinking the output images are real. I used these images to train StyleGAN2 ADA from NVidia, which has a generator and discriminator network. I then used the CLIP model from OpenAI to filter the images to keep the “good ones.” I gathered images of Impressionist landscape paintings from and processed the images to adjust the aspect ratio. I’ll discuss the details of each component later in the article. Here is a brief, high-level overview of the components used in GANscapes. GANscapes System Components, Diagram by Author Overview of Components īelow is a sample of paintings from StyleGAN, SAPGAN, Lightweight GAN, and GANscapes. created a Lightweight GAN in their paper, “Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis”. ![]() Alice Xue created SAPGAN described in her paper, “End-to-End Chinese Landscape Painting Creation Using Generative Adversarial Networks”. There’s Drew Flaherty’s master’s thesis from the Queensland University of Technology entitled “Artistic approaches to machine learning,” where he used the original StyleGAN. There have been several projects and papers that show how to use GANs to create landscape paintings. You can create new landscape paintings using the Google Colab here. I posted all of the source code for GANscapes on GitHub and posted the original paintings on Kaggle. The first two articles focused on creating abstract art by using image augmentation, but this one focuses on creating Impressionist landscape paintings. This is my third article on experimenting with Generative Adversarial Networks (GANs) to create fine art. GANscapes Sample Output, Images by Author ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |