This AI Transforms Sketches Into Landscape Photos

By Kehl Bayern / March 18, 2019

Last Updated on by

NVIDIA is really at the forefront of developing next-gen artificial intelligence technology. Given its huge presence in video games, among other industries, that's really no surprise.

At NVIDIA GTC 2019, the company unveiled a Microsoft Paint-like application called GauGAN that uses generative adversarial networks to transform your simple paint sketch into a realistic-looking photo.

Image via Martin Damboldt from

There are three basic tools that users can use in order to paint, including a paint bucket, pen, and a pencil. Using these tools in coordination with presets such as snow, water, and road, GauGAN can interpret user input but doesn’t rely upon “image stamps” as TechCrunch points out – everything it generates is unique to that particular picture.

Further, GauGAN interprets the same input in a unique way each in and every time. This what TechCrunch calls “multimodal” and the result is a wealth of unique content from one program.

What NVIDIA showed off was occurring in real time and took a lot of computing power to accomplish. Even then, the results weren’t perfect but “good enough” to pass as photo realistic upon first glance.

Interestingly, NVIDIA used Flickr to train the neural network powering GauGAN. For those of you who read our article about Flickr and Creative Commons rights, you’ll know this probably raised a few eyebrows (you can read that article here to find out why).

To calm any fears, Bryan Catanzaro, NVIDIA’s VP of Applied Deep Learning Research, said that all of the images they used were used with permission.

The company hopes to have it available for their AI Playground but wasn’t specific on a timeline for that. There are no plans to commercialize the software but TechCrunch says that a public trial could happen in the future.

If you’d like to learn more about the GauGAN project, you can watch a video about it here on YouTube.


About the author

Kehl Bayern

Kehl Bayern is our staff news writer and has over a decade of experience in online media and publishing. In terms of photography, he is interested in architecture and modern design. Kehl Bayern is also the author of science fiction thriller Animus Proxy. He is based in Boston, Massachusetts and studied politics at the University of Virginia and, later, Harvard University for graduate school. He spends much of his time traveling up and down the east coast of the United States. You can follow him on Instagram, Twitter and Facebook.

Leave a comment: