Latent Garden

What is latent space?

Latent space can be thought of as the imagination of a generative neural network such as StyleGAN3 that can synthesise photo realistic potraits of non existing people. These networks are trained on existing images to generate artificial images resembling the real ones. The space into which these networks inscribe their understanding of images is the latent space.

Initially this space has no meaning, it is the particular training data and the setup of the network that interpret the latent space and thus give it meaning. In the context of neural networks each latent space is unique. These networks create endlessly varying landscapes of imagery for which latent spaces serve as the source of inspiration.

Walking in latent gardens

The goal of this project is develop intuitive ways to explore and navigate these imagination landscapes. Over the course of next 6 months I will create a series of prototypical interfaces and data visualisations for intuitive exploration of latent spaces and the artificial images they create.

My aim is to create experiences similar to a walk in the garden instead of a tool for analytical observations of latent spaces. I draw inspiration from experimental video games, interactive data visualizations and non-linear interactive fiction. This website serves as the journal for documenting this journey.

This project is made possible with the generous support of Creative Industries Fund NL.

Contact

I’m Matúš Solčány a digital designer and creative coder, if you’re interested in the project get in touch at matus.solcany@protonmail.com

• Mar 15 '23

• Nov 1 '22

  

• Oct 17 '22

  

• Oct 5 '22

  I simplified the website layout today and added some recent updatets.

• Sep 3 '22

  Can nature guide the movement in the latent space?

• Aug 10 '22

• Jul 25 '22

  Each room can expand or shrink to reveal a larger or smaller sample from the latent space.

• Jun 19 '22

  Artificial gallery

  The first prototype represents the latent space as an artificial gallery where each room represents a segment of the latent space. Each room is a sample from the latent space.

  The network is mostly familiar with the samples from the center of the latent space. There’s an infinite amount of rooms in this gallery, however farther out from the center the visitor ventures, the less recognisable images will they discover.

• May 21 '22

  

• Apr 10 '22

  

  I’ve been experimenting with different image generating networks in since January, I felt like the images from the portrait network weren’t very interesting.

  I tried training a new one from scratch and customised some existing ones. I realised that a well trained image network and means to explore the latent space of one are two separate projects. I feel somewhat burnt out at this point. I’ll be working with existing networks from now on.

• Jan 27 '22

  The goals for February

  Most of the prototypes I’ve made so far are all using pretrained image generators. These are usually trained on single class of data: painted portraits, portrait photographs, flowers, animals and so on. I’m curious to see if a generator can be simultenously trained on for example painted portraits and flowers and create images that resemble both portraits and flowers.

  In February I’ll be training a generator with a multi class dataset. With a lot of technical issues out of the way and the engine working, I’m hoping spend more time exploring the conceptual possibilities of these prototypes.

• Jan 18 '22

  First person exploration

  What if latent space is explored from inside? In December and January I created a prototype for first person navigation in the spirit of a shooter video game. There’s a bug where the images farther back render over the ones in the front, I kept it as a visually interesting anomaly for now.

  I’m still using the generator trained on painted portraits for the demo above. To better show variety these neural networks can create I’m training a new network on custom and more varied data.

• Dec 24 '21

  I updated the website text to better reflect the artistic direction of this project. I also dropped the initial project name Latent Space Cartography as it was suggesting analytical direction instead of the experiential one. I’m currently working on the first interactive prototype, I will soon have footage to show the demo here.

• Dec 8 '21

  Two different perspectives on 1000 images created by a generator trained on painted portraits dataset. The first has the images clustered by their visual similarity. The second clusters the respective latent points - that is the seeds that the generator interpreted as the images. I was expecting that similar latent points would also produce visually similar images but this isn’t the case.

• Nov 29 '21

  The goal for December

  The goal for december is to develop a tool to be used as the foundation for the navigational prototypes. This tool will generate images, preserve the latent noise and visualise the images and the noise in 3d space.

  Initially I imagined that these navigational prototypes would be metaphorically based on submarines, boats and planes. But after discussions with friends I’m reconsidering to use these prototypes to explore concepts like direction, distance, region and boundary in the context of latent spaces.

• Nov 24 '21

  Seeing multidimensional noise

  I created this visualisation of a sample from a multidimensional space. Each point in the sphere represents a random point of a 128 dimensional space reduced to 3 dimensions. Each point is a seed that a generative neural network could interpret as an image. The goal here is to have a basic visual representation of the space I’m working witth.

• Nov 1 '21

  

  The project starts today. In the beginning I plan to create basic prototypes to visualize large samples from pretrained generative networks.