_1.png)
Creating artistic works has never been straightforward. For every final piece hanging in a gallery or delivered to a client, there is a tremendous amount of time and effort in its conception, design, and preparation.
The current narrative around Artificial Intelligence (AI) based art creation, especially text-to-image tools like Dalle-2, is fear about artists getting replaced, and the resulting society losing a piece of its humanity. But there’s another story here, one where AI tools are embedded into the creative process and make artists’ lives better. In this story, these tools save creatives time and aid in numerous ways, such as reducing ‘writer’s block’ and improving existing workflows. These AI tools of the near-future have the opportunity to allow a true collaboration and lead to a more fulfilling, richer process, better communication across teams, and ultimately stronger work.
Technology and Art
Art has a complicated relationship with technology. Historically, when new technology enters the art world there are concerns and cultural shifts. When photography initially gained popularity in the early 1800’s, Théophile Gautier wrote that “Certain minds see the admirable discovery as posing a danger for art; they were afraid lest the human hand might forget its skill, knowing that a machine exists that can do its work.” He goes on to say that this did not come to pass. Instead, the art world changed in unexpected ways. Impressionism became popular as painting was no longer required to save an image of reality. Artists’ ability to make highly realistic drawings improved in a ways that weren’t previously possible before high definition photography could be a reference. More recent examples, like Adobe Illustrator and Adobe Photoshop, have greatly augmented artists’ abilities to mockup designs and to make rapid iterations to their works.
AI, primarily machine learning, is opening up a new set of technologies for artists that are fundamentally different from previous developments. This stems from the fact that AI is far less deterministic than most existing technologies used in creative industries. With most AI discussed here, and unlike the tools mentioned above, identical inputs will produce an open number of different results. As opposed to traditional tools that will execute a specific task in the same exact way every time, many AI methods are better geared toward capturing the artist’s intent and maintain the work’s context, while operating with the freedom to complete the task in different, often surprising, ways.
Three such machine learning based technologies are Disco Diffusion, Dalle-2, and Midjourney. Disco Diffusion is an open source project (the code and data used to make it are freely available) and Midjourney and Dalle-2 are proprietary, but do similar things. Give one a text prompt (“a blue cat playing a harmonica on the surface of the moon in the style of Vincent Van Gogh”) and, based on an enormous amount of training data, each will attempt to create images that adhere to the prompt.
_5.png)
A New Creative Partner
Because of the non-deterministic nature of many AI workflows, there is enormous potential for these tools to be used as creative partners for artists to explore with. In this use, the AI steps in as a collaborator during the planning and creation stages of a creative work, creating hundreds of potential ideas on the spot to inspire new thinking and help artists break through creative blocks. Friend and talented industrial designer Saloni Bedi provided prompts (“water bottles in the shape of gourds”) that were fed into Disco Diffusion, with the hope of helping her enhance her own creative process, giving her forms that she might not have imagined on her own.
_17.png)
_29.png)
_22.png)
To better understand the creative process, and where AI tools might fit, I spent a month working with Lauren Trager, a professional artist out of NYC. She had just started creating a series of sculptures based on the idea of a fountain of human consciousness.
She had already developed a series of words to guide the work: Plastic, Single-use, Reuse, Fountain, Consciousness, Immediately, Environment, Consumption, Water. She also knew that she wanted to create a fountain out of materials that would normally be considered trash, but wasn’t yet sure about the form that would take. With those initial keywords and description Midjourney was used to generate images with the goal of providing inspiration for the actual physical design of the fountain.
While many of these prompts used other artists as style inspiration, it’s also possible for artists like Lauren to use the open-source potential of Disco Diffusion to fine-tune a model, training it on a body of their own work. This will allow them to generate new images in their own artistic style.
Disco Diffusion is unique in that it has an option to add an “initial image” that the prompt is then generated from. This allows artists to draw a quick sketch, describe the ideal final product, and see hundreds of potential directions that could develop from that sketch. Using this capability, we incorporated photos of Lauren’s materials to generate different ways that they could be turned into the sculptures in her imagination.
Lauren’s next step was a surprise; instead of asking for new images or returning an image to be modified further she used these images as a jumping off point in her own research into the current art world. Her research was augmented by the collaboration, expanding her thinking to incorporate ideas and work she hadn’t previously considered.
The early brainstorming and research phase of the creative process is a powerful place for disruption. In addition to my work with Lauren, I interviewed Phoebe Warner, a muralist and mentor at Artists for Humanity. When she starts working on a project, clients often are not sure what they’re looking for. In this stage, all work takes place digitally using Photoshop. They’ll go back and forth until they approve a design, and only then do they start actually drawing the version that will be turned into a final mural.
In that first digital step there is great potential for time-saving AI-based tools and products. One example might be a tool that will generate a collection of images to serve as a moodboard. While the copyright conversation around machine-learning generated art is still ongoing, the most commonly accepted conclusion is that these AI-generated images are part of the public domain. It isn’t too far from the current iteration of these tools to add a series of keywords and generate a moodboard of license-less images to easily share with coworkers and clients.
With tools like the ones that supported Lauren’s practice, an artist could upload a sketch or photograph and generate hundreds of possible directions to share with a client, saving them time during those initial conversations. Feedback between the artist and the AI, such as selecting images to make more of a certain style, or "painting in" a prompt with a capability called in-painting, expand the possibilities of a true AI-human collaboration even further.
Creating the Future
Lauren kept working on this series beyond our collaboration (her show opens tomorrow as I publish this piece). Looking back, the ideas were all her own. Her early AI collaboration opened her to artists, ideas, and forms that she previously hadn’t considered. She found new shapes in the world around her that reflected Midjourney-generated fountains, and was able to use the output of Disco Diffusion as both a jumping off point to the next phase of her sculptures and as a way to visualize different ways they might look at the end without building everything out.
There is no way to know exactly what the world is going to look like as machine learning technologies, including realistic text-to-image, improve. However, I believe in creating a future where new technologies don’t take away our humanity by filling museums with generative shadows. We can choose to develop technology that works within human processes to help us become more creative, frees up our time, and ultimately makes us more human.
