How to use Prompts for AI Imaging using Stable Diffusion

This post is intended to help you create images using Stable Diffusion.Have you been amazed at the awesome images some creative folks create sing prompts for AI Imaging? In this post, I will introduce you to some commands or prompts and some basics of image generation using generative AI. Table of Contents

Introduction

1. What is Stable Diffusion and where to get it?

2. How to Generate AI Image Using Stable Diffusion?

3. Example : Painting of a Tabby Cat

4. Wrapping it up: AI image generation is an Art and a Science

5. Bonus: More Image prompts for AI Image generation

---

Introduction

The arrival of Artificial Intelligence (AI) based image generation or AI Imaging tools over the past several months have taken the world of AI Art  by storm. Dalle2, Midjourney and Stable Diffusion have become well known names (and tools) in this space in a short span of time. I began experimenting with AI imaging tools from early August 2022. Since then, I have created thousands of images. Majority of them have been generated using Stable Diffusion. There are several tools such as lexica.ai, prompthero, and the prompt database at Stability.ai, which can help the beginner figure their way around with generating AI images. At the time of publishing this post, the most recent version of Stable Diffusion is Version 2.1. We will use the Image generator form Stable  Diffusion for this post. This Cat Does Not Exist (in real world, that is!) Below post is based on my own experimentation and experience when I was exploring AI Image generation techniques for my upcoming book, titled

An Eye for AI : How to create amazing images using AI Imaging

You can learn more about this book by visiting artwithai.in

---

What is Stable Diffusion and where to get it?

Stable Diffusion is a powerful AI imaging tool that allows users to generate stunning images quickly and easily, with minimal effort. You can experiment with Stable Diffusion to generate AI images using different materials, art styles and lighting. There are many tools for generating AI based images. Stable Diffusion is open source, and that has prompted many developers to create tools that use Stable Diffusion at the back end. These include free, freemium and premium tools. PlaygroundAI and Stable Diffusion space on Huggingface are example of free sites. You can generate images for free on these sites. Thumbsnap, Starryai and Nightcafe are examples of freemium sites. They all offer a certain amount of free credits that can be used to generate images. Supermachine is an example of a premium or paid tool that offers Stable Diffusion and many other imaging algorithms or apps. For the sake of simplicity, we will use an existing web-based tool that is either free or freemium. For generating images for gaathastory and my blog, I typically use Supermachine. I have a paid subscription for this awesome AI Image generation service. In the free or freemium space, my personal choices, in no particular order, are Thumbsnap, Nightcafe, and PlaygroundAI. For first timers looking to step into the world of AI Imaging tools, the Stable Diffusion web version could be a good starting point. Advanced users might want to try out more complex prompts at Dreambooth or Midjourney, two more awesome AI Imaging sites.

How to Generate AI Image Using Stable Diffusion?

In recent weeks, there have been many posts, tutorials and other information on the importance of something called as “Prompt Engineering”. Let us quickly understand what this term means. In plain and simple terms, prompt engineering refers to the art and science of crafting the right combination of commands that will help you generate great images. Some folks suggest providing as detailed and clear set of instructions as possible. While others suggest, starting with very simple commands, and slowly adding complexity or details, one or two modifications at a time. I like the latter approach. If we start with very complex set of commands, not everyone might understand what each command stands for, or how it can enhance an image. In particular, when we are dealing with many moving parts such as medium, material, lighting and art style. Confused? Let us try and look at the above statement with a simple example.

Example of Generative Art: Painting of a Tabby Cat

First of all, how about we do something different say we desire a watercolour painting this time around? The prompt will read something like below:

Oil Painting of a Tabby cat

Oil Painting of a Tabby cat, generated by Stable Diffusion Note: You will notice that Stable diffusion web tool can generate upto 4 images at a time, for the rest of our example we will only use one of the four options.

Adding more elements

Remember what I mentioned earlier about starting simple and adding one or two elements at a time? Let us give it a try ! We refine our command by defining the background, lighting, and finish. The modified command is as follows:

Oil painting of tabby cat, dark background, matte finish, volumetric lighting

The result? Below is the best of the four options that the AI tool gave us. Not too bad, but not stunning either.

Tabby Cat with a dark background Now, we refine our command further, as below:

Oil painting of a tabby cat, dark background, highly detailed, volumetric lighting, 8k

Followed by

A stunning oil colour painting of a tabby cat, dark background, highly detailed, 4k

What if we change the material from oil to ink?

A stunning ink painting of a tabby cat, dark background, highly detailed, volumetric lighting, 8k

Ink Painting of a Tabby cat, generated using Stable Diffusion 2.1 And another trial, back to oil painting

A stunning oil colour painting of a tabby cat, dark background, highly detailed, 4k, digital art, concept art

Painting of a cat, generated using Stable Diffusion 2.1 The cat in the above picture looks a little lost or sad. We can do better. Therefore we now add a painting style or an artist’s name to the prompt. You may find the names Greg Rutkowski, WLOP, Alphonse Mucha and artgem mentioned many times. For our painting, we will try a couple of artists,namely, Raja Ravi Verma and Roberto Ferri. For the final iteration, I decided to try another tool, Thumbsnap. I also changed the aspect ratio of the image to a portrait style, instead of a square. The result was rather delightful, and I could not choose between the two images that were generated. Therefore, presenting both of them below.

Portrait of a cat generated using Stable Diffusion 2.1

Another portrait of a cat generated using Stable Diffusion 2.1

Wrapping it up: AI image generation is an Art and a Science

In the above example, you might have noticed that generating an awesome image requires a bit of trial and error. In our case, it took us about 4 or 5 attempts to get the right look and style for the cat. It gets more complicated if we try to render images of humans or more complex patterns such as a spaceship or a building. In case of AI generated images of humans, the common grips is that the hands or face is deformed, or there are simply too many limbs. But remember, AI image generation is a new albeit rapidly evolving field. I am sure that in a few more versions, the AI generation tools will get things right. Tip: In the above example, you can try adding options such as biopunk or synthwave to get some lovely variations in the images. Once you are satisfied with the results, you can download the image in png or jpg format. Some image rendering sites such as Nightcafe and StarryAI also allow you to share the images via social media or email.

Did you find this introduction to AI Image generation using Stable Diffusion useful? Would you like me to write similar short intros to Midjourney, Dalle2 and some other AI imaging tools? I would love to hear from you ! -Amar Vyas

Bonus: More Image prompts for AI Image generation

I have mentioned earlier in this post that you can experiment with materials, lighting, art form and artists, and image quality when you are generating prompts for AI Imaging using stable diffusion. We will create images using different materials (such as watercolour, ink, acrylic), art styles (such as baroque, renaissance, steampunk, synthwave) and experiment with image quality (such as 4k, matte finish) and lighting (such as octane render, volumetric lighting).

Parameter	Examples
Material	Watercolour, ink, acrylic, crayons
Art Styles	Baroque, Steampunk, Cyberpunk, Fantasy
Image Quality	4k, Matte Finish, HD, aperture speed (in case of photo)
Lighting	Octane Render, Volumetric Lighting, soft focus
Artists	Will depend on the image or art technique
Aspect Ratio	Landscape, Portrait, Square, custom aspect ratio

How to mix and match AI image generation parameters using stable diffusion. What if we change the artist or the style or painting? Try the following:

A beautiful oil painting of a cute tabby cat, by Paul Gaughin, biopunk style, 4k, matte finish, digital art, highly detailed

or

A beautiful painting of a cute tabby cat, by Edvard Munch, synthwave style, 4k, volumetric lighting, digital art

Have fun trying out AI imaging! I would love to see what images you can come up with.

End Note: For dog Lovers

Don’t like cats/ Or you think your doggo is feeling left out? Try the below prompts on Stable Diffusion or Bing. Have fun!

More prompts for AI imaging using Stable Diffusion

• Indian Shepherd Dog in Watercolor Art with Baroque Style
• High-Quality 4K Image of an Indian Shepherd Dog using Octane Render and Soft Focus Lighting
• Ink Drawing of an Indian Shepherd Dog by a Famous Artist in a Cyberpunk Setting
• Acrylic Painting of an Indian Shepherd Dog with Custom Aspect Ratio Composition
• HD Photo of an Indian Shepherd Dog with High Aperture Speed Captured in Natural Light
• Matte Finished Digital Illustration of an Indian Shepherd Dog in a Fantasy Landscape Scene
• Crayon Sketch of an Indian Shepherd Dog Using Volumetric Lighting Effects
• Portrait Photography of an Indian Shepherd Dog with Shallow Depth of Field and Soft Background Blur
• Steampunk Concept Art Showcasing an Indian Shepherd Dog Wearing Gear and Mechanical Accessories
• Customized Aspect Ratio Animation of an Indian Shepherd Dog Interacting with Other Animals in a Forest Environment

---

About the Author Amar Vyas is a writer, speaker and Sustainability Professional. He can be reached at contact @amarvyas.in. To learn more about Amar, visit here. For more of Amar’s posts on images, click here.