From sketch to image with Stable Diffusion and ControlNet
My own drawings and simple Blender renderings as structural input for Stable Diffusion.
- Year
- 2023
- Context
- Own project
- My role
- Drawings, 3D models, image generation
- Tools
- Stable Diffusion, ControlNet, Blender
Approach
Two series. In the first, a drawing is the input for ControlNet in img2img mode. In the second, a simple Blender rendering of soft, clay-like forms takes that place.
The prompts stay the same within a series. For the drawings: “trash mountain, complex, highly detailed, intricate, … crepuscular rays, luminescence”. The drawing sets the composition, the prompt sets material and light.
Each input produces several variants. The gallery shows input and result side by side.
Result
This project shaped me and led me to AI art. Getting control over the AI process was one of the most exciting turning points in my work. After that I experimented more and more and kept working with AI. It was also the starting point for my current job.
