How To Generate ControlNet And Canny Edge In Stable Diffusion

Hyeat

In the fascinating realm of AI-generated imagery, the power to control poses and compositions has reached new heights with technologies like ControlNet and Canny Edge. This blog post delves into the remarkable capabilities of these tools, showcasing how they can breathe life into even the wildest visions. From seamlessly transferring human poses to animals to crafting captivating wildlife compositions, the possibilities are as limitless as the wilderness itself.

SOFTWARE

We will use AUTOMATIC1111 Stable Diffusion GUI to generate animal images. You can use this GUI on WindowsMac, or Google Colab.

SUBJECT

ere are some prompts based on different subjects:

  • Photo of a Solitary Lion, its golden mane illuminated by the setting sun, exuding strength and serenity.
  • Photo of a mystical Pack of Wolves emerging from the misty depths of an ancient forest.
  • Photo of a mischievous Red Panda peeking out from behind colorful leaves, its eyes sparkling with curiosity.
  • Photo of a Regal Peacock caught mid-dance, feathers spread wide in a kaleidoscope of iridescent hues.
  • Photo of a Tiny teacup kitty perched on an oversized chair, looking like the ruler of its whimsical domain.

Feel free to customize these prom

SCENE WEAVING

Every photograph is a story waiting to be told, and the scene sets the stage for that narrative. Scene keywords hold the power to transport viewers into the heart of the wilderness, where nature’s dramas unfold:

  • Blanketed in snow, a wolf’s determined gaze pierces through the winter’s veil.
  • A river’s gentle whispers provide the soundtrack to a bear’s fishing expedition.
  • Ancient trees stand as guardians of the forest’s secrets.
  • A grassland canvas paints a serene backdrop for a deer’s graceful leap.
  • Amidst a realm of emerald, a lion’s roar asserts dominance.

CAPTIVATING LIGHTING

As the brushstrokes of a painter define their masterpiece, lighting molds the essence of an image. Lighting keywords are the artist’s palette, each stroke enhancing the narrative:

  • The dark studio unveils hidden mysteries, igniting the imagination.
  • Rim lighting dances around a stalking predator, outlining nature’s prowess.
  • Sunset’s golden hues kiss the horizon, infusing warmth into every pixel.
  • Dramatic lighting casts nature’s actors in a captivating chiaroscuro.

HARMONIOUS EXTRAS

An image’s symphony is composed not just of scenes and lighting, but also the subtle extras that lend authenticity. Keywords, like well-chosen notes, enrich the composition:

  • The camera’s lens captures the soul of the moment, a DSLR’s heartbeat in every click.
  • Ultra quality elevates pixels to emotions, a visual symphony in high fidelity.
  • Film grain embraces nostalgia, weaving stories through the fabric of time.
  • 8K UHD reveals every whisper of nature’s tapestry, immersing viewers in detail.

AESTHETIC EMBELLISHMENTS 

Breathe life into your photographs with phrases that resonate, lending an aesthetic rhythm:

  • “National Geographic Wildlife photo of the year” crowns the pinnacle of nature’s artistry.
  • “The American Landscape Contest” paints patriotism across panoramic horizons.
  • “Wildlife photography contest” ignites a communal celebration of untamed beauty.

CAPTIVATING ANIME ELEGANCE

Step into the realm of anime-inspired vector art with our secret weapon: the MeinaMix model. It’s not just about animals; it’s about capturing their essence in a style that’s both timeless and entrancing. Your prompt is your wand, guiding the model’s creative flow. But wait! To ensure that you get pure animal marvels, wield the power of the negative prompt. Wave away elements like “bad art,” “amateur,” and even “girl” or “people.” It’s like creating a portal to a world where only your chosen animal exists.

Controlling Poses with ControlNet

ControlNet emerges as an artist’s palette for crafting compositions. With its ingenious algorithms, it becomes possible to shape the very essence of an image. Even the audacious task of transferring human poses to animals is within its grasp, though the outcome might provoke curiosity due to the inherent differences between human and animal physiques. A harmony of art and science, ControlNet empowers creators to dictate visual narratives and blur the boundaries of the imaginable.

Face Close-Ups with OpenPose

A challenge within a challenge arises when capturing the essence of a face, especially in the context of wildlife imagery. Enter OpenPose, a solution that requires a reference image of a close-up human face to meticulously replicate the subtleties of expression. The blog post unveils the delicate dance between technology and artistry, painting a vivid picture of how a mere reference can birth the most remarkable portraits.

Reference image.

ControlNet

OpenPose

model

Realistic Vision v2

Prompting Possibilities

National Geographic Wildlife photo of the year, A regal cat perched against a Mountain, in morning, day light, where depth of field the world, focusing solely on untamed elegance.

NEGATIVE PROMPT

In a powerful image, a deformed and disfigured tree stands resilient amid a serene forest, serving as a testament to nature’s enduring beauty even in the face of adversity.  

Here are some close-up images of animals generated.

Transcending Realism with Canny Edge

In the pursuit of authenticity, Canny Edge emerges as a bridge between reality and generated art. The technique of transferring the very essence of a wildlife photograph using Canny Edge becomes a focal point. The post unveils the intricate settings—Canny preprocessor, model, control weight, and control steps—that dictate the transformation. By narrating the journey from ordinary to extraordinary, readers can appreciate the marriage of technology and creativity.

ControlNet Setting:

  • Preprocessor: Canny
  • Model: Canny-fp16
  • Control Weight: 0.65
  • Starting control step: 0
  • Ending control Step: 0.5

Prompting Possibilities

“In the acclaimed National Geographic Wildlife Photo of the Year, a deer takes center stage. Bathed in the gentle glow of sunset, its silhouette is outlined by a delicate rim of light. A portrait of nature’s serenity in the evening’s embrace.”

NEGATIVE PROMPT

In a powerful image, a deformed and disfigured, woman, man, people

Reference image for ControlNet:

Reference image.

Generated images (various animals):

CONLUSION

The blog post stands as a tribute to the harmonious symphony between technology and human creativity. It captures the essence of commanding poses, orchestrating compositions, and weaving dreams into pixels. With ControlNet and Canny Edge, the wild is tamed, reshaped, and spun into something new—a testament to the boundless potential of AI-guided artistry.

Leave a Comment