Greying red ape with a machine as a tool
🦧 🦍 📺 CG / 3D / AI / VFX / #dailyXYZ / #diffusetogether / #6079ai
Find Oranguerillatan on their website, Instagram, YouTube, TikTok, X (Twitter), Threads, AiAnimation, LinkedInARTIST: Peter Gabriel
CREATOR: Oranguerillatan
SONG: The Court (Dark-Side Mix) FROM i/o
Seeing the 'Sledgehammer' stop-motion video by Peter Gabriel was a pivotal moment in my life, igniting a love for animation that led me to study Animation/Illustration at University over 20 years ago, essentially setting me on the track to where I am now.
As a long term admirer of Peter Gabriel's musical/visual contribution to the world and Tim Shaw's magnificent sculptural masterpieces, I entered this competition with an AI-animated music video for ‘The Court’ that attempts to embody the spirit of their recent artistic collaboration (see the beautiful artwork by Tim that accompanies the song).
In creating my video, I wanted to pay tribute to the visual and musical genius of Peter and Tim. Both artists relentlessly convey fascinating messages and deep meanings with their bold audio/visual outputs.
Drawing visual inspiration from Tim’s profound sculptural work, I sought to capture its essence without directly emulating it, using text prompts containing aesthetics and subject matter rather than specifically using his name. I hope my video stands alongside Tim’s fantastic artwork in a respectful manner, instilling discomfort and empowering viewers much like his pieces do.
The resulting video aims to capture the duality of the human experience, at once gritty and hopeful, conveying the daily struggles of humanity while also highlighting our collective hope for a brighter future. At the same time, I sought to capture Peter’s vision of a just and peaceful world, where embattled people can rise together towards a better future.
In creating my video, I wanted to pay tribute to both Peter and Tim. Both artists are geniuses that relentlessly convey fascinating messages and deep meanings with their bold audio/visual outputs.
-Orangerillian
The Court is a track taken from Peter Gabriel's album i/o, released on 1 December 2023. This is the Dark-Side Mix, by Tchad Blake.
Words and music by Peter Gabriel. Produced by Peter Gabriel.
Published by Real World Music Ltd. / Sony Music Publishing.
The Court began with a Cuban loop that I was playing around with on my keyboard. I’m an old drummer, so I still love to find a groove that makes me want to dance and then build ideas around it.
In the lyrics, the chorus refrain ‘the court will rise’ came early, and this free-form, impressionistic lyric around justice evolved around it. There’s a sense of urgency there. A lot of life is a struggle between order and chaos and in some senses the justice or legal system is something that we impose to try and bring some element of order to the chaos. That’s often abused, it’s often unfair and discriminatory but at the same time it’s probably an essential part of a civilised society. But we do need to think sometimes about how it is actually realised and employed and if justice could be applied.
-pg
ARTIST: Peter Gabriel
CREATOR: Oranguerillatan
SONG: Olive Tree (Bright-Side Mix) FROM i/o
ARTIST: Peter Gabriel
CREATOR: Oranguerillatan
SONG: Olive Tree (Bright-Side Mix) FROM i/o
Video info
Date created: Dec 2023-Jan 2024
ChatGPT: Concept and narrative themes / Initial shot-list.
StoryboardHero.ai: Initial storyboard frames
Adobe Premiere Pro: Board-O-Matic / 2D animation / Final Edit
RunwayML Gen-2 / PIKA Labs: Additional AI img2vid motion to bring the source material to life.
Mixamo / Maya: Additional 3D character/camera animation
ComfyUI: AI vid2vid processing, utilising SDXL / AnimateDiff / IPAdapters and ControlNets.
Nuke: Colour grading, VFX compositing, shot cleanup.
TopazVideoAI: Upscaling of ComfyUI outputs.
The creative process, man meets machine:
Starting in December 2023, the creative voyage of this video was a sophisticated fusion of cutting-edge AI technology and traditional 3D/VFX tools, with collaboration between humans and machines pivotal at every stage. From conceptualization to final execution, the synergy between these advanced tools and the creative vision behind the project was instrumental in bringing it to life.
To kick start the process, Storyboardhero.ai was utilised for initial AI-generated storyboard frames, guided by the shot-list provided during creative discussions with ChatGPT. These frames were compiled into a timed "board-o-matic" using Adobe Premiere Pro, allowing for the pacing and overall feel of the video to be gauged. This process involved refining the shots iteratively, with input from both AI tools and human creatives.
Once the timings were established, the process of adding motion to the source material began, utilising various methods. Some simple 2D animation was applied to certain storyboard frames using Nuke and Premiere. However, the primary focus was on running the boards through multiple AI tools individually to imbue them with life. Working closely with Graeme Cornies from Kodiak Arcade, storyboard frames were processed using img2vid tools like RunwayML’s Gen2 and PIKA Labs to enhance their dynamism. Additionally, 3D software such as Mixamo and Maya was employed by Oranguerillatan to introduce character animation, camera movements, and depth to select shots. The addition of a few real-life video clips at this stage further enriched the visual narrative.
The previous steps focused on animating still frames, but they didn't address the final aesthetic. Enter ComfyUI. Using a vid2vid setup developed with input from some very smart members of the AI community. Each of the 200+ shots underwent a ComfyUI transformation, resulting in the distinctive claymation style evident in the final piece. This tactile aesthetic, reminiscent of stop-motion animation and 'old-tech' craftsmanship, contrasts with the modern AI technology used in production. Most shots were generated multiple times, resulting in over 1000 generations being created during this stage. The cutting room floor was covered in binned generations that didn’t quite make it.
After the shots were generated and the edit refined over several weeks, the process entered its final stages: colour grading using Nuke and AI upscaling with TopazVideoAI. The colour grading was a painstaking process, with each shot meticulously matched to its surroundings to maintain cohesion and evoke the intended mood for each section of the video. From the cool, dark tones of winter nights to the vibrant hues of summer days, each season was depicted with its own distinct visual palette, enriching the narrative and emotional journey of the video.
Exploring conceptual and narrative themes with AI:
Olive Tree’ stands out as a six-minute musical masterpiece that exudes uplifting energy. Its distinct sections intricately capture the essence of hope, interconnectedness, and curiosity. Peter's discussion of his inspirations and themes in his August 2023 "Full Moon" video provided fascinating insight into his creative process while composing and recording the track.
After immersing in the track and Peter's insightful discussions about it, a dialogue between Oranguerillatan and ChatGPT commenced to delve further into the conceptual and narrative dimensions. Through collaborative exchanges, AI served as an illuminating guide, shedding light on creative possibilities and thematic depths. By engaging in these discussions, a deeper understanding between man and machine emerged, enriching the creative process and broadening perspectives.
Delving deeper into the intended themes and content of the video, discussions revolved around the different sections of the song. The song's composition mirrored an epic journey across time, seamlessly transitioning from mystique and child-like curiosity to scepticism, growth, transformation, and ultimately, unabashed joy and euphoria. This musical evolution resonated with the notion of ever-changing seasons and human emotions' daily fluctuations, highlighting life's unpredictable nature.
Strategically incorporating changing seasons and time of day into the video became a key feature of aligning the visuals with the evolving narrative and emotional arc of the song. As the track progresses through its various verses, choruses, bridges, and instrumental sections, the time of day and season seamlessly shift multiple times, enhancing the immersive experience for the viewer.
Shifting seasons - a journey through time:
As our hero embarks on their journey through the changing seasons and the passing of time, each transition mirrors the evolving emotions and narrative of the song. From the mysterious allure of winter nights to the vibrant energy of summer days, the visual landscape shifts in tandem with the song's tempo and mood.
The journey commences in the depths of winter and the cover of night, setting the stage for the mysterious and enchanting ambience of the track's introduction and opening verse. This wintry landscape serves as the perfect backdrop for introducing the protagonist and unveiling their intriguing discovery propelling the narrative forward. Each post-chorus transition to a new verse brings the viewer back to the dark, chilly embrace of winter night, accompanied by the timeless symbolism of the full moon, a motif dear to Peter.
As the song progresses, the visuals transition to the bright hues of sunrise and the promise of a new day, signifying spring's arrival. This season symbolises growth and renewal, reflected in budding flora and the protagonist's burgeoning curiosity as they interact with a mysterious device in their backyard.
The arrival of the climactic chorus comes with the midday summer sun. The video explodes into a vibrant display of sunshine and revelry, characterised by uninhibited joy and boundless energy. The colorful and euphoric imagery captures the essence of interconnectivity and togetherness, embodying Peter's concept of ‘the melting of the solid world.’ Amidst the chaos and fun, the underlying theme of unity and interconnectedness shines through, offering a respite from the cynicism of social disconnection.
As the video transitions into the post-chorus autumnal sections, viewers witness the serene beauty of dusk and sunset, signalling a moment of introspection for our protagonist. Here, amidst the fading light of the day, they reflect on recent experiences and contemplate the implications of their newfound interconnectedness. The juxtaposition of light and shadow prompts viewers to ponder the balance between safety and uncertainty in embracing this vivid interconnectivity.
With each cycle, our protagonist undergoes a transformation, evolving from a curious observer to a determined explorer of the interconnected world around him. As they delve deeper into this realm, their initial scepticism gives way to a sense of purpose and connection, guided by the ever-present motif of the full moon; a symbol of continuity and cyclical renewal. Throughout this journey, time becomes fluid, transcending conventional notions of age and linear progression.
-Oranguerillatan
Song info:
Olive Tree is a track taken from Peter Gabriel's album i/o, released on 1 December 2023. This is the Bright-Side Mix, by Mark 'Spike' Stent.
Words and music by Peter Gabriel. Produced by Peter Gabriel.
Published by Real World Music Ltd. / Sony Music Publishing.