Animation doesn’t need to remain a big studio prerogative. Deep Learning and AI driven tools and techniques can slash both time and cost of creating computer graphics and visual effects to increase accessibility.

Once upon a time, a young boy named Ed nursed a wild ambition to become a feature-film animator. He loved Disney’s Peter Pan and Pinocchio, and could not imagine a more magical way of leading life. But by the time he reached college, he realised there was one big problem: he couldn’t draw well enough.

Both Peter Pan and Pinocchio were flat, hand-sketched, two-dimensional movies. This meant that its artists had worked long and hard at sketching each frame. They had painstakingly drawn each character posing slightly differently than in the previous scene, and then some more, and some more.

The young boy loved animation, but in the absence of a prodigious artistic talent, he turned towards two areas he did have skills in: physics and computer science. And he forged a brand new ambition — to make the first computer-generated animated film.

So at a time when most animators were sketching with their pencils, Ed tossed the sketchbook out the window, and pushed a chair near his computer.

He created a model of his left hand that he would painstakingly digitize and laboriously animate in a self-written three-dimensional animation program for a 1976 film, Futureworld.

The young boy’s “A Computer Animated Hand” would be considered the world’s first giant leap towards three-dimensional (3D) animation. And more than 40 years later, the boy, Edwin Catmull, would retire as the co-founder and President of Pixar Studios.

Catmull’s success is not all hunky-dory. Many, including bosses at Disney, rejected his work and were unwilling to accept that computer animation was the future. Or that one day, even Mickey Mouse would take the digital plunge into 3D animation.

Drumrolls for 3D

2D to 3D was path-breaking. Artists could now add a lot more detail to their characters (think Toy Story to Frozen). They could easily model them and treat them as physical objects. And instead of an artist repeatedly sketching characters in different angles, the camera could move around the model seamlessly. Lights, shadows and texture could be used in a sophisticated manner…the benefits were simply enormous.

But it’s been close to five decades since 3D was introduced and while the technology has evolved, 3D animation remains time-consuming and prohibitively expensive.

“The process begins by drawing a character’s sketch on paper, and then creating an expression chart for how the character will look when it grins, laughs or cries,” explains Manish Jha, VFX expert at Myelin Foundry.

“The real deal is 3D modelling and texturing, and the most complex part is rigging, where a skeleton is created for each part of the character’s body.”

Not all of an animator’s work is artistically stimulating. It also requires a significant amount of repetitive labour, back-breaking attention to detail and precision, which Manish believes, “may not need a lot of creativity”.

It can very well be automated.

There are steep costs involved, too. A minute of CGI for Game of Thronesrequired 1600 human hours which led to a cost of $80k.

Game of Thrones has extravagant budgets, but the story is quite different in India. If an animator in the US animates half a second or a second’s worth of a show or film, Indians would be expected to do three to five minute a day, Manish says. “If I am asked to animate several times more, naturally, I might compromise on the quality,” he explains.

Indian audiences are consuming global content and understandably, enhanced visual effects and computer-generated imagery for realistic scenes is now becoming a must.

However, repeatedly, there’s news about how VFX artists aren’t able to deliver on time. Famously, Baahubali’s Rajamoulli tweeted in 2015 about his troubles with VFX, and the same challenge persisted more recently for Rajinikanth-Akshay Kumar’s 2.0.

Intelligent Animation

Recent advances in AI and Deep Learning have made it easier to automate some of the laborious processes involved in animation that spike the required time and costs. It’s possible to automate and speed up rigging, for example, by developing a basic skeleton for a human or a four-legged animal character. Deep learning can also be used to present suggestive rigs for other characters.

Creating new characters is often one of the most challenging jobs of an animator. Neural network technology, more specifically Generative Adversarial Networks (GANs), can assist here. If an existing set of characters (think wide from Shrek to Aslan) are fed to GANs, it can come up with new samples by learning from them. A more relatable example: GAN can easily be used to auto-generate and animize anime characters.

Similarly, when it’s time to animate the character, deep learning and AI can help. Just as it learned about the shape of the face, body, or nose of thousands of characters, it can learn about movement — from dancing to combat.

Style transfer is another optimization technique that can be of aid. It can let animators transform what they’re creating in the style of someone whose work they admire, like Van Gogh or Dante. Imagine a character inspired by Virgil or Beatrice from Dante’s Inferno.

The question of creativity

It’s possible that while reading about the tools and techniques that AI and Deep Learning can promulgate, you’re wondering about creativity and originality.

When animators picked the mouse over their pencils, many worried that machines were taking over, but since then, not only do animators have more control over what they produce, the quality of visual effects and animation has grown by leaps and bounds.

“If I am thinking of a genie, Aladdin’s genie dominates my mind,” Manish says. “But if I could use AI to match genie to random objects like a tail or a hat, or if I can feed the machine with all kinds of genie characters that have ever existed, the newer characters it would generate could help me think afresh or simply break my mental block. They would work as brainstorming triggers and eventually, the animator can tweak and change everything.”

Along with creative cues, the vision of expanding new technologies in animation, Gopichand Katragadda, the Founder and CEO of Myelin Foundry says, “is to democratize computer graphics so that it doesn’t stay limited to the big studios. AI and Deep Learning can slash time and costs spent in computer graphics and visual effects to 1/10th. It can stir up a revolution”.

Pushing the envelope

Virtual scene creation is the next area Deep Learning and AI can explore. The neural network can observe and learn from existing images of say, cities or scenic landscapes, and synthesise new ones just as an artist would draw something.

Say, you have videos and images of a city’s landscape — full of grids, high rises and parks. The image could be converted into a 3D model without having to put in hours and hours drawing shapes, extruding and developing the world manually. Nvidia has been doing some exciting work in creating AI rendered 3D scenes.

The artist, however, would remain the boss, and free to blend, merge, or reject it altogether.

Either way, AI and Deep Learning is pushing the envelope in computer graphics and visual effects.

It is expanding the scope of animation by keeping tedious and manually intensive tasks at bay, and liberating the artist to chase brilliance. Most notably, it is building a future where more people can engage in and make art through animation and visual effects.