News/Tech News

Open-source AI tool for animating child and amateur drawings of human figures

Published on April 24, 2023

Meta AI Research, Tencent America, MIT CSAIL, and Carnegie Mellon have released Animated Drawings, a tool for creating animations from hand drawn human-like characters.

In order to achieve its goal, Animated Drawings utilizes a variety of techniques, including object detection to determine a tight bounding box surrounding the figure, image segmentation to distinguish pixels belonging to a human figure from pixels belonging to the background, and pose estimation to identify skeletal joints. In this manner, it is possible to create a digital version of a drawing that can then be animated using computer graphics techniques by applying motion data from preselected clips obtained from human actors.

In order to collect data from creators, the researchers set up a Web-based tool called Animated Drawings Demo after an initial unsuccessful attempt to construct a dataset from synthetic data. In spite of their initial goal of collecting about 10,000 drawings, the overwhelming response from users resulted in the collection of over 3 million drawings, of which approximately 200,000 were incorporated into the final dataset.

Thanks to Meta open-sourcing an implementation of the algorithm described in the paper, the results of the research, which are presented in a paper just accepted for publication in ACM Transactions on Graphics, are now available for experimental use.

Animated Drawings has been published on GitHub and has been tested on macOS Ventura 13.2.1 and Ubuntu 18.04.

To make use of additional drawings, the user must first annotate them, which can be done manually or more easily by using scripts that run a humanoid figure detector and pose estimator. Scripts are based on TorchServe and can be deployed using a Dockerfile provided. Meta notes that the annotation files may require some manual adjustment, for which the container includes a Web-based tool.

In the final step, characters can be animated using a motion clip in BVH format. There are a number of BVH animations provided by Meta, including jumping, waving hello, and others. A tool such as Rokoko can be used to create additional BVH animations from an RGB video.

The Animated Drawings system is only a first attempt at an AI-based drawing-to-animation system, and there are numerous possibilities for improvement. In particular, the researchers recommend improving segmentation by using pre-animation classification of the human figure to improve pose estimation and animation by better handling distortion, lighting effects, and shadows.

Tech News

ChatGPT Is Fun, but the Future Is Fully Autonomous AI for Code at QCon London img

ChatGPT Is Fun, but the Future Is Fully Autonomous AI for Code at QCon London

A presentation on artificial intelligence (AI) for code writing was given by Mathew Lodge, CEO of DiffBlue, at the…

New Java SE Universal Subscription from Oracle img

New Java SE Universal Subscription from Oracle

Since January 2023, Oracle has announced the new Java SE Universal subscription and pricing, which will replace ..

Our Latest Blog

Unlock Your Potential with a Level 5 Diploma in Business London's Top Courses img

Unlock Your Potential with a Level 5 Diploma in Business: London’s Top Courses

Are you looking to enhance your knowledge and skills in the field of business? Do...
Read More
Unlock Your Potential with Level 4 Diploma in Business Courses in London img

Unlock Your Potential with Level 4 Diploma in Business Courses in London

Are you looking for a comprehensive course to take your business career to the next...
Read More

Follow Us


Browse LSET presentations to understand interesting…

Explore Now

Get complete guides to empower yourself academically…

Explore Now

Learn about information technology and business…

Explore Now