I turned doodles into video clips in minutes with Runway's new AI video tool - here's how

AI transformed doodles of birds into scary winged creatures.

Webb Wright/ZDNET/via Runway AI

Follow ZDNET: Add america arsenic a preferred source on Google.

ZDNET's cardinal takeaways

• A caller diagnostic from Runway lets you make video from doodles.
• It's impressive, but it performs champion with immoderate cautious hand-holding.
• Access requires paying for (at least) a Standard Runway account. 

A household of settlers, admiring a herd of bison, breaks into a tally arsenic 2 demonic winged creatures alert overhead. The small miss passes done a fencepost similar a ghost.

This surreal five-second country was created acknowledgment to a caller diagnostic wrong AI level Runway called Motion Sketch. As the sanction suggests, it generates videos from elemental doodles that you tin marque straight connected apical of a inactive image. 

In the lawsuit of the clip below, I started by generating an representation utilizing Google's Nano Banana Pro -- 1 of the text-to-image models offered done Runway's level -- past simply drew a smattering of bird-like shapes and 2 arrows pointing the absorption successful which the household and the bison should flee. I added immoderate substance prompts for a small other guidance, and the exemplary did the rest.

Runway iterpreted these doodles to animate a abbreviated video.

Webb Wright/ZDNET/via Runway AI

The diagnostic inactive needs immoderate hand-holding, successful overmuch the aforesaid mode that ChatGPT often requires aggregate finely crafted prompts to make the output you're looking for. All the same, it could people a large measurement guardant for AI-generated video, and for professionals successful originative industries who similar to deliberation much visually than verbally. 

"The diagnostic is adjuvant for each types of users," Runway Product Manager Aditi Poduval told ZDNET. "Ultimately, we're helping radical get precise question without needing to trade a circumstantial written prompt."

Here's what I've learned aft dabbling with Motion Sketch.

My experiments

After my archetypal experimentation with the representation of the bedraggled settler family, I decided to up the sketchiness, truthful to speak.

I'd generated the representation of the household utilizing a substance prompt. But Nano Banana Pro via Runway tin besides make an representation based solely connected a doodle, which you tin marque straight connected the level à la Microsoft Paint. I'm a writer, not a ocular artist, but I did my champion to sketch retired what I hoped vaguely looked similar a snake hanging from the subdivision of a tree. 

A sketch of a snake slithering successful a tree.

Webb Wright/ZDNET/via Runway AI

Then, I turned to Motion Sketch. As mentioned, I'd met the exemplary halfway successful the settler household experimentation by adding immoderate substance prompts successful summation to my doodles, but this clip I wanted to spot however good it would execute without immoderate written guidance. I hoped to make a clip of the snake slithering to the right, truthful I drew immoderate elemental motion-directing arrows on the magnitude of its body. Any six-year-old would beryllium capable to grasp what those arrows mean, but however would AI perform? I acceptable the magnitude to 10 seconds, selected Runway's Gen-4.5 video exemplary (Google's Veo 3 and Veo 3.1 are besides available), and clicked Generate.

I was expecting a comparatively creaseless clip of the snake moving on the subdivision and exiting the bottom-right of the shot, but what I got was overmuch trippier. The snake near down portion of its assemblage to the near of the shot, and different snake emerged from its assemblage and fell retired of the tree. At 1 point, it appeared to turn alligator-like feet, which it utilized to propulsion itself on the branch. The archetypal arrows I'd drawn besides concisely appeared astatine the opening of the clip earlier disintegrating.

To beryllium fair, snakes are a biologic marvel, and accurately modeling their question is nary tiny task for immoderate animator -- quality oregon machine. And the exemplary seemed to misinterpret my aggregate arrows arsenic an acquisition for the snake to determination successful aggregate directions simultaneously, which intelligibly confused it. So I went backmost and added a elemental substance punctual to my archetypal representation with the arrows: "The snake slithers on the branch."

The last merchandise this clip was a spot person to what I'd had successful caput (no abrupt duplication of snake bodies oregon sprouting of legs). The question inactive looked awkward, but again, snake question is simply a reasonably gangly ask, with tons of analyzable fine-grained physics involved. 

One much experiment: I wanted to spot however good the exemplary could enactment with existent photos, truthful I uploaded 1 from my laptop: a changeable I'd taken immoderate months earlier of Prospect Bark successful Brooklyn. 

I drew what's efficaciously a cosmopolitan icon for fire: wavy reddish lines inlaid with orangish and radiating upwards. Again, I marque nary assertion to beryllium a talented illustrator, but astir humans would beryllium capable to admit what I was going for with this. Runway's exemplary picked up connected it too, and dropped a monolithic bonfire into the halfway of the tranquil lakeside country captured successful my photo. Again, my sketch concisely appeared successful the clip earlier disappearing successful a psychedelic question of gradually shortening wiggly lines. And possibly I'm imagining it, but the trees adjacent to the bonfire adjacent seemed to beryllium blackening nether its heat.

The takeaway

Like immoderate different generative AI tool, Motion Sketch has its imperfections and glitches (the small miss passing done a woody beam, the snake sprouting a leg, the wavy lines of my archetypal sketches concisely appearing successful the last video clips). Still, I'm impressed.

Here is simply a glimpse of an wholly caller format for generating video with AI, 1 that could little the obstruction betwixt imaginativeness and accumulation much than ever before. "THIS is what I've been waiting for," 1 X user posted beneath Runway's Tuesday announcement. "Drawing ur imaginativeness past watching it render arsenic video? Absolute crippled changer for creators who can't punctual technologist their full life."

Right now, it inactive requires a just spot of trial-and-error if you person a circumstantial imaginativeness successful mind, but each the same, it's an breathtaking preview of what the aboriginal mightiness hold. And it's decidedly worthy playing astir with, adjacent -- and possibly particularly -- if you're conscionable going into it successful the tone of creativity, with nary clear-cut imaginativeness successful caput of what you'd really similar to produce.

How to access

Motion Sketch requires (at minimum) a Standard subscription to Runway, which costs $12 per idiosyncratic per period and comes with 625 monthly credits.

To entree from your idiosyncratic dashboard, click connected "App" successful the left-hand menu, past prime Motion Sketch either beneath the "Explore Gen-4.5" Collection, oregon beneath the preview model connected the close fractional of the screen, wherever you'll spot a fastener labeled "Try it now." 

From there, conscionable upload your representation and click "Sketch" —> "Export sketch" —> Generate.