Перейти к содержимому

AI Can Now Animate Any Monster. Opus 5.5 Built the Tool

Can It Code?

0:00 / 0:00

AI Can Now Animate Any Monster. Opus 5.5 Built the Tool

18 693 просмотра · 8 ч назад
Can It Code?
14,9 тыс. подписчиков
18 693 просмотра · 8 ч назад
My Patronite:   / canitcode   A troll, a wyvern, a centaur with a sword. For people there is motion capture and Mixamo. For monsters there is nothing. So Claude Opus 5.5 and I built our own way: Seedance films our monster moving, and our tool finds every joint in that video and puts the move on the monster's own skeleton in Blender, ready for the game. In this video: Where it started: a deer from our game, one camera, and legs that kept crossing Seedance with two cameras in one picture Every tool we tried and why it broke: point tracking from Google DeepMind, SAM 2, Depth Anything, painted legs, animal pose (AP-10K), MoCapAnything The rethink: one problem to solve, every joint in every frame A skeleton for any 3D model with UniRig DINO from Meta, taught on our troll in about 30 minutes, reads every joint The exam every change to the tool has to pass Depth, take two From A to Z: model, skeleton, Seedance, joints, Blender, game Before: 4 days with Opus checking every frame. Now: minutes What it still can't do Would you like to try it? Tell me in the comments. My last video, NVIDIA Kimodo: [LINK] Patreon:   / canitcode   Chapters 00:00 Intro 00:42 Every way we tried 01:20 Where it started: the deer 02:27 Seedance with two cameras 03:22 Shopping: point tracking 03:48 SAM 2 outlines 04:06 Depth Anything 04:28 Painted legs 05:00 Animal pose 05:14 MoCapAnything 05:29 The rethink 05:54 A skeleton with UniRig 06:27 DINO reads every joint 07:40 The exam 08:32 Depth, take two 09:08 From A to Z 09:32 4 days vs minutes 09:48 The limits Sources AI video: Seedance 2.5 (ByteDance Seed). 3D models: Tripo (tripo3d.ai). Skeleton: UniRig (github.com/VAST-AI-Research/UniRig). Joint reader: DINOv2 from Meta (github.com/facebookresearch/dinov2). Tools we tried: BootsTAPIR point tracking from Google DeepMind (github.com/google-deepmind/tapnet), SAM 2 from Meta (github.com/facebookresearch/sam2), Depth Anything V2 (github.com/DepthAnything/Depth-Anything-V2), ViTPose++ with AP-10K, MoCapAnything. In the intro: NVIDIA Kimodo (github.com/nv-tlabs/kimodo) and Mixamo. Built in Blender and Godot 4.7. Code and scripts: Claude Opus 5.5. Music: Nocturnal Pads.