从一句话到一个能玩的世界 VAST 把整条链路开源了From One Sentence to a Playable World: VAST Open-Sources the Entire Pipeline


AI free token/AI免费积分

在游戏里,如果你走到一扇门前,不确定能不能进去,通常的做法是先转身往回走两步,再跳两下,看看能不能翻过旁边的墙。

这些都是游戏里再普通不过的操作,可是想象一下,如果眼前的画面由 AI 生成,模型就得跟上玩家的每一次临时起意——突然停下,又突然转身,又突然跑跳。这期间,每一帧都得稳定在线。

这就是 VAST 刚刚开源的实时世界模型 EdenLive 所能做到的。它是 Project Eden 系列中的实时生成式渲染器,游戏、动画预览等可交互内容的规则和状态仍由上游交互平台维护,白模、2D/3D 情形,或真实录像,都能作为生成画面的参考。你让角色往前走,或者把镜头转向身后,交互平台将更新后的状态转化为灰盒画面与情形描述,输入 EdenLive,由模型实时渲染出对应的视觉成效让你一边玩,一边看到 AI 实时生成的世界。

参考不必是一套已经做完美术的游戏——任何视觉素材都可以。一个粗糙的 3D 白模、一段 2D 情形,甚至一段已有录像,都可以给出结构和运动。同一段参考,换一张首帧、一句提示词,就能得到写实、手绘线稿、卡通、像素或油画等完全不同的外观。

Image

拿任意视觉参考,边交互边生成画面

EdenLive 堪称是全能,可以跟着玩家的操作持续生成游戏画面,也可以保留一段视频的运动轨迹和镜头、把人物与环境换成另一种风格;到了具身情形,第一人称的人手操作视频还能改写成机械臂视角的版本。

为了做到这一点,EdenLive 达成的是一种「分工合作」:参考画面决定结构、运动和镜头,生成模型负责重写这些内容最终「看起来是什么」。EdenLive 的三类输入分工明确:参考视频决定情形布局、镜头运动、物体轨迹和事件发生的时间;目标首帧确定主体、材质、配色和画风;文本提示词补充风格和细节,交互中还能在事件发生时追加一句描述。这最终让生成式渲染从离线的视频生成,进入长时游戏交互、视频画风转换、具身数据增强等不同的情形。

Image

在交互持续开展所需的长时生成能力上,EdenLive 取得了不错的突破。基于 Gemini-3.8-Flash 对 45 个长时案例的两两盲评(每组交换顺序各评一次,再按维度计算 Elo 分数),在参考遵循、首帧一致性、物理合理性、画质和避免外观泄漏五个维度上,EdenLive 14B 和 5B 全部排在第一和第二。以参考遵循为例,14B 得分 1704,比 AgentGarten 高出约 200 分。

从一句话到一个能玩的世界 VAST 把整条链路开源了

更强悍的是 EdenLive 对于实时的跟随能力。离线视频生成模型要先拿到完整输入,再整段生成,接不进实时交互,EdenLive 却可以追着不断变化的参考画面往前走。按照团队披露,EdenLive 5B 在单张消费级显卡 RTX 5090 上能以超过 24 FPS 生成 720P 画面,并兼容分钟级连续生成。
这对游戏制作来说尤其有意义,玩家按下攻击键之后,最先工作的并不是 EdenLive。武器有没有击中对手、血量应该减少多少、角色又移动到了哪里,都会先由上游交互平台作出判断,并更新到游戏状态里。

Image

在情形和动画预览中,参考可以是白模情形、美术做的粗动画,或由精细模型渲染出的法线图视频。参考不变,只换首帧和提示词,就能在同一个情形、同一段动画上比较不同的材质、光照和配色。镜头离开一处再转回来,参考里的结构还在原处,EdenLive 按同样的结构继续渲染,不必从过去的画面里重新推测。

Image

在游戏里这种连续和一致性,还需要与玩家的操作协同。例如在进入升降装置之后,随着装置下降,整个围栏和网格不能走形,但外面的环境却需要缓缓变化。

面向游戏开发,VAST 同步开源了 EdenGolem(v0.1),一款给编程智能体使用的 skill。只要输入一段玩法描述,智能体便会分四个阶段,依次完成玩法外观、情形搭建、交互逻辑达成与界面完善,生成可独立运行的 3D 灰盒游戏。当下,EdenGolem 已兼容解谜、射击、驾驶、冒险、竞技场、格斗等多种游戏类型。

除了依据简单素材生成,EdenLive 也可以理解现实素材,并达成转换。像「闪身步」是真人教学,在保留构图和运镜的同时,人物、画面和笔触都动画化。模型沿着原视频的动作和空间关系,重新生成每一帧,同时不照搬原视频的质感。

Image

还有一个容易被忽略的用途:具身数据增强。

Image

机器人训练中,并不缺「人类怎样擦桌子」的知识,缺乏的是「从机器人的眼睛看出去,机械手怎样靠近物体、抓住物体并完成动作」的训练素材。

取得类似视角倒也不难,找一个人戴着头戴相机,擦桌子、开抽屉、倒水,一天可以拍很多段。但真人录像里浮现的是人的手臂和五指,机器人部署时看到的却是机械臂和夹爪,两者的外形、运动范围和遮挡方式都不同。模型如果主要用真人视频训练,到了机器人摄像头里,可能会遇到显著的视觉分布差异。

Image

借助 EdenLive 把真人的手替换成机械手,从而把素材转换成更接近机器人实际工作时会看到的数据画面。例如,人拿起水壶的动作、水壶与桌面的相对位置、第一人称镜头的移动都保留,只把执行者改成机械手,可以协助模型适应「机器人看到的画面」。

不同的参考素材类型有不同的成效,自身也有不同的限制, 现成录像只能给出镜头和动作,经过重渲染以后仍然是一段视频,可玩性来自上游代码。但 EdenLive 背后有一套分工与规则,因而生成结果可以随操作继续变化。

世界模型需要解耦状态与渲染

看这些演示,很容易以为玩家正在直接操纵一个视频模型。实际上,EdenLive 并不判断游戏里发生了什么,也不负责让这个世界运转。在今年 5 月发布的 Project Eden 中,VAST 将世界模型研究分为三层架构:持续演化的世界状态,记录物体位置与属性、交互规则和动作的后果;状态到画面的接口,把当前状态转换成某个视角下的布局、几何与事件约束;生成式渲染器,依据这些约束生成高保真画面。EdenLive 达成后两层,不限定第一层。

以格斗游戏为例,玩家按下攻击键,指令仍然先进入上游交互平台。它判断这一脚有没有踢中对手、应该扣掉多少血量,随后更新两名角色的状态和血条。

Image

视频模型擅长生成看起来合理的画面,却无法独自维护血量、碰撞、物体身份和胜负条件等规则,因而 EdenLive 把确定性状态留给上游交互平台,把视觉展现交给生成模型。

交互平台会把每次变化渲染成新的灰盒画面,一个专门微调的视觉编码器从中提取布局、镜头和动作;它不需要精细的皮肤、衣服和光影,只要把空间布局、运动轨迹,以及镜头怎样移动交代清楚。

Image

「规则归交互平台,画面归模型」,这套分工应对了视频模型不擅长长期保存规则的难题,保证视觉元素的稳定展现。当然,这套分工仍要经过更长时间、更复杂交互的检验。

世界模型不缺演示,缺的是用法

2026 年 1 月牛津学者的综述把「视觉混淆」列为世界模型要应对的挑战:画面逼真不代表模型理解了因果和物理规律,并主张世界模型从生成画面的视觉引擎,转向可以表示动态与约束的可操作模拟器。

在这条路线尚未成为世界模型的主流叙事时,今年 5 月 VAST 发布的 Project Eden 已经把交互平台维护世界状态、生成模型负责视觉展现写进完整架构;几个月后,EdenLive 又把其中的生成式渲染推进到实时、连续运行。

Image

在市面上,类似思路到 8、9 月才开始密集浮现。

此次开源涵盖 5B、14B 模型权重(各含双向生成与少步流式生成两个版本)、实时流式推理代码、EdenGolem v0.1 和前端工作台,同时公开工艺报告。开发者可以接入自己的参考来源,测量延迟、稳定性与参考遵循程度。

伴随 EdenLive 一同发布的,还有面向全球青年算法与研究人才的 V·STAR 顶尖人才计划。招募方向覆盖 3D 生成模型、3D 理解与 Agentic AI、世界模型三个方向。无论是全职还是实习生,V·STAR 将有机会直接参与核心项目,并拿到比头部大厂更具竞争力的薪资,特别出色者上不封顶,还有长期期权。

Image

过去的世界模型演示,往往用一段视频证明模型可以想象一个世界。EdenLive 则往前走了关键的一步。而当更多人把自己的代码、情形和参考画面接进来,工艺也将逐步发展为真正可用的创作工作流,世界将迎来更多可能性。

详细项目信息可前往👇:主页:https://vast-eden.ai/?p=edenlive代码:https://github.com/VAST-AI-Research/EdenLive

模型权重:https://huggingface.co/VAST-AI/EdenLive

工艺报告:https://assets.vast-eden.ai/assets/edenlive/EdenLive-tech-report.pdf

Unbound Loop:http://loop.unbound.io

参与 V·STAR 顶尖人才计划,可发送简历至 vstar.talent@vastai3d.com

本文来源:https://www.ifanr.com/1685059
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。

In the game, if you walk to a door and are not sure if you can enter, the usual practice is to turn around and take two steps back, then jump twice to see if you can climb over the wall next to it.

These are the most common operations in the game, but imagine that if the screen in front of you is generated by AI, the model will have to keep up with the player's every improvisation - suddenly stopping, suddenly turning around, and suddenly jumping. During this time, each frame must be stable online.

That's what vast's just open-sourced real-time world model, EdenLive, can do. It is a real-time generative renderer in the Project Eden series. The rules and state of interactive content such as games and animation previews are still maintained by the upstream interactive platform. White mode, 2D/3D situations, or real video can be used as a reference for generating pictures. You let the character walk forward or turn the camera behind you. The interactive platform converts the updated state into a gray box screen and a situation description. Enter EdenLive, and the model renders the corresponding visual effect in real time. Let you play while watching the world generated by AI in real time.

Image

Reference doesn't have to be a set of games that have done the perfect job - any visual material will do. A rough 3D white pattern, a 2D situation, or even an existing video can give structure and movement. In the same paragraph, you can change the first frame and a prompt word to get a completely different appearance, such as realistic, hand-drawn line manuscript, cartoon, pixel or oil painting.

EdenLive is omnipotent. It can continuously generate game images following the player's operation. It can also retain the motion trajectory and footage of a video, and change the characters and environment into another style. When it comes to the actual situation, the first-person manual operation video can also be rewritten into a version of the robotic arm perspective.

To do this, EdenLive achieves a kind of "division of labor cooperation": the reference screen determines the structure, movement and shot, and the generative model is responsible for rewriting these contents and ultimately "what looks like". EdenLive's three types of inputs have a clear division of labor: refer to the video to determine the situation layout, camera movement, object trajectory, and time of the event; the target first frame determines the subject, material, color scheme, and painting style; the text prompts supplement the style and details, and the interaction can also add a sentence description when the event occurs. This finally allows generative rendering to be generated from offline video into different situations such as long-time game interaction, video style conversion, and personalized data enhancement.

Image

EdenLive has made a good breakthrough in the long-term generation capabilities required for the continuous development of interactions. Based on Gemini-3.8-Flash's two-way blind evaluation of 45 long-term cases (each group was evaluated once in exchange order, and the Elo score was calculated by dimension), EdenLive 14b and 5B were all ranked first and second in the five dimensions of reference compliance, first frame consistency, physical rationality, image quality, and avoidance of appearance leakage. Taking reference compliance as an example, 14b scores 1704, which is about 200 points higher than AgentGarten.

In scenarios and animation previews, references can be white-mode scenes, coarse animations made by fine art, or normal graphic videos rendered by fine models. Refer to the same, just change the first frame and prompt, you can compare different materials, lighting and color schemes in the same situation and on the same animation. The camera left a place and then turned back. The structure in the reference is still there. EdenLive continues to render according to the same structure, without having to re-infer from the past picture.

This continuity and consistency in the game also requires coordination with the player's actions. For example, after entering the lifting device, as the device descends, the entire fence and grid can not be out of shape, but the outside environment needs to change slowly.

For game development, vast synchronously open-sourced EdenGolem (v0.1), a skill for programming agents. As long as you enter a gameplay description, the agent will be divided into four stages, complete the gameplay appearance, situation construction, interaction logic and interface improvement, and generate a 3D gray box game that can run independently. Today, EdenGolem is compatible with puzzle solving, shooting, driving, adventure, arena, fighting and many other game types.

From One Sentence to a Playable World: VAST Open-Sources the Entire Pipeline

In addition to generating based on simple materials, EdenLive can also understand realistic materials and achieve conversions. Like "shining step" is a real person teaching, while retaining the composition and mirror, the characters, pictures and brush strokes are animated. The model regenerates each frame along the motion and spatial relationship of the original video, without copying the texture of the original video.

There is another use that is easily overlooked: embodied data augmentation.

In robot training, there is no lack of knowledge of "how humans wipe the table". What is lacking is the training material of "how robots approach objects, grasp objects and complete movements from the robot's eyes".

It's not hard to get a similar perspective, find a person wearing a head-mounted camera, wipe the table, open the drawer, pour water, you can take a lot of photos in a day. However, the human arm and five fingers appear in the live-action video, while the robot deploys to see the robotic arm and claws, which have different shapes, range of motion, and occlusion methods. If the model is mainly trained with live video, it may encounter significant visual distribution differences when it comes to the robot camera.

Use EdenLive to replace a real person's hand with a robotic hand, thus converting the material into a data picture that is closer to what the robot will see when it actually works. For example, the action of a person picking up a kettle, the relative position of the kettle to the table top, and the movement of the first-person camera are all retained, and only the actuator is changed to a robotic arm, which can assist the model to adapt to the "picture seen by the robot".

Image

Different reference material types have different effects, and they also have different limitations. Ready-made video can only give shots and actions. After re-rendering, it is still a video, and playability comes from upstream code. However, EdenLive has a set of division of labor and rules behind it, so the generated results can continue to change with the operation.

With these demos, it's easy to assume that the player is directly manipulating a video model. In fact, EdenLive does not judge what is happening in the game, nor is it responsible for making the world run. In Project Eden released in May this year, vast divided the world model research into three layers of architecture: the continuously evolving world state, which records the position and properties of objects, interaction rules and the consequences of actions; the state-to-image interface, which converts the current state into layout, geometry and event constraints under a certain perspective; and the generative renderer, which generates high-fidelity images based on these constraints. EdenLive achieves the last two layers and does not limit the first layer.

Taking a fighting game as an example, the player presses the attack key, and the command still enters the upstream interactive platform first. It determines whether this foot has hit the opponent, how much blood should be deducted, and then updates the status and blood bars of the two characters.

Video models are good at generating images that look reasonable, but cannot independently maintain rules such as blood volume, collision, object identity, and win-lose conditions, so EdenLive leaves the deterministic state to the upstream interaction platform and hands the visual presentation to the generative model.

The interactive platform renders each change into a new gray-box image, from which a specially fine-tuned visual encoder extracts layouts, shots, and movements; it does not require delicate skin, clothing, and lighting, as long as the spatial layout, movement trajectory, and how the lens moves are clearly explained.

"Rules are attributed to the interactive platform, and pictures are attributed to the model". This set of division of labor addresses the problem that video models are not good at preserving rules for a long time, and ensures the stable presentation of visual elements. Of course, this division of labor is still subject to the test of longer and more complex interactions.

Image

A review by Oxford Scholars in January 2026 listed "visual confusion" as a challenge for the world model: realistic pictures do not mean that the model understands cause and effect and physical laws, and advocated that the world model should shift from the visual engine that generates pictures to an operable simulator that can represent dynamics and constraints.

When this route has not yet become the mainstream narrative of the world model, Project Eden released by vast in May this year has written the interactive platform to maintain the state of the world and the generative model is responsible for visual presentation into the complete architecture; a few months later, EdenLive has promoted the generative rendering to real-time and continuous operation.

In the market, similar ideas began to emerge in August and September.

The open source covers 5B and 14b model weights (each including two-way generation and less-step generation), real-time streaming inference code, EdenGolem v0.1, and front-end workbench, and discloses process reports. Developers can access their own reference sources to measure latency, stability, and reference compliance.

Along with the release of EdenLive, there is the V · star Top Talent Program for global young algorithmic and research talents. Recruitment directions cover three directions: 3D generation model, 3D understanding, Agentic AI, and world model. Whether they are full-time or interns, V · star will have the opportunity to directly participate in core projects and get a more competitive salary than the top manufacturers, especially the best ones, and long-term options.

Past world model demonstrations often use a video to prove that the model can imagine a world. EdenLive takes a critical step forward. And as more people incorporate their own codes, situations, and reference frames, craftsmanship will gradually evolve into a truly usable creative workflow, and the world will usher in more possibilities.

Model weight: https://huggingface.co/VAST-AI/EdenLive

Process Report: https://assets.vast-eden.ai/assets/edenlive/EdenLive-tech-report.pdf

Image

Unbound Loop: http://loop.unbound.io

Participate in the V · star Top Talent Program by sending your CV to vstar.talent@vastai3d.com

This article is from: https://www.ifanr.com/1685059 For copyright issues, please write to: 13951431913#139.com or submit a review on the right.

ImageImageImageImageImageImage

暂无评论,快来发表第一条评论吧!No comments yet. Be the first!

📮 需求咨询📮 Inquiry