渲染管线基础:保留模式、脏矩形与合成Rendering Pipeline Fundamentals: Retained Mode, Dirty Rects and Compositing


AI free token/AI免费积分

渲染管线基础:保留模式、脏矩形与合成

本文来源:https://parc.yolo.scapegoat.dev/note/research/kb/fundamentals/rendering-pipeline-fundamentals
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。

我们构建的每一个显示系统背后的架构模式:维护一棵视觉元素树,计算布局,跟踪哪些区域发生了变化(脏矩形),合并这些区域以尽量减少显示更新,并且只渲染发生变化的部分。本条目涵盖了支撑我们的电子墨水屏、LCD 和热敏打印渲染工作的理论基础。

渲染管线将应用状态(数据、时间、用户输入)转换为显示屏上的像素。对于资源受限的显示屏(电子墨水屏、小型 LCD、热敏打印机),管线必须尽量减少写入的像素数量,因为每次写入的代价都很高——电子墨水屏的局部刷新每个区域需要 50–200 毫秒,串行 LCD 受带宽限制,而热敏打印头一次只能加热一行。

该管线包含四个阶段,它们出现在我们的每一个显示项目中:

布局(Layout):计算每个视觉元素的位置和尺寸。

更新(Update):应用数据变更,将受影响的元素标记为脏(dirty)。

收集 + 合并(Collect + Merge):收集脏矩形,并将重叠/相邻的矩形合并。

渲染 + 刷新(Render + Refresh):只重绘合并后的脏区域,并将其发送到显示屏。

入门路径:电子墨水屏显示驱动——电子墨水屏的局部刷新是脏矩形优化的经典用例。全刷新很慢(300 毫秒–1 秒)并且会产生可见的闪烁。局部刷新很快(50–200 毫秒)但会累积残影。管线必须在两者之间做出选择。

部落知识:ESP-IDF 固件模式——我们的固件渲染遵循这一四阶段模式。Gnosis 显式地使用了它;SToMS3R 则隐式地使用(浏览器负责布局,ESP32 负责“渲染”阶段)。

如果我们不把这套管线记录为一种可复用的模式,那么我们构建的每一个显示项目都会重新发明它。

带有脏矩形跟踪的保留模式渲染是资源受限显示屏的最优架构。其关键特性在于:系统维护着视觉树的持久化模型(即“保留”模式),因此能够计算出帧与帧之间发生变化的最小像素集合。

这不同于立即模式渲染(许多游戏 UI 和终端模拟器使用),后者在每个周期都从头重绘整帧。立即模式实现起来更简单,但每帧都会发送每一个像素——这对 60Hz 的 LCD 尚可接受,对电子墨水屏和热敏打印机则不可接受。

想象一下,你正在一块白板上画一个仪表盘,上面有一个时钟、一个仪表和一个标签。每一秒,时钟都在变化,仪表的指针都在移动。

立即模式:擦掉整块白板,把一切重新画一遍。简单,但你擦除并重画了 95% 未发生变化的内容。

带脏矩形的保留模式:只擦除并重画时钟数字和仪表指针,其余部分保持不变。更快,但你需要确切知道时钟和仪表占据了哪些像素。

脏矩形系统就是“确切知道哪些像素发生了变化”的那部分。在基于树的布局中:

每个节点都有一个边界矩形(由布局阶段计算得出)。

当某个节点的值发生变化时,其边界矩形会被标记为脏。

相邻的脏矩形会被合并(两个重叠的 10×10 矩形合并为一个 15×15 的矩形——比两个独立的矩形产生更少的显示更新)。

只重绘并刷新合并后的脏区域。

PaperS3 上的 Gnosis 引擎正是采用了这种方法。Loupedeck 驱动在它的 60×360 LCD 上使用这一方法。reMarkable 的管线将它用于文档渲染。

Gnosis 的前身(固件 0075–0077):使用临时的、即席的脏跟踪——特定的方法各自知道自己负责的屏幕区域。添加一个按钮意味着要在四处修改坐标。移动一个面板意味着要连锁式地重新计算。布局引擎用基于树的位置和自动脏跟踪取代了这种做法。

当只有时钟发生变化时对电子墨水屏进行全屏刷新:没有脏矩形跟踪时,唯一安全的选择就是刷新整个屏幕。这需要 800 毫秒以上,并会产生可见的闪烁。有了脏矩形跟踪,只需 80 毫秒刷新时钟区域,且没有闪烁。

热敏打印机因光栅打印中途暂停而产生的条带:“脏区域”的概念在打印机层面同样适用。如果你把一幅位图拆分成多条光栅命令(即分带打印),那么带与带之间的接缝就会成为可见的瑕疵——带间走纸与点阵网格无法完美对齐。解决办法是:将整幅位图作为一条连续的光栅命令发送(即“整幅缓冲”模式)。

《Gnosis Layout Engine for PaperS3 - A UI Operating System on E-Ink》(PaperS3 的 Gnosis 布局引擎——电子墨水屏上的 UI 操作系统)——电子墨水屏上带脏矩形优化的四阶段管线

《Loupedeck Live Hello World - Serial Go Driver》(Loupedeck Live Hello World——串口 Go 驱动)——60×360 LCD 上带区域合并的渲染调度器

Skala, A.(2019)《Rendering and Compositing》(渲染与合成)——对 UI 框架中保留模式渲染、脏矩形和合成的实用讲解。

Gnome/GTK 文档——GTK 的渲染模型是带有脏区域跟踪的保留模式。其架构文档详细解释了这一四阶段管线。

本 PARC 资料库中的 Gnosis Layout Engine 项目报告——我们在 ESP32-S3 电子墨水屏上对四阶段管线的实现。

Rendering Pipeline Fundamentals: Retained Mode, Dirty Rects and Compositing

本文来源:https://parc.yolo.scapegoat.dev/note/research/kb/fundamentals/rendering-pipeline-fundamentals
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。

The architectural pattern behind every display system we build: maintain a tree of visual elements, compute layout, track which regions changed (dirty rectangles), merge those regions to minimize display updates, and render only what changed. This entry covers the theory that makes our e-ink, LCD, and thermal rendering work.

A rendering pipeline transforms application state (data, time, user input) into pixels on a display. For constrained displays (e-ink, small LCDs, thermal printers), the pipeline must minimize the number of pixels written because each write is expensive — e-ink partial refresh takes 50–200ms per region, serial LCDs are bandwidth-limited, and thermal printheads heat one row at a time.

The pipeline has four stages that appear in every one of our display projects:

Layout : Compute the position and size of every visual element.

Update : Apply data changes, mark affected elements as dirty.

Collect + Merge : Gather dirty rectangles and merge overlapping/adjacent ones.

Render + Refresh : Redraw only the merged dirty regions and send them to the display.

On-Ramp: E-Ink Display Driving — E-ink partial refresh is the canonical use case for dirty-rect optimization. Full refresh is slow (300ms–1s) and causes visible flashing. Partial refresh is fast (50–200ms) but accumulates ghosting. The pipeline must choose between them.

Tribal: ESP-IDF Firmware Patterns — Our firmware rendering follows this four-stage pattern. Gnosis uses it explicitly; SToMS3R uses it implicitly (the browser does layout, the ESP32 does the "render" stage).

Every display project we build reinvents this pipeline unless we document it as a reusable pattern.

Retained-mode rendering with dirty-rectangle tracking is the optimal architecture for constrained displays. The key property: the system maintains a persistent model of the visual tree (the "retained" mode), so it can compute the minimal set of pixels that changed between frames.

This is distinct from immediate-mode rendering (used by many game UIs and terminal emulators), where the entire frame is redrawn from scratch each cycle. Immediate mode is simpler to implement but sends every pixel every frame — acceptable for 60Hz LCDs, unacceptable for e-ink and thermal printers.

Think of a whiteboard where you're drawing a dashboard with a clock, a gauge, and a label. Every second, the clock changes and the gauge needle moves.

Immediate mode : Erase the entire whiteboard and redraw everything. Simple, but you erase and redraw 95% unchanged content.

Retained mode with dirty rects : Only erase and redraw the clock digits and the gauge needle. Everything else stays. Faster, but you need to know exactly which pixels the clock and gauge occupy.

The dirty-rect system is the "knowing exactly which pixels changed" part. In a tree-based layout:

Each node has a bounding rectangle (computed by the layout pass).

When a node's value changes, its bounding rectangle is marked dirty.

Adjacent dirty rectangles are merged (two overlapping 10×10 rects become one 15×15 rect — fewer display updates than two separate ones).

Only the merged dirty regions are redrawn and refreshed.

The Gnosis engine on PaperS3 uses exactly this approach. The Loupedeck driver uses it for its 60×360 LCD. The reMarkable pipeline uses it for document rendering.

Gnosis predecessor (firmware 0075–0077) : Used ad-hoc dirty tracking — specific methods knew their own screen regions. Adding a button meant changing coordinates in four places. Moving a panel meant cascading arithmetic. The layout engine replaced this with tree-based positions and automatic dirty tracking.

Full-screen refresh on e-ink when only a clock changed : Without dirty-rect tracking, the only safe option is to refresh the entire screen. This takes 800ms+ and causes a visible flash. With dirty-rect tracking, only the clock region is refreshed in 80ms with no flash.

Thermal printer banding from mid-raster pauses : The "dirty region" concept applies at the printer level too. If you split a bitmap into multiple raster commands (banding), the seam between bands is a visible artifact — the paper feed between bands is not perfectly aligned with the dot grid. The fix: send the entire bitmap as one continuous raster command (the "full-body buffer" pattern).

Gnosis Layout Engine for PaperS3 - A UI Operating System on E-Ink — four-stage pipeline on e-ink with dirty-rect optimization

Loupedeck Live Hello World - Serial Go Driver — render scheduler with region coalescing on 60×360 LCD

Skala, A. (2019) . Rendering and Compositing . — Practical coverage of retained-mode rendering, dirty rectangles, and compositing for UI frameworks.

Gnome/GTK documentation — GTK's rendering model is retained-mode with dirty-region tracking. The architecture documents explain the four-stage pipeline in detail.

Gnosis Layout Engine project report in this PARC library — Our implementation of the four-stage pipeline on an ESP32-S3 e-ink display.

暂无评论,快来发表第一条评论吧!No comments yet. Be the first!

📮 需求咨询📮 Inquiry