留住记忆:电子墨水相册折腾记Preserve Memories: Tinkering with an E-Ink Photo Frame


AI free token/AI免费积分

本文来源:https://mp.weixin.qq.com/s/qMz9hnclmHXWzNp1zsU43A
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。

我们拍了很多照片,往往它最高光的时刻就是按下快门的那一瞬间,此后便长眠于我们的手机或相机存储卡中。iPhone 的推荐时常给我惊喜和感动,把我尘封的记忆扰动,可是我们是否可以“让美好持续发生”?

初衷:我就是想要

如我博客的副标题:那些岁月,如果不记录,就像风一样飘逝了,然后了无痕迹,仿佛未曾发生。

我一直有个比较“强迫”的习惯,记录一些瞬间——不管未来用不用得上。这是我从大学到毕业不停买相机的原因,老实说拍照理论学了一通,但摄影水平有限。所幸生活并不全是艺术,真实的瞬间便足够回味。于是手机相册常常满满的,iCloud 得开通 2TB 会员才能容纳。

我也时常分享一些有趣或有意义的照片给家人,当有了娃后,特别是当父母不在身边时,他们只能从我零星发的照片里看到孙辈,我在想我或许需要灵活的相册,我也不用那么主动和刻意的分享,有一个相册他们随时可以看到。你或许会说,这不是群相册、QQ 相册之流就可以解决的吗?遗憾的是老人上了年纪较难主动获取这些信息了,有了门槛效果就没有多大意义了。

原本我还在想如何解决此问题,直到某天在 B 站看到 老戴Donald 的这个视频。我毫不犹豫地下单了。

硬件:成品它不香

起初我打算买零散部件自己在家焊,再次锻炼一下金工能力,但挑选再三发现各个部件的组装成本远高于直接买成品硬件,于是就直接在微雪下单了一款。钱的差额是小事,主要担心缺少个别部件到时反复折腾等待快递就破坏了美妙的心情,当时时间还是五一放假前夕(没错,又拖更了很久),于是果断先成品折腾。 它是 ESP32-S3 7.3寸 E6 全彩色电子墨水屏,看画面显示效果还不错,诸位别被骗了,放大了看,颗粒度还是清晰的,毕竟 PPI 摆在那里。

硬件成品
硬件成品

当然我不是来带货的,直接买了成品降低了我折腾的乐趣,硬件咱就没得玩了。下单后第二天就到了,看了一下发货地深圳,果然,深圳速度~

软件:私人化定制

收到货物后,官方给的固件集成了几个模式,还带了小智对话模式,我之前也折腾过小智机器人(见让你的小智AI机器人起飞),有种莫名的熟悉感。 但是带的相册功能和上面老戴的那种 Feel 就差一大截了,仅有图片轮播等基础玩法,既不文艺也不艺术。这一点都不慌,啥年代了,不就是写一些驱动、编译固件、刷机嘛?

❯改造一:迁移 InkTime

人家已经开源了InkTime,让 AI “借鉴”一下是很快的。我先在原固件上添加了一个新模式,将 InkTime 的代码集成进去。老戴的模式中特别亮点的就是 AI 根据图片的内容评分和打标签,而我是不想将大量图片做贡献给云端大厂们作训练素材的。“众所周知”,我有一台 Mac Mini M4,这正是它发挥的时候。用 LMStudio 再召唤 qwen3.5 9B,效果已经不错了。虽然说处理一张图片大概要 50-90 秒,但这个设备功耗极低,又保障了隐私,咱们就忍着点,图不出户才是最重要的:)

LMStudio
LMStudio

迁移完可以用了,测试了一下效果,图片筛选和文案看了后表示满意:

InkTime 效果
InkTime 效果

不过原来的架构是这样的:

InkTime原架构
InkTime原架构

这里有一个比较严重的问题,server 需要绑定在 NAS 侧获取图片再渲染,这样就不方便我们将电子相框带到外面去时也可以查看图片。所以我们需要调整架构,使图片处理在内网,图片分享拉取在外网:

云端一体架构
云端一体架构

当然我们的服务在拉取图片时需要支持身份认证,这里我让服务返回了 COS 的预签名链接,使相框固件拿到链接后便可直接安全拉取,保护隐私。

❯改造二:支持 iPhone 相册追踪

原项目中假定了我们 NAS 中的图片都需要分享和唤起,它负责可以扫描某个目录进行处理。而我还有一个场景就是最近发生的事情(照片),隔几天会冒出来让你回忆下。我希望我可以将某些有意义的照片放到可被追踪的相簿中,我家人也可以随时分享它觉得有意义的照片,当满足一定条件,它会自动出现在相框中。这显然是 iPhone 共享相簿擅长的场景,怎么整合进来呢?

“众所周知”(你现在应该知道了),我有一台 Mac Mini M4。我找到一个开源软件,用 osxphotos 将 macOS 照片 app 中的 iCloud 共享相簿导出到本地目录,再通过 rsync 同步到我们的图片处理服务中(调用 VLM 识图及筛选),这个功能我是放在 NAS 上的。为了能够比较及时的感知到相簿的变化,可通过 launchd 定时后台同步。

然后我们的图片处理服务负责监听目录下的文件变化,有新文件过来就会交给VLM进一步处理。

这些工具我都设计为一个个小仓库,方便后续的维护和扩展。本来想整理一下放 GitHub 的,但现在 AI 时代,如果你要折腾也很容易搞定,有需要的朋友欢迎关注我,给我留言。

❯改造三:丰富 AI 生图

我家娃年幼无知,沉迷奥特曼打怪兽,所以给小智扩展了一下,让它对接国内几个图像生成和编辑的模型,这里我先选择火山引擎豆包的模型(seedream 5.0/4.5),因为有免费额度,而且效果也不错,最重要的是速度还比较快(我之前有篇文章国内外生图编辑图片横评对比过各个生图效果,完全可以接其它模型)。

于是我在娃面前说,帮我生成迪迦在大海中对战一个章鱼怪兽,在一小会等待之后,屏幕忽闪忽闪刷新后,一张让娃兴奋的图片出现了。

小智语音绘图
小智语音绘图

因为支持了编辑能力,我们还可以让它继续修改,在小智 AI 的帮助下,娃也可以自己通过描述来定制自己想要的图片了。我看着他好奇又开心的样子,或许科技本该让人如此愉悦。

思考:AI 擅长做,你自己才是大脑

在当前阶段,AI去实现功能已经很快捷了,并且基本你只要描述清楚了,做出来返工都比较少。这并不意味着你就废了,以这个项目为例,有几点可以值得聊聊的。

❯云控

原来固件中有一些配置,它是放在 SD 卡中再加载的,这样不方便调整。在几次拆卡读卡后,我便难以忍受了。我把配置重新梳理了一下,让 SD 卡中只有最基本的配置,只要达到联网能力和找到我们的 API 即可,其它都通过网络拉取。同时为了减少 SD 卡的读取,WiFi 等 SSID 信息我们直接保存到 ESP32 的 NVS 中。

{  "wifi_ssid": "wifi1;wifi2",  "wifi_password": "password1;password2",  "wifi_override": true,  "publish_service_url": "https://example.com",  "publish_device_id": "device-001",  "publish_token": "token-001"}

不过这样运行中当前到底跑的是什么配置,最新配置是否生效了呢?我们不妨来下个调试界面:

调试界面
调试界面

❯通用图片展示框

既然我们已经在云端进行渲染和图片管理,有这样一套统一的管理和展示逻辑,那么也不再局限于只展示我们特定照片了。有云端 API 后,我让 AI 将其创建了一个skill,然后我们可以随时将要展示的信息让各个 Agent 可以调用在它上面呈现。

我想最近关注 Token 消耗比较多,到底订阅的套餐被我们薅了多少本回来呢?

Token周消耗监控
Token周消耗监控

当每天早上醒来时,看到你的 Token 用量,是不是又有冲动继续折腾 AI 了呢?

电子相框上的Token监控
电子相框上的Token监控
这个 Token 统计支持多机汇总,使用了开源方案 openusage。当时看界面不错,支持的 Agent 比较多,于是给它扩展了一下多机汇总的 Hub 功能,作者很给力也很认真 Review PR,已经合入了。

后记

当然还有不少折腾的细节,限于篇幅不再展开了。这个项目当前比较稳定运行,一个相框静静放在那里,时不时有一些小惊喜。我在想是放在每天出入的门口,家庭的温馨程度能否+1。

但从技术上,还是有些遗憾:涉及部分硬件的事情,时常要烧录固件再测试,为了降低调试成本,除了给项目的核心功能添加完善的测试用例外,我尝试在本地搭建一个模拟器,希望打通全流程这样 AI 可以自闭环。遗憾的是因为过往经验较少,加上这里的电路等还没研究清楚,这个想法最终没有成功。

如果你有更好的点子,欢迎留言交流。如果你懂上面的硬件模拟,欢迎指导我一下,谢谢。

PS:相关代码暂时躺在 Github 私有仓库中,若你有需要,给我留言。记得点赞、关注哟。

我是个爱折腾技术的工程师,也乐于分享。欢迎点赞、关注、分享,更欢迎一起探讨技术问题,共同学习,共同进步。为了获得更及时的文章推送,欢迎关注我的公众号:爱折腾的风

 

Source of this article: https://mp.weixin.qq.com/s/qMz9hnclmHXWzNp1zsU43A. If there are copyright issues, please inform us via email: 13951431913#139.com or submit a comment on the right.

We take many photos, and often their most brilliant moment is the instant the shutter is pressed, after which they lie dormant in our phone or camera memory cards. iPhone Recommendations often bring me surprises and moving moments, stirring up my dusted memories, but can we "let beauty continue to happen"?

Original Intention: I Just Want It

As the subtitle of my blog says: Those years, if not recorded, drift away like the wind, leaving no trace, as if they never happened.

I've always had a somewhat "compulsive" habit of recording moments—regardless of whether they'll be useful in the future. This is the reason I kept buying cameras from university until graduation. Honestly, I learned a lot about photography theory, but my photography skills are limited. Fortunately, life isn't all about art; real moments are enough to savor. So my phone album is often full, and I have to subscribe to a 2TB iCloud membership to accommodate it.

I also frequently share some interesting or meaningful photos with my family. After having a child, especially when my parents are not around, they can only see their grandchildren from the sporadic photos I send. I thought perhaps I need a flexible album, so I don't have to share so actively and deliberately; there should be an album they can view at any time. You might say, can't this be solved by group albums or QQ albums? Unfortunately, elderly people find it difficult to actively access this information due to their age; with a threshold, the effect loses much of its meaning.

I was originally thinking about how to solve this problem until I saw this video by 老戴Donald (Lao Dai Donald) on Bilibili one day. I ordered it without hesitation.

Hardware: Finished Products Are Not Fragrant

At first, I planned to buy scattered components to solder at home, exercising my metalworking skills again. However, after careful selection, I found that the assembly cost of individual components was far higher than buying finished hardware directly, so I ordered a model directly from Weixue. The price difference is a minor issue; the main concern was that missing individual components would lead to repeated hassle and waiting for delivery, ruining the wonderful mood. At that time, it was just before the May Day holiday (yes, another long delay in updates), so I decisively started with the finished product. It is an ESP32-S3 with a 7.3-inch E6 full-color electronic ink screen. The display effect looks quite good, but don't be deceived; when zoomed in, the granularity is still clear, after all, the PPI is what it is.

Of course, I'm not here to sell anything. Buying the finished product directly reduced my fun of tinkering, so there's no hardware playing for us. It arrived the day after I ordered it. I checked the shipping origin: Shenzhen. Indeed, Shenzhen speed~

Software: Personalized Customization

After receiving the goods, the official firmware integrated several modes and included the Xiao Zhi dialogue mode. I had also tinkered with the Xiao Zhi robot before (see "Let Your Xiao Zhi AI Robot Take Off"), so there was a strange sense of familiarity. However, the included album function was a far cry from the "feel" of Lao Dai's version mentioned above, offering only basic features like image slideshow, neither literary nor artistic. No need to panic; what era is it? It's just writing some drivers, compiling firmware, and flashing, right?

❯ Modification 1: Migrate InkTime

InkTime is already open source, so letting AI "borrow" from it is very fast. I first added a new mode to the original firmware, integrating the InkTime code. The particularly bright spot in Lao Dai's mode is AI scoring and tagging images based on content, but I don't want to contribute a large number of images to cloud giants as training material. "As everyone knows," I have a Mac Mini M4, and this is exactly when it comes into play. Summoning qwen3.5 9B via LMStudio yields good results. Although processing one image takes about 50-90 seconds, this device has extremely low power consumption and protects privacy, so we just bear with it. Staying at home is the most important: )

After migration, it's ready to use. I tested the effect, and the image screening and copy were satisfactory:

However, the original architecture was like this:

Here is a relatively serious problem: the server needs to be bound to the NAS side to fetch images and then render them, making it inconvenient to view images when we take the electronic photo frame outside. So we need to adjust the architecture so that image processing happens on the intranet, while image sharing and fetching occur on the extranet:

Of course, our service needs to support identity authentication when fetching images. Here, I had the service return a COS pre-signed link, allowing the frame firmware to fetch directly and securely after obtaining the link, protecting privacy.

❯ Modification 2: Support iPhone Album Tracking

The original project assumed that all images in our NAS needed to be shared and invoked, responsible for scanning a certain directory for processing. I have another scenario: recent events (photos) that pop up every few days to make you reminisce. I hope I can place certain meaningful photos into a trackable album, and my family can also share photos they find meaningful at any time. When certain conditions are met, they will automatically appear on the frame. This is clearly a scenario where iPhone Shared Albums excel. How to integrate it?

"As everyone knows" (you should know now), I have a Mac Mini M4. I found an open-source software, using osxphotos to export iCloud Shared Albums from the macOS Photos app to a local directory, then syncing them to our image processing service via rsync (calling VLM for image recognition and screening). I placed this function on the NAS. To perceive changes in the album relatively timely, we can use launchd for scheduled background syncing.

Then our image processing service is responsible for listening to file changes in the directory; when a new file arrives, it will be handed over to the VLM for further processing.

I designed all these tools as separate small repositories to facilitate future maintenance and expansion. I originally wanted to organize and put them on GitHub, but in the current AI era, if you want to tinker, it's also very easy to handle. Friends in need are welcome to follow me and leave me a message.

❯ Modification 3: Enrich AI Image Generation

My child is young and ignorant, addicted to Ultraman fighting monsters, so I extended Xiao Zhi a bit, letting it connect to several domestic image generation and editing models. Here, I first chose the model from Volcengine's Doubao (seedream 5.0/4.5) because it has a free quota, the effect is good, and most importantly, the speed is quite fast (I previously wrote an article "Comparison of Domestic and Foreign Image Generation and Editing" comparing various generation effects, so it can definitely connect to other models).

So I said in front of my child, "Help me generate an image of Tiga fighting an octopus monster in the sea." After a short wait, the screen flashed and refreshed, and an image that excited my child appeared.

Because editing capabilities are supported, we can also let it continue modifying. With the help of Xiao Zhi AI, my child can also customize the images they want through descriptions themselves. Watching his curious and happy expression, perhaps technology is supposed to make people feel this joyful.

Reflection: AI is Good at Doing, But You Are the Brain

At the current stage, AI implementing functions is already very fast, and basically, as long as you describe it clearly, the amount of rework is minimal. This doesn't mean you're useless. Taking this project as an example, there are several points worth discussing.

❯ Cloud Control

The original firmware had some configurations stored on the SD card and then loaded, which was inconvenient to adjust. After several times of removing the card and reading it, I could no longer bear it. I reorganized the configuration so that the SD card only holds the most basic settings, just enough to achieve network connectivity and find our API, with everything else fetched via the network. At the same time, to reduce SD card reads, WiFi SSID information is directly saved in the ESP32's NVS.

However, during operation, how do we know what configuration is currently running and whether the latest configuration has taken effect? Let's add a debugging interface:

❯ Universal Image Display Frame

Since we are already rendering and managing images in the cloud, with such a unified management and display logic, we are no longer limited to displaying only our specific photos. After having a cloud API, I had AI create a skill for it, allowing us to have various Agents call and present information to be displayed on it at any time.

I've been paying a lot of attention to Token consumption recently. How much of the subscribed package have we actually saved back?

When you wake up every morning and see your Token usage, do you feel the urge to continue tinkering with AI?

This Token statistics feature supports multi-device aggregation, using the open-source solution openusage. At the time, I thought the interface was good and it supported many Agents, so I extended it with a multi-device aggregation Hub function. The author was very helpful and seriously reviewed the PR, and it has already been merged.

Postscript

Of course, there are many more details of tinkering, but due to space limitations, I won't expand on them. This project is currently running quite stably. A photo frame sits quietly there, occasionally offering some small surprises. I'm wondering if placing it at the entrance we pass through daily can increase the family's warmth by +1.

However, from a technical perspective, there are still some regrets: involving some hardware matters, I often have to flash firmware and test again. To reduce debugging costs, besides adding comprehensive test cases to the core functions of the project, I tried to set up a simulator locally to hope for a full-process loop so AI can close the loop autonomously. Unfortunately, due to limited past experience and the fact that I haven't fully researched the circuits here yet, this idea ultimately did not succeed.

If you have better ideas, welcome to leave a message for discussion. If you understand the hardware simulation above, welcome to guide me, thank you.

PS: The related code is temporarily lying in a private GitHub repository. If you need it, leave me a message. Remember to like and follow.

I am an engineer who loves tinkering with technology and is also happy to share. Welcome to like, follow, and share. Even more welcome to discuss technical issues together, learn together, and progress together. To receive more timely article pushes, welcome to follow my WeChat official account: 爱折腾的风 (Love Tinkering Wind).

硬件成品

LMStudio

InkTime 效果

InkTime原架构

云端一体架构

小智语音绘图

调试界面

Token周消耗监控

电子相框上的Token监控

暂无评论,快来发表第一条评论吧!No comments yet. Be the first!

📮 需求咨询📮 Inquiry