feat: add layered bedroom motion review sample #1

Open
leefer wants to merge 2 commits from leefer/hel-711-layered-bedroom AGit into main
16 changed files with 2166 additions and 0 deletions
Showing only changes of commit 444ee56c6a - Show all commits
+7
View File
@@ -0,0 +1,7 @@
node_modules/
dist/
public/assets/*
!public/assets/manifest.json
evidence/*
!evidence/.gitkeep
__pycache__/
+39
View File
@@ -0,0 +1,39 @@
HEL-711 最小动态校对小样,基于 main `1e9b08a`。仅在 `samples/bedroom` 内工作;不连接人格、记忆、天气或语音。采用 HEL-708 的环境/人物/交谈分层边界,以及 CG-001~026 的当前需求。旧演示不变。
## 本地复核
1. 将本卡附件 `HEL-711-review-bundle.zip` 中的 `public/assets` 解压到此目录对应位置。
2. `npm ci --ignore-scripts --no-audit --no-fund`,沿用旧前端分支的 package.json/package-lock.json/tsconfig.json,无新依赖。
3. `npm run build`(包含严格 TypeScript 检查)。
4. `npm run dev -- --port 5173`,本机浏览器打开终端输出的地址。
5. `python3 tools/browser_check.py` 使用已安装的 Playwright/Chromium,自动启停本机测试服务并保存证据。此运行机需要给该命令设置 `LD_LIBRARY_PATH=/home/xiaomo/pwlibs/root/usr/lib/x86_64-linux-gnu`,复用了已有浏览器库,未安装系统依赖。
素材不进 Git,缺失时页面明确提示;manifest 在 Git 中保存来源、原图 SHA256、坐标、编码和问题。附件内含完整 assets、已构建 dist、浏览器连续录屏、原始试片、透明试片、分层校对图及生成工作流/日志。
## 代码与边界
- `src/main.ts`:一个持久 video 元素驱动透明 canvas;环境按钮只改环境亮度,不重建/seek 视频。人物、前景遮挡、接触阴影独立开关;静态近景不替代真实交谈。
- `src/tokens.css`:原样复用旧前端唯一 token 表。`style.css` 为独立工程校对台,无第二套同类组件。
- `tools/extract_layers.py`:读取现有 SAM vit-b 模型的原始 modeling API,避免可选 predictor 的 OpenCV 依赖;不伪造 cv2,不下载。原图 RGB 保留,只生成 alpha;封闭孔洞填充、0.65px 羽化。
- `tools/contact_layers.py`:从原入景图提取毯子前景,另建可关闭的暂定接触阴影;未宣称物理阴影校准完成。
- `tools/generate_once.py`:原 ComfyUI venv + 原生 Wan 节点,单片段、离线、自定义节点禁用。临时服务器仅监听回环 18189,finally 终止自己启动的精确 PID。不修改现有服务、系统 Python、驱动或模型路径配置。
- `tools/matte_video.py`:49 帧逐帧 SAM 分割,原有 PyAV 编码 VP9 alpha。原始视频未抠像的背景不用于浏览器人物合成。
- `tools/browser_check.py`:真实 Chromium 检查、三种视口截图和可下载连续录屏。页面隐藏的事件处理做了合成 visibilitychange 检查;真实系统切后台/锁屏仍待体验环境手测。
## 素材处理命令
处理目录:`.12:/media/lucian/Downloads/ComfyUI/output/hel711-task`,输入复制到其 `assets`,脚本放此目录;没有改变现有项目输入。原图由 Multica 鉴权 attachment download 获取。运行时使用原 `/media/lucian/Downloads/ComfyUI/venv/bin/python`。
```bash
python extract_layers.py assets \
--sam-source /media/lucian/Downloads/ComfyUI/custom_nodes/was-node-suite-comfyui/repos/SAM/segment_anything \
--checkpoint /media/lucian/Downloads/ComfyUI/models/sams/sam_vit_b_01ec64.pth
python generate_once.py
python matte_video.py
```
回传处理后的素材到本地 `public/assets`,运行 `python3 tools/contact_layers.py`。手工蒙版和坐标均保留在脚本中。`workflow.json` 是本次实际运行的请求,最终脚本仅补了目录创建/日志关闭等清理逻辑;日志和 history 可交叉核对。执行新试片前必须重新检查显存和端口;禁止停止已有服务腾资源。
## 当前实测与未通过项
当前结果见 `REPORT.md`,属于可审核候选,不代表 D1–D6 全部通过。图层是真实独立合成;没有用重生成整幅图代替分层。近景仍用已确认 v2 静态方向图;没有真人语音或口型。
+52
View File
@@ -0,0 +1,52 @@
2026-10-20 HEL-711 审核记录
## 实际结果
已完成原始像素透明人物层、无人环境层、前景毯子/蒙版、暂定接触阴影、静态合成和独立开关。坐标系为 1672×941,固定右侧 552×552 区域(1120,200),人物未平移/缩放伪装动作。原参考和透明层在素材包中,可检查 alpha。
仅生成一段 Wan2.2 真实阅读动作:512×512 原始生成、49 帧、16fps、3.0625s;逐帧分割后恢复为 552×552 裁切区域,VP9 WebM alpha。人物从微歪头看向镜头变为低头读书,衣服和坐姿保持;这次没有实现再次抬头。四张连续取样帧与原始视频均交付,不用 RGB 差分独自证明人物动作。
实测推理日志 `Prompt executed in 57.18 seconds`,不包含环境启动、分层和浏览器录屏耗时。首尾 RGB 平均差约 11.998/255,仅是诊断数据。首尾姿态有明显差异,硬循环会跳变;没有反向播放伪造完整动作。
## .12 工具核查
- GPU RTX 5090 D,nvidia-smi 总显存 32607MiB;生成/抠像前后既有占用均为 5633MiB,未记录精确峰值,不能声称峰值或共存 SLA。磁盘可用约 754GiB。
- Desktop 安装 `/home/lucian/ComfyUI-Installs/ComfyUI` 和 `/home/lucian/ComfyUI-Shared` 确实存在,但 8188 未监听。
- 实际可运行安装为 `/media/lucian/Downloads/ComfyUI`,venv PyTorch `2.14.1+cu130`、torchvision `0.29.1+cu130`、PyAV `19.0.1`;缺 cv2/rembg/timm,系统 ffmpeg 不在 PATH。未安装任何依赖。
- 原有 models 内已存在 Wan2.2 I2V 高/低噪声 fp8 各约 14.29GB、UMT5 fp8 约 6.74GB、Wan VAE 约 254MB、对应 LightX2V LoRA 各约 1.23GB,以及 SAM vit-b 约 375MB。并非只见到文件就宣称可用:本次已实际跑通原生节点和单片段生成。
- 临时 ComfyUI PID 596473,日志在素材包 `evidence/server.log`,已由启动脚本 finally 停止;18189 已无监听。旧 18080 及 ASR/TTS 监听保持。没有驻留生成工作,没有部署。
- SSH 使用已确认的主机密钥及既有凭据;没有输出凭据。没有下载大模型、购买接口、修改开机启动或运行 GUI。
## 验证结果
- `npm run build`:通过,包含 TypeScript 严格检查;生产构建约 3.2KB CSS / 3.6KB JS(未 gzip)。
- `python3 -m pytest -q`:原有 78 项全部通过。
- `python3 -m ruff check src tests samples/bedroom/tools`:通过。
- 真实 Chromium 解码 alpha:背景像素 alpha=0、人物内部 alpha=255。
- 环境开关前后同一视频 source、同一 currentTime;实测样例 0.915222s→0.915222s。录屏包含人物播放、环境开关、无人画面、蒙版、阴影/前景开关和静态近景进入/退出。
- 1600×900、1280×800、390×844:无横向溢出、脸部在画面内,控制区不遮脸。手机以全画面包含方式展示,人物较小,未宣称手机沉浸效果验收通过。
- 暂停/恢复、减少动效切静态、动态素材加载失败回退静态:通过。浏览器 pageerror=0。
- 隐藏暂停/恢复处理经合成 visibilitychange 事件验证;headless 环境不能代表真实切后台/系统锁屏,仍需设备手测。未测 Safari/iOS 的 VP9 alpha 支持,解码失败会显式回退。
## 待修与验收边界
1. 循环首尾头部跳变,未建立可接受切点。需后续在获准后做闭环片段或经美术确认的自然选片衔接,本轮不批量重试。
2. SAM 是分割加羽化,不是精细发丝抠像;细发、针织边缘、书本遮挡可能闪烁/缺损,需逐帧美术复核。49 帧蒙版面积范围 49312~50563 像素,不能据此宣称边缘无抖动。
3. 原无人/有人参考沙发与抱枕几何略有差异。人物固定落位与独立遮挡已实现,残余靠背、接触边缘和前景拼接仍需美术校正。接触阴影是暂定独立工程层。
4. 小尺寸手部/书页可辨但不保证解剖正确;没有完成逐帧身份评分,脸部细节与跨镜头一致性待审。
5. 环境减光只验证人物/房间独立,人物本身未做对应重打光;按钮标注手动演示,未接真实城市天气。
6. 近景为 v2 静态切换,进退是即时校对切镜,衔接未润色;不能算真实交谈、语音或口型通过。
按 Emil review-animations 复核界面控制:
| Before | After | Why |
| --- | --- | --- |
| 自动循环可能让减少动效用户持续看到人物运动 | 默认手动播放,减少动效时暂停并显示静态层 | 用户控制和无运动回退已验证 |
| 近景或环境切换可能重新装载视频 | 保留唯一 video,仅切独立层的可见性/环境亮度 | 切换可打断,保持实际播放位置 |
| 片段首尾头姿不同 | 仍标注待修,不添加平移/缩放掩盖跳变 | 真实视频循环没有通过观感验收 |
界面播放控制 Approve;循环衔接与自然近景体验 Block,交总管/美术继续验收。本卡交付可审核的最小真实片段,不表示整体动态体验通过。
## 仓库
Multica checkout 返回仓库未配置,因此使用既有 SSH 权限直接 clone 指定仓库,建立独立 `hel-711-layered-bedroom`,基于明确指定 main `1e9b08a`。旧 `hel-700-rain-room` 未改动。`git ls-remote refs/pull/*/head` 无结果;SSH refs 不能证明私有 Gitea 没有 PR,未通过未知 API 凭据枚举 PR。此次不合并、不部署,提交号见本卡最终结果评论。
+38
View File
@@ -0,0 +1,38 @@
<!doctype html>
<html lang="zh-CN">
<head><meta charset="UTF-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>卧室分层校对 · HEL-711</title></head>
<body>
<main>
<header><h1>卧室分层校对</h1><p>工程小样 · 手动光照演示 · 近景仅静态参考,未接语音或口型</p></header>
<div id="stage" role="img" aria-label="固定机位卧室人物合成">
<div id="frame">
<img id="environment" src="./assets/environment.png" alt="无人高层卧室">
<img id="shadow" src="./assets/contact-shadow.png" alt="" class="layer">
<img id="person" src="./assets/person.png" alt="阅读人物透明层" class="layer">
<canvas id="motion" width="552" height="552" hidden></canvas>
<img id="occlusion" src="./assets/occlusion.png" alt="" class="layer">
<img id="mask" src="./assets/person-mask.png" alt="人物蒙版" class="layer" hidden>
<img id="talk" src="./assets/talk-reference.png" alt="交谈静态近景参考" class="layer" hidden>
</div>
</div>
<video id="source" muted playsinline loop preload="metadata" hidden></video>
<section aria-label="校对控制">
<fieldset><legend>图层开关</legend>
<label><input id="show-person" type="checkbox" checked>人物</label>
<label><input id="show-shadow" type="checkbox" checked>接触阴影</label>
<label><input id="show-occlusion" type="checkbox" checked>前景毯子</label>
<label><input id="show-mask" type="checkbox">仅看人物蒙版</label>
</fieldset>
<div class="controls">
<button id="lighting" type="button" aria-pressed="false">光照演示:原始</button>
<button id="play" type="button" disabled>读取动作素材</button>
<button id="closeup" type="button" aria-pressed="false">进入静态近景</button>
</div>
<p id="status" role="status">读取素材清单…</p>
<output id="time">动作位置:未载入</output>
<p id="fallback" role="alert" hidden></p>
<p>人物、房间和近景来自已确认的 v2 参考。图层独立不等于几何、发丝与循环已经通过验收。</p>
</section>
</main>
<script type="module" src="./src/main.ts"></script>
</body></html>
+1166
View File
File diff suppressed because it is too large Load Diff
+19
View File
@@ -0,0 +1,19 @@
{
"name": "cybergril-rain-room",
"private": true,
"version": "0.1.0",
"type": "module",
"scripts": {
"dev": "vite --host 127.0.0.1",
"typecheck": "tsc --noEmit",
"build": "tsc --noEmit && vite build",
"preview": "vite preview --host 127.0.0.1"
},
"dependencies": {
"pixi.js": "^8.0.0"
},
"devDependencies": {
"typescript": "~5.9.3",
"vite": "^5.4.0"
}
}
+138
View File
@@ -0,0 +1,138 @@
{
"schema": "hel711-review-1",
"status": "review_candidate",
"reference_sha256": {
"reference.png": "226d7922e042e22e4b8a8241a7cf6f7aea50c301029978e4ca26cad9b3d56359",
"environment.png": "b5d4771eb010dc1d0a41d1b698e45edaa45d2f90bb441378bc32cf5c812e76a2",
"talk-reference.png": "0ae9b1a1cb9c590ab89a81ce4efd25de843a600ff227c184663db23e9e9c16b3"
},
"camera": {
"canvas": [
1672,
941
],
"generation_crop": [
1120,
200,
552,
552
],
"person_anchor": [
1390,
530
],
"coordinate_units": "source pixels, top left origin",
"transform": "none; crop reinserted into original position",
"clothing": "v2 cream cardigan, charcoal trousers"
},
"layers": [
{
"file": "environment.png",
"source": "approved 02-v2 original",
"alpha": false
},
{
"file": "contact-shadow.png",
"source": "engineered ellipse 65/255, 12px blur, not physically calibrated",
"alpha": true
},
{
"file": "person.png",
"source": "original 03-v2 RGB + existing SAM vit-b alpha; enclosed mask holes filled",
"alpha": true
},
{
"file": "occlusion.png",
"source": "03-v2 blanket pixels, hand-authored polygon matte",
"alpha": true
}
],
"motion": {
"status": "真实生成;循环与抠像待修",
"file": "reading-alpha.webm",
"alpha": true,
"maskFile": null,
"original": "reading_00001_.webm",
"generator": "installed ComfyUI native Wan2.2 high/low 14B fp8 + installed LightX2V 4-step LoRAs",
"seed": 711,
"dimensions": [
552,
552
],
"generation_dimensions": [
512,
512
],
"frames": 49,
"fps": 16,
"duration_seconds": 3.0625,
"codec": "VP9 yuva420p / WebM",
"matting": "existing SAM vit-b, per frame, filled enclosed holes, 0.65px feather",
"start_pose": "seated holding open book, head slightly tilted toward camera",
"end_pose": "same seat and clothes, head down at book",
"loop": "hard loop at 49 frames; visible head jump, not seamless",
"cut_point": "none accepted"
},
"closeup": {
"file": "talk-reference.png",
"type": "static_reference_only",
"identity_verified": false,
"lipsync": false,
"source": "approved v2 static direction reference"
},
"known_issues": [
"Original empty/occupied sofa and pillow geometry differs; residual contact geometry needs art review.",
"SAM hard segmentation + feather is not hair matting; edge texture loss/flicker possible.",
"Book and fingers are small; hand anatomy/identity not certified.",
"Start/end head poses differ; hard loop jumps visibly.",
"Manual environment dimming changes background only; subject relighting not implemented.",
"Static near-shot is a generated reference; cross-shot face continuity not certified.",
"No real weather, speech, lipsync or life-state integration."
],
"files": {
"talk-reference.png": {
"bytes": 2246098,
"sha256": "0ae9b1a1cb9c590ab89a81ce4efd25de843a600ff227c184663db23e9e9c16b3"
},
"reading-alpha.webm": {
"bytes": 472707,
"sha256": "d28ef9a062744a8b362abfc6a4bc4d374a74128eea7c59d8441a4b77f7c14d98"
},
"person.png": {
"bytes": 2522355,
"sha256": "a904f4081069ba97019d3c679d8dd59102118560471f8d6337f7b29d60380f24"
},
"composite.png": {
"bytes": 2576324,
"sha256": "4354716558858d0e9ac9a4c4a025d77f4042a0e106f10fc37e500c7157e77456"
},
"person-mask.png": {
"bytes": 7264,
"sha256": "7569160b3c87ebf1d5bb66792ed6d2db6b41fde006fad0473e7db5337d18b0f7"
},
"environment.png": {
"bytes": 2442707,
"sha256": "b5d4771eb010dc1d0a41d1b698e45edaa45d2f90bb441378bc32cf5c812e76a2"
},
"occlusion-mask.png": {
"bytes": 4838,
"sha256": "dc22d1823a02df4fa5f3bf484f157c49b270f100185c0a35b3c31f28880dd3f0"
},
"generation-input.png": {
"bytes": 508812,
"sha256": "f3922903fb28c5940a2e272b70be6e33dc094039b4e275724cf3ff1aa4af1584"
},
"occlusion.png": {
"bytes": 2522991,
"sha256": "5f0f4aa7ae90ad863f694365eaa93d7d7558841ecca08c95d415731450226c4f"
},
"reference.png": {
"bytes": 2384917,
"sha256": "226d7922e042e22e4b8a8241a7cf6f7aea50c301029978e4ca26cad9b3d56359"
},
"contact-shadow.png": {
"bytes": 16144,
"sha256": "948f88c1298638158f9a21ffe107bf403106dcf6e58c72a3d6e738fe0c7cc003"
}
}
}
+141
View File
@@ -0,0 +1,141 @@
import './style.css';
type Manifest = {
motion: { status: string; file: string | null; alpha: boolean; maskFile: string | null };
};
const get = <T extends HTMLElement>(id: string) => document.getElementById(id) as T;
const video = get<HTMLVideoElement>('source');
const canvas = get<HTMLCanvasElement>('motion');
const context = canvas.getContext('2d')!;
const play = get<HTMLButtonElement>('play');
const person = get<HTMLImageElement>('person');
const status = get('status');
const reduced = matchMedia('(prefers-reduced-motion: reduce)');
const missingImages = new Set<string>();
let ready = false;
let wantsPlay = false;
let near = false;
let failed = false;
let frameHandle = 0;
function fallback(message: string) {
const node = get('fallback');
node.hidden = false;
node.textContent = message;
}
for (const image of document.querySelectorAll<HTMLImageElement>('#frame img')) {
image.addEventListener('error', () => {
missingImages.add(image.id);
image.hidden = true;
fallback(`素材缺失:${image.id}。请解压任务素材包到 public/assets;当前内容不能作为验收结果。`);
});
}
function visibility() {
const visible = get<HTMLInputElement>('show-person').checked;
const maskOnly = get<HTMLInputElement>('show-mask').checked;
const activeMotion = ready && !failed && !reduced.matches;
person.hidden = !visible || maskOnly || near || activeMotion;
canvas.hidden = !visible || maskOnly || near || !activeMotion;
get('environment').hidden = maskOnly;
get('mask').hidden = !maskOnly || near;
get('shadow').hidden = !visible || maskOnly || near || !get<HTMLInputElement>('show-shadow').checked;
get('occlusion').hidden = !visible || maskOnly || near || !get<HTMLInputElement>('show-occlusion').checked;
get('talk').hidden = !near;
for (const id of missingImages) get(id).hidden = true;
}
function paint() {
if (video.readyState >= 2) {
context.clearRect(0, 0, canvas.width, canvas.height);
context.globalCompositeOperation = 'source-over';
context.drawImage(video, 0, 0, canvas.width, canvas.height);
}
get('time').textContent = ready
? `动作位置:${video.currentTime.toFixed(2)} / ${Number.isFinite(video.duration) ? video.duration.toFixed(2) : '…'} 秒`
: '动作位置:未载入';
}
function draw() {
paint();
if (!video.paused && !document.hidden) frameHandle = requestAnimationFrame(draw);
}
async function syncPlayback() {
cancelAnimationFrame(frameHandle);
const run = wantsPlay && ready && !failed && !document.hidden && !reduced.matches;
if (run) {
try {
await video.play();
draw();
} catch {
wantsPlay = false;
video.pause();
fallback('浏览器未允许播放;请点击播放重试。');
}
} else { video.pause(); paint(); }
play.textContent = ready ? (video.paused ? '播放真实试片' : '暂停真实试片') : '真实动作尚不可用';
visibility();
}
function layout() {
const stage = get('frame');
const scale = Math.min(stage.clientWidth / 1672, stage.clientHeight / 941);
const left = (stage.clientWidth - 1672 * scale) / 2;
const top = (stage.clientHeight - 941 * scale) / 2;
Object.assign(canvas.style, { left: `${left + 1120 * scale}px`, top: `${top + 200 * scale}px`,
width: `${552 * scale}px`, height: `${552 * scale}px` });
}
new ResizeObserver(layout).observe(get('frame'));
for (const input of document.querySelectorAll('input')) input.addEventListener('change', visibility);
get('lighting').addEventListener('click', () => {
const button = get<HTMLButtonElement>('lighting');
const dim = button.getAttribute('aria-pressed') !== 'true';
button.setAttribute('aria-pressed', String(dim));
button.textContent = `光照演示:${dim ? '环境减光' : '原始'}`;
get('environment').style.setProperty('--demo-environment-brightness', dim ? '0.65' : '1');
// No source assignment, load(), currentTime mutation or player recreation here.
});
get('closeup').addEventListener('click', () => {
near = !near;
const button = get<HTMLButtonElement>('closeup');
button.setAttribute('aria-pressed', String(near));
button.textContent = near ? '退出静态近景' : '进入静态近景';
visibility();
});
play.addEventListener('click', () => { wantsPlay = !wantsPlay; void syncPlayback(); });
document.addEventListener('visibilitychange', () => { void syncPlayback(); });
reduced.addEventListener('change', () => {
if (reduced.matches) wantsPlay = false;
void syncPlayback();
});
video.addEventListener('error', () => {
failed = true; ready = false; wantsPlay = false; play.disabled = true;
fallback('动作素材缺失或解码失败,回退静态人物;未通过动态验证。');
void syncPlayback();
});
async function init() {
try {
const response = await fetch('./assets/manifest.json');
if (!response.ok) throw new Error('manifest');
const manifest: Manifest = await response.json();
const motion = manifest.motion;
if (!motion.file || !motion.alpha) {
status.textContent = '静态分层可校对;无合格的透明动作素材,动态验收未通过。';
play.disabled = true; play.textContent = '真实动作尚不可用'; return;
}
video.src = `./assets/${motion.file}`;
video.addEventListener('loadeddata', () => {
ready = true; play.disabled = false;
status.textContent = `真实生成试片 · ${motion.status} · 循环切点与身份一致性以检查报告为准`;
paint(); void syncPlayback();
}, { once: true });
} catch {
fallback('素材清单不可用;保留静态分层校对。');
play.textContent = '真实动作尚不可用';
}
}
visibility(); layout(); void init();
+22
View File
@@ -0,0 +1,22 @@
@import './tokens.css';
* { box-sizing: border-box; }
[hidden] { display: none !important; }
body { margin: 0; background: var(--scene-night-deep); color: var(--ink-primary); font: var(--fs-body) var(--font-sans); }
main { max-width: 1600px; margin: auto; }
header, section { padding: var(--space-4) var(--space-6); }
h1 { font-size: var(--fs-title); margin: 0; }
p { color: var(--ink-secondary); line-height: 1.5; }
#stage { position: relative; overflow: hidden; height: min(60vh, 640px); background: var(--glass-bg-strong); }
#frame { position: absolute; width: 100%; height: 100%; }
#environment, .layer { position: absolute; width: 100%; height: 100%; object-fit: contain; }
#environment { filter: brightness(var(--demo-environment-brightness, 1)); }
#motion { position: absolute; }
fieldset { border: 1px solid var(--glass-border); border-radius: var(--r-control); padding: var(--space-3); display: flex; flex-wrap: wrap; gap: var(--space-4); }
label, button { cursor: pointer; }
.controls { display: flex; flex-wrap: wrap; gap: var(--space-2); margin-top: var(--space-3); }
button { color: var(--ink-primary); background: var(--glass-bg-strong); border: 1px solid var(--glass-highlight); border-radius: var(--r-control); padding: var(--space-3) var(--space-4); font: inherit; }
button[aria-pressed=true] { border-color: var(--accent); }
button:disabled { color: var(--ink-disabled); cursor: default; }
button:focus-visible, input:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; }
output { font-variant-numeric: tabular-nums; color: var(--accent); }
@media (max-width: 600px) { header, section { padding: var(--space-3); } #stage { height: 38vh; } }
+73
View File
@@ -0,0 +1,73 @@
/* Reused verbatim from hel-700-rain-room; no alternative product theme. */
:root {
--scene-night-deep:#16233c;
--scene-city-blue:#305481;
--scene-city-mid:#5a76a2;
--scene-tower:#2b456b;
--scene-glass-blue:#718fc2;
--scene-amber:#e8a45c;
--scene-amber-soft:#f2c894;
--scene-wood:#59361d;
--scene-cream:#e8d9c8;
--scene-blanket:#362928;
--ink-primary:#f2efe9;
--ink-secondary:rgba(242,239,233,.64);
--ink-tertiary:rgba(242,239,233,.4);
--ink-disabled:rgba(242,239,233,.28);
--accent:#e8a45c;
--accent-hover:#f2b879;
--accent-pressed:#c9853f;
--accent-ink:#1a1208;
--glass-bg:rgba(14,18,30,.55);
--glass-bg-strong:rgba(14,18,30,.72);
--glass-border:rgba(255,255,255,.08);
--glass-highlight:rgba(255,255,255,.14);
--bubble-user:rgba(120,144,180,.2);
--bubble-user-border:rgba(255,255,255,.07);
--bubble-her:rgba(255,255,255,.06);
--bubble-failed:rgba(200,90,80,.14);
--state-danger:#e07a6a;
--font-sans:"PingFang SC","HarmonyOS Sans SC","MiSans","Source Han Sans SC",system-ui,-apple-system,sans-serif;
--fs-caption:12px;
--fs-small:13px;
--fs-body:14px;
--fs-chat:15px;
--fs-nav:15px;
--fs-title:20px;
--fs-brand:16px;
--fs-time:13px;
--space-1:4px;
--space-2:8px;
--space-3:12px;
--space-4:16px;
--space-5:20px;
--space-6:24px;
--space-8:32px;
--space-10:40px;
--r-panel:24px;
--r-bubble:16px;
--r-card:16px;
--r-control:12px;
--r-pill:999px;
--shadow-panel:0 24px 80px rgba(0,0,0,.45);
--shadow-pop:0 8px 24px rgba(0,0,0,.35);
--glow-accent:0 0 12px rgba(232,164,92,.45);
--glass-blur:blur(28px) saturate(1.2);
--z-scene-far:0;
--z-scene-rain:1;
--z-scene-near:2;
--z-caption:10;
--z-nav:10;
--z-topbar:20;
--z-panel:30;
--z-toast:40;
--ease-out:cubic-bezier(.23,1,.32,1);
--ease-in-out:cubic-bezier(.77,0,.175,1);
--ease-drawer:cubic-bezier(.32,.72,0,1);
--lamp-opacity:.12;
--panel-width:clamp(360px,30vw,460px);
--panel-top:64px;
--panel-inset:24px;
--scene-height:330px;
--drawer-height:30dvh;
}
+144
View File
@@ -0,0 +1,144 @@
"""Foreground Chromium interaction checks, screenshots and downloadable recording."""
import functools
import http.server
import json
import threading
from pathlib import Path
from playwright.sync_api import sync_playwright
ROOT = Path(__file__).resolve().parents[1]
EVIDENCE = ROOT / 'evidence'
class Quiet(http.server.SimpleHTTPRequestHandler):
def log_message(self, *_args):
pass
def main():
EVIDENCE.mkdir(exist_ok=True)
handler = functools.partial(Quiet, directory=str(ROOT / 'dist'))
server = http.server.ThreadingHTTPServer(('127.0.0.1', 0), handler)
thread = threading.Thread(target=server.serve_forever)
thread.start()
results = {}
try:
with sync_playwright() as p:
browser = p.chromium.launch()
url = f'http://127.0.0.1:{server.server_port}'
context = browser.new_context(viewport={'width': 1280, 'height': 800},
record_video_dir=str(EVIDENCE),
record_video_size={'width': 1280, 'height': 800})
page = context.new_page()
errors = []
page.on('pageerror', lambda error: errors.append(str(error)))
page.goto(url)
page.wait_for_function('!document.querySelector("#play").disabled')
assert page.locator('#source').evaluate('(v) => v.paused')
page.click('#play')
page.wait_for_timeout(800)
alpha = page.locator('#motion').evaluate('(c) => {const x=c.getContext("2d"); return [x.getImageData(0,0,1,1).data[3], x.getImageData(300,220,1,1).data[3]]}')
assert alpha[0] == 0 and alpha[1] > 240, alpha
results['vp9_alpha_decoded_by_chromium'] = alpha
page.wait_for_function('document.querySelector("#source").currentTime > 0.9 && document.querySelector("#source").currentTime < 1.4')
positions = page.evaluate('''() => {
const video=document.querySelector('#source'); const before=video.currentTime;
const source=video.currentSrc; document.querySelector('#lighting').click();
return {before, after:video.currentTime, sourcePreserved: source===video.currentSrc};
}''')
assert positions['before'] == positions['after'] and positions['sourcePreserved']
results['environment_toggle_no_seek'] = positions
page.wait_for_timeout(650)
assert page.locator('#source').evaluate('(v) => !v.paused')
page.click('#show-person')
assert page.locator('#motion').is_hidden() and page.locator('#person').is_hidden()
page.wait_for_timeout(600)
page.screenshot(path=str(EVIDENCE / 'environment-only.png'))
page.click('#show-person')
page.click('#show-occlusion')
assert page.locator('#occlusion').is_hidden()
page.click('#show-occlusion')
page.click('#show-shadow')
assert page.locator('#shadow').is_hidden()
page.click('#show-shadow')
page.click('#show-mask')
assert page.locator('#mask').is_visible() and page.locator('#environment').is_hidden()
page.wait_for_timeout(500)
page.click('#show-mask')
results['independent_layers'] = True
page.click('#closeup')
assert page.locator('#talk').is_visible()
assert page.locator('#source').evaluate('(v) => !v.paused')
page.wait_for_timeout(700)
page.screenshot(path=str(EVIDENCE / 'static-near.png'))
page.click('#closeup')
assert page.locator('#talk').is_hidden()
page.wait_for_timeout(3400) # Includes a real hard loop.
results['near_is_static_player_continues'] = True
page.click('#play')
t = page.locator('#source').evaluate('(v) => v.currentTime')
page.wait_for_timeout(250)
assert page.locator('#source').evaluate('(v) => v.currentTime') == t
results['manual_pause'] = True
# Chromium lifecycle API freezes the page after making it hidden.
session = context.new_cdp_session(page)
session.send('Page.setWebLifecycleState', {'state': 'active'})
# Visibility handler unit probe. Browser screenshot/recording checks above are real.
page.click('#play')
page.evaluate('''() => {
Object.defineProperty(document, 'hidden', {configurable:true, get:()=>true});
document.dispatchEvent(new Event('visibilitychange'));
}''')
assert page.locator('#source').evaluate('(v) => v.paused')
page.evaluate('''() => {
delete document.hidden; document.dispatchEvent(new Event('visibilitychange'));
}''')
page.wait_for_function('!document.querySelector("#source").paused')
results['visibility_pause_resume_handler_synthetic_event'] = True
for width, height in [(1600, 900), (1280, 800), (390, 844)]:
page.set_viewport_size({'width': width, 'height': height})
page.wait_for_timeout(180)
bounds = page.evaluate('''() => {
const r=document.querySelector('#motion').getBoundingClientRect();
const s=document.querySelector('#stage').getBoundingClientRect();
return {faceX:r.x+305/552*r.width, faceY:r.y+135/552*r.height,
stage:{left:s.left,right:s.right,top:s.top,bottom:s.bottom},
scroll:document.documentElement.scrollWidth, width:innerWidth};
}''')
assert bounds['scroll'] <= width
assert bounds['stage']['left'] <= bounds['faceX'] <= bounds['stage']['right']
assert bounds['stage']['top'] <= bounds['faceY'] <= bounds['stage']['bottom']
page.screenshot(path=str(EVIDENCE / f'viewport-{width}x{height}.png'))
results[f'viewport_{width}x{height}'] = bounds
page.emulate_media(reduced_motion='reduce')
page.wait_for_timeout(100)
assert page.locator('#source').evaluate('(v) => v.paused')
assert page.locator('#person').is_visible() and page.locator('#motion').is_hidden()
results['reduced_motion_static_fallback'] = True
page.emulate_media(reduced_motion='no-preference')
assert page.locator('#source').evaluate('(v) => v.paused')
video_path = Path(page.video.path())
context.close()
video_path.rename(EVIDENCE / 'browser-continuous.webm')
missing = browser.new_page()
missing.route('**/reading-alpha.webm', lambda route: route.abort())
missing.goto(url)
missing.wait_for_function('!document.querySelector("#fallback").hidden')
assert missing.locator('#person').is_visible()
assert missing.locator('#play').is_disabled()
results['missing_motion_static_fallback'] = True
missing.close()
assert not errors, errors
results['page_errors'] = errors
browser.close()
finally:
server.shutdown()
thread.join()
server.server_close()
(EVIDENCE / 'browser-results.json').write_text(json.dumps(results, ensure_ascii=False, indent=2))
print(json.dumps(results, ensure_ascii=False, indent=2))
if __name__ == '__main__':
main()
+33
View File
@@ -0,0 +1,33 @@
"""Reproduce the separately switchable foreground and provisional contact shadow."""
from pathlib import Path
from PIL import Image, ImageDraw, ImageFilter
def main():
assets = Path(__file__).resolve().parents[1] / 'public/assets'
source = Image.open(assets / 'reference.png').convert('RGBA')
mask = Image.new('L', source.size)
ImageDraw.Draw(mask).polygon([
(1240, 582), (1282, 566), (1316, 556), (1360, 552), (1403, 532),
(1438, 526), (1486, 514), (1517, 521), (1539, 544), (1559, 556),
(1565, 634), (1520, 707), (1407, 747), (1260, 703), (1227, 650),
], fill=255)
mask = mask.filter(ImageFilter.GaussianBlur(1))
source.putalpha(mask)
source.save(assets / 'occlusion.png')
mask.save(assets / 'occlusion-mask.png')
mask = Image.new('L', source.size)
ImageDraw.Draw(mask).ellipse((1230, 511, 1512, 595), fill=65)
mask = mask.filter(ImageFilter.GaussianBlur(12))
shadow = Image.new('RGBA', source.size, (44, 28, 20, 0))
shadow.putalpha(mask)
shadow.save(assets / 'contact-shadow.png')
composite = Image.open(assets / 'environment.png').convert('RGBA')
for name in ['contact-shadow.png', 'person.png', 'occlusion.png']:
composite = Image.alpha_composite(composite, Image.open(assets / name))
composite.save(assets / 'composite.png')
if __name__ == '__main__':
main()
+74
View File
@@ -0,0 +1,74 @@
"""Use installed SAM modeling modules directly; no predictor/OpenCV dependency.
Run in the existing ComfyUI venv. No downloads or dependency changes.
"""
import argparse
import importlib
import sys
import types
from pathlib import Path
import numpy as np
import torch
from PIL import Image, ImageChops, ImageDraw, ImageFilter
def load_sam(source: Path, checkpoint: Path):
# The optional predictor package imports cv2. We only use the original model API.
package = types.ModuleType("hel711_sam")
package.__path__ = [str(source)]
sys.modules[package.__name__] = package
builder = importlib.import_module("hel711_sam.build_sam")
return builder.build_sam_vit_b(str(checkpoint)).eval()
def segment(model, image):
width, height = image.size
scale = 1024 / max(width, height)
size = (round(width * scale), round(height * scale))
tensor = torch.from_numpy(np.array(image.resize(size))).permute(2, 0, 1).to(model.device)
# Crop coordinate system: fixed right-hand seat, 552 x 552.
points = torch.tensor([[[305, 140], [300, 245], [200, 300], [307, 301],
[80, 100], [450, 400], [330, 450]]], dtype=torch.float)
points = (points * scale).to(model.device)
labels = torch.tensor([[1, 1, 1, 1, 0, 0, 0]], device=model.device)
box = torch.tensor([[100, 60, 420, 380]], dtype=torch.float, device=model.device) * scale
with torch.inference_mode():
result = model([dict(image=tensor, original_size=(height, width),
point_coords=points, point_labels=labels, boxes=box)],
multimask_output=False)[0]
mask = result['masks'][0, 0].cpu().numpy().astype(np.uint8) * 255
mask = Image.fromarray(mask)
# The book/knit texture produces enclosed SAM holes. Fill enclosed holes only.
exterior = mask.copy()
ImageDraw.floodfill(exterior, (0, 0), 255)
mask = ImageChops.lighter(mask, ImageChops.invert(exterior))
return mask.filter(ImageFilter.GaussianBlur(0.65))
def main():
parser = argparse.ArgumentParser()
parser.add_argument('assets', type=Path)
parser.add_argument('--sam-source', type=Path, required=True)
parser.add_argument('--checkpoint', type=Path, required=True)
args = parser.parse_args()
torch.set_num_threads(8)
model = load_sam(args.sam_source, args.checkpoint)
source = Image.open(args.assets / 'reference.png').convert('RGB')
region = (1120, 200, 1672, 752)
crop = source.crop(region)
mask = segment(model, crop)
full_mask = Image.new('L', source.size)
full_mask.paste(mask, region[:2])
full_mask.save(args.assets / 'person-mask.png')
layer = source.convert('RGBA')
layer.putalpha(full_mask)
layer.save(args.assets / 'person.png')
crop.save(args.assets / 'generation-input.png')
environment = Image.open(args.assets / 'environment.png').convert('RGBA')
Image.alpha_composite(environment, layer).save(args.assets / 'composite.png')
print('Saved original-pixel person layer, SAM mask, crop and real composite.', flush=True)
if __name__ == '__main__':
main()
+141
View File
@@ -0,0 +1,141 @@
"""One foreground job using the installed ComfyUI native Wan nodes.
Owns and stops its exact server PID in finally. Does not modify the installation.
"""
import json
import os
import subprocess
import time
import urllib.error
import urllib.request
from pathlib import Path
ROOT = Path('/media/lucian/Downloads/ComfyUI')
TASK = ROOT / 'output/hel711-task'
URL = 'http://127.0.0.1:18189'
def request(path, payload=None):
data = None if payload is None else json.dumps(payload).encode()
req = urllib.request.Request(URL + path, data=data,
headers={'Content-Type': 'application/json'})
with urllib.request.urlopen(req, timeout=20) as response:
return json.load(response)
def graph():
nodes = {}
def add(key, kind, **inputs):
nodes[str(key)] = dict(class_type=kind, inputs=inputs)
for key, mode in [(1, 'high'), (2, 'low')]:
add(key, 'UNETLoader', unet_name=f'wan2.2_i2v_{mode}_noise_14B_fp8_scaled.safetensors',
weight_dtype='default')
add(key + 2, 'LoraLoaderModelOnly', model=[str(key), 0],
lora_name=f'wan2.2_i2v_lightx2v_4steps_lora_v1_{mode}_noise.safetensors',
strength_model=1.0)
add(key + 4, 'ModelSamplingSD3', model=[str(key + 2), 0], shift=5.0)
add(7, 'CLIPLoader', clip_name='umt5_xxl_fp8_e4m3fn_scaled.safetensors',
type='wan', device='cpu')
add(8, 'VAELoader', vae_name='wan_2.1_vae.safetensors')
add(9, 'LoadImage', image='generation-input.png')
add(10, 'CLIPTextEncode', clip=['7', 0], text=(
'Locked off camera, photorealistic young adult woman with short dark hair, '
'cream cardigan and dark trousers seated on the same sofa in the same bedroom. '
'She quietly reads the book in her hands, slowly lowers her gaze to the page '
'then gently raises her head a little. Small natural eye blink and breathing. '
'Her hands keep holding the book. Fixed seat position, stationary room and furniture.'))
add(11, 'CLIPTextEncode', clip=['7', 0],
text='camera movement, zoom, changing face, changing clothes, standing up, '
'extra fingers, distorted hands, deformed face, moving furniture, subtitles')
add(12, 'WanImageToVideo', positive=['10', 0], negative=['11', 0], vae=['8', 0],
width=512, height=512, length=49, batch_size=1, start_image=['9', 0])
add(13, 'KSamplerAdvanced', model=['5', 0], add_noise='enable', noise_seed=711,
steps=4, cfg=1.0, sampler_name='euler', scheduler='simple',
positive=['12', 0], negative=['12', 1], latent_image=['12', 2],
start_at_step=0, end_at_step=2, return_with_leftover_noise='enable')
add(14, 'KSamplerAdvanced', model=['6', 0], add_noise='disable', noise_seed=711,
steps=4, cfg=1.0, sampler_name='euler', scheduler='simple',
positive=['12', 0], negative=['12', 1], latent_image=['13', 0],
start_at_step=2, end_at_step=4, return_with_leftover_noise='disable')
add(15, 'VAEDecode', samples=['14', 0], vae=['8', 0])
add(16, 'SaveWEBM', images=['15', 0], codec='vp9', fps=16.0,
filename_prefix='reading', crf=20.0)
return nodes
def main():
for name in ['user', 'temp']:
(TASK / name).mkdir(exist_ok=True)
os.environ['HF_HUB_OFFLINE'] = '1'
os.environ['TRANSFORMERS_OFFLINE'] = '1'
command = [str(ROOT / 'venv/bin/python'), str(ROOT / 'main.py'),
'--listen', '127.0.0.1', '--port', '18189', '--disable-all-custom-nodes',
'--offline', '--disable-manager-ui', '--disable-assets',
'--reserve-vram', '8', '--disable-dynamic-vram',
'--input-directory', str(TASK / 'assets'),
'--output-directory', str(TASK), '--user-directory', str(TASK / 'user'),
'--temp-directory', str(TASK / 'temp'), '--database-url', 'sqlite:///:memory:']
# Refuse to attach to an unrelated listener.
import socket
with socket.socket() as sock:
if sock.connect_ex(('127.0.0.1', 18189)) == 0:
raise RuntimeError('Task port already occupied; no process started')
with open(TASK / 'server.log', 'w') as log:
server = subprocess.Popen(command, cwd=ROOT, stdout=log, stderr=subprocess.STDOUT)
(TASK / 'process.json').write_text(json.dumps(dict(pid=server.pid, command=command)))
try:
deadline = time.monotonic() + 90
while True:
if server.poll() is not None:
raise RuntimeError(f'ComfyUI exited: {server.returncode}; see server.log')
try:
info = request('/object_info')
break
except (OSError, urllib.error.URLError):
if time.monotonic() > deadline:
raise TimeoutError('Startup exceeded 90 seconds') from None
time.sleep(1)
workflow = graph()
(TASK / 'workflow.json').write_text(json.dumps(workflow, indent=2))
required = {n['class_type']: info.get(n['class_type']) for n in workflow.values()}
(TASK / 'nodes.json').write_text(json.dumps(required, indent=2))
if any(v is None for v in required.values()):
raise RuntimeError('Required native node unavailable; see nodes.json')
result = request('/prompt', {'prompt': workflow})
(TASK / 'submission.json').write_text(json.dumps(result))
prompt_id = result['prompt_id']
print(f'Generating one clip: {prompt_id}; owned server PID {server.pid}', flush=True)
deadline = time.monotonic() + 2400
last_report = 0
while time.monotonic() < deadline:
history = request('/history/' + prompt_id)
if prompt_id in history:
(TASK / 'history.json').write_text(json.dumps(history, indent=2))
print(json.dumps(history[prompt_id]['status']), flush=True)
if history[prompt_id]['status']['status_str'] != 'success':
raise RuntimeError('Generation failed; see history.json')
break
if time.monotonic() - last_report > 30:
print('Single generation still active; existing services untouched.', flush=True)
last_report = time.monotonic()
time.sleep(3)
else:
request('/interrupt', {})
raise TimeoutError('Single generation exceeded 40 minutes')
except urllib.error.HTTPError as error:
(TASK / 'api-error.json').write_bytes(error.read())
raise
finally:
server.terminate()
try:
server.wait(timeout=20)
except subprocess.TimeoutExpired:
server.kill()
server.wait()
print(f'Stopped owned server PID {server.pid}', flush=True)
if __name__ == '__main__':
main()
+59
View File
@@ -0,0 +1,59 @@
"""Per-frame real SAM segmentation, then VP9 alpha encoding with existing PyAV."""
import json
from pathlib import Path
import av
import numpy as np
import torch
from extract_layers import load_sam, segment
from PIL import Image
ROOT = Path('/media/lucian/Downloads/ComfyUI')
TASK = ROOT / 'output/hel711-task'
def main():
torch.set_num_threads(8)
model = load_sam(
ROOT / 'custom_nodes/was-node-suite-comfyui/repos/SAM/segment_anything',
ROOT / 'models/sams/sam_vit_b_01ec64.pth').to('cuda')
output = av.open(str(TASK / 'reading-alpha.webm'), 'w')
stream = output.add_stream('libvpx-vp9', rate=16)
stream.width = 552
stream.height = 552
stream.pix_fmt = 'yuva420p'
stream.options = {'crf': '20', 'b': '0', 'auto-alt-ref': '0'}
metrics = []
previews = []
source = av.open(str(TASK / 'reading_00001_.webm'))
previous = None
first = None
for index, decoded in enumerate(source.decode(video=0)):
image = decoded.to_image().resize((552, 552), Image.Resampling.LANCZOS)
mask = segment(model, image)
frame = image.convert('RGBA')
frame.putalpha(mask)
pixels = np.array(image).astype(float)
if first is None:
first = pixels
metrics.append(dict(frame=index, mask_pixels=int((np.array(mask) > 127).sum()),
mean_absolute_rgb_change=0 if previous is None else float(np.abs(pixels-previous).mean())))
previous = pixels
if index in [0, 16, 32, 48]:
previews.append(frame)
frame.save(TASK / f'matted-{index:02}.png')
for packet in stream.encode(av.VideoFrame.from_ndarray(np.array(frame), format='rgba')):
output.mux(packet)
if index % 16 == 0:
print(f'Matted frame {index}', flush=True)
for packet in stream.encode():
output.mux(packet)
output.close()
source.close()
(TASK / 'frame-metrics.json').write_text(json.dumps(dict(
frames=metrics, loop_rgb_difference=float(np.abs(previous-first).mean()),
interpretation='Real generated frames; RGB differences alone are not proof of valid anatomy or identity.'), indent=2))
if __name__ == '__main__':
main()
+20
View File
@@ -0,0 +1,20 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "Bundler",
"lib": [
"ES2022",
"DOM",
"DOM.Iterable"
],
"strict": true,
"noEmit": true,
"skipLibCheck": true,
"noUnusedLocals": true,
"noUnusedParameters": true
},
"include": [
"src"
]
}