feat: add layered bedroom motion review sample #1
@@ -0,0 +1,7 @@
|
||||
node_modules/
|
||||
dist/
|
||||
public/assets/*
|
||||
!public/assets/manifest.json
|
||||
evidence/*
|
||||
!evidence/.gitkeep
|
||||
__pycache__/
|
||||
@@ -0,0 +1,39 @@
|
||||
HEL-711 最小动态校对小样,基于 main `1e9b08a`。仅在 `samples/bedroom` 内工作;不连接人格、记忆、天气或语音。采用 HEL-708 的环境/人物/交谈分层边界,以及 CG-001~026 的当前需求。旧演示不变。
|
||||
|
||||
## 本地复核
|
||||
|
||||
1. 将本卡附件 `HEL-711-review-bundle.zip` 中的 `public/assets` 解压到此目录对应位置。
|
||||
2. `npm ci --ignore-scripts --no-audit --no-fund`,沿用旧前端分支的 package.json/package-lock.json/tsconfig.json,无新依赖。
|
||||
3. `npm run build`(包含严格 TypeScript 检查)。
|
||||
4. `npm run dev -- --port 5173`,本机浏览器打开终端输出的地址。
|
||||
5. `python3 tools/browser_check.py` 使用已安装的 Playwright/Chromium,自动启停本机测试服务并保存证据。此运行机需要给该命令设置 `LD_LIBRARY_PATH=/home/xiaomo/pwlibs/root/usr/lib/x86_64-linux-gnu`,复用了已有浏览器库,未安装系统依赖。
|
||||
|
||||
素材不进 Git,缺失时页面明确提示;manifest 在 Git 中保存来源、原图 SHA256、坐标、编码和问题。附件内含完整 assets、已构建 dist、浏览器连续录屏、原始试片、透明试片、分层校对图及生成工作流/日志。
|
||||
|
||||
## 代码与边界
|
||||
|
||||
- `src/main.ts`:一个持久 video 元素驱动透明 canvas;环境按钮只改环境亮度,不重建/seek 视频。人物、前景遮挡、接触阴影独立开关;静态近景不替代真实交谈。
|
||||
- `src/tokens.css`:原样复用旧前端唯一 token 表。`style.css` 为独立工程校对台,无第二套同类组件。
|
||||
- `tools/extract_layers.py`:读取现有 SAM vit-b 模型的原始 modeling API,避免可选 predictor 的 OpenCV 依赖;不伪造 cv2,不下载。原图 RGB 保留,只生成 alpha;封闭孔洞填充、0.65px 羽化。
|
||||
- `tools/contact_layers.py`:从原入景图提取毯子前景,另建可关闭的暂定接触阴影;未宣称物理阴影校准完成。
|
||||
- `tools/generate_once.py`:原 ComfyUI venv + 原生 Wan 节点,单片段、离线、自定义节点禁用。临时服务器仅监听回环 18189,finally 终止自己启动的精确 PID。不修改现有服务、系统 Python、驱动或模型路径配置。
|
||||
- `tools/matte_video.py`:49 帧逐帧 SAM 分割,原有 PyAV 编码 VP9 alpha。原始视频未抠像的背景不用于浏览器人物合成。
|
||||
- `tools/browser_check.py`:真实 Chromium 检查、三种视口截图和可下载连续录屏。页面隐藏的事件处理做了合成 visibilitychange 检查;真实系统切后台/锁屏仍待体验环境手测。
|
||||
|
||||
## 素材处理命令
|
||||
|
||||
处理目录:`.12:/media/lucian/Downloads/ComfyUI/output/hel711-task`,输入复制到其 `assets`,脚本放此目录;没有改变现有项目输入。原图由 Multica 鉴权 attachment download 获取。运行时使用原 `/media/lucian/Downloads/ComfyUI/venv/bin/python`。
|
||||
|
||||
```bash
|
||||
python extract_layers.py assets \
|
||||
--sam-source /media/lucian/Downloads/ComfyUI/custom_nodes/was-node-suite-comfyui/repos/SAM/segment_anything \
|
||||
--checkpoint /media/lucian/Downloads/ComfyUI/models/sams/sam_vit_b_01ec64.pth
|
||||
python generate_once.py
|
||||
python matte_video.py
|
||||
```
|
||||
|
||||
回传处理后的素材到本地 `public/assets`,运行 `python3 tools/contact_layers.py`。手工蒙版和坐标均保留在脚本中。`workflow.json` 是本次实际运行的请求,最终脚本仅补了目录创建/日志关闭等清理逻辑;日志和 history 可交叉核对。执行新试片前必须重新检查显存和端口;禁止停止已有服务腾资源。
|
||||
|
||||
## 当前实测与未通过项
|
||||
|
||||
当前结果见 `REPORT.md`,属于可审核候选,不代表 D1–D6 全部通过。图层是真实独立合成;没有用重生成整幅图代替分层。近景仍用已确认 v2 静态方向图;没有真人语音或口型。
|
||||
@@ -0,0 +1,52 @@
|
||||
2026-10-20 HEL-711 审核记录
|
||||
|
||||
## 实际结果
|
||||
|
||||
已完成原始像素透明人物层、无人环境层、前景毯子/蒙版、暂定接触阴影、静态合成和独立开关。坐标系为 1672×941,固定右侧 552×552 区域(1120,200),人物未平移/缩放伪装动作。原参考和透明层在素材包中,可检查 alpha。
|
||||
|
||||
仅生成一段 Wan2.2 真实阅读动作:512×512 原始生成、49 帧、16fps、3.0625s;逐帧分割后恢复为 552×552 裁切区域,VP9 WebM alpha。人物从微歪头看向镜头变为低头读书,衣服和坐姿保持;这次没有实现再次抬头。四张连续取样帧与原始视频均交付,不用 RGB 差分独自证明人物动作。
|
||||
|
||||
实测推理日志 `Prompt executed in 57.18 seconds`,不包含环境启动、分层和浏览器录屏耗时。首尾 RGB 平均差约 11.998/255,仅是诊断数据。首尾姿态有明显差异,硬循环会跳变;没有反向播放伪造完整动作。
|
||||
|
||||
## .12 工具核查
|
||||
|
||||
- GPU RTX 5090 D,nvidia-smi 总显存 32607MiB;生成/抠像前后既有占用均为 5633MiB,未记录精确峰值,不能声称峰值或共存 SLA。磁盘可用约 754GiB。
|
||||
- Desktop 安装 `/home/lucian/ComfyUI-Installs/ComfyUI` 和 `/home/lucian/ComfyUI-Shared` 确实存在,但 8188 未监听。
|
||||
- 实际可运行安装为 `/media/lucian/Downloads/ComfyUI`,venv PyTorch `2.14.1+cu130`、torchvision `0.29.1+cu130`、PyAV `19.0.1`;缺 cv2/rembg/timm,系统 ffmpeg 不在 PATH。未安装任何依赖。
|
||||
- 原有 models 内已存在 Wan2.2 I2V 高/低噪声 fp8 各约 14.29GB、UMT5 fp8 约 6.74GB、Wan VAE 约 254MB、对应 LightX2V LoRA 各约 1.23GB,以及 SAM vit-b 约 375MB。并非只见到文件就宣称可用:本次已实际跑通原生节点和单片段生成。
|
||||
- 临时 ComfyUI PID 596473,日志在素材包 `evidence/server.log`,已由启动脚本 finally 停止;18189 已无监听。旧 18080 及 ASR/TTS 监听保持。没有驻留生成工作,没有部署。
|
||||
- SSH 使用已确认的主机密钥及既有凭据;没有输出凭据。没有下载大模型、购买接口、修改开机启动或运行 GUI。
|
||||
|
||||
## 验证结果
|
||||
|
||||
- `npm run build`:通过,包含 TypeScript 严格检查;生产构建约 3.2KB CSS / 3.6KB JS(未 gzip)。
|
||||
- `python3 -m pytest -q`:原有 78 项全部通过。
|
||||
- `python3 -m ruff check src tests samples/bedroom/tools`:通过。
|
||||
- 真实 Chromium 解码 alpha:背景像素 alpha=0、人物内部 alpha=255。
|
||||
- 环境开关前后同一视频 source、同一 currentTime;实测样例 0.915222s→0.915222s。录屏包含人物播放、环境开关、无人画面、蒙版、阴影/前景开关和静态近景进入/退出。
|
||||
- 1600×900、1280×800、390×844:无横向溢出、脸部在画面内,控制区不遮脸。手机以全画面包含方式展示,人物较小,未宣称手机沉浸效果验收通过。
|
||||
- 暂停/恢复、减少动效切静态、动态素材加载失败回退静态:通过。浏览器 pageerror=0。
|
||||
- 隐藏暂停/恢复处理经合成 visibilitychange 事件验证;headless 环境不能代表真实切后台/系统锁屏,仍需设备手测。未测 Safari/iOS 的 VP9 alpha 支持,解码失败会显式回退。
|
||||
|
||||
## 待修与验收边界
|
||||
|
||||
1. 循环首尾头部跳变,未建立可接受切点。需后续在获准后做闭环片段或经美术确认的自然选片衔接,本轮不批量重试。
|
||||
2. SAM 是分割加羽化,不是精细发丝抠像;细发、针织边缘、书本遮挡可能闪烁/缺损,需逐帧美术复核。49 帧蒙版面积范围 49312~50563 像素,不能据此宣称边缘无抖动。
|
||||
3. 原无人/有人参考沙发与抱枕几何略有差异。人物固定落位与独立遮挡已实现,残余靠背、接触边缘和前景拼接仍需美术校正。接触阴影是暂定独立工程层。
|
||||
4. 小尺寸手部/书页可辨但不保证解剖正确;没有完成逐帧身份评分,脸部细节与跨镜头一致性待审。
|
||||
5. 环境减光只验证人物/房间独立,人物本身未做对应重打光;按钮标注手动演示,未接真实城市天气。
|
||||
6. 近景为 v2 静态切换,进退是即时校对切镜,衔接未润色;不能算真实交谈、语音或口型通过。
|
||||
|
||||
按 Emil review-animations 复核界面控制:
|
||||
|
||||
| Before | After | Why |
|
||||
| --- | --- | --- |
|
||||
| 自动循环可能让减少动效用户持续看到人物运动 | 默认手动播放,减少动效时暂停并显示静态层 | 用户控制和无运动回退已验证 |
|
||||
| 近景或环境切换可能重新装载视频 | 保留唯一 video,仅切独立层的可见性/环境亮度 | 切换可打断,保持实际播放位置 |
|
||||
| 片段首尾头姿不同 | 仍标注待修,不添加平移/缩放掩盖跳变 | 真实视频循环没有通过观感验收 |
|
||||
|
||||
界面播放控制 Approve;循环衔接与自然近景体验 Block,交总管/美术继续验收。本卡交付可审核的最小真实片段,不表示整体动态体验通过。
|
||||
|
||||
## 仓库
|
||||
|
||||
Multica checkout 返回仓库未配置,因此使用既有 SSH 权限直接 clone 指定仓库,建立独立 `hel-711-layered-bedroom`,基于明确指定 main `1e9b08a`。旧 `hel-700-rain-room` 未改动。`git ls-remote refs/pull/*/head` 无结果;SSH refs 不能证明私有 Gitea 没有 PR,未通过未知 API 凭据枚举 PR。此次不合并、不部署,提交号见本卡最终结果评论。
|
||||
@@ -0,0 +1,38 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head><meta charset="UTF-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>卧室分层校对 · HEL-711</title></head>
|
||||
<body>
|
||||
<main>
|
||||
<header><h1>卧室分层校对</h1><p>工程小样 · 手动光照演示 · 近景仅静态参考,未接语音或口型</p></header>
|
||||
<div id="stage" role="img" aria-label="固定机位卧室人物合成">
|
||||
<div id="frame">
|
||||
<img id="environment" src="./assets/environment.png" alt="无人高层卧室">
|
||||
<img id="shadow" src="./assets/contact-shadow.png" alt="" class="layer">
|
||||
<img id="person" src="./assets/person.png" alt="阅读人物透明层" class="layer">
|
||||
<canvas id="motion" width="552" height="552" hidden></canvas>
|
||||
<img id="occlusion" src="./assets/occlusion.png" alt="" class="layer">
|
||||
<img id="mask" src="./assets/person-mask.png" alt="人物蒙版" class="layer" hidden>
|
||||
<img id="talk" src="./assets/talk-reference.png" alt="交谈静态近景参考" class="layer" hidden>
|
||||
</div>
|
||||
</div>
|
||||
<video id="source" muted playsinline loop preload="metadata" hidden></video>
|
||||
<section aria-label="校对控制">
|
||||
<fieldset><legend>图层开关</legend>
|
||||
<label><input id="show-person" type="checkbox" checked>人物</label>
|
||||
<label><input id="show-shadow" type="checkbox" checked>接触阴影</label>
|
||||
<label><input id="show-occlusion" type="checkbox" checked>前景毯子</label>
|
||||
<label><input id="show-mask" type="checkbox">仅看人物蒙版</label>
|
||||
</fieldset>
|
||||
<div class="controls">
|
||||
<button id="lighting" type="button" aria-pressed="false">光照演示:原始</button>
|
||||
<button id="play" type="button" disabled>读取动作素材</button>
|
||||
<button id="closeup" type="button" aria-pressed="false">进入静态近景</button>
|
||||
</div>
|
||||
<p id="status" role="status">读取素材清单…</p>
|
||||
<output id="time">动作位置:未载入</output>
|
||||
<p id="fallback" role="alert" hidden></p>
|
||||
<p>人物、房间和近景来自已确认的 v2 参考。图层独立不等于几何、发丝与循环已经通过验收。</p>
|
||||
</section>
|
||||
</main>
|
||||
<script type="module" src="./src/main.ts"></script>
|
||||
</body></html>
|
||||
Generated
+1166
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"name": "cybergril-rain-room",
|
||||
"private": true,
|
||||
"version": "0.1.0",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite --host 127.0.0.1",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"build": "tsc --noEmit && vite build",
|
||||
"preview": "vite preview --host 127.0.0.1"
|
||||
},
|
||||
"dependencies": {
|
||||
"pixi.js": "^8.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"typescript": "~5.9.3",
|
||||
"vite": "^5.4.0"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
{
|
||||
"schema": "hel711-review-1",
|
||||
"status": "review_candidate",
|
||||
"reference_sha256": {
|
||||
"reference.png": "226d7922e042e22e4b8a8241a7cf6f7aea50c301029978e4ca26cad9b3d56359",
|
||||
"environment.png": "b5d4771eb010dc1d0a41d1b698e45edaa45d2f90bb441378bc32cf5c812e76a2",
|
||||
"talk-reference.png": "0ae9b1a1cb9c590ab89a81ce4efd25de843a600ff227c184663db23e9e9c16b3"
|
||||
},
|
||||
"camera": {
|
||||
"canvas": [
|
||||
1672,
|
||||
941
|
||||
],
|
||||
"generation_crop": [
|
||||
1120,
|
||||
200,
|
||||
552,
|
||||
552
|
||||
],
|
||||
"person_anchor": [
|
||||
1390,
|
||||
530
|
||||
],
|
||||
"coordinate_units": "source pixels, top left origin",
|
||||
"transform": "none; crop reinserted into original position",
|
||||
"clothing": "v2 cream cardigan, charcoal trousers"
|
||||
},
|
||||
"layers": [
|
||||
{
|
||||
"file": "environment.png",
|
||||
"source": "approved 02-v2 original",
|
||||
"alpha": false
|
||||
},
|
||||
{
|
||||
"file": "contact-shadow.png",
|
||||
"source": "engineered ellipse 65/255, 12px blur, not physically calibrated",
|
||||
"alpha": true
|
||||
},
|
||||
{
|
||||
"file": "person.png",
|
||||
"source": "original 03-v2 RGB + existing SAM vit-b alpha; enclosed mask holes filled",
|
||||
"alpha": true
|
||||
},
|
||||
{
|
||||
"file": "occlusion.png",
|
||||
"source": "03-v2 blanket pixels, hand-authored polygon matte",
|
||||
"alpha": true
|
||||
}
|
||||
],
|
||||
"motion": {
|
||||
"status": "真实生成;循环与抠像待修",
|
||||
"file": "reading-alpha.webm",
|
||||
"alpha": true,
|
||||
"maskFile": null,
|
||||
"original": "reading_00001_.webm",
|
||||
"generator": "installed ComfyUI native Wan2.2 high/low 14B fp8 + installed LightX2V 4-step LoRAs",
|
||||
"seed": 711,
|
||||
"dimensions": [
|
||||
552,
|
||||
552
|
||||
],
|
||||
"generation_dimensions": [
|
||||
512,
|
||||
512
|
||||
],
|
||||
"frames": 49,
|
||||
"fps": 16,
|
||||
"duration_seconds": 3.0625,
|
||||
"codec": "VP9 yuva420p / WebM",
|
||||
"matting": "existing SAM vit-b, per frame, filled enclosed holes, 0.65px feather",
|
||||
"start_pose": "seated holding open book, head slightly tilted toward camera",
|
||||
"end_pose": "same seat and clothes, head down at book",
|
||||
"loop": "hard loop at 49 frames; visible head jump, not seamless",
|
||||
"cut_point": "none accepted"
|
||||
},
|
||||
"closeup": {
|
||||
"file": "talk-reference.png",
|
||||
"type": "static_reference_only",
|
||||
"identity_verified": false,
|
||||
"lipsync": false,
|
||||
"source": "approved v2 static direction reference"
|
||||
},
|
||||
"known_issues": [
|
||||
"Original empty/occupied sofa and pillow geometry differs; residual contact geometry needs art review.",
|
||||
"SAM hard segmentation + feather is not hair matting; edge texture loss/flicker possible.",
|
||||
"Book and fingers are small; hand anatomy/identity not certified.",
|
||||
"Start/end head poses differ; hard loop jumps visibly.",
|
||||
"Manual environment dimming changes background only; subject relighting not implemented.",
|
||||
"Static near-shot is a generated reference; cross-shot face continuity not certified.",
|
||||
"No real weather, speech, lipsync or life-state integration."
|
||||
],
|
||||
"files": {
|
||||
"talk-reference.png": {
|
||||
"bytes": 2246098,
|
||||
"sha256": "0ae9b1a1cb9c590ab89a81ce4efd25de843a600ff227c184663db23e9e9c16b3"
|
||||
},
|
||||
"reading-alpha.webm": {
|
||||
"bytes": 472707,
|
||||
"sha256": "d28ef9a062744a8b362abfc6a4bc4d374a74128eea7c59d8441a4b77f7c14d98"
|
||||
},
|
||||
"person.png": {
|
||||
"bytes": 2522355,
|
||||
"sha256": "a904f4081069ba97019d3c679d8dd59102118560471f8d6337f7b29d60380f24"
|
||||
},
|
||||
"composite.png": {
|
||||
"bytes": 2576324,
|
||||
"sha256": "4354716558858d0e9ac9a4c4a025d77f4042a0e106f10fc37e500c7157e77456"
|
||||
},
|
||||
"person-mask.png": {
|
||||
"bytes": 7264,
|
||||
"sha256": "7569160b3c87ebf1d5bb66792ed6d2db6b41fde006fad0473e7db5337d18b0f7"
|
||||
},
|
||||
"environment.png": {
|
||||
"bytes": 2442707,
|
||||
"sha256": "b5d4771eb010dc1d0a41d1b698e45edaa45d2f90bb441378bc32cf5c812e76a2"
|
||||
},
|
||||
"occlusion-mask.png": {
|
||||
"bytes": 4838,
|
||||
"sha256": "dc22d1823a02df4fa5f3bf484f157c49b270f100185c0a35b3c31f28880dd3f0"
|
||||
},
|
||||
"generation-input.png": {
|
||||
"bytes": 508812,
|
||||
"sha256": "f3922903fb28c5940a2e272b70be6e33dc094039b4e275724cf3ff1aa4af1584"
|
||||
},
|
||||
"occlusion.png": {
|
||||
"bytes": 2522991,
|
||||
"sha256": "5f0f4aa7ae90ad863f694365eaa93d7d7558841ecca08c95d415731450226c4f"
|
||||
},
|
||||
"reference.png": {
|
||||
"bytes": 2384917,
|
||||
"sha256": "226d7922e042e22e4b8a8241a7cf6f7aea50c301029978e4ca26cad9b3d56359"
|
||||
},
|
||||
"contact-shadow.png": {
|
||||
"bytes": 16144,
|
||||
"sha256": "948f88c1298638158f9a21ffe107bf403106dcf6e58c72a3d6e738fe0c7cc003"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,141 @@
|
||||
import './style.css';
|
||||
|
||||
type Manifest = {
|
||||
motion: { status: string; file: string | null; alpha: boolean; maskFile: string | null };
|
||||
};
|
||||
const get = <T extends HTMLElement>(id: string) => document.getElementById(id) as T;
|
||||
const video = get<HTMLVideoElement>('source');
|
||||
const canvas = get<HTMLCanvasElement>('motion');
|
||||
const context = canvas.getContext('2d')!;
|
||||
const play = get<HTMLButtonElement>('play');
|
||||
const person = get<HTMLImageElement>('person');
|
||||
const status = get('status');
|
||||
const reduced = matchMedia('(prefers-reduced-motion: reduce)');
|
||||
const missingImages = new Set<string>();
|
||||
let ready = false;
|
||||
let wantsPlay = false;
|
||||
let near = false;
|
||||
let failed = false;
|
||||
let frameHandle = 0;
|
||||
|
||||
function fallback(message: string) {
|
||||
const node = get('fallback');
|
||||
node.hidden = false;
|
||||
node.textContent = message;
|
||||
}
|
||||
|
||||
for (const image of document.querySelectorAll<HTMLImageElement>('#frame img')) {
|
||||
image.addEventListener('error', () => {
|
||||
missingImages.add(image.id);
|
||||
image.hidden = true;
|
||||
fallback(`素材缺失:${image.id}。请解压任务素材包到 public/assets;当前内容不能作为验收结果。`);
|
||||
});
|
||||
}
|
||||
|
||||
function visibility() {
|
||||
const visible = get<HTMLInputElement>('show-person').checked;
|
||||
const maskOnly = get<HTMLInputElement>('show-mask').checked;
|
||||
const activeMotion = ready && !failed && !reduced.matches;
|
||||
person.hidden = !visible || maskOnly || near || activeMotion;
|
||||
canvas.hidden = !visible || maskOnly || near || !activeMotion;
|
||||
get('environment').hidden = maskOnly;
|
||||
get('mask').hidden = !maskOnly || near;
|
||||
get('shadow').hidden = !visible || maskOnly || near || !get<HTMLInputElement>('show-shadow').checked;
|
||||
get('occlusion').hidden = !visible || maskOnly || near || !get<HTMLInputElement>('show-occlusion').checked;
|
||||
get('talk').hidden = !near;
|
||||
for (const id of missingImages) get(id).hidden = true;
|
||||
}
|
||||
|
||||
function paint() {
|
||||
if (video.readyState >= 2) {
|
||||
context.clearRect(0, 0, canvas.width, canvas.height);
|
||||
context.globalCompositeOperation = 'source-over';
|
||||
context.drawImage(video, 0, 0, canvas.width, canvas.height);
|
||||
|
||||
}
|
||||
get('time').textContent = ready
|
||||
? `动作位置:${video.currentTime.toFixed(2)} / ${Number.isFinite(video.duration) ? video.duration.toFixed(2) : '…'} 秒`
|
||||
: '动作位置:未载入';
|
||||
}
|
||||
|
||||
function draw() {
|
||||
paint();
|
||||
if (!video.paused && !document.hidden) frameHandle = requestAnimationFrame(draw);
|
||||
}
|
||||
|
||||
async function syncPlayback() {
|
||||
cancelAnimationFrame(frameHandle);
|
||||
const run = wantsPlay && ready && !failed && !document.hidden && !reduced.matches;
|
||||
if (run) {
|
||||
try {
|
||||
await video.play();
|
||||
draw();
|
||||
} catch {
|
||||
wantsPlay = false;
|
||||
video.pause();
|
||||
fallback('浏览器未允许播放;请点击播放重试。');
|
||||
}
|
||||
} else { video.pause(); paint(); }
|
||||
play.textContent = ready ? (video.paused ? '播放真实试片' : '暂停真实试片') : '真实动作尚不可用';
|
||||
visibility();
|
||||
}
|
||||
|
||||
function layout() {
|
||||
const stage = get('frame');
|
||||
const scale = Math.min(stage.clientWidth / 1672, stage.clientHeight / 941);
|
||||
const left = (stage.clientWidth - 1672 * scale) / 2;
|
||||
const top = (stage.clientHeight - 941 * scale) / 2;
|
||||
Object.assign(canvas.style, { left: `${left + 1120 * scale}px`, top: `${top + 200 * scale}px`,
|
||||
width: `${552 * scale}px`, height: `${552 * scale}px` });
|
||||
}
|
||||
new ResizeObserver(layout).observe(get('frame'));
|
||||
for (const input of document.querySelectorAll('input')) input.addEventListener('change', visibility);
|
||||
get('lighting').addEventListener('click', () => {
|
||||
const button = get<HTMLButtonElement>('lighting');
|
||||
const dim = button.getAttribute('aria-pressed') !== 'true';
|
||||
button.setAttribute('aria-pressed', String(dim));
|
||||
button.textContent = `光照演示:${dim ? '环境减光' : '原始'}`;
|
||||
get('environment').style.setProperty('--demo-environment-brightness', dim ? '0.65' : '1');
|
||||
// No source assignment, load(), currentTime mutation or player recreation here.
|
||||
});
|
||||
get('closeup').addEventListener('click', () => {
|
||||
near = !near;
|
||||
const button = get<HTMLButtonElement>('closeup');
|
||||
button.setAttribute('aria-pressed', String(near));
|
||||
button.textContent = near ? '退出静态近景' : '进入静态近景';
|
||||
visibility();
|
||||
});
|
||||
play.addEventListener('click', () => { wantsPlay = !wantsPlay; void syncPlayback(); });
|
||||
document.addEventListener('visibilitychange', () => { void syncPlayback(); });
|
||||
reduced.addEventListener('change', () => {
|
||||
if (reduced.matches) wantsPlay = false;
|
||||
void syncPlayback();
|
||||
});
|
||||
video.addEventListener('error', () => {
|
||||
failed = true; ready = false; wantsPlay = false; play.disabled = true;
|
||||
fallback('动作素材缺失或解码失败,回退静态人物;未通过动态验证。');
|
||||
void syncPlayback();
|
||||
});
|
||||
|
||||
async function init() {
|
||||
try {
|
||||
const response = await fetch('./assets/manifest.json');
|
||||
if (!response.ok) throw new Error('manifest');
|
||||
const manifest: Manifest = await response.json();
|
||||
const motion = manifest.motion;
|
||||
if (!motion.file || !motion.alpha) {
|
||||
status.textContent = '静态分层可校对;无合格的透明动作素材,动态验收未通过。';
|
||||
play.disabled = true; play.textContent = '真实动作尚不可用'; return;
|
||||
}
|
||||
video.src = `./assets/${motion.file}`;
|
||||
video.addEventListener('loadeddata', () => {
|
||||
ready = true; play.disabled = false;
|
||||
status.textContent = `真实生成试片 · ${motion.status} · 循环切点与身份一致性以检查报告为准`;
|
||||
paint(); void syncPlayback();
|
||||
}, { once: true });
|
||||
} catch {
|
||||
fallback('素材清单不可用;保留静态分层校对。');
|
||||
play.textContent = '真实动作尚不可用';
|
||||
}
|
||||
}
|
||||
visibility(); layout(); void init();
|
||||
@@ -0,0 +1,22 @@
|
||||
@import './tokens.css';
|
||||
* { box-sizing: border-box; }
|
||||
[hidden] { display: none !important; }
|
||||
body { margin: 0; background: var(--scene-night-deep); color: var(--ink-primary); font: var(--fs-body) var(--font-sans); }
|
||||
main { max-width: 1600px; margin: auto; }
|
||||
header, section { padding: var(--space-4) var(--space-6); }
|
||||
h1 { font-size: var(--fs-title); margin: 0; }
|
||||
p { color: var(--ink-secondary); line-height: 1.5; }
|
||||
#stage { position: relative; overflow: hidden; height: min(60vh, 640px); background: var(--glass-bg-strong); }
|
||||
#frame { position: absolute; width: 100%; height: 100%; }
|
||||
#environment, .layer { position: absolute; width: 100%; height: 100%; object-fit: contain; }
|
||||
#environment { filter: brightness(var(--demo-environment-brightness, 1)); }
|
||||
#motion { position: absolute; }
|
||||
fieldset { border: 1px solid var(--glass-border); border-radius: var(--r-control); padding: var(--space-3); display: flex; flex-wrap: wrap; gap: var(--space-4); }
|
||||
label, button { cursor: pointer; }
|
||||
.controls { display: flex; flex-wrap: wrap; gap: var(--space-2); margin-top: var(--space-3); }
|
||||
button { color: var(--ink-primary); background: var(--glass-bg-strong); border: 1px solid var(--glass-highlight); border-radius: var(--r-control); padding: var(--space-3) var(--space-4); font: inherit; }
|
||||
button[aria-pressed=true] { border-color: var(--accent); }
|
||||
button:disabled { color: var(--ink-disabled); cursor: default; }
|
||||
button:focus-visible, input:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; }
|
||||
output { font-variant-numeric: tabular-nums; color: var(--accent); }
|
||||
@media (max-width: 600px) { header, section { padding: var(--space-3); } #stage { height: 38vh; } }
|
||||
@@ -0,0 +1,73 @@
|
||||
/* Reused verbatim from hel-700-rain-room; no alternative product theme. */
|
||||
:root {
|
||||
--scene-night-deep:#16233c;
|
||||
--scene-city-blue:#305481;
|
||||
--scene-city-mid:#5a76a2;
|
||||
--scene-tower:#2b456b;
|
||||
--scene-glass-blue:#718fc2;
|
||||
--scene-amber:#e8a45c;
|
||||
--scene-amber-soft:#f2c894;
|
||||
--scene-wood:#59361d;
|
||||
--scene-cream:#e8d9c8;
|
||||
--scene-blanket:#362928;
|
||||
--ink-primary:#f2efe9;
|
||||
--ink-secondary:rgba(242,239,233,.64);
|
||||
--ink-tertiary:rgba(242,239,233,.4);
|
||||
--ink-disabled:rgba(242,239,233,.28);
|
||||
--accent:#e8a45c;
|
||||
--accent-hover:#f2b879;
|
||||
--accent-pressed:#c9853f;
|
||||
--accent-ink:#1a1208;
|
||||
--glass-bg:rgba(14,18,30,.55);
|
||||
--glass-bg-strong:rgba(14,18,30,.72);
|
||||
--glass-border:rgba(255,255,255,.08);
|
||||
--glass-highlight:rgba(255,255,255,.14);
|
||||
--bubble-user:rgba(120,144,180,.2);
|
||||
--bubble-user-border:rgba(255,255,255,.07);
|
||||
--bubble-her:rgba(255,255,255,.06);
|
||||
--bubble-failed:rgba(200,90,80,.14);
|
||||
--state-danger:#e07a6a;
|
||||
--font-sans:"PingFang SC","HarmonyOS Sans SC","MiSans","Source Han Sans SC",system-ui,-apple-system,sans-serif;
|
||||
--fs-caption:12px;
|
||||
--fs-small:13px;
|
||||
--fs-body:14px;
|
||||
--fs-chat:15px;
|
||||
--fs-nav:15px;
|
||||
--fs-title:20px;
|
||||
--fs-brand:16px;
|
||||
--fs-time:13px;
|
||||
--space-1:4px;
|
||||
--space-2:8px;
|
||||
--space-3:12px;
|
||||
--space-4:16px;
|
||||
--space-5:20px;
|
||||
--space-6:24px;
|
||||
--space-8:32px;
|
||||
--space-10:40px;
|
||||
--r-panel:24px;
|
||||
--r-bubble:16px;
|
||||
--r-card:16px;
|
||||
--r-control:12px;
|
||||
--r-pill:999px;
|
||||
--shadow-panel:0 24px 80px rgba(0,0,0,.45);
|
||||
--shadow-pop:0 8px 24px rgba(0,0,0,.35);
|
||||
--glow-accent:0 0 12px rgba(232,164,92,.45);
|
||||
--glass-blur:blur(28px) saturate(1.2);
|
||||
--z-scene-far:0;
|
||||
--z-scene-rain:1;
|
||||
--z-scene-near:2;
|
||||
--z-caption:10;
|
||||
--z-nav:10;
|
||||
--z-topbar:20;
|
||||
--z-panel:30;
|
||||
--z-toast:40;
|
||||
--ease-out:cubic-bezier(.23,1,.32,1);
|
||||
--ease-in-out:cubic-bezier(.77,0,.175,1);
|
||||
--ease-drawer:cubic-bezier(.32,.72,0,1);
|
||||
--lamp-opacity:.12;
|
||||
--panel-width:clamp(360px,30vw,460px);
|
||||
--panel-top:64px;
|
||||
--panel-inset:24px;
|
||||
--scene-height:330px;
|
||||
--drawer-height:30dvh;
|
||||
}
|
||||
@@ -0,0 +1,144 @@
|
||||
"""Foreground Chromium interaction checks, screenshots and downloadable recording."""
|
||||
import functools
|
||||
import http.server
|
||||
import json
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
EVIDENCE = ROOT / 'evidence'
|
||||
|
||||
|
||||
class Quiet(http.server.SimpleHTTPRequestHandler):
|
||||
def log_message(self, *_args):
|
||||
pass
|
||||
|
||||
|
||||
def main():
|
||||
EVIDENCE.mkdir(exist_ok=True)
|
||||
handler = functools.partial(Quiet, directory=str(ROOT / 'dist'))
|
||||
server = http.server.ThreadingHTTPServer(('127.0.0.1', 0), handler)
|
||||
thread = threading.Thread(target=server.serve_forever)
|
||||
thread.start()
|
||||
results = {}
|
||||
try:
|
||||
with sync_playwright() as p:
|
||||
browser = p.chromium.launch()
|
||||
url = f'http://127.0.0.1:{server.server_port}'
|
||||
context = browser.new_context(viewport={'width': 1280, 'height': 800},
|
||||
record_video_dir=str(EVIDENCE),
|
||||
record_video_size={'width': 1280, 'height': 800})
|
||||
page = context.new_page()
|
||||
errors = []
|
||||
page.on('pageerror', lambda error: errors.append(str(error)))
|
||||
page.goto(url)
|
||||
page.wait_for_function('!document.querySelector("#play").disabled')
|
||||
assert page.locator('#source').evaluate('(v) => v.paused')
|
||||
page.click('#play')
|
||||
page.wait_for_timeout(800)
|
||||
alpha = page.locator('#motion').evaluate('(c) => {const x=c.getContext("2d"); return [x.getImageData(0,0,1,1).data[3], x.getImageData(300,220,1,1).data[3]]}')
|
||||
assert alpha[0] == 0 and alpha[1] > 240, alpha
|
||||
results['vp9_alpha_decoded_by_chromium'] = alpha
|
||||
page.wait_for_function('document.querySelector("#source").currentTime > 0.9 && document.querySelector("#source").currentTime < 1.4')
|
||||
positions = page.evaluate('''() => {
|
||||
const video=document.querySelector('#source'); const before=video.currentTime;
|
||||
const source=video.currentSrc; document.querySelector('#lighting').click();
|
||||
return {before, after:video.currentTime, sourcePreserved: source===video.currentSrc};
|
||||
}''')
|
||||
assert positions['before'] == positions['after'] and positions['sourcePreserved']
|
||||
results['environment_toggle_no_seek'] = positions
|
||||
page.wait_for_timeout(650)
|
||||
assert page.locator('#source').evaluate('(v) => !v.paused')
|
||||
page.click('#show-person')
|
||||
assert page.locator('#motion').is_hidden() and page.locator('#person').is_hidden()
|
||||
page.wait_for_timeout(600)
|
||||
page.screenshot(path=str(EVIDENCE / 'environment-only.png'))
|
||||
page.click('#show-person')
|
||||
page.click('#show-occlusion')
|
||||
assert page.locator('#occlusion').is_hidden()
|
||||
page.click('#show-occlusion')
|
||||
page.click('#show-shadow')
|
||||
assert page.locator('#shadow').is_hidden()
|
||||
page.click('#show-shadow')
|
||||
page.click('#show-mask')
|
||||
assert page.locator('#mask').is_visible() and page.locator('#environment').is_hidden()
|
||||
page.wait_for_timeout(500)
|
||||
page.click('#show-mask')
|
||||
results['independent_layers'] = True
|
||||
page.click('#closeup')
|
||||
assert page.locator('#talk').is_visible()
|
||||
assert page.locator('#source').evaluate('(v) => !v.paused')
|
||||
page.wait_for_timeout(700)
|
||||
page.screenshot(path=str(EVIDENCE / 'static-near.png'))
|
||||
page.click('#closeup')
|
||||
assert page.locator('#talk').is_hidden()
|
||||
page.wait_for_timeout(3400) # Includes a real hard loop.
|
||||
results['near_is_static_player_continues'] = True
|
||||
page.click('#play')
|
||||
t = page.locator('#source').evaluate('(v) => v.currentTime')
|
||||
page.wait_for_timeout(250)
|
||||
assert page.locator('#source').evaluate('(v) => v.currentTime') == t
|
||||
results['manual_pause'] = True
|
||||
# Chromium lifecycle API freezes the page after making it hidden.
|
||||
session = context.new_cdp_session(page)
|
||||
session.send('Page.setWebLifecycleState', {'state': 'active'})
|
||||
# Visibility handler unit probe. Browser screenshot/recording checks above are real.
|
||||
page.click('#play')
|
||||
page.evaluate('''() => {
|
||||
Object.defineProperty(document, 'hidden', {configurable:true, get:()=>true});
|
||||
document.dispatchEvent(new Event('visibilitychange'));
|
||||
}''')
|
||||
assert page.locator('#source').evaluate('(v) => v.paused')
|
||||
page.evaluate('''() => {
|
||||
delete document.hidden; document.dispatchEvent(new Event('visibilitychange'));
|
||||
}''')
|
||||
page.wait_for_function('!document.querySelector("#source").paused')
|
||||
results['visibility_pause_resume_handler_synthetic_event'] = True
|
||||
for width, height in [(1600, 900), (1280, 800), (390, 844)]:
|
||||
page.set_viewport_size({'width': width, 'height': height})
|
||||
page.wait_for_timeout(180)
|
||||
bounds = page.evaluate('''() => {
|
||||
const r=document.querySelector('#motion').getBoundingClientRect();
|
||||
const s=document.querySelector('#stage').getBoundingClientRect();
|
||||
return {faceX:r.x+305/552*r.width, faceY:r.y+135/552*r.height,
|
||||
stage:{left:s.left,right:s.right,top:s.top,bottom:s.bottom},
|
||||
scroll:document.documentElement.scrollWidth, width:innerWidth};
|
||||
}''')
|
||||
assert bounds['scroll'] <= width
|
||||
assert bounds['stage']['left'] <= bounds['faceX'] <= bounds['stage']['right']
|
||||
assert bounds['stage']['top'] <= bounds['faceY'] <= bounds['stage']['bottom']
|
||||
page.screenshot(path=str(EVIDENCE / f'viewport-{width}x{height}.png'))
|
||||
results[f'viewport_{width}x{height}'] = bounds
|
||||
page.emulate_media(reduced_motion='reduce')
|
||||
page.wait_for_timeout(100)
|
||||
assert page.locator('#source').evaluate('(v) => v.paused')
|
||||
assert page.locator('#person').is_visible() and page.locator('#motion').is_hidden()
|
||||
results['reduced_motion_static_fallback'] = True
|
||||
page.emulate_media(reduced_motion='no-preference')
|
||||
assert page.locator('#source').evaluate('(v) => v.paused')
|
||||
video_path = Path(page.video.path())
|
||||
context.close()
|
||||
video_path.rename(EVIDENCE / 'browser-continuous.webm')
|
||||
missing = browser.new_page()
|
||||
missing.route('**/reading-alpha.webm', lambda route: route.abort())
|
||||
missing.goto(url)
|
||||
missing.wait_for_function('!document.querySelector("#fallback").hidden')
|
||||
assert missing.locator('#person').is_visible()
|
||||
assert missing.locator('#play').is_disabled()
|
||||
results['missing_motion_static_fallback'] = True
|
||||
missing.close()
|
||||
assert not errors, errors
|
||||
results['page_errors'] = errors
|
||||
browser.close()
|
||||
finally:
|
||||
server.shutdown()
|
||||
thread.join()
|
||||
server.server_close()
|
||||
(EVIDENCE / 'browser-results.json').write_text(json.dumps(results, ensure_ascii=False, indent=2))
|
||||
print(json.dumps(results, ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,33 @@
|
||||
"""Reproduce the separately switchable foreground and provisional contact shadow."""
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image, ImageDraw, ImageFilter
|
||||
|
||||
|
||||
def main():
|
||||
assets = Path(__file__).resolve().parents[1] / 'public/assets'
|
||||
source = Image.open(assets / 'reference.png').convert('RGBA')
|
||||
mask = Image.new('L', source.size)
|
||||
ImageDraw.Draw(mask).polygon([
|
||||
(1240, 582), (1282, 566), (1316, 556), (1360, 552), (1403, 532),
|
||||
(1438, 526), (1486, 514), (1517, 521), (1539, 544), (1559, 556),
|
||||
(1565, 634), (1520, 707), (1407, 747), (1260, 703), (1227, 650),
|
||||
], fill=255)
|
||||
mask = mask.filter(ImageFilter.GaussianBlur(1))
|
||||
source.putalpha(mask)
|
||||
source.save(assets / 'occlusion.png')
|
||||
mask.save(assets / 'occlusion-mask.png')
|
||||
mask = Image.new('L', source.size)
|
||||
ImageDraw.Draw(mask).ellipse((1230, 511, 1512, 595), fill=65)
|
||||
mask = mask.filter(ImageFilter.GaussianBlur(12))
|
||||
shadow = Image.new('RGBA', source.size, (44, 28, 20, 0))
|
||||
shadow.putalpha(mask)
|
||||
shadow.save(assets / 'contact-shadow.png')
|
||||
composite = Image.open(assets / 'environment.png').convert('RGBA')
|
||||
for name in ['contact-shadow.png', 'person.png', 'occlusion.png']:
|
||||
composite = Image.alpha_composite(composite, Image.open(assets / name))
|
||||
composite.save(assets / 'composite.png')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,74 @@
|
||||
"""Use installed SAM modeling modules directly; no predictor/OpenCV dependency.
|
||||
|
||||
Run in the existing ComfyUI venv. No downloads or dependency changes.
|
||||
"""
|
||||
import argparse
|
||||
import importlib
|
||||
import sys
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import torch
|
||||
from PIL import Image, ImageChops, ImageDraw, ImageFilter
|
||||
|
||||
|
||||
def load_sam(source: Path, checkpoint: Path):
|
||||
# The optional predictor package imports cv2. We only use the original model API.
|
||||
package = types.ModuleType("hel711_sam")
|
||||
package.__path__ = [str(source)]
|
||||
sys.modules[package.__name__] = package
|
||||
builder = importlib.import_module("hel711_sam.build_sam")
|
||||
return builder.build_sam_vit_b(str(checkpoint)).eval()
|
||||
|
||||
|
||||
def segment(model, image):
|
||||
width, height = image.size
|
||||
scale = 1024 / max(width, height)
|
||||
size = (round(width * scale), round(height * scale))
|
||||
tensor = torch.from_numpy(np.array(image.resize(size))).permute(2, 0, 1).to(model.device)
|
||||
# Crop coordinate system: fixed right-hand seat, 552 x 552.
|
||||
points = torch.tensor([[[305, 140], [300, 245], [200, 300], [307, 301],
|
||||
[80, 100], [450, 400], [330, 450]]], dtype=torch.float)
|
||||
points = (points * scale).to(model.device)
|
||||
labels = torch.tensor([[1, 1, 1, 1, 0, 0, 0]], device=model.device)
|
||||
box = torch.tensor([[100, 60, 420, 380]], dtype=torch.float, device=model.device) * scale
|
||||
with torch.inference_mode():
|
||||
result = model([dict(image=tensor, original_size=(height, width),
|
||||
point_coords=points, point_labels=labels, boxes=box)],
|
||||
multimask_output=False)[0]
|
||||
mask = result['masks'][0, 0].cpu().numpy().astype(np.uint8) * 255
|
||||
mask = Image.fromarray(mask)
|
||||
# The book/knit texture produces enclosed SAM holes. Fill enclosed holes only.
|
||||
exterior = mask.copy()
|
||||
ImageDraw.floodfill(exterior, (0, 0), 255)
|
||||
mask = ImageChops.lighter(mask, ImageChops.invert(exterior))
|
||||
return mask.filter(ImageFilter.GaussianBlur(0.65))
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('assets', type=Path)
|
||||
parser.add_argument('--sam-source', type=Path, required=True)
|
||||
parser.add_argument('--checkpoint', type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
torch.set_num_threads(8)
|
||||
model = load_sam(args.sam_source, args.checkpoint)
|
||||
source = Image.open(args.assets / 'reference.png').convert('RGB')
|
||||
region = (1120, 200, 1672, 752)
|
||||
crop = source.crop(region)
|
||||
mask = segment(model, crop)
|
||||
full_mask = Image.new('L', source.size)
|
||||
full_mask.paste(mask, region[:2])
|
||||
full_mask.save(args.assets / 'person-mask.png')
|
||||
layer = source.convert('RGBA')
|
||||
layer.putalpha(full_mask)
|
||||
layer.save(args.assets / 'person.png')
|
||||
crop.save(args.assets / 'generation-input.png')
|
||||
environment = Image.open(args.assets / 'environment.png').convert('RGBA')
|
||||
Image.alpha_composite(environment, layer).save(args.assets / 'composite.png')
|
||||
print('Saved original-pixel person layer, SAM mask, crop and real composite.', flush=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,141 @@
|
||||
"""One foreground job using the installed ComfyUI native Wan nodes.
|
||||
|
||||
Owns and stops its exact server PID in finally. Does not modify the installation.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path('/media/lucian/Downloads/ComfyUI')
|
||||
TASK = ROOT / 'output/hel711-task'
|
||||
URL = 'http://127.0.0.1:18189'
|
||||
|
||||
|
||||
def request(path, payload=None):
|
||||
data = None if payload is None else json.dumps(payload).encode()
|
||||
req = urllib.request.Request(URL + path, data=data,
|
||||
headers={'Content-Type': 'application/json'})
|
||||
with urllib.request.urlopen(req, timeout=20) as response:
|
||||
return json.load(response)
|
||||
|
||||
|
||||
def graph():
|
||||
nodes = {}
|
||||
|
||||
def add(key, kind, **inputs):
|
||||
nodes[str(key)] = dict(class_type=kind, inputs=inputs)
|
||||
|
||||
for key, mode in [(1, 'high'), (2, 'low')]:
|
||||
add(key, 'UNETLoader', unet_name=f'wan2.2_i2v_{mode}_noise_14B_fp8_scaled.safetensors',
|
||||
weight_dtype='default')
|
||||
add(key + 2, 'LoraLoaderModelOnly', model=[str(key), 0],
|
||||
lora_name=f'wan2.2_i2v_lightx2v_4steps_lora_v1_{mode}_noise.safetensors',
|
||||
strength_model=1.0)
|
||||
add(key + 4, 'ModelSamplingSD3', model=[str(key + 2), 0], shift=5.0)
|
||||
add(7, 'CLIPLoader', clip_name='umt5_xxl_fp8_e4m3fn_scaled.safetensors',
|
||||
type='wan', device='cpu')
|
||||
add(8, 'VAELoader', vae_name='wan_2.1_vae.safetensors')
|
||||
add(9, 'LoadImage', image='generation-input.png')
|
||||
add(10, 'CLIPTextEncode', clip=['7', 0], text=(
|
||||
'Locked off camera, photorealistic young adult woman with short dark hair, '
|
||||
'cream cardigan and dark trousers seated on the same sofa in the same bedroom. '
|
||||
'She quietly reads the book in her hands, slowly lowers her gaze to the page '
|
||||
'then gently raises her head a little. Small natural eye blink and breathing. '
|
||||
'Her hands keep holding the book. Fixed seat position, stationary room and furniture.'))
|
||||
add(11, 'CLIPTextEncode', clip=['7', 0],
|
||||
text='camera movement, zoom, changing face, changing clothes, standing up, '
|
||||
'extra fingers, distorted hands, deformed face, moving furniture, subtitles')
|
||||
add(12, 'WanImageToVideo', positive=['10', 0], negative=['11', 0], vae=['8', 0],
|
||||
width=512, height=512, length=49, batch_size=1, start_image=['9', 0])
|
||||
add(13, 'KSamplerAdvanced', model=['5', 0], add_noise='enable', noise_seed=711,
|
||||
steps=4, cfg=1.0, sampler_name='euler', scheduler='simple',
|
||||
positive=['12', 0], negative=['12', 1], latent_image=['12', 2],
|
||||
start_at_step=0, end_at_step=2, return_with_leftover_noise='enable')
|
||||
add(14, 'KSamplerAdvanced', model=['6', 0], add_noise='disable', noise_seed=711,
|
||||
steps=4, cfg=1.0, sampler_name='euler', scheduler='simple',
|
||||
positive=['12', 0], negative=['12', 1], latent_image=['13', 0],
|
||||
start_at_step=2, end_at_step=4, return_with_leftover_noise='disable')
|
||||
add(15, 'VAEDecode', samples=['14', 0], vae=['8', 0])
|
||||
add(16, 'SaveWEBM', images=['15', 0], codec='vp9', fps=16.0,
|
||||
filename_prefix='reading', crf=20.0)
|
||||
return nodes
|
||||
|
||||
|
||||
def main():
|
||||
for name in ['user', 'temp']:
|
||||
(TASK / name).mkdir(exist_ok=True)
|
||||
os.environ['HF_HUB_OFFLINE'] = '1'
|
||||
os.environ['TRANSFORMERS_OFFLINE'] = '1'
|
||||
command = [str(ROOT / 'venv/bin/python'), str(ROOT / 'main.py'),
|
||||
'--listen', '127.0.0.1', '--port', '18189', '--disable-all-custom-nodes',
|
||||
'--offline', '--disable-manager-ui', '--disable-assets',
|
||||
'--reserve-vram', '8', '--disable-dynamic-vram',
|
||||
'--input-directory', str(TASK / 'assets'),
|
||||
'--output-directory', str(TASK), '--user-directory', str(TASK / 'user'),
|
||||
'--temp-directory', str(TASK / 'temp'), '--database-url', 'sqlite:///:memory:']
|
||||
# Refuse to attach to an unrelated listener.
|
||||
import socket
|
||||
with socket.socket() as sock:
|
||||
if sock.connect_ex(('127.0.0.1', 18189)) == 0:
|
||||
raise RuntimeError('Task port already occupied; no process started')
|
||||
with open(TASK / 'server.log', 'w') as log:
|
||||
server = subprocess.Popen(command, cwd=ROOT, stdout=log, stderr=subprocess.STDOUT)
|
||||
(TASK / 'process.json').write_text(json.dumps(dict(pid=server.pid, command=command)))
|
||||
try:
|
||||
deadline = time.monotonic() + 90
|
||||
while True:
|
||||
if server.poll() is not None:
|
||||
raise RuntimeError(f'ComfyUI exited: {server.returncode}; see server.log')
|
||||
try:
|
||||
info = request('/object_info')
|
||||
break
|
||||
except (OSError, urllib.error.URLError):
|
||||
if time.monotonic() > deadline:
|
||||
raise TimeoutError('Startup exceeded 90 seconds') from None
|
||||
time.sleep(1)
|
||||
workflow = graph()
|
||||
(TASK / 'workflow.json').write_text(json.dumps(workflow, indent=2))
|
||||
required = {n['class_type']: info.get(n['class_type']) for n in workflow.values()}
|
||||
(TASK / 'nodes.json').write_text(json.dumps(required, indent=2))
|
||||
if any(v is None for v in required.values()):
|
||||
raise RuntimeError('Required native node unavailable; see nodes.json')
|
||||
result = request('/prompt', {'prompt': workflow})
|
||||
(TASK / 'submission.json').write_text(json.dumps(result))
|
||||
prompt_id = result['prompt_id']
|
||||
print(f'Generating one clip: {prompt_id}; owned server PID {server.pid}', flush=True)
|
||||
deadline = time.monotonic() + 2400
|
||||
last_report = 0
|
||||
while time.monotonic() < deadline:
|
||||
history = request('/history/' + prompt_id)
|
||||
if prompt_id in history:
|
||||
(TASK / 'history.json').write_text(json.dumps(history, indent=2))
|
||||
print(json.dumps(history[prompt_id]['status']), flush=True)
|
||||
if history[prompt_id]['status']['status_str'] != 'success':
|
||||
raise RuntimeError('Generation failed; see history.json')
|
||||
break
|
||||
if time.monotonic() - last_report > 30:
|
||||
print('Single generation still active; existing services untouched.', flush=True)
|
||||
last_report = time.monotonic()
|
||||
time.sleep(3)
|
||||
else:
|
||||
request('/interrupt', {})
|
||||
raise TimeoutError('Single generation exceeded 40 minutes')
|
||||
except urllib.error.HTTPError as error:
|
||||
(TASK / 'api-error.json').write_bytes(error.read())
|
||||
raise
|
||||
finally:
|
||||
server.terminate()
|
||||
try:
|
||||
server.wait(timeout=20)
|
||||
except subprocess.TimeoutExpired:
|
||||
server.kill()
|
||||
server.wait()
|
||||
print(f'Stopped owned server PID {server.pid}', flush=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Per-frame real SAM segmentation, then VP9 alpha encoding with existing PyAV."""
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import av
|
||||
import numpy as np
|
||||
import torch
|
||||
from extract_layers import load_sam, segment
|
||||
from PIL import Image
|
||||
|
||||
ROOT = Path('/media/lucian/Downloads/ComfyUI')
|
||||
TASK = ROOT / 'output/hel711-task'
|
||||
|
||||
|
||||
def main():
|
||||
torch.set_num_threads(8)
|
||||
model = load_sam(
|
||||
ROOT / 'custom_nodes/was-node-suite-comfyui/repos/SAM/segment_anything',
|
||||
ROOT / 'models/sams/sam_vit_b_01ec64.pth').to('cuda')
|
||||
output = av.open(str(TASK / 'reading-alpha.webm'), 'w')
|
||||
stream = output.add_stream('libvpx-vp9', rate=16)
|
||||
stream.width = 552
|
||||
stream.height = 552
|
||||
stream.pix_fmt = 'yuva420p'
|
||||
stream.options = {'crf': '20', 'b': '0', 'auto-alt-ref': '0'}
|
||||
metrics = []
|
||||
previews = []
|
||||
source = av.open(str(TASK / 'reading_00001_.webm'))
|
||||
previous = None
|
||||
first = None
|
||||
for index, decoded in enumerate(source.decode(video=0)):
|
||||
image = decoded.to_image().resize((552, 552), Image.Resampling.LANCZOS)
|
||||
mask = segment(model, image)
|
||||
frame = image.convert('RGBA')
|
||||
frame.putalpha(mask)
|
||||
pixels = np.array(image).astype(float)
|
||||
if first is None:
|
||||
first = pixels
|
||||
metrics.append(dict(frame=index, mask_pixels=int((np.array(mask) > 127).sum()),
|
||||
mean_absolute_rgb_change=0 if previous is None else float(np.abs(pixels-previous).mean())))
|
||||
previous = pixels
|
||||
if index in [0, 16, 32, 48]:
|
||||
previews.append(frame)
|
||||
frame.save(TASK / f'matted-{index:02}.png')
|
||||
for packet in stream.encode(av.VideoFrame.from_ndarray(np.array(frame), format='rgba')):
|
||||
output.mux(packet)
|
||||
if index % 16 == 0:
|
||||
print(f'Matted frame {index}', flush=True)
|
||||
for packet in stream.encode():
|
||||
output.mux(packet)
|
||||
output.close()
|
||||
source.close()
|
||||
(TASK / 'frame-metrics.json').write_text(json.dumps(dict(
|
||||
frames=metrics, loop_rgb_difference=float(np.abs(previous-first).mean()),
|
||||
interpretation='Real generated frames; RGB differences alone are not proof of valid anatomy or identity.'), indent=2))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "Bundler",
|
||||
"lib": [
|
||||
"ES2022",
|
||||
"DOM",
|
||||
"DOM.Iterable"
|
||||
],
|
||||
"strict": true,
|
||||
"noEmit": true,
|
||||
"skipLibCheck": true,
|
||||
"noUnusedLocals": true,
|
||||
"noUnusedParameters": true
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user