diff --git a/README.md b/README.md index f57c22d5..08ffae84 100644 --- a/README.md +++ b/README.md @@ -164,6 +164,23 @@ Auto is safe even when scenes are queued in one batch: Calliope's queue renders The workflow pattern (per [kat3ri/ComfyUI-MiniMax-H3-Extend](https://github.com/kat3ri/ComfyUI-MiniMax-H3-Extend)) is a `LoadVideo (Input:video)` node feeding the MiniMax H3 extend patched nodes (`MiniMaxH3EncodeAVPatched` → `MiniMaxH3VideoExtendPatched`) with the `(Output:video)` node at the end. Recommended starting settings from that repo: `context_frames` **2**, `ref_spacing` **1–2**, `ref_decay` **0.3**, `ref_ramp` **3–4** (5–6 if the prior clip had heavy motion). +### Review the prompt before you generate + +Generate no longer fires blind. Hitting **Generate clip** first opens a prompt preview: the exact text that will land on the workflow's `(Input:prompt)` node — your saved draft if there is one, otherwise a fresh MiniMax H3 rewrite (six-section format) or the prose scene prompt. + +- **Edit it inline** — typos, camera notes, pacing, anything. The edited text is what gets sent. +- **Regenerate** re-runs the H3 rewrite for a different take. +- **Save draft** keeps it on the scene; future generates (single or **Generate all**) reuse the draft instead of calling the LLM again. A hint appears when the draft predates changes to the scene. +- **Cancel** aborts with nothing enqueued. + +After a render, **View prompt & inputs** opens the scene's render history: every job as a chip, the payload each one actually sent to ComfyUI, and **Copy settings to form** to pull a past job's input values back into the live form. + +Your video-stage setup (workflow choice, input values, clip source) auto-saves per scene and comes back after a reload or app restart. **Generate all** honors every scene's saved setup and drafts — the toast reports how many drafts were used. + +### Better ComfyUI errors + +When ComfyUI rejects a workflow, the job error now names the actual cause and node — e.g. `ComfyUI rejected the workflow (400): prompt_outputs_failed_validation; node 12: Invalid audio file: "voice.m4a"` — instead of a bare status code. Audio reference inputs upload to ComfyUI's flat input directory and work with both stock `LoadAudio` and VHS's `VHS_LoadAudio`. + ### 3. Export the workflow In ComfyUI, use **Save (API Format)** — not the regular UI workflow graph format. Calliope only understands API Format JSON. diff --git a/calliope-backend/pyproject.toml b/calliope-backend/pyproject.toml index 23b75661..f58746cd 100644 --- a/calliope-backend/pyproject.toml +++ b/calliope-backend/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "calliope" -version = "1.2.3" +version = "1.3.0" description = "Local-first AI story-to-video studio" requires-python = ">=3.11" dependencies = [ diff --git a/calliope-backend/src/calliope/agent/video_agent.py b/calliope-backend/src/calliope/agent/video_agent.py index 0d07ba47..d380974c 100644 --- a/calliope-backend/src/calliope/agent/video_agent.py +++ b/calliope-backend/src/calliope/agent/video_agent.py @@ -1,6 +1,7 @@ """Enqueue per-scene video generation jobs.""" from __future__ import annotations +import hashlib import json import logging import sqlite3 @@ -145,12 +146,122 @@ def _get_workflow(workflow_id: int | None = None) -> dict[str, Any] | None: conn.close() +def _scene_video_settings(scene: dict[str, Any]) -> dict[str, Any]: + """Parsed scenes.video_settings_json — {} when unset or malformed.""" + raw = scene.get("video_settings_json") + if not raw: + return {} + try: + data = json.loads(raw) + except (json.JSONDecodeError, TypeError): + return {} + return data if isinstance(data, dict) else {} + + +def _stored_input_values(scene: dict[str, Any]) -> dict[str, Any]: + """Saved per-scene form values (input_values inside video_settings).""" + values = _scene_video_settings(scene).get("input_values") + if isinstance(values, dict): + return {k: v for k, v in values.items() if v not in (None, "")} + return {} + + +def _stored_prompt_draft(scene: dict[str, Any]) -> str | None: + draft = _scene_video_settings(scene).get("prompt_draft") + return draft if isinstance(draft, str) and draft.strip() else None + + +def _scene_prompt_hash(scene: dict[str, Any]) -> str: + """Cheap fingerprint of the inputs a draft was based on (stale detection).""" + basis = "|".join( + str(scene.get(k) or "") + for k in ("heading", "action", "dialog", "duration_sec", "location_id") + ) + chars = ",".join(str(c) for c in sorted(scene.get("character_ids") or [])) + return hashlib.sha256((basis + "|" + chars).encode()).hexdigest()[:16] + + +async def preview_scene_prompt( + project_id: int, + scene_id: int, + workflow_id: int | None = None, +) -> dict[str, Any]: + """Resolve the exact prompt a Generate would send — without enqueueing. + + Returns {"prompt": str, "profile": str, "from_draft": bool, "based_on": str} + so the UI can show/edit/save it before the user commits to a render. + """ + conn = get_db(settings.db_path) + try: + row = conn.execute( + "SELECT * FROM scenes WHERE id = ? AND project_id = ?", + (scene_id, project_id), + ).fetchone() + if not row: + raise ValueError(f"Scene {scene_id} not found in project {project_id}") + scene = row_to_dict(row) + char_rows = conn.execute( + """ + SELECT c.* FROM characters c + JOIN scene_characters sc ON sc.character_id = c.id + WHERE sc.scene_id = ? + """, + (scene_id,), + ).fetchall() + characters = [row_to_dict(r) for r in char_rows] + loc_image = scene.get("env_image_path") + loc_row: dict[str, Any] | None = None + if scene.get("location_id"): + loc = conn.execute( + "SELECT name, description, consistency_prompt, reference_image_path " + "FROM locations WHERE id = ?", + (scene["location_id"],), + ).fetchone() + if loc: + loc_row = row_to_dict(loc) + if not loc_image: + loc_image = loc_row["reference_image_path"] + finally: + conn.close() + + wf_id = workflow_id or scene.get("workflow_id") + workflow = _get_workflow(wf_id) + if not workflow: + raise ValueError("No enabled video workflow found — configure one in Settings") + inputs = parse_dynamic_inputs(_workflow_json(workflow)) + profile = workflow.get("prompt_profile") or "prose" + based_on = _scene_prompt_hash(scene) + + if profile == "minimax_h3_ref": + # Fresh saved draft short-circuits the LLM call + draft = _stored_prompt_draft(scene) + if draft: + meta = _scene_video_settings(scene).get("prompt_draft_meta") or {} + if meta.get("based_on") == based_on: + return { + "prompt": draft, + "profile": profile, + "from_draft": True, + "based_on": based_on, + } + subjects, _ = _h3_subjects(characters, loc_row, loc_image, inputs) + await event_bus.publish( + "agent.thinking", + {"message": f"H3 prompt rewrite · scene {scene.get('order_index')}", "project_id": project_id}, + ) + prompt = await _h3_rewrite(scene, subjects) + else: + prompt = scene_video_prompt(scene, characters) + return {"prompt": prompt, "profile": profile, "from_draft": False, "based_on": based_on} + + async def enqueue_video_jobs( project_id: int, *, scene_ids: list[int] | None = None, workflow_id: int | None = None, input_values_override: dict[str, Any] | None = None, + prompts: dict[int, str] | None = None, ) -> list[dict[str, Any]]: await event_bus.publish( "agent.thinking", {"message": "Queuing video jobs…", "project_id": project_id} @@ -221,33 +332,67 @@ async def enqueue_video_jobs( loc_image = loc_row["reference_image_path"] profile = (workflow or {}).get("prompt_profile") or "prose" + # Merge base: saved per-scene setup first, explicit request wins on + # top — so batch Generate-all honors persisted form setups. + stored_values = _stored_input_values(scene) + extra_values: dict[str, Any] = {**stored_values} + if input_values_override: + extra_values.update( + {k: v for k, v in input_values_override.items() if v not in (None, "")} + ) if profile == "minimax_h3_ref": subjects, ref_paths = _h3_subjects(characters, loc_row, loc_image, inputs) - await event_bus.publish( - "agent.thinking", - { - "message": f"H3 prompt rewrite · scene {scene.get('order_index')}", - "project_id": project_id, - }, - ) - prompt = await _h3_rewrite(scene, subjects) + # Prompt precedence: explicit request → saved (fresh) draft → LLM. + explicit_prompt = (prompts or {}).get(scene["id"]) + fresh_draft = None + if explicit_prompt is None: + candidate = _stored_prompt_draft(scene) + if candidate: + meta = _scene_video_settings(scene).get("prompt_draft_meta") or {} + if meta.get("based_on") == _scene_prompt_hash(scene): + fresh_draft = candidate + if explicit_prompt is not None: + prompt = explicit_prompt + elif fresh_draft: + prompt = fresh_draft + else: + await event_bus.publish( + "agent.thinking", + { + "message": f"H3 prompt rewrite · scene {scene.get('order_index')}", + "project_id": project_id, + }, + ) + prompt = await _h3_rewrite(scene, subjects) values = smart_fill_inputs( inputs, prompt=prompt, ref_images=ref_paths, duration=duration, - extra=input_values_override, + extra=extra_values, ) else: - prompt = scene_video_prompt(scene, characters) + prompt = (prompts or {}).get(scene["id"]) or scene_video_prompt( + scene, characters + ) values = smart_fill_inputs( inputs, prompt=prompt, character_image=char_image, location_image=loc_image, duration=duration, - extra=input_values_override, + extra=extra_values, ) + # Stored per-scene setups override smart-fill's context choices + # (e.g. an edited duration). smart_fill skips duration-role nodes + # in `extra` by design — re-apply them here, request values win. + explicit_final = { + **stored_values, + **{k: v for k, v in (input_values_override or {}).items() if v not in (None, "")}, + } + for k, v in explicit_final.items(): + if v not in (None, ""): + values[str(k)] = v payload: dict[str, Any] = {"input_values": values, "prompt": prompt} if scene.get("chain_from_prev"): video_input = _video_input(inputs) diff --git a/calliope-backend/src/calliope/comfyui/client.py b/calliope-backend/src/calliope/comfyui/client.py index 691777be..0a724e6a 100644 --- a/calliope-backend/src/calliope/comfyui/client.py +++ b/calliope-backend/src/calliope/comfyui/client.py @@ -14,6 +14,44 @@ from calliope.comfyui.registry import IMAGE_CLASSES, AUDIO_CLASSES, VIDEO_CLASSE logger = logging.getLogger("calliope.comfyui") +def _surface_error(prefix: str, resp: httpx.Response) -> RuntimeError: + """Raise with ComfyUI's own error body instead of an opaque status line. + + Comfy's /prompt replies carry {"error": {...}, "node_errors": {node_id: {...}}} + naming the exact node and message (e.g. 'Invalid audio file'). raise_for_status() + discards all of it, which turns diagnosable failures into one-line bug reports. + """ + detail = "" + try: + body = resp.json() + except Exception: + body = None + if isinstance(body, dict): + parts: list[str] = [] + err = body.get("error") + if isinstance(err, dict): + err_type = err.get("type") or err.get("message") + if err_type: + parts.append(str(err_type)) + elif isinstance(err, str) and err: + parts.append(err) + node_errors = body.get("node_errors") + if isinstance(node_errors, dict): + for node_id, node_err in node_errors.items(): + if not isinstance(node_err, dict): + continue + errors = node_err.get("errors") + if isinstance(errors, list): + for e in errors: + if isinstance(e, dict): + msg = e.get("message") or e.get("details") + if msg: + parts.append(f"node {node_id}: {msg}") + detail = "; ".join(parts) + suffix = f": {detail}" if detail else "" + return RuntimeError(f"{prefix} ({resp.status_code}){suffix}") + + class ComfyUIClient: def __init__(self, base_url: str | None = None) -> None: self.base_url = (base_url or settings.comfyui_base_url).rstrip("/") @@ -35,18 +73,28 @@ class ComfyUIClient: files = {"image": (path.name, data, "application/octet-stream")} form = {"overwrite": "true", "subfolder": subfolder} resp = await self._http.post(f"{self.base_url}/upload/image", files=files, data=form) - resp.raise_for_status() + if resp.status_code >= 400: + raise _surface_error("ComfyUI image upload failed", resp) result = resp.json() name = result.get("name", path.name) sub = result.get("subfolder") or subfolder return f"{sub}/{name}" if sub else name - async def upload_audio(self, path: Path, subfolder: str = "calliope") -> str: + async def upload_audio(self, path: Path, subfolder: str = "") -> str: + """Upload audio via Comfy's /upload/image route (there is no /upload/audio + in core ComfyUI — its own frontend posts audio there too, field `image`). + + Flat input dir + bare filename: LoadAudio's file list and VALIDATE_INPUTS + existence check are most reliable without a subfolder prefix. + """ data = path.read_bytes() - files = {"image": (path.name, data, "application/octet-stream")} # Comfy uses image field - form = {"overwrite": "true", "subfolder": subfolder, "type": "input"} + files = {"image": (path.name, data, "application/octet-stream")} + form = {"overwrite": "true", "type": "input"} + if subfolder: + form["subfolder"] = subfolder resp = await self._http.post(f"{self.base_url}/upload/image", files=files, data=form) - resp.raise_for_status() + if resp.status_code >= 400: + raise _surface_error("ComfyUI audio upload failed", resp) result = resp.json() name = result.get("name", path.name) sub = result.get("subfolder") or subfolder @@ -58,7 +106,8 @@ class ComfyUIClient: files = {"image": (path.name, data, "application/octet-stream")} form = {"overwrite": "true", "subfolder": subfolder, "type": "input"} resp = await self._http.post(f"{self.base_url}/upload/image", files=files, data=form) - resp.raise_for_status() + if resp.status_code >= 400: + raise _surface_error("ComfyUI video upload failed", resp) result = resp.json() name = result.get("name", path.name) sub = result.get("subfolder") or subfolder @@ -79,12 +128,20 @@ class ComfyUIClient: inputs["image"] = await self.upload_image(path) node["inputs"] = inputs elif class_type in AUDIO_CLASSES: - audio = inputs.get("audio") - if isinstance(audio, str) and self._looks_like_local_path(audio): - path = Path(audio) - if path.exists(): - inputs["audio"] = await self.upload_audio(path) - node["inputs"] = inputs + # VHS_LoadAudio names its widget "audio:" (with colon); stock + # LoadAudio uses "audio". Probe both so the file is uploaded + # whichever variant the workflow uses. + audio_key = next( + (k for k in ("audio", "audio:") if isinstance(inputs.get(k), str)), + None, + ) + if audio_key: + audio = inputs[audio_key] + if isinstance(audio, str) and self._looks_like_local_path(audio): + path = Path(audio) + if path.exists(): + inputs[audio_key] = await self.upload_audio(path) + node["inputs"] = inputs elif class_type in VIDEO_CLASSES: field = "file" if class_type in VIDEO_FILE_CLASSES else "video" media = inputs.get(field) @@ -105,7 +162,8 @@ class ComfyUIClient: async def queue_prompt(self, workflow: dict[str, Any]) -> str: payload = {"prompt": workflow, "client_id": self.client_id} resp = await self._http.post(f"{self.base_url}/prompt", json=payload) - resp.raise_for_status() + if resp.status_code >= 400: + raise _surface_error("ComfyUI rejected the workflow", resp) data = resp.json() prompt_id = data.get("prompt_id") if not prompt_id: diff --git a/calliope-backend/src/calliope/comfyui/patcher.py b/calliope-backend/src/calliope/comfyui/patcher.py index 32275b9d..13f8be59 100644 --- a/calliope-backend/src/calliope/comfyui/patcher.py +++ b/calliope-backend/src/calliope/comfyui/patcher.py @@ -7,6 +7,25 @@ from typing import Any from calliope.comfyui.registry import class_to_patch_field +def _resolve_field(field: str, inputs: dict[str, Any]) -> str: + """Map the computed patch field onto a key that exists on this node. + + Known variants: + - text ↔ value (PrimitiveString-style nodes expose `value`, not `text`). + - audio ↔ audio: (VHS_LoadAudio names its widget `audio:` with a colon). + The fallback stays guarded to these exact sibling pairs — never a fuzzy + match — so an unknown node can't have its values written to some + unrelated key ComfyUI would silently ignore. + """ + if field in inputs: + return field + siblings = {"text": "value", "value": "text", "audio": "audio:", "audio:": "audio"} + alt = siblings.get(field) + if alt and alt in inputs: + return alt + return field + + def patch_workflow( base: dict[str, Any], values_by_node_id: dict[str, Any], @@ -20,11 +39,7 @@ def patch_workflow( if not isinstance(node, dict): continue inputs = dict(node.get("inputs") or {}) - field = class_to_patch_field(node.get("class_type", "")) - if field == "text" and "text" not in inputs and "value" in inputs: - field = "value" - if field == "value" and "text" in inputs and "value" not in inputs: - field = "text" + field = _resolve_field(class_to_patch_field(node.get("class_type", "")), inputs) inputs[field] = value node["inputs"] = inputs patched[key] = node diff --git a/calliope-backend/src/calliope/db.py b/calliope-backend/src/calliope/db.py index c7d717c8..02553f64 100644 --- a/calliope-backend/src/calliope/db.py +++ b/calliope-backend/src/calliope/db.py @@ -75,6 +75,7 @@ CREATE TABLE IF NOT EXISTS scenes ( env_image_path TEXT, location_id INTEGER, video_path TEXT, + video_settings_json TEXT, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); @@ -192,6 +193,8 @@ async def migrate_db(db_path: Path) -> None: conn.execute( "ALTER TABLE scenes ADD COLUMN chain_from_prev INTEGER NOT NULL DEFAULT 0" ) + if "video_settings_json" not in scene_cols: + conn.execute("ALTER TABLE scenes ADD COLUMN video_settings_json TEXT") project_cols = {r[1] for r in conn.execute("PRAGMA table_info(projects)").fetchall()} if "cover_path" not in project_cols: conn.execute("ALTER TABLE projects ADD COLUMN cover_path TEXT") @@ -272,4 +275,31 @@ def rebase_stale_asset_paths(conn: sqlite3.Connection, data_dir: Path, assets_di (json.dumps(new_paths), row["id"]), ) count += 1 + # Scene video settings embed absolute asset paths (ref images, uploaded + # clips) in input_values — rebase those too so a moved install keeps the + # saved setups working. + rows = conn.execute( + "SELECT id, video_settings_json FROM scenes WHERE video_settings_json IS NOT NULL" + ).fetchall() + for row in rows: + try: + data = json.loads(row["video_settings_json"] or "{}") + except json.JSONDecodeError: + continue + if not isinstance(data, dict): + continue + values = data.get("input_values") + if isinstance(values, dict): + data["input_values"] = { + k: (_rebase_path(v, data_dir, assets_dir) or v if isinstance(v, str) else v) + for k, v in values.items() + } + draft = data.get("prompt_draft") + if isinstance(draft, str) and not draft: + data.pop("prompt_draft", None) + conn.execute( + "UPDATE scenes SET video_settings_json = ? WHERE id = ?", + (json.dumps(data), row["id"]), + ) + count += 1 return count diff --git a/calliope-backend/src/calliope/models/schemas.py b/calliope-backend/src/calliope/models/schemas.py index ec6bddfe..42cc9cd7 100644 --- a/calliope-backend/src/calliope/models/schemas.py +++ b/calliope-backend/src/calliope/models/schemas.py @@ -142,6 +142,9 @@ class SceneUpdate(BaseModel): location_id: int | None = None video_path: str | None = None chain_from_prev: bool | None = None + # Persisted video-stage setup: form input_values, clip source, prompt + # draft + metadata. Serialized into scenes.video_settings_json. + video_settings: dict[str, Any] | None = None class SceneReorder(BaseModel): @@ -201,3 +204,11 @@ class GenerateVideosRequest(BaseModel): scene_ids: list[int] | None = None workflow_id: int | None = None input_values: dict[str, Any] | None = None + # Scene-id → confirmed prompt text (from the review modal). Wins over the + # LLM rewrite and over any saved draft. + prompts: dict[int, str] | None = None + + +class PreviewPromptRequest(BaseModel): + scene_id: int + workflow_id: int | None = None diff --git a/calliope-backend/src/calliope/routers/jobs.py b/calliope-backend/src/calliope/routers/jobs.py index c2c3386d..b4f3159d 100644 --- a/calliope-backend/src/calliope/routers/jobs.py +++ b/calliope-backend/src/calliope/routers/jobs.py @@ -5,13 +5,17 @@ from typing import Any from fastapi import APIRouter, HTTPException, Query -from calliope.agent.video_agent import enqueue_video_jobs +from calliope.agent.video_agent import enqueue_video_jobs, preview_scene_prompt from calliope.comfyui.client import ComfyUIClient from calliope.config import settings from calliope.db import get_db from calliope.events.bus import event_bus from calliope.export.runner import kill_running -from calliope.models.schemas import GenerateVideosRequest, JobCreate +from calliope.models.schemas import ( + GenerateVideosRequest, + JobCreate, + PreviewPromptRequest, +) from calliope.queue.manager import queue_manager router = APIRouter() @@ -63,12 +67,24 @@ async def generate_videos( scene_ids=body.scene_ids, workflow_id=body.workflow_id, input_values_override=body.input_values, + prompts=body.prompts, ) except ValueError as exc: raise HTTPException(status_code=400, detail=str(exc)) from exc return {"ok": True, "jobs": [_job_public(j) for j in jobs]} +@router.post("/projects/{project_id}/preview-prompt") +async def preview_prompt(project_id: int, payload: PreviewPromptRequest) -> dict[str, Any]: + """HITL review step: the exact prompt a Generate would send, no enqueue.""" + try: + return await preview_scene_prompt( + project_id, payload.scene_id, workflow_id=payload.workflow_id + ) + except ValueError as exc: + raise HTTPException(status_code=400, detail=str(exc)) from exc + + @router.post("/projects/{project_id}/export") async def export_film(project_id: int) -> dict[str, Any]: conn = get_db(settings.db_path) diff --git a/calliope-backend/src/calliope/routers/playground.py b/calliope-backend/src/calliope/routers/playground.py index eada8424..a6c793f0 100644 --- a/calliope-backend/src/calliope/routers/playground.py +++ b/calliope-backend/src/calliope/routers/playground.py @@ -114,6 +114,9 @@ UPLOAD_KIND_BY_EXT: dict[str, str] = { ".wav": "audio", ".flac": "audio", ".ogg": "audio", + # Note: core ComfyUI's LoadAudio file list is typically wav/mp3/ogg/flac/aiff. + # .m4a uploads succeed here and land in Comfy's input dir, but some builds + # reject it at /prompt validation — prefer wav/mp3 for ref audio. ".m4a": "audio", } diff --git a/calliope-backend/src/calliope/routers/scenes.py b/calliope-backend/src/calliope/routers/scenes.py index 368a5bed..6bef5320 100644 --- a/calliope-backend/src/calliope/routers/scenes.py +++ b/calliope-backend/src/calliope/routers/scenes.py @@ -1,5 +1,6 @@ from __future__ import annotations +import json from typing import Any from fastapi import APIRouter, HTTPException @@ -25,6 +26,14 @@ def _scene_with_chars(conn, scene_row) -> dict[str, Any]: ).fetchall() scene["characters"] = [row_to_dict(c) for c in chars] scene["character_ids"] = [c["id"] for c in scene["characters"]] + raw_settings = scene.pop("video_settings_json", None) + if raw_settings: + try: + scene["video_settings"] = json.loads(raw_settings) + except (json.JSONDecodeError, TypeError): + scene["video_settings"] = None + else: + scene["video_settings"] = None return scene @@ -111,11 +120,22 @@ async def update_scene(project_id: int, scene_id: int, payload: SceneUpdate) -> raise HTTPException(status_code=404, detail="Scene not found") data = payload.model_dump(exclude_unset=True) char_ids = data.pop("character_ids", None) + # video_settings arrives as a dict — the generic UPDATE path below only + # handles scalars, so serialize it into its JSON column (or NULL it). + video_settings = data.pop("video_settings", None) data = {k: v for k, v in data.items() if v is not None} if data: fields = ", ".join(f"{k} = :{k}" for k in data) data["id"] = scene_id + if video_settings is not None: + data["video_settings_json"] = json.dumps(video_settings) + fields += ", video_settings_json = :video_settings_json" conn.execute(f"UPDATE scenes SET {fields} WHERE id = :id", data) + elif video_settings is not None: + conn.execute( + "UPDATE scenes SET video_settings_json = ? WHERE id = ?", + (json.dumps(video_settings), scene_id), + ) if char_ids is not None: conn.execute("DELETE FROM scene_characters WHERE scene_id = ?", (scene_id,)) for cid in char_ids: diff --git a/calliope-backend/tests/test_pipeline.py b/calliope-backend/tests/test_pipeline.py index 695b82cb..8484b9e4 100644 --- a/calliope-backend/tests/test_pipeline.py +++ b/calliope-backend/tests/test_pipeline.py @@ -1,6 +1,7 @@ from __future__ import annotations import json +from pathlib import Path from calliope.comfyui.parser import parse_dynamic_inputs, parse_dynamic_outputs from calliope.comfyui.patcher import patch_workflow @@ -53,6 +54,127 @@ def test_patch_workflow_by_node_id(): assert patched["10"]["inputs"]["image"] == "ref.png" +def test_patch_workflow_vhs_audio_colon_widget(): + """VHS_LoadAudio's widget is literally `audio:` — the patcher must write it.""" + wf = { + "5": { + "inputs": {"audio:": "old.wav"}, + "class_type": "VHS_LoadAudio", + "_meta": {"title": "Ref Audio (Input:audio)"}, + } + } + patched = patch_workflow(wf, {"5": "C:/assets/new.wav"}) + assert patched["5"]["inputs"]["audio:"] == "C:/assets/new.wav" + # no bogus `audio` key gets added alongside + assert "audio" not in patched["5"]["inputs"] + + +def test_patch_workflow_no_fuzzy_fallback(): + """Unknown field on a node with unrelated keys must not get rewritten.""" + wf = { + "7": { + "inputs": {"aspect_ratio": "16:9", "megapixels": 1.0}, + "class_type": "ResolutionSelector", + "_meta": {"title": "Resolution (Input)"}, + } + } + patched = patch_workflow(wf, {"7": "9:16"}) + # computed field 'value' doesn't exist and has no sibling pair — stays put + assert patched["7"]["inputs"].get("value") == "9:16" + assert patched["7"]["inputs"]["aspect_ratio"] == "16:9" + + +def test_queue_prompt_surfaces_comfy_error_body(monkeypatch): + """A 400 from Comfy /prompt must name the node, not just the status code.""" + import httpx + from calliope.comfyui.client import ComfyUIClient + + body = { + "error": {"type": "prompt_outputs_failed_validation"}, + "node_errors": { + "12": {"errors": [{"message": "Invalid audio file: ref.m4a"}]}, + }, + } + + def fake_send(request, **kwargs): + return httpx.Response(400, json=body, request=request) + + transport = httpx.MockTransport(fake_send) + client = ComfyUIClient(base_url="http://comfy.test") + client._http = httpx.AsyncClient(transport=transport) + import asyncio + + async def run(): + try: + await client.queue_prompt({"1": {"class_type": "X", "inputs": {}}}) + except RuntimeError as exc: + return str(exc) + finally: + await client.close() + + msg = asyncio.run(run()) + assert "prompt_outputs_failed_validation" in msg + assert "node 12" in msg + assert "Invalid audio file" in msg + + +def test_upload_audio_flat_no_subfolder(tmp_path, monkeypatch): + """Audio uploads go to the flat input dir and return a bare filename.""" + import asyncio + import httpx + from calliope.comfyui.client import ComfyUIClient + + sent: dict = {} + + def fake_send(request, **kwargs): + sent["url"] = str(request.url) + sent["content_type"] = request.headers.get("content-type", "") + return httpx.Response(200, json={"name": "ref.wav", "subfolder": ""}, request=request) + + transport = httpx.MockTransport(fake_send) + client = ComfyUIClient(base_url="http://comfy.test") + client._http = httpx.AsyncClient(transport=transport) + + audio = tmp_path / "ref.wav" + audio.write_bytes(b"wav-placeholder") + + async def run(): + try: + return await client.upload_audio(audio) + finally: + await client.close() + + name = asyncio.run(run()) + assert name == "ref.wav" # bare filename, no calliope/ prefix + assert sent["url"].endswith("/upload/image") + + +def test_prepare_media_inputs_vhs_audio(tmp_path): + """VHS_LoadAudio with an `audio:` widget gets its file uploaded in place.""" + import asyncio + from calliope.comfyui.client import ComfyUIClient + + audio = tmp_path / "voice.wav" + audio.write_bytes(b"wav-placeholder") + + client = ComfyUIClient.__new__(ComfyUIClient) # skip __init__ HTTP client + + async def fake_upload(path, subfolder=""): + assert Path(path) == audio + return "voice.wav" + + client.upload_audio = fake_upload # type: ignore[method-assign] + wf = { + "5": { + "inputs": {"audio:": str(audio)}, + "class_type": "VHS_LoadAudio", + "_meta": {"title": "Ref Audio (Input:audio)"}, + } + } + out = asyncio.run(client.prepare_media_inputs(wf)) + assert out["5"]["inputs"]["audio:"] == "voice.wav" + + def test_workflow_analyze_and_create(client): r = client.post("/api/workflows/analyze", json={"workflow_json": SAMPLE_WORKFLOW}) assert r.status_code == 200 diff --git a/calliope-backend/tests/test_scene_settings.py b/calliope-backend/tests/test_scene_settings.py new file mode 100644 index 00000000..5149e21d --- /dev/null +++ b/calliope-backend/tests/test_scene_settings.py @@ -0,0 +1,312 @@ +"""Scene video_settings persistence (issue #28): PATCH round-trip, enqueue merge.""" +from __future__ import annotations + +import json + +from calliope.agent.video_agent import enqueue_video_jobs +from calliope.config import settings +from calliope.db import get_db + + +def _mk_project(client, title: str) -> int: + return client.post("/api/projects", json={"title": title}).json()["id"] + + +def _add_scene(client, pid: int, order: int, **extra) -> dict: + payload = {"order_index": order, "heading": f"S{order}", **extra} + return client.post(f"/api/projects/{pid}/scenes", json=payload).json() + + +def test_scene_patch_video_settings_roundtrip(client): + pid = _mk_project(client, "Settings RT") + scene = _add_scene(client, pid, 1) + + settings_payload = { + "input_values": {"102": 1280, "209": "my prompt"}, + "clip_source": "auto", + "form_workflow_id": 3, + } + r = client.patch( + f"/api/projects/{pid}/scenes/{scene['id']}", + json={"video_settings": settings_payload}, + ) + assert r.status_code == 200 + body = r.json() + assert body["video_settings"] == settings_payload + + # GET path parses the column back out too + r = client.get(f"/api/projects/{pid}/scenes") + scenes = r.json()["scenes"] + assert scenes[0]["video_settings"] == settings_payload + + +def test_scene_patch_video_settings_alongside_other_fields(client): + pid = _mk_project(client, "Settings mixed") + scene = _add_scene(client, pid, 1) + + r = client.patch( + f"/api/projects/{pid}/scenes/{scene['id']}", + json={"heading": "Renamed", "video_settings": {"input_values": {"5": "x.wav"}}}, + ) + assert r.status_code == 200 + body = r.json() + assert body["heading"] == "Renamed" + assert body["video_settings"]["input_values"]["5"] == "x.wav" + + +def test_video_settings_survive_migration_column(client): + """Existing DBs get the column via migrate_db — writes/reads work on it.""" + conn = get_db(settings.db_path) + try: + cols = {r[1] for r in conn.execute("PRAGMA table_info(scenes)").fetchall()} + assert "video_settings_json" in cols + finally: + conn.close() + + +def _insert_h3_workflow(conn, name: str) -> int: + wf = { + "10": { + "class_type": "CLIPTextEncode", + "inputs": {"text": ""}, + "_meta": {"title": "Main Prompt (Input:prompt)"}, + }, + "20": { + "class_type": "PrimitiveInt", + "inputs": {"value": 6}, + "_meta": {"title": "Duration (Input:duration)"}, + }, + } + cur = conn.execute( + """ + INSERT INTO workflows (name, kind, workflow_json, input_schema, output_schema, + prompt_profile, is_enabled) + VALUES (?, 'video', ?, '[]', '[]', 'minimax_h3_ref', 1) + """, + (name, json.dumps(wf)), + ) + conn.commit() + return cur.lastrowid + + +def test_enqueue_merges_stored_input_values(client, monkeypatch): + """Batch enqueue with no form values must honor the saved per-scene setup.""" + pid = _mk_project(client, "Merge stored") + scene = _add_scene(client, pid, 1) + + conn = get_db(settings.db_path) + try: + wf_id = _insert_h3_workflow(conn, "H3 merge") + stored = {"input_values": {"20": 10}} + conn.execute( + "UPDATE scenes SET workflow_id = ?, video_settings_json = ? WHERE id = ?", + (wf_id, json.dumps(stored), scene["id"]), + ) + conn.commit() + finally: + conn.close() + + # Skip the LLM rewrite — the fallback template is deterministic + async def fake_rewrite(scene_, subjects): + return "fallback" + + monkeypatch.setattr("calliope.agent.video_agent._h3_rewrite", fake_rewrite) + + from calliope.queue.manager import queue_manager + + queue_manager.paused = True + try: + jobs = asyncio_run( + enqueue_video_jobs(pid, scene_ids=[scene["id"]]) + ) + assert len(jobs) == 1 + raw = json.loads(jobs[0]["payload_json"]) + values = raw["input_values"] + assert values["20"] == 10 # stored setup applied + assert raw["prompt"] == "fallback" + finally: + queue_manager.paused = False + + +def test_preview_prompt_endpoint_h3_profile(client, monkeypatch): + """Preview resolves the H3 rewrite without enqueueing anything.""" + from calliope.queue.manager import queue_manager + + pid = _mk_project(client, "Preview H3") + scene = _add_scene(client, pid, 1) + + conn = get_db(settings.db_path) + try: + wf_id = _insert_h3_workflow(conn, "H3 preview") + conn.execute( + "UPDATE scenes SET workflow_id = ? WHERE id = ?", (wf_id, scene["id"]) + ) + conn.commit() + finally: + conn.close() + + async def fake_rewrite(scene_, subjects): + return "H3 REWRITE" + + monkeypatch.setattr("calliope.agent.video_agent._h3_rewrite", fake_rewrite) + + queue_manager.paused = True + try: + r = client.post( + f"/api/jobs/projects/{pid}/preview-prompt", + json={"scene_id": scene["id"]}, + ) + assert r.status_code == 200 + body = r.json() + assert body["prompt"] == "H3 REWRITE" + assert body["profile"] == "minimax_h3_ref" + assert body["from_draft"] is False + assert body["based_on"] + # No jobs were created by a preview + jobs = client.get(f"/api/jobs?project_id={pid}").json() + video_jobs = [j for j in jobs if j["kind"] == "video"] + assert video_jobs == [] + finally: + queue_manager.paused = False + + +def test_preview_prompt_fresh_draft_shortcircuits_llm(client, monkeypatch): + """A saved fresh draft is returned as-is — no LLM call.""" + from calliope.agent.video_agent import preview_scene_prompt + + pid = _mk_project(client, "Preview draft") + scene = _add_scene(client, pid, 1) + + conn = get_db(settings.db_path) + try: + wf_id = _insert_h3_workflow(conn, "H3 draft") + conn.execute( + "UPDATE scenes SET workflow_id = ? WHERE id = ?", (wf_id, scene["id"]) + ) + conn.commit() + finally: + conn.close() + + # Compute the hash the backend would, then store the draft with it + from calliope.agent.video_agent import _scene_prompt_hash + + conn = get_db(settings.db_path) + try: + fresh_row = conn.execute( + "SELECT * FROM scenes WHERE id = ?", (scene["id"],) + ).fetchone() + from calliope.db import row_to_dict + + fresh_scene = row_to_dict(fresh_row) + # character_ids aren't on the row — hash treats them via scene dict + fresh_scene["character_ids"] = [] + stored = { + "prompt_draft": "MY SAVED DRAFT", + "prompt_draft_meta": {"based_on": _scene_prompt_hash(fresh_scene)}, + } + conn.execute( + "UPDATE scenes SET video_settings_json = ? WHERE id = ?", + (json.dumps(stored), scene["id"]), + ) + conn.commit() + finally: + conn.close() + + called = [] + + async def fake_rewrite(scene_, subjects): + called.append(1) + return "SHOULD NOT BE USED" + + monkeypatch.setattr("calliope.agent.video_agent._h3_rewrite", fake_rewrite) + + result = asyncio_run(preview_scene_prompt(pid, scene["id"])) + assert result["prompt"] == "MY SAVED DRAFT" + assert result["from_draft"] is True + assert called == [] # LLM never invoked + + +def test_enqueue_prompts_override(client, monkeypatch): + """Confirmed prompt from the review modal wins over the rewrite.""" + pid = _mk_project(client, "Prompt override") + scene = _add_scene(client, pid, 1) + + conn = get_db(settings.db_path) + try: + wf_id = _insert_h3_workflow(conn, "H3 override") + conn.execute( + "UPDATE scenes SET workflow_id = ? WHERE id = ?", (wf_id, scene["id"]) + ) + conn.commit() + finally: + conn.close() + + async def fake_rewrite(scene_, subjects): + return "LLM VERSION" + + monkeypatch.setattr("calliope.agent.video_agent._h3_rewrite", fake_rewrite) + + from calliope.queue.manager import queue_manager + + queue_manager.paused = True + try: + jobs = asyncio_run( + enqueue_video_jobs(pid, scene_ids=[scene["id"]], prompts={scene["id"]: "CONFIRMED"}) + ) + raw = json.loads(jobs[0]["payload_json"]) + assert raw["prompt"] == "CONFIRMED" + finally: + queue_manager.paused = False + + +def test_preview_prompt_prose_profile(client): + """Prose workflows return the deterministic scene_video_prompt.""" + pid = _mk_project(client, "Preview prose") + scene = _add_scene(client, pid, 1, action="A knight rides at dawn.") + + conn = get_db(settings.db_path) + try: + wf = { + "10": { + "class_type": "CLIPTextEncode", + "inputs": {"text": ""}, + "_meta": {"title": "Main Prompt (Input:prompt)"}, + } + } + cur = conn.execute( + """ + INSERT INTO workflows (name, kind, workflow_json, input_schema, output_schema, is_enabled) + VALUES (?, 'video', ?, '[]', '[]', 1) + """, + ("Prose WF", json.dumps(wf)), + ) + conn.execute( + "UPDATE scenes SET workflow_id = ? WHERE id = ?", (cur.lastrowid, scene["id"]) + ) + conn.commit() + finally: + conn.close() + + r = client.post( + f"/api/jobs/projects/{pid}/preview-prompt", + json={"scene_id": scene["id"]}, + ) + assert r.status_code == 200 + body = r.json() + assert body["profile"] == "prose" + assert "knight" in body["prompt"] + + +def test_preview_prompt_missing_scene_400(client): + pid = _mk_project(client, "Preview missing") + r = client.post( + f"/api/jobs/projects/{pid}/preview-prompt", + json={"scene_id": 99999}, + ) + assert r.status_code == 400 + + +def asyncio_run(coro): + import asyncio + + return asyncio.run(coro) diff --git a/calliope-web/package-lock.json b/calliope-web/package-lock.json index f465c1ba..d1fccb26 100644 --- a/calliope-web/package-lock.json +++ b/calliope-web/package-lock.json @@ -1,12 +1,12 @@ { "name": "calliope-web", - "version": "1.2.3", + "version": "1.3.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "calliope-web", - "version": "1.2.3", + "version": "1.3.0", "dependencies": { "@tanstack/svelte-query": "^5.66.0" }, diff --git a/calliope-web/package.json b/calliope-web/package.json index e3e599ad..9580c37c 100644 --- a/calliope-web/package.json +++ b/calliope-web/package.json @@ -1,6 +1,6 @@ { "name": "calliope-web", - "version": "1.2.3", + "version": "1.3.0", "private": true, "scripts": { "dev": "vite dev", diff --git a/calliope-web/src/lib/api.ts b/calliope-web/src/lib/api.ts index e2784343..fcdb3c5f 100644 --- a/calliope-web/src/lib/api.ts +++ b/calliope-web/src/lib/api.ts @@ -104,8 +104,15 @@ export interface Settings { dry_run: boolean; } -import type { Job, Scene, Workflow, ComfyDynamicInput, ComfyDynamicOutput } from './comfy/types'; -export type { Job, Scene, Workflow, ComfyDynamicInput, ComfyDynamicOutput }; +import type { + Job, + Scene, + SceneVideoSettings, + Workflow, + ComfyDynamicInput, + ComfyDynamicOutput, +} from './comfy/types'; +export type { Job, Scene, SceneVideoSettings, Workflow, ComfyDynamicInput, ComfyDynamicOutput }; async function api(path: string, init?: RequestInit): Promise { const res = await fetch(`${API_BASE}${path}`, { @@ -274,12 +281,26 @@ export const jobsApi = { scene_ids?: number[]; workflow_id?: number; input_values?: Record; + prompts?: Record; } = {}, ) => api<{ ok: boolean; jobs: Job[] }>(`/api/jobs/projects/${projectId}/generate-videos`, { method: 'POST', body: JSON.stringify(payload), }), + previewPrompt: ( + projectId: number, + payload: { scene_id: number; workflow_id?: number }, + ) => + api<{ + prompt: string; + profile: string; + from_draft: boolean; + based_on: string; + }>(`/api/jobs/projects/${projectId}/preview-prompt`, { + method: 'POST', + body: JSON.stringify(payload), + }), exportFilm: (projectId: number) => api<{ ok: boolean; job: Job }>(`/api/jobs/projects/${projectId}/export`, { method: 'POST' }), }; diff --git a/calliope-web/src/lib/comfy/types.ts b/calliope-web/src/lib/comfy/types.ts index fc1a843d..393e85ab 100644 --- a/calliope-web/src/lib/comfy/types.ts +++ b/calliope-web/src/lib/comfy/types.ts @@ -53,6 +53,16 @@ export interface Job { retry_count: number; } +export interface SceneVideoSettings { + input_values?: Record; + /** 'auto' | 'upload' | target scene id as string. */ + clip_source?: string; + form_workflow_id?: number; + prompt_draft?: string; + prompt_draft_meta?: { based_on?: string; saved_at?: string }; + [key: string]: unknown; +} + export interface Scene { id: number; project_id: number; @@ -66,6 +76,7 @@ export interface Scene { location_id: number | null; video_path: string | null; chain_from_prev?: number | boolean | null; + video_settings?: SceneVideoSettings | null; character_ids: number[]; characters: Array<{ id: number; diff --git a/calliope-web/src/lib/components/QueueStage.svelte b/calliope-web/src/lib/components/QueueStage.svelte index ec536937..54dd5c94 100644 --- a/calliope-web/src/lib/components/QueueStage.svelte +++ b/calliope-web/src/lib/components/QueueStage.svelte @@ -19,6 +19,7 @@ import type { AssetOption } from '$lib/assetPicker'; import { progressFor } from '$lib/jobProgress'; import SafeMedia from './SafeMedia.svelte'; + import PromptPreviewModal from './video/PromptPreviewModal.svelte'; import VideoEditWorkspace from './video/VideoEditWorkspace.svelte'; import Button from './ui/Button.svelte'; import Icon from './ui/Icon.svelte'; @@ -170,9 +171,78 @@ if (id === lastFormScene) return; if (lastFormScene != null) formCache.set(lastFormScene, formValues); lastFormScene = id; - formValues = id != null ? { ...(formCache.get(id) ?? seedSceneDefaults(selected)) } : {}; + if (id != null) { + const stored = formCache.get(id); + if (stored) { + formValues = { ...stored }; + } else if (selected?.video_settings?.input_values) { + // First open of a persisted setup: hydrate from the scene row. + formValues = { + ...seedSceneDefaults(selected), + ...selected.video_settings.input_values, + }; + } else { + formValues = { ...seedSceneDefaults(selected) }; + } + } else { + formValues = {}; + } }); + // --- Auto-save scene setup (issue #28) --- + // Debounced write-through: in-memory formCache stays source of truth for + // the session; the scene row makes it survive restarts. Hash-compare so + // polling refetches never trigger PATCH churn. + let saveTimer: ReturnType | null = null; + let lastSavedHash = $state(''); + + function currentVideoSettings(): Record { + if (!selected) return {}; + const out: Record = { + input_values: compactInputValues(formValues), + }; + const wfId = selectedWorkflow[selected.id]; + if (wfId) out.form_workflow_id = wfId; + const src = clipSource[selected.id]; + if (src) out.clip_source = src; + const draft = selected.video_settings?.prompt_draft; + const meta = selected.video_settings?.prompt_draft_meta; + if (draft) { + out.prompt_draft = draft; + if (meta) out.prompt_draft_meta = meta; + } + return out; + } + + function settingsHash(obj: Record): string { + return JSON.stringify(obj); + } + + $effect(() => { + // Track the pieces that make up the persisted settings. + const _unused = [formValues, selectedWorkflow, clipSource, selected?.id]; + void _unused; + if (!selected || saveTimer) return; + saveTimer = setTimeout(() => { + saveTimer = null; + if (!selected) return; + const next = currentVideoSettings(); + const hash = settingsHash(next); + if (hash === lastSavedHash) return; + lastSavedHash = hash; + projects + .updateScene(projectId, selected.id, { video_settings: next }) + .catch(() => { + /* transient — next change retries */ + }); + }, 800); + }); + + async function onFormChangePersist() { + // formCache write-through happens in the switch effect; autosave effect + // handles persistence. Kept as an explicit hook for future callers. + } + // Context-aware defaults for a freshly opened scene form (user edits and the // workflow's static defaults must not override these). Duration-role inputs // seed from the scene's estimated duration; ComfyDynamicForm's own prefill @@ -209,13 +279,15 @@ } const generateOne = createMutation({ - mutationFn: (sceneId: number) => { + mutationFn: (vars: { sceneId: number; prompt?: string }) => { + const { sceneId } = vars; const scene = scenes.find((s) => s.id === sceneId); const wf = scene ? workflowFor(scene) : undefined; return jobsApi.generateVideos(projectId, { scene_ids: [sceneId], workflow_id: selectedWorkflow[sceneId] ?? wf?.id, input_values: compactInputValues(formValues), + prompts: vars.prompt ? { [String(sceneId)]: vars.prompt } : undefined, }); }, onSuccess: async () => { @@ -226,6 +298,19 @@ onError: (err) => toast.error(err instanceof Error ? err.message : String(err)), }); + // --- HITL review gate (issue #27) --- + let previewOpen = $state(false); + + function beginGenerate() { + if (!selected) return; + previewOpen = true; + } + + function onGenerateConfirmed(prompt: string) { + if (!selected) return; + $generateOne.mutate({ sceneId: selected.id, prompt }); + } + // --- Batch generate: queue clips one by one, in timeline order. --- // One POST per scene (the H3 prompt rewrite runs synchronously inside each // request, so a single all-scenes POST could take minutes and time out). @@ -255,6 +340,7 @@ if (batching || batchTargets.length === 0) return; batching = true; let queued = 0; + let drafted = 0; const targets = [...batchTargets].sort((a, b) => a.order_index - b.order_index); for (let i = 0; i < targets.length; i++) { const scene = targets[i]; @@ -263,11 +349,20 @@ // Resolve like the per-scene button does: session pick → scene's stored // workflow → first enabled video workflow. A scene whose stored workflow // was deleted would otherwise enqueue a job doomed to "No workflow found". + // Saved prompt drafts ride along; un-drafted scenes get the backend's + // auto-rewrite (deterministic template on LLM failure). + const draft = scene.video_settings?.prompt_draft; + const draftFresh = + draft && scene.video_settings?.prompt_draft_meta?.based_on + ? scene.video_settings.prompt_draft_meta.based_on + : null; await jobsApi.generateVideos(projectId, { scene_ids: [scene.id], workflow_id: workflowFor(scene)?.id, + prompts: draft ? { [String(scene.id)]: draft } : undefined, }); queued++; + if (draft) drafted++; client.invalidateQueries({ queryKey: ['jobs'] }); client.invalidateQueries({ queryKey: ['scenes'] }); } catch (err) { @@ -278,7 +373,12 @@ } batching = false; batchNote = ''; - if (queued > 0) toast.success(`${queued} clip${queued === 1 ? '' : 's'} queued — rendering in sequence`); + if (queued > 0) { + const draftNote = drafted > 0 ? ` · ${drafted} saved draft${drafted === 1 ? '' : 's'}` : ''; + toast.success( + `${queued} clip${queued === 1 ? '' : 's'} queued — rendering in sequence${draftNote}`, + ); + } await client.invalidateQueries({ queryKey: ['jobs'] }); await client.invalidateQueries({ queryKey: ['scenes'] }); } @@ -578,6 +678,10 @@ progress={selProg} error={selError} errorLong={selErrorLong} + job={selJob} + sceneJobs={($jobsQuery.data ?? []).filter( + (j) => j.scene_id === selected.id && j.kind === 'video', + )} workflow={selWf} workflows={enabledWorkflows} bind:formValues @@ -624,8 +728,18 @@ onWorkflowChange={(id) => { selectedWorkflow = { ...selectedWorkflow, [selected.id]: Number(id) }; }} - onGenerate={() => $generateOne.mutate(selected.id)} - /> + onGenerate={() => $generateOne.mutate({ sceneId: selected.id })} + onPreviewPrompt={beginGenerate} + /> + + {/if} {:else}
diff --git a/calliope-web/src/lib/components/video/JobInputsDrawer.svelte b/calliope-web/src/lib/components/video/JobInputsDrawer.svelte new file mode 100644 index 00000000..2cfc5082 --- /dev/null +++ b/calliope-web/src/lib/components/video/JobInputsDrawer.svelte @@ -0,0 +1,399 @@ + + + + {#if !activeJob} +

No render jobs recorded for this scene yet.

+ {:else} + {#if historyJobs.length > 1} +
+ {#each historyJobs as j (j.id)} + + {/each} +
+ {/if} + +
+ Job #{activeJob.id} + · + {activeJob.status} + {#if activeJob.completed_at} + · + {new Date(activeJob.completed_at).toLocaleString()} + {/if} + + {#if onCopySettings && activeJob.payload?.input_values} + + {/if} + +
+ + {#if prompt} +
+
+

Prompt

+ +
+ {#if workflow?.prompt_profile === 'minimax_h3_ref'} +

MiniMax H3 six-section rewrite — this is the exact text queued on the (Input:prompt) node.

+ {/if} +
{prompt}
+
+ {:else} +

No prompt recorded on this job.

+ {/if} + + {#if refRows.length > 0} +
+

References

+
    + {#each refRows as row (row.nodeId)} +
  • + {roleLabel(row.role)} + {baseName(row.value)} +
  • + {/each} +
+
+ {/if} + + {#if otherRows.length > 0} +
+

Other inputs

+
    + {#each otherRows as row (row.nodeId)} +
  • + {row.label} + {row.value} +
  • + {/each} +
+
+ {/if} + {/if} +
+ + diff --git a/calliope-web/src/lib/components/video/PromptPreviewModal.svelte b/calliope-web/src/lib/components/video/PromptPreviewModal.svelte new file mode 100644 index 00000000..31c991c0 --- /dev/null +++ b/calliope-web/src/lib/components/video/PromptPreviewModal.svelte @@ -0,0 +1,235 @@ + + + + {#if !scene} +

No scene selected.

+ {:else if $preview.isPending} +
+ + Resolving prompt{workflow?.prompt_profile === 'minimax_h3_ref' ? ' (H3 rewrite)' : ''}… +
+ {:else} +
+ Scene #{scene.order_index} · {scene.heading || 'Untitled'} + {workflow?.name ?? 'Default workflow'} +
+ + {#if stale} +
+ + Saved draft is based on older scene content — regenerate to refresh it. +
+ {/if} + + + + {#if fromDraft} +

Loaded from your saved draft. Regenerate re-runs the rewrite.

+ {:else if workflow?.prompt_profile === 'minimax_h3_ref'} +

MiniMax H3 six-section rewrite. Edit freely — this exact text goes to the (Input:prompt) node.

+ {:else} +

Scene prompt (prose profile). Edit freely before generating.

+ {/if} + {/if} + + {#snippet footer()} + + + + + {/snippet} +
+ + diff --git a/calliope-web/src/lib/components/video/VideoEditWorkspace.svelte b/calliope-web/src/lib/components/video/VideoEditWorkspace.svelte index 12e47a8b..28bb5aa2 100644 --- a/calliope-web/src/lib/components/video/VideoEditWorkspace.svelte +++ b/calliope-web/src/lib/components/video/VideoEditWorkspace.svelte @@ -4,12 +4,14 @@ * Hero monitor + filmstrip + meta strip + docked Omni composer. * Does not reuse the old top two-card clip-stage layout. */ - import type { Scene, Workflow } from '$lib/api'; + import type { Job, Scene, Workflow } from '$lib/api'; import OmniComposer from '$lib/components/OmniComposer.svelte'; import type { AssetOption } from '$lib/assetPicker'; import Icon from '$lib/components/ui/Icon.svelte'; import ClipMonitor from './ClipMonitor.svelte'; import ClipSourceModal from './ClipSourceModal.svelte'; + import JobInputsDrawer from './JobInputsDrawer.svelte'; + import PromptPreviewModal from './PromptPreviewModal.svelte'; import SceneFilmstrip from './SceneFilmstrip.svelte'; import SceneScriptDrawer from './SceneScriptDrawer.svelte'; @@ -44,6 +46,10 @@ progress?: Progress | null; error?: string; errorLong?: boolean; + /** Latest video job for the scene — drives the "what was sent" drawer. */ + job?: Job | null; + /** All video jobs for the scene (history strip in the drawer). */ + sceneJobs?: Job[]; workflow: Workflow | undefined; workflows: Workflow[]; formValues: Record; @@ -52,11 +58,14 @@ /** Disable Generate: scene continues from the previous video but the workflow cannot accept it. */ generateDisabled?: boolean; generateDisabledReason?: string; - /** Where this continue scene's video input comes from (auto / upload / a timeline clip). */ - clipSource?: ClipSourceConfig; - onClipSourceChange?: (value: string) => void; - /** Upload file picked in the source modal — caller opens the file dialog. */ - onClipSourceUpload?: () => void; + /** Where this continue scene's video input comes from (auto / upload / a timeline clip). */ + clipSource?: ClipSourceConfig; + onClipSourceChange?: (value: string) => void; + /** Upload file picked in the source modal — caller opens the file dialog. */ + onClipSourceUpload?: () => void; + /** HITL prompt review before Generate: caller resolves + shows the modal. */ + onPreviewPrompt?: () => void; + generateLabel?: string; chained?: (scene: Scene) => boolean; submitting?: boolean; statusOf: (scene: Scene) => string; @@ -78,6 +87,8 @@ progress = null, error = '', errorLong = false, + job = null, + sceneJobs = [], workflow, workflows, formValues = $bindable(), @@ -88,6 +99,8 @@ clipSource, onClipSourceChange, onClipSourceUpload, + onPreviewPrompt, + generateLabel = 'Generate clip', chained = () => false, submitting = false, statusOf, @@ -101,6 +114,15 @@ }: Props = $props(); let clipSourceOpen = $state(false); + let inputsOpen = $state(false); + + const hasJobPayload = $derived( + Boolean( + job && + ((typeof job.payload?.prompt === 'string' && job.payload.prompt) || + job.payload?.input_values), + ), + ); /** The label shown on the Video source trigger. */ const clipSourceLabel = $derived.by(() => { @@ -176,12 +198,36 @@ onupload={() => onClipSourceUpload?.()} /> {/if} - {#if assetOptions.length === 0} -

- No refs yet. Generate character sheets or environments in Assets, or upload a video/audio - file here. -

- {/if} + {#if assetOptions.length === 0} +

+ No refs yet. Generate character sheets or environments in Assets, or upload a video/audio + file here. +

+ {/if} + {#if hasJobPayload} +
+ +
+ { + formValues = { ...formValues, ...values }; + onFormChange?.({ ...formValues }); + }} + /> + {/if} {:else}
@@ -321,6 +367,40 @@ line-height: 1.4; } + .job-inputs-row { + display: flex; + justify-content: flex-end; + margin: 0 0 6px; + } + + .job-inputs-trigger { + display: inline-flex; + align-items: center; + gap: 6px; + padding: 4px 10px; + font: inherit; + font-size: 12px; + font-weight: 500; + color: var(--text-secondary); + background: transparent; + border: 1px solid var(--border); + border-radius: var(--radius-sm); + cursor: pointer; + transition: + color 150ms ease, + border-color 150ms ease; + } + + .job-inputs-trigger:hover { + color: var(--text-primary); + border-color: var(--text-muted); + } + + .job-inputs-trigger:focus-visible { + outline: 2px solid var(--accent); + outline-offset: 2px; + } + .no-wf { padding: 20px; text-align: center; diff --git a/docs/changelogs/2026-08-31.md b/docs/changelogs/2026-08-31.md new file mode 100644 index 00000000..a875106d --- /dev/null +++ b/docs/changelogs/2026-08-31.md @@ -0,0 +1,31 @@ +# 2026-08-31 — Calliope 1.3.0 + +Issue-driven update: prompt visibility (#27), ComfyUI error surfacing + audio fixes (#33), and scene-setup persistence (#28). + +## Prompt review before Generate (#27) + +- **Review gate**: Generate on a scene now opens a prompt preview modal first. It resolves the exact prompt that would be sent (saved draft → MiniMax H3 rewrite → deterministic fallback), lets you edit it, and only enqueues after you confirm. +- **Editable + saveable**: edit the text inline, Regenerate to re-run the H3 rewrite, or Save draft so future generates (single or batch) reuse it. A stale hint appears when the draft was saved against older scene content. +- **Render history**: the per-scene drawer (View prompt & inputs) is now a full job history — newest jobs for the scene as selectable chips, each showing the exact payload sent to ComfyUI, plus "Copy settings to form" to load a past job's input values back into the live form. +- Batch **Generate all** skips the modal: it uses each scene's saved draft when present and auto-rewrites the rest, and the completion toast reports how many drafts were applied. + +## Better ComfyUI errors + audio input fixes (#33) + +- Failed `/prompt` calls no longer surface as an opaque 400. ComfyUI's own `error.type` and per-node messages (e.g. `node 12: Invalid audio file: "x.m4a"`) are parsed and shown on the job. +- Audio uploads go to ComfyUI's flat input directory and return a bare filename, matching what `LoadAudio`'s file list expects. +- `VHS_LoadAudio` support: its widget is literally named `audio:` (with the colon) — both the media-uploader probe and the workflow patcher now handle it. The patcher's sibling-key fallback is strictly guarded to `text ↔ value` and `audio ↔ audio:` pairs. +- Upload failures (`/upload/image`) surface the same detailed error bodies. + +## Saved scene setups (#28) + +- The video stage's per-scene form (workflow pick, input values, video source) auto-saves to the scene and hydrates on reload — no more losing settings on refresh or app restart. +- **Generate all** honors each scene's saved setup, so batch renders keep per-scene tweaks. +- Prompt drafts persist per scene alongside the settings. +- Moved installs stay working: asset paths inside saved settings are rebased by the existing migration when the data directory changes. + +## Under the hood + +- `scenes.video_settings_json` column (additive migration, no data loss). +- New endpoint `POST /api/jobs/projects/{id}/preview-prompt`. +- `GenerateVideosRequest` accepts an optional `prompts` map (scene-id → confirmed text). +- 262 backend tests green; `svelte-check` clean. diff --git a/docs/changelogs/README.md b/docs/changelogs/README.md index 52ed6fb8..46e344ec 100644 --- a/docs/changelogs/README.md +++ b/docs/changelogs/README.md @@ -2,6 +2,7 @@ Dated notes for GitHub releases. Newest first. +- [2026-08-31](./2026-08-31.md) — Prompt review gate, ComfyUI error surfacing, saved scene setups (1.3.0) - [2026-08-29](./2026-08-29.md) — Model per agent + clip-native export fps + video-extend continue (1.2.3) - [2026-08-24](./2026-08-24.md) — Agent + Assets Management (1.2.1) - [2026-08-23](./2026-08-23.md) — 1.2.0 / 1.2.1 video, H3, picker, release notes