diff --git a/README.md b/README.md index 0b200324..f0fcfe40 100644 --- a/README.md +++ b/README.md @@ -46,8 +46,9 @@ Open `http://127.0.0.1:5173`. The dev server proxies `/api` to the backend on `1 Open the app, go to **Settings**, and set: -1. **LLM** — base URL, model name, and API key of your OpenAI-compatible endpoint +1. **LLM** — one or more OpenAI-compatible endpoints (base URL, model name, API key). Save several, then pick which one is **Active**. 2. **ComfyUI** — the base URL of your running ComfyUI (e.g. `http://127.0.0.1:8188`) +3. **Agent** *(optional)* — assign a specific LLM per agent role (see below) Leave **Dry-run** off — it is meant for testing and produces placeholder results instead of real generations. @@ -60,6 +61,13 @@ In **Settings → Queue** you can tune how the worker talks to ComfyUI: - **Poll timeout (seconds)** — how long Calliope keeps waiting on ComfyUI for a single job before failing it. **Default is `1800` (30 minutes).** Long video generations can easily exceed 10 minutes, so raise this for heavy workflows — or set it to **`0` to wait indefinitely** (until the job finishes or you cancel it). - **Max retries** — automatic retries before a job is marked failed. +### Agent settings + +In **Settings → Agent**: + +- **Model per agent** — assign any saved LLM to each role: **Main agent, Planner, Story agent, Script agent, Assets agent, Video agent**. Blank means the **Active LLM** from Settings → LLM applies, so a single-endpoint setup needs no configuration here. A common setup is a strong cloud model for the main agent and planner, and a fast local model for the sub-agents. The Video agent's assignment also drives the MiniMax H3 prompt rewrite. +- **System-prompt rules (hardening)** — operator rules appended to every agent system prompt. Leave blank to disable. + ## Using the app The app walks a project through four stages — **Story, Assets, Script, Video**: @@ -68,7 +76,7 @@ The app walks a project through four stages — **Story, Assets, Script, Video** - **Assets:** each character, location, and item has its own **Image prompt**. Pick a workflow and shared settings (width/height/etc.) at the top, then click Generate per entity to produce reference images on your ComfyUI. Regenerate any single entity without touching the others. - **Script:** **Regenerate Script** also opens a project-linked Agents chat (pre-filled) to rewrite the per-scene script. Scenes link back to the characters and locations from the Story stage. - **Video:** each scene gets a **Generate** button that queues a clip job on ComfyUI with the right prompt and reference images (plus optional video/audio file refs). Import a FirstMotion + NextMotion H3 pair to chain clips: clip 1 starts a motion, clips 2+ continue the last 22 frames and 1s of audio from the previous latent so the export stitch reads as one long take. -- **Film view:** once scenes have clips, **Export film** stitches them with ffmpeg: every clip is normalized to 1080p30, joined with 0.5s crossfades, and loudness-normalized into one final file. +- **Film view:** once scenes have clips, **Export film** stitches them with ffmpeg: clips are normalized to 1080p at the **majority frame rate of the clips themselves** (24 fps clips export at 24 fps; mixed-rate projects conform to whichever rate most clips use), joined with 0.5s crossfades, and loudness-normalized into one final file. - When everything is done the project is automatically marked **Completed**. **Playground** is a free-form generation page outside the project pipeline: run any imported workflow with arbitrary inputs, upload your own files (image / video / audio) as inputs, and optionally attach a result to a project as an asset. diff --git a/calliope-backend/.gitignore b/calliope-backend/.gitignore new file mode 100644 index 00000000..cc7d8bca --- /dev/null +++ b/calliope-backend/.gitignore @@ -0,0 +1,18 @@ +# --- Secrets & user data (NEVER commit) --- +calliope_config.json +data/ +*.db +*.db-wal +*.db-shm +.env +.env.* + +# --- Python --- +.venv/ +__pycache__/ +*.py[cod] +.pytest_cache/ +.ruff_cache/ +build/ +dist/ +*.egg-info/ diff --git a/calliope-web/.gitignore b/calliope-web/.gitignore new file mode 100644 index 00000000..1d845166 --- /dev/null +++ b/calliope-web/.gitignore @@ -0,0 +1,9 @@ +# --- Node / Svelte --- +node_modules/ +build/ +.svelte-kit/ + +# --- Env / logs --- +.env +.env.* +*.log diff --git a/docs/changelogs/2026-08-24.md b/docs/changelogs/2026-08-24.md new file mode 100644 index 00000000..ab62b1d4 --- /dev/null +++ b/docs/changelogs/2026-08-24.md @@ -0,0 +1,78 @@ +# Calliope changelog — 24 August 2026 + +**Agent + Assets Management** update. Ships in Windows zip **1.2.1**. + +Posted in both places: + +- [Release v1.2.1](https://github.com/benjiyaya/Calliope/releases/tag/v1.2.1) — zip download +- [Discussion #20](https://github.com/benjiyaya/Calliope/discussions/20) — comments + +Video / H3 / Generate-all: [2026-08-23](https://github.com/benjiyaya/Calliope/blob/main/docs/changelogs/2026-08-23.md). + +--- + +## Agents + +Agents chat is a plugin harness with an event-sourced session log. UI rows and LLM history are derived from that log — they are not written on their own. + +### Modes + +- **Project-linked** — tools stay on that film (story, scenes, assets, jobs). +- **Sandbox** — no project. `@workflow` / generate queues on hidden **Playground scratch**. The agent does **not** `create_project` just to make an image. `create_project` / `link_project` only when the user wants a film; the session then re-scopes. + +### Tools the agent may use + +- Workspace: `create_project`, `link_project`, `list_projects`. +- Story / script: draft and edit beats, characters, locations, **misc. items**, scenes. +- Assets / render: `enqueue_asset_jobs`, `enqueue_video_jobs`, `run_workflow`, `wait_for_jobs`, `attach_asset`. +- `attach_asset` files a finished generate (`job_id` or path) onto a film as a character sheet, environment, or misc. item. Refuses Playground scratch. + +Comfy MCP `comfy_*` generate tools are **not loaded**. Agents use Calliope HTTP tools only. + +### Human-in-the-loop (1.2.1) + +- Render tools (`run_workflow`, `enqueue_asset_jobs`, `enqueue_video_jobs`) stay **hidden** until the user’s latest message asks to generate or confirms an offer. +- Tagging `@workflow` alone is not permission. Creating a misc. item (or any text edit) is not permission. +- Intent is read from user prose only. The machine `[Calliope context]` appendix is stripped first (`kind=image` in the appendix cannot unlock renders). +- More than three video clips needs an explicit “all scenes / every clip” — a bare “yes” after a 2-clip offer must not enqueue the whole timeline. +- Destructive `replace=true` regeneration on a non-empty project still needs an explicit confirm (“yes, replace” / “from scratch”). + +Helpers live in `harness/policy.py`. + +### Operator controls + +- Settings → Agent: **hardening rules** appended last on the system prompt (blank disables). Same text on swarm sub-agents. +- Step budget: `agent_max_steps`. +- Composer: `@` mentions (workflows), file attachments. Welcome chips stay on a newly created empty session. + +--- + +## Assets management + +Three entity types on Project → Assets. All three are LLM-extracted in Draft Storyline, editable in Story, and generated as reference images here. + +| Type | Stored as | Video refs | +|---|---|---| +| Characters | `characters` | Yes — sheets / portraits | +| Environments | `locations` | Yes — location / env | +| **Misc. Items** | `items` | Picker can use the image; items are **not** auto-attached to scenes | + +Items match locations in the form (`name`, `description`, `consistency_prompt`, `reference_image_path`). Image prompt on the card maps to `(Input:prompt)` by **role**, not node id. + +### Generate (Assets tab) + +- Per-entity **Image prompt** + workflow inputs (width/height/seed…). Shared top-form Text Prompt stays empty — use the entity prompt. +- Worker writes the output path onto the entity. Items now persist `reference_image_path` (they used to finish in the queue and stay Pending on the card). Queue label: `Name · item`. +- Characters / locations were already attached this way. + +### Playground → project + +- **Add to project** can add as Character sheet, Background / location, or **Misc. item**. +- Misc. item **creates a new row**. It does not replace an existing item on the project list. Name field (prefilled from the file); existing items stay unchanged. +- Character / location still pick an existing row. + +### Agent path + +- Sandbox generate → Playground job → `attach_asset` onto a real film (sheet / env / item). +- Or enqueue from a linked project after the user asks to generate. + diff --git a/docs/changelogs/2026-08-29.md b/docs/changelogs/2026-08-29.md new file mode 100644 index 00000000..4b074c3a --- /dev/null +++ b/docs/changelogs/2026-08-29.md @@ -0,0 +1,50 @@ +# Calliope changelog — 29 August 2026 + +**Per-agent LLMs + clip-native export** update. Development branch `test-pr-25`. + +Full changelog index: [README](./README.md). Previous: [2026-08-24](./2026-08-24.md). + +--- + +## Model per agent (Settings → Agent) + +You can now choose **which LLM each agent uses** — no more one model for everything. + +- **Settings → Agent → "Model per agent"** sits above the hardening rules. Six rows: + **Main agent, Planner, Story agent, Script agent, Assets agent, Video agent**. +- Each dropdown lists every saved LLM from **Settings → LLM** plus a **(use Active LLM)** default. Blank = Active LLM, so existing setups behave exactly as before. +- Where the assignments apply: + - **Main agent** — the chat loop that answers you and calls tools. + - **Planner** — decides single vs. swarm, and writes the final swarm summary. + - **Story / Script / Assets / Video agents** — the four swarm sub-agents. + - The **Video agent** assignment also drives the MiniMax H3 prompt rewrite. +- Typical setup: a strong cloud model for the main agent and planner, a fast local model for sub-agents. + +Details: + +- Assignments are stored in the local `calliope_config.json` (profile **ids**, never API keys) and survive restarts. +- Deleting an LLM profile in Settings → LLM automatically clears any assignment that pointed at it. +- The API validates assignments: unknown roles are ignored; assigning a non-existent profile id returns `400 Unknown LLM profile for role: `. +- Settings → Agent's stacked panels now have proper spacing between them. + +## Export film at the clips' own FPS + +Film export no longer forces everything to 30 fps. + +- Each scene clip is probed for its real frame rate; the export targets the **majority fps across clips** (e.g. 24 fps clips export at 24 fps). +- Ties resolve to the lower rate; if no rate can be read, the old default (30) applies. +- Mixed-rate projects are conformed to the majority rate — the 0.5s crossfade chain needs one common timebase. +- The export progress message now reports the rate: **"Exporting film (24 fps)…"**. +- Everything else in the pipeline is unchanged: 1080p normalize → 0.5s crossfades → loudness normalize. + +## Housekeeping + +- Added `.gitignore` files inside `calliope-backend/` and `calliope-web/` (venv, caches, SQLite data, node_modules, build output) so each sub-folder is self-protecting in addition to the repo-root rules. + +## For developers + +- `LLMClient` accepts optional `base_url` / `model` / `api_key` overrides and gains `LLMClient.for_role(role)`; unknown roles and dangling profile ids fall back to the Active LLM. +- New config knob: `agent_llm_assignments` (role → profile id or null), exposed via `GET/POST /api/settings`. +- `POST /api/settings` accepts `agent_llm_assignments` with server-side validation. +- Export: `probe()` now returns `fps` (from ffprobe `r_frame_rate`), `parse_rate()` parses rational rates like `24000/1001`, `target_fps()` votes the conform rate, and `build_ffmpeg_cmd(..., fps=...)` takes an optional override. +- Full backend suite: **248 passed**. diff --git a/docs/changelogs/README.md b/docs/changelogs/README.md index f1c44d25..2a7537b7 100644 --- a/docs/changelogs/README.md +++ b/docs/changelogs/README.md @@ -2,4 +2,6 @@ Dated notes for GitHub releases. Newest first. -- [2026-08-23](./2026-08-23.md) — 1.2.1 (tabbed video refs, HITL, item attach) over 1.2.0 +- [2026-08-29](./2026-08-29.md) — Model per agent + clip-native export fps +- [2026-08-24](./2026-08-24.md) — Agent + Assets Management (1.2.1) +- [2026-08-23](./2026-08-23.md) — 1.2.0 / 1.2.1 video, H3, picker, release notes