From 419148999570b3921012c90d8c4c74bde92ca981 Mon Sep 17 00:00:00 2001 From: mozempk Date: Thu, 27 Aug 2026 09:52:24 +0200 Subject: [PATCH] docs: platform notes - macOS 13 vs 14 split, backend assumptions, issue #121 --- README.md | 8 +- docs/platform-notes.md | 64 ++ extensions/imgen/index.ts | 798 +++++++++++++++++++ extensions/imgen/tools/index.ts | 1265 +++++++++++++++++++++++++++++++ templates/centred-stack.typ | 5 +- templates/split.typ | 12 +- 6 files changed, 2148 insertions(+), 4 deletions(-) create mode 100644 docs/platform-notes.md create mode 100644 extensions/imgen/index.ts create mode 100644 extensions/imgen/tools/index.ts diff --git a/README.md b/README.md index 771dc8a..535979e 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,10 @@ veri. Due conseguenze pratiche: ## Requisiti -- **macOS su Apple Silicon.** Sviluppato per un Mac mini M4 base con 16 GB. +- **macOS su Apple Silicon** (obbligatorio: esistono solo build arm64). Sviluppato per un + Mac mini M4 base con 16 GB. **macOS 13** è il minimo; con **macOS 13** funziona tutto + tranne lo scontorno in un passaggio, che richiede macOS 14 e in mancanza ripiega su + un'alternativa — vedi [`docs/platform-notes.md`](docs/platform-notes.md). - [`draw-things-cli`](https://github.com/drawthingsai/draw-things-community) e [`typst`](https://typst.app) — installati da `install.sh`. - Spazio per i modelli (~10 GB). Possono stare su un **disco esterno**: il percorso è @@ -79,7 +82,8 @@ configurazione. | Impaginazione | Typst 0.15 — un solo binario, PDF di stampa nativo, sillabazione italiana | | Caratteri | 22 famiglie OFL/Apache incluse nel pacchetto ([THIRD-PARTY-FONTS.md](THIRD-PARTY-FONTS.md)) | -Dettagli verificati sperimentalmente in [`docs/typst-verified.md`](docs/typst-verified.md). +Dettagli verificati sperimentalmente in [`docs/typst-verified.md`](docs/typst-verified.md); +requisiti di sistema e assunzioni non verificate in [`docs/platform-notes.md`](docs/platform-notes.md). ## Licenza diff --git a/docs/platform-notes.md b/docs/platform-notes.md new file mode 100644 index 0000000..b9741a6 --- /dev/null +++ b/docs/platform-notes.md @@ -0,0 +1,64 @@ +# Platform requirements and unverified assumptions + +Written down because these bite on someone else's machine, not on the one where the code +was authored. Everything below was established by reading source or vendor docs; items +marked **UNVERIFIED** need the actual Mac to settle and are flagged in the code too. + +## macOS version matrix + +The stack does **not** have one minimum version — it has three, and the highest one only +affects an optional feature. + +| Component | Minimum | If unavailable | +|---|---|---| +| `draw-things-cli` | **macOS 13** (Homebrew Core, `arm64` bottles only) | Nothing works — this is the floor | +| `typst` | macOS 11 | Nothing renders | +| `native/cutout.swift` (Apple Vision) | **macOS 14** | Exits **3**; `cpu.ts` reports it in Italian and falls back to `rembg` | + +⚠️ **The cutout helper needs a newer macOS than everything else.** It uses +`VNGenerateForegroundInstanceMaskRequest`, which is macOS 14+, while the rest of the +stack runs on 13. So on macOS 13 everything works *except* one-step background removal, +which degrades rather than failing: + +- `/logo` still generates and vectorises; the cutout falls back to `rembg` if installed. +- ⚠️ If falling back, **never accept rembg's default model**: `bria-rmbg` is + **CC BY-NC 4.0** and not usable commercially. Pass `isnet` or `u2net` explicitly. +- rembg also **never touches the Neural Engine** unless given the undocumented + `-x '{"providers":["CoreMLExecutionProvider"]}'` — silently slow otherwise. + +Apple Silicon is required outright: `draw-things-cli` ships `arm64` bottles only, and +there is no Intel build. `install.sh` refuses non-arm64 with a clear message. + +`native/cutout.swift` is **not compile-tested** — this repo was authored on Linux, with +no Swift toolchain and no Vision framework. It is written in the single-file `swiftc` +pattern, but its first real compile will happen on the Mac. + +## Unverified assumptions in the backend + +| Assumption | Risk if wrong | +|---|---| +| **Units of `decodingTileWidth`/`decodingTileHeight`** — pixels assumed, matching the 512/512/64 defaults | If they are *latent* units, each tile covers 8× the area and the memory saving is smaller than intended. Flagged on `tilingPayload()`. | +| **No `--strength` CLI flag exists**, so img2img strength travels inside `--config-json` as the `strength` key | Strength silently ignored; edits come back too weak or too strong | +| **Which lever an upscale model expects** — the configured model is driven as the main `--model` over `--image` at low strength; ESRGAN-family names *also* set `upscaler`/`upscalerScaleFactor` | Upscale falls back to a Lanczos3 resample via sharp, announced in Italian, with `model: "lanczos(sharp)"` in the result so the caller can tell | +| **Daemon argv** assumed ` --no-tls --port N [--cpu-offload]`, models dir positional and first | Daemon fails to start; `doctor` reports it | +| **Readiness = a TCP accept** on the port. Without FlatBuffers we cannot ask the daemon anything over gRPC | "Porta aperta" means *reachable*, **not warm**. The first generation may still pay a cold model load. | +| **A Node `fetch` download does not carry `com.apple.quarantine`** | `xattr -d` runs best-effort and its failure is non-fatal | +| **Download integrity is structural, not cryptographic** — size ≥ 1 MiB plus a Mach-O magic number; no published checksum exists to pin | A tampered-but-valid Mach-O would pass | +| **Timeouts** (60 s ready budget, 5 s SIGTERM grace) chosen without a machine to measure on | The ready budget is the one most likely to need raising on a cold first model load | + +Also deliberate: **`-w/--weights-cache` is never passed.** Its 0 GiB default is correct on +16 GB — raising it steals exactly the RAM the VAE decode needs. + +Draw Things works in **64 px units**, so requested sizes are snapped **up** to that grid +(never down: art slightly larger than the frame can be cropped, art slightly smaller +would have to be upscaled). Real dimensions are read back from the produced file, so a +caller is never told a size the image does not have. + +## Still open upstream + +**Draw Things issue #121** — img2img crashes with `EXC_BREAKPOINT` on 16 GB Macs on the +current build, via both the API and the UI, unresolved. txt2img is unaffected. Detection +is graded rather than binary: an abnormal signal or an explicit Mach-exception fingerprint +gives confident Italian wording, a bare dead-transport marker gives hedged wording +("con ogni probabilità"), and the diagnosis is only ever offered for runs that passed +`--image`. See `docs/OPEN-DEFECTS.md` for what still works when it bites. diff --git a/extensions/imgen/index.ts b/extensions/imgen/index.ts new file mode 100644 index 0000000..7197a78 --- /dev/null +++ b/extensions/imgen/index.ts @@ -0,0 +1,798 @@ +/** + * index.ts — the extension entry point, and the ONLY place where the pieces meet. + * + * Everything below is wiring. There is no business logic here on purpose: each command + * owns its own flow, each backend owns its own process, and this file's single job is to + * build one config, one backend, one renderer and one director, hand the same objects to + * every command and tool, and start/stop the resident daemon at the right moment. + * + * THE ONE RULE THIS FILE EXISTS TO OBEY: + * an extension factory may run in invocations that never start a session (`pi --help`, + * package listing, a config dump). So the factory below starts NOTHING — no process, + * no socket, no watcher, no timer. It reads a JSON file and registers callbacks. + * The daemon starts in `session_start`, in the background, and is torn down — once, + * idempotently — in `session_shutdown`. + * + * The second rule is graceful degradation: an unplugged model disk, a missing binary or + * a render module that is not there yet must surface as one Italian sentence at the point + * of use. Nothing in here may throw at load time. + */ + +import { basename } from "node:path"; + +import { defineTool, getAgentDir } from "@earendil-works/pi-coding-agent"; +import type { + AgentToolResult, + AgentToolUpdateCallback, + ExtensionAPI, + ExtensionCommandContext, + ExtensionContext, +} from "@earendil-works/pi-coding-agent"; +import { StringEnum } from "@earendil-works/pi-ai"; +import type { ImageContent, TextContent } from "@earendil-works/pi-ai"; +import { Type, type Static } from "typebox"; + +import { loadConfig, type ImgenConfig, type Preset } from "./config.ts"; +import type { Backend, ProgressFn } from "./backends/types.ts"; +import { createDrawThingsBackend } from "./backends/drawthings.ts"; +import { startForSession, type DrawThingsServer } from "./backends/server.ts"; +import { removeBackground, upscaleRealesrgan, vectorize } from "./backends/cpu.ts"; +import * as director from "./design/director.ts"; +import type { Brief, DirectorContext } from "./design/director.ts"; +import { FORMATS, TEMPLATES, type Format } from "./design/spec.ts"; +import { formatReport, runDoctor, sessionBanner } from "./doctor.ts"; +import { openInFinder, outputFile } from "./job.ts"; +import { S, duration, errorText, fileLabel, fill } from "./ui/strings.ts"; + +import { + registerPoster, + runPoster, + type Director, + type PosterDeps, + type PosterResult, + type RenderOutcome, + type RenderRequest, + type Renderer, +} from "./commands/poster.ts"; +import { registerLogo, type LogoDeps } from "./commands/logo.ts"; +import { + SOCIAL_FORMATS, + registerSocial, + runSocial, + type SocialDeps, + type SocialRenderOutcome, + type SocialRenderRequest, + type SocialRenderedFile, + type SocialRenderer, + type SocialResult, +} from "./commands/social.ts"; +import { + previewImage, + registerRetouch, + type PosterHandoff, + type RetouchCropper, + type RetouchDeps, +} from "./commands/retouch.ts"; +import { registerPresets, savePreset, type PresetsDeps } from "./commands/presets.ts"; + +// --------------------------------------------------------------------------- +// Strings this file needs that ui/strings.ts does not have yet +// --------------------------------------------------------------------------- + +/** + * TODO: these belong in `ui/strings.ts` (as `S.commands.doctor` and `S.errors.*`). + * They live here only because strings.ts is owned by another module and editing it in + * parallel would conflict. Everything already in `S` is reused rather than duplicated. + */ +const T = { + doctorDescription: "Controlla che ci sia tutto", + doctorUsage: "Scrivi /doctor per sapere cosa manca e come sistemarlo.", + rendererMissing: + "Manca il pezzo che compone le scritte, quindi non posso preparare i file.", + rendererMissingFix: "Reinstalla l'estensione, poi scrivi /doctor per ricontrollare.", + cropperMissing: "Manca il pezzo che ritaglia le immagini per i social.", + handoffAskTitle: "Di che evento si tratta?", + nothingProduced: "Non è uscito nessun file.", +} as const; + +const MESSAGE_TYPE = "imgen"; + +/** How long session_start waits for the daemon before giving up and going cold. */ +const SERVER_READY_MS = 90_000; + +// --------------------------------------------------------------------------- +// The factory +// --------------------------------------------------------------------------- + +export default function imgen(pi: ExtensionAPI): void { + // ---- config: one object, shared by reference ------------------------------ + // /presets MUTATES this object in place so that every command holding the same + // reference — and the backend, which keeps it too — sees a new preset at once. + const piConfigDir = getAgentDir(); + const loaded = loadConfig(piConfigDir); + const config: ImgenConfig = loaded.config; + const configProblems: string[] = loaded.problems; + + // ---- resident daemon: declared here, started in session_start ------------- + let server: DrawThingsServer | null = null; + let serverStart: Promise<{ ok: boolean; message: string }> | null = null; + let serverReady = false; + let teardown: Promise | null = null; + + /** + * Idempotent, memoised, never throws. Called from session_start (fire and forget) and + * from the first generation (which awaits it), so a tool invoked in a session that + * never fired session_start still gets a warm model. + * + * A failed start stays memoised: retrying the spawn on every generation would hammer a + * broken install. `/doctor` is how he finds out why. + */ + const startServer = (): Promise<{ ok: boolean; message: string }> => { + if (!config.server.enabled) return Promise.resolve({ ok: false, message: "" }); + serverStart ??= startForSession(config, piConfigDir, { readyTimeoutMs: SERVER_READY_MS }) + .then((res) => { + server = res.server; + serverReady = res.ok; + return { ok: res.ok, message: res.message }; + }) + // startForSession is documented never to throw; this is belt and braces, because a + // rejected promise here would surface as an unhandled rejection at session start. + .catch((e: unknown) => ({ + ok: false, + message: errorText(S.errors.serverStartFailed((e as Error)?.message)), + })); + return serverStart; + }; + + /** + * Called before every model run. The daemon is an optimisation, never a requirement: + * `drawthings.ts` probes the port itself and runs cold when nothing answers, so all we + * do here is avoid racing a start that is already under way. + */ + const waitForServer = async (onProgress?: ProgressFn): Promise => { + if (!config.server.enabled) return; + if (!serverReady) onProgress?.(S.progress.warmup); + const res = await startServer(); + if (!res.ok && res.message) onProgress?.(res.message); + }; + + const stopServer = async (): Promise => { + const running = server; + server = null; + serverStart = null; + serverReady = false; + if (!running) return; + try { + // `stop()` leaves a daemon it did not spawn alone: killing his own Draw Things + // session out from under him would be rude. + await running.stop(); + } catch { + /* a teardown that throws would take the whole shutdown down with it */ + } + }; + + // ---- the backend --------------------------------------------------------- + // Constructing it is pure: no probe, no spawn, no filesystem. The wrapper's only job + // is to let the first generation wait for the daemon that session_start kicked off. + const rawBackend = createDrawThingsBackend(config); + const backend: Backend = { + name: rawBackend.name, + probe: () => rawBackend.probe(), + async generate(opts, onProgress, signal) { + await waitForServer(onProgress); + return rawBackend.generate(opts, onProgress, signal); + }, + async upscale(input, outPath, factor, onProgress, signal) { + await waitForServer(onProgress); + return rawBackend.upscale(input, outPath, factor, onProgress, signal); + }, + }; + + // ---- the renderer -------------------------------------------------------- + // + // TODO — ASSUMED SIGNATURES. `render/typst.ts` is being written in parallel and owns + // resolveSpec() / render() / renderAllFormats(). This file is the single adapter, so a + // mismatch is fixed here and nowhere else. What is assumed: + // - `render(req)` and `renderAllFormats(req)` take the poster `RenderRequest` shape + // (spec, artPath, outDir, formats, config, pdf, ppi, signal, onProgress); + // - they return `{ files, warnings? }`, where `files` is either absolute paths or + // `{ format, path }` records — both are accepted below; + // - the renderer resolves the spec itself (contrast, geometry, font paths) and writes + // under job.ts's OUTPUT_BASENAMES. + // The module is imported lazily so that a missing or broken renderer is one Italian + // sentence at the point of use rather than a crash while pi is still booting. + type TypstFile = string | { format?: Format; path: string; warnings?: readonly string[] }; + interface TypstOutcome { + files?: readonly TypstFile[]; + warnings?: readonly string[]; + } + interface TypstApi { + render(req: RenderRequest): Promise; + renderAllFormats(req: RenderRequest): Promise; + } + + let typstApi: Promise | null = null; + const typst = (): Promise => { + typstApi ??= import("./render/typst.ts") + .then((mod) => { + const api = mod as unknown as Partial; + if (typeof api.render !== "function" || typeof api.renderAllFormats !== "function") { + throw new Error("render/typst.ts: render()/renderAllFormats() mancanti"); + } + return api as TypstApi; + }) + .catch((e: unknown) => { + typstApi = null; // let a later call try again, e.g. after a reinstall + throw new Error( + errorText({ + message: T.rendererMissing, + fix: T.rendererMissingFix, + detail: (e as Error)?.message, + }), + ); + }); + return typstApi; + }; + + const pathOf = (f: TypstFile): string => (typeof f === "string" ? f : f.path); + + /** basename -> format, so a returned path can be labelled without the renderer's help. */ + const FORMAT_BY_FILE = new Map(); + for (const f of FORMATS) { + FORMAT_BY_FILE.set(outputFile(f, "png"), f); + FORMAT_BY_FILE.set(outputFile(f, "pdf"), f); + } + + const collectWarnings = (out: TypstOutcome): string[] => { + const warnings = [...(out.warnings ?? [])]; + for (const f of out.files ?? []) { + if (typeof f !== "string" && f.warnings) warnings.push(...f.warnings); + } + return warnings; + }; + + const toRenderOutcome = (out: TypstOutcome): RenderOutcome => { + const files = (out.files ?? []).map(pathOf); + const warnings = collectWarnings(out); + return warnings.length > 0 ? { files, warnings } : { files }; + }; + + const renderer: Renderer = { + async render(req) { + return toRenderOutcome(await (await typst()).render(req)); + }, + async renderAllFormats(req) { + return toRenderOutcome(await (await typst()).renderAllFormats(req)); + }, + }; + + /** + * The same renderer seen through /social's eyes: it wants one record per format and + * carries `print` overrides instead of a whole config. Never asks for a PDF — /social + * only ever produces screen sizes. + */ + const socialRenderer: SocialRenderer = { + async renderAllFormats(req: SocialRenderRequest): Promise { + const out = await (await typst()).renderAllFormats({ + spec: req.spec, + artPath: req.artPath, + outDir: req.outDir, + formats: req.formats, + config: req.print ? { ...config, print: { ...config.print, ...req.print } } : config, + pdf: false, + signal: req.signal, + onProgress: req.onProgress, + }); + + const files: SocialRenderedFile[] = []; + const returned = out.files ?? []; + returned.forEach((f, i) => { + const path = pathOf(f); + const named = typeof f === "string" ? undefined : f.format; + // Fall back to the filename, then to the request order: the renderer may not + // label its output, but the names are ours and the order is the one we asked for. + const format = named ?? FORMAT_BY_FILE.get(basename(path)) ?? req.formats[i]; + if (!format) return; + const perFile = typeof f === "string" ? undefined : f.warnings; + files.push(perFile && perFile.length > 0 ? { format, path, warnings: perFile } : { format, path }); + }); + + const warnings = out.warnings ?? []; + return warnings.length > 0 ? { files, warnings } : { files }; + }, + }; + + /** + * PDF preflight. TODO — ASSUMED SIGNATURE: `render/preflight.ts` exports + * `checkPdf(pdfPath, config?) -> { ok, problems }`. A preflight that cannot run is not + * a broken PDF: the files are still produced, they are simply not inspected. + */ + const preflight = async (pdfPath: string): Promise<{ ok: boolean; problems: string[] }> => { + try { + const mod = (await import("./render/preflight.ts")) as unknown as { + checkPdf?: (path: string, cfg?: ImgenConfig) => Promise<{ ok: boolean; problems: string[] }>; + }; + if (typeof mod.checkPdf !== "function") return { ok: true, problems: [] }; + return await mod.checkPdf(pdfPath, config); + } catch { + return { ok: true, problems: [] }; + } + }; + + /** + * Saliency-aware crop for /retouch. TODO — ASSUMED SIGNATURE: `render/contrast.ts` + * exports `smartCrop(input, out, {width,height}, opts?)`. + */ + const crop: RetouchCropper = { + async smartCrop(input, out, size, opts) { + let mod: { + smartCrop?: RetouchCropper["smartCrop"]; + }; + try { + mod = (await import("./render/contrast.ts")) as unknown as { smartCrop?: RetouchCropper["smartCrop"] }; + } catch (e) { + throw new Error( + errorText({ message: T.cropperMissing, fix: T.rendererMissingFix, detail: (e as Error)?.message }), + ); + } + if (typeof mod.smartCrop !== "function") { + throw new Error(errorText({ message: T.cropperMissing, fix: T.rendererMissingFix })); + } + return mod.smartCrop(input, out, size, opts); + }, + }; + + // ---- shared deps --------------------------------------------------------- + + const posterDeps: PosterDeps = { + config, + backend, + renderer, + director: director as Director, + preflight, + openFolder: openInFinder, + }; + + const presetsDeps: PresetsDeps = { + config, + configDir: piConfigDir, + backend, + renderer, + director, + onConfigChange: (updated) => { + // Keep the ONE shared object: half the extension holds this reference. + if (updated !== config) Object.assign(config, updated); + // /presets only ever writes palette, fonts, logo, tone and defaultPreset — it never + // touches `server`, so the running daemon stays valid. + }, + }; + + const logoDeps: LogoDeps = { + config, + backend, + renderer, + director, + cpu: { removeBackground, vectorize }, + piConfigDir, + // Wired to the /presets writer so the two can never disagree about the file format. + savePreset: (key: string, preset: Preset) => savePreset(presetsDeps, key, preset), + }; + + const socialDeps: SocialDeps = { + config, + renderer: socialRenderer, + director, + // "Parti da zero" is the only branch of /social that may reach a model, and it goes + // through the /poster pipeline rather than growing a second one. + createJobFromBrief: async (brief, ctx, signal) => { + const result = await runPoster(posterDeps, brief, directorContext(ctx, signal), { signal }); + return { job: result.job, spec: result.spec }; + }, + openFolder: openInFinder, + }; + + /** /retouch → /poster: the photo becomes the artwork, the pipeline does the rest. */ + const startPoster: PosterHandoff = async (opts, ctx) => { + let text = opts.freeText?.trim(); + if (!text && ctx.hasUI) { + text = (await ctx.ui.input(T.handoffAskTitle, S.commands.poster.usage))?.trim(); + } + if (!text) { + ctx.ui.notify(S.common.cancelled, "info"); + return; + } + const parsed = await director.parseFreeText(text, directorContext(ctx), config).catch(() => undefined); + const fields = parsed?.fields ?? {}; + const brief: Brief = { + kind: "poster", + title: fields.title, + subtitle: fields.subtitle, + date: fields.date, + venue: fields.venue, + details: fields.details, + price: fields.price, + footer: fields.footer, + freeText: parsed?.freeText || text, + format: parsed?.format, + photoPath: opts.photoPath, + }; + ctx.ui.setStatus(MESSAGE_TYPE, S.progress.thinking); + try { + const result = await runPoster(posterDeps, brief, directorContext(ctx), { + onProgress: (m) => ctx.ui.setStatus(MESSAGE_TYPE, m), + }); + say(pi, describePoster(result)); + ctx.ui.notify(S.notify.finished, "info"); + } catch (e) { + say(pi, italianError(e)); + ctx.ui.notify(S.notify.failed, "error"); + } finally { + ctx.ui.setStatus(MESSAGE_TYPE, undefined); + } + }; + + const retouchDeps: RetouchDeps = { + config, + backend, + cpu: { removeBackground, upscaleRealesrgan }, + crop, + director, + startPoster, + }; + + // ---- commands ------------------------------------------------------------ + + registerPoster(pi, posterDeps); + registerLogo(pi, logoDeps); + registerSocial(pi, socialDeps); + registerRetouch(pi, retouchDeps); + registerPresets(pi, presetsDeps); + + /** + * /doctor. Half the error messages in the string table end with "poi scrivi /doctor per + * ricontrollare", so the command has to exist for those sentences to be true. + */ + pi.registerCommand("doctor", { + description: `${T.doctorDescription}. ${T.doctorUsage}`, + handler: async (args: string, ctx: ExtensionCommandContext): Promise => { + ctx.ui.setStatus(MESSAGE_TYPE, S.doctor.running); + try { + const report = await runDoctor({ piConfigDir, config, configProblems }); + say(pi, formatReport(report, { verbose: /(-v|--verbose|dettagl)/i.test(args) })); + } catch (e) { + say(pi, italianError(e)); + } finally { + ctx.ui.setStatus(MESSAGE_TYPE, undefined); + } + }, + }); + + // ---- tools --------------------------------------------------------------- + // + // /logo and /retouch register their own (imgen_logo, imgen_retouch) inside their + // register functions. The three below are the ones whose implementation lives in a + // command module but which have no tool of their own yet. + // + // Every tool that can reach the diffusion model is `sequential`: two concurrent runs + // would OOM a 16GB machine. + + const PosterParams = Type.Object({ + title: Type.Optional(Type.String({ description: "Event title, exactly as it must be typeset." })), + subtitle: Type.Optional(Type.String()), + date: Type.Optional(Type.String({ description: "Italian date as it should appear, e.g. «12 settembre»." })), + venue: Type.Optional(Type.String()), + details: Type.Optional(Type.String({ description: "Extra line: line-up, opening time, booking." })), + price: Type.Optional(Type.String()), + footer: Type.Optional(Type.String({ description: "Organiser, sponsors, small print." })), + free_text: Type.Optional( + Type.String({ description: "Anything else: tone, audience, what must NOT appear." }), + ), + format: Type.Optional(StringEnum(FORMATS, { description: "Main format. Defaults to A3 portrait." })), + template: Type.Optional(StringEnum(TEMPLATES, { description: "Layout. Omit to let the art director choose." })), + preset: Type.Optional(Type.String({ description: "Key of a saved brand preset in the user's config." })), + photo_path: Type.Optional( + Type.String({ description: "Absolute path to an existing photo to use instead of generated artwork." }), + ), + formats: Type.Optional( + Type.Array(StringEnum(FORMATS), { description: "Every format to export. Defaults to A3 plus the social sizes." }), + ), + }); + + interface PosterToolDetails { + status: string; + slug?: string; + dir?: string; + files?: string[]; + warnings?: string[]; + elapsedMs?: number; + } + + pi.registerTool( + defineTool({ + name: "imgen_poster", + label: S.toolLabels.generateArt, + description: + "Create a finished event poster: the art director turns the brief into a design spec, the local " + + "diffusion model paints ARTWORK ONLY (never a letter), and Typst typesets the real Italian text " + + "on top, exporting a print-ready A3 PDF plus the social sizes. Text is data, so accents and dates " + + "are always correct. Slow (minutes) and holds the machine: call it once, never in parallel. " + + "Use imgen_social to re-render an existing job — that path costs seconds and touches no model.", + parameters: PosterParams, + executionMode: "sequential", + async execute( + _toolCallId: string, + params: Static, + signal: AbortSignal | undefined, + onUpdate: AgentToolUpdateCallback | undefined, + ctx: ExtensionContext, + ): Promise> { + const brief: Brief = { + kind: "poster", + title: params.title, + subtitle: params.subtitle, + date: params.date, + venue: params.venue, + details: params.details, + price: params.price, + footer: params.footer, + freeText: params.free_text, + format: params.format, + template: params.template, + preset: params.preset, + photoPath: params.photo_path, + }; + + const result = await runPoster(posterDeps, brief, directorContext(ctx, signal), { + formats: params.formats, + signal, + onProgress: (message) => + onUpdate?.({ content: [{ type: "text", text: message }], details: { status: message } }), + }); + + return { + content: await presentation(describePoster(result), result.files), + details: { + status: "done", + slug: result.job.slug, + dir: result.job.dir, + files: result.files, + warnings: result.warnings, + elapsedMs: result.elapsedMs, + }, + }; + }, + }), + ); + + const SocialParams = Type.Object({ + slug: Type.String({ description: "Folder name of an existing job under the output directory." }), + formats: Type.Optional( + Type.Array(StringEnum(FORMATS), { description: "Screen formats to write. Defaults to all of them." }), + ), + new_title: Type.Optional(Type.String({ description: "Replacement title, verbatim as it must be typeset." })), + new_date: Type.Optional(Type.String({ description: "Replacement date, verbatim, e.g. «11 settembre»." })), + fork: Type.Optional( + Type.Boolean({ + description: + "When the text changes, write into a NEW folder (default true). False overwrites a job that may " + + "already have gone to the print shop.", + }), + ), + caption: Type.Optional( + Type.Boolean({ + description: + "Write caption.txt with Italian social copy and hashtags. Only ever true after the user was asked.", + }), + ), + }); + + interface SocialToolDetails { + status: string; + slug?: string; + dir?: string; + forked?: boolean; + files?: string[]; + warnings?: string[]; + usedModel?: boolean; + elapsedMs?: number; + } + + pi.registerTool( + defineTool({ + name: "imgen_social", + label: S.toolLabels.export, + description: + "Re-render an existing job into the social formats (Instagram post and story, Facebook cover, " + + "YouTube thumbnail), optionally with a new title or date. This is a pure Typst pass over the saved " + + "spec: no diffusion model runs, it costs seconds, and changing the text forks a new folder so last " + + "year's file survives. Prefer this over imgen_poster whenever the artwork already exists.", + parameters: SocialParams, + executionMode: "sequential", + async execute( + _toolCallId: string, + params: Static, + signal: AbortSignal | undefined, + onUpdate: AgentToolUpdateCallback | undefined, + ctx: ExtensionContext, + ): Promise> { + const result: SocialResult = await runSocial( + { + slug: params.slug, + formats: params.formats ?? SOCIAL_FORMATS, + newTitle: params.new_title, + newDate: params.new_date, + fork: params.fork, + caption: params.caption, + onProgress: (message) => + onUpdate?.({ content: [{ type: "text", text: message }], details: { status: message } }), + }, + socialDeps, + { ...directorContext(ctx, signal), signal }, + ); + + return { + content: await presentation(describeFiles(result.dir, result.files, result.warnings, result.elapsedMs), result.files), + details: { + status: "done", + slug: result.slug, + dir: result.dir, + forked: result.forked, + files: result.files, + warnings: result.warnings, + usedModel: result.usedModel, + elapsedMs: result.elapsedMs, + }, + }; + }, + }), + ); + + const DoctorParams = Type.Object({ + verbose: Type.Optional(Type.Boolean({ description: "List every item, including the healthy ones." })), + }); + + interface DoctorToolDetails { + ok: boolean; + warnings: boolean; + problems: string[]; + checks: { id: string; level: string; message: string; fix?: string }[]; + } + + pi.registerTool( + defineTool({ + name: "imgen_doctor", + label: S.toolLabels.doctor, + description: + "Check that everything the poster pipeline needs is in place: config, model disk, model files, " + + "Draw Things, the resident server, Typst, fonts and the output folder. Read-only, fast, never " + + "throws. Run it first whenever a generation failed, and repeat what it says — the report is in " + + "Italian and already contains the remedy.", + parameters: DoctorParams, + executionMode: "parallel", + async execute( + _toolCallId: string, + params: Static, + ): Promise> { + const report = await runDoctor({ piConfigDir, config, configProblems }); + return { + content: [{ type: "text", text: formatReport(report, { verbose: params.verbose === true }) }], + details: { + ok: report.ok, + warnings: report.warnings, + problems: report.problems, + checks: report.checks.map((c) => ({ + id: c.id, + level: c.level, + message: c.message, + ...(c.fix ? { fix: c.fix } : {}), + })), + }, + }; + }, + }), + ); + + // ---- lifecycle ----------------------------------------------------------- + + pi.on("session_start", (_event, ctx) => { + // A previous session in this same process may have torn everything down. + teardown = null; + + // Deliberately NOT awaited: session start must not wait on a disk check, let alone + // on a model load. Both the report and the daemon land when they land. + void (async () => { + try { + // `fast: true` skips the external --version calls, so this stays instant. + const report = await runDoctor({ piConfigDir, config, configProblems, fast: true }); + + // sessionBanner() returns null when everything is fine: a health check that + // greets him every morning is a health check he stops reading. + const banner = sessionBanner(report); + if (banner && ctx.hasUI) ctx.ui.notify(banner, report.ok ? "warning" : "error"); + + // Only warm the daemon when the machine can actually paint: spawning + // gRPCServerCLI against an unplugged model disk just adds a second failure. + if (report.ok && config.server.enabled) void startServer(); + } catch { + // A health check that crashes the session is worse than no health check. + } + })(); + }); + + pi.on("session_shutdown", () => { + teardown ??= stopServer(); + return teardown; + }); + + // ---- small helpers ------------------------------------------------------- + + function describePoster(result: PosterResult): string { + return describeFiles(result.job.dir, result.files, result.warnings, result.elapsedMs); + } +} + +// --------------------------------------------------------------------------- +// Free functions — no state, nothing to inject +// --------------------------------------------------------------------------- + +function directorContext(ctx: ExtensionContext, signal?: AbortSignal): DirectorContext { + return { modelRegistry: ctx.modelRegistry, model: ctx.model, signal: signal ?? ctx.signal }; +} + +function say(pi: ExtensionAPI, text: string): void { + pi.sendMessage({ customType: MESSAGE_TYPE, content: text, display: true }); +} + +/** The closing report: what was made, where it is, what went less than perfectly. */ +function describeFiles( + dir: string, + files: readonly string[], + warnings: readonly string[], + elapsedMs: number, +): string { + if (files.length === 0) return T.nothingProduced; + const lines = [S.done.header, fill(S.done.folder, { cartella: dir }), "", S.done.filesHeader]; + for (const f of files) lines.push(` • ${fileLabel(basename(f))}`); + if (warnings.length > 0) lines.push("", ...warnings.map((w) => ` ! ${w}`)); + lines.push("", fill(S.done.elapsed, { tempo: duration(elapsedMs) })); + return lines.join("\n"); +} + +/** + * Both halves of a tool result: the image so he SEES it in the terminal, and the text so + * the model gets the paths. A preview that cannot be built is survivable — the text still + * names every file. + */ +async function presentation( + text: string, + files: readonly string[], +): Promise<(TextContent | ImageContent)[]> { + const preview = pickPreview(files); + const image = preview ? await previewImage(preview) : undefined; + return image ? [image, { type: "text", text }] : [{ type: "text", text }]; +} + +/** The most representative PNG: the square-ish social post first, then the poster. */ +function pickPreview(files: readonly string[]): string | undefined { + const byName = (name: string): string | undefined => files.find((f) => basename(f) === name); + return ( + byName(outputFile("ig-post")) ?? + byName(outputFile("a3-portrait")) ?? + byName(outputFile("a4-portrait")) ?? + files.find((f) => f.toLowerCase().endsWith(".png")) + ); +} + +/** + * Every module in this extension raises errors whose `italian` field is what the user + * should read; anything else gets the generic sentence rather than a stack trace. + */ +function italianError(e: unknown): string { + const err = e as { italian?: unknown; message?: unknown }; + if (typeof err?.italian === "string" && err.italian) return err.italian; + const message = typeof err?.message === "string" ? err.message : String(e); + // Commands already format their own errors with errorText(); those arrive pre-Italian. + return message.includes("\n") || /[àèéìòù]/i.test(message) + ? message + : errorText(S.errors.unknown(message)); +} diff --git a/extensions/imgen/tools/index.ts b/extensions/imgen/tools/index.ts new file mode 100644 index 0000000..f9f41a5 --- /dev/null +++ b/extensions/imgen/tools/index.ts @@ -0,0 +1,1265 @@ +/** + * tools/index.ts — the raw, agent-callable surface of pi-imgen. + * + * The slash commands in `commands/` are for the human at the keyboard: they open forms, + * show drafts, ask questions. THIS file is for the other lane — when he simply describes + * what he wants in the chat and the model has to do the work without a single dialog. + * + * Two rules shape everything below. + * + * 1. NO SECOND IMPLEMENTATION. Every tool is a thin parameter adapter over the exact + * function the corresponding command calls: `applyInstantRefinement()` and the + * injected `Renderer` for the instant lane, `runSocial()` / `writeSocialCaption()` + * for re-editions and captions, `runRetouch()` for every image operation, + * `backends/cpu.ts` for the vectoriser. If a tool here starts growing pipeline logic, + * that logic belongs in `commands/` or `backends/` and should be moved back. + * + * 2. ONE HEAVY JOB AT A TIME. `executionMode: "sequential"` on every tool that can reach + * the diffusion model. Two concurrent runs OOM a 16GB Mac mini before either finishes + * its VAE decode. `backends/serialize.ts` enforces the same rule one layer down; this + * is the belt to its braces, because the agent is perfectly capable of firing four + * tool calls in one batch. + * + * Every tool returns BOTH an image block (so the render appears inline in his terminal) + * and a text block naming the absolute paths (so the model has a handle to act on). + * Every failure THROWS: returning an error-shaped object never sets `isError` in pi. + * + * Progress text streamed through `onUpdate` is ITALIAN — he reads it. Tool descriptions, + * parameter descriptions and prompt guidelines are ENGLISH — the model reads those. + */ + +import { existsSync, renameSync } from "node:fs"; +import { basename, extname, isAbsolute, join, resolve } from "node:path"; + +import { Type, type Static } from "typebox"; +import { StringEnum } from "@earendil-works/pi-ai"; +import type { ImageContent, TextContent } from "@earendil-works/pi-ai"; +import { + defineTool, + withFileMutationQueue, + type ExtensionAPI, + type ExtensionContext, +} from "@earendil-works/pi-coding-agent"; + +import type { ImgenConfig } from "../config.ts"; +import type { Backend } from "../backends/types.ts"; +import { BackendError } from "../backends/types.ts"; +import { + vectorize as vectorizeImpl, + type CpuOpResult, + type VectorizeOptions, +} from "../backends/cpu.ts"; +import { + BLOCK_ROLES, + DesignSpecSchema, + FORMATS, + TEMPLATES, + type DesignSpec, + type Format, + type TemplateName, +} from "../design/spec.ts"; +import { bodyFamilies, displayFamilies } from "../design/fonts.ts"; +import { PALETTE_NAMES } from "../design/palettes.ts"; +import { ensureNegative, scrubArtPrompt, type DirectorContext } from "../design/director.ts"; +import { + ART_FILENAME, + createJob, + describeJob, + jobFile, + listJobs, + loadSpec, + normaliseSlug, + openInFinder, + openJob, + outputFile, + saveSpec, + slugify, + tryLoadSpec, + type Job, + type JobSummary, +} from "../job.ts"; +import { + generationSize, + needsUpscale, + upscaleFactor, + type PixelSize, +} from "../render/formats.ts"; +import { + applyInstantRefinement, + exportFormats, + type PosterDeps, + type RenderOutcome, + type RenderRequest, +} from "../commands/poster.ts"; +import { + runSocial, + writeSocialCaption, + SOCIAL_FORMATS, + type SocialDeps, +} from "../commands/social.ts"; +import { + normaliseImagePath, + previewImage, + runRetouch, + type RetouchDeps, + type RetouchResult, +} from "../commands/retouch.ts"; +import { S, bullets, duration, errorText, fileLabel, fill, type ErrorMessage } from "../ui/strings.ts"; + +// --------------------------------------------------------------------------- +// Italian, local to this module +// +// TODO: these belong in `S.tools` in ui/strings.ts, exactly like retouch.ts's `R` and +// logo.ts's `LOGO_S`. They are collected here, and nowhere else in the file, so moving +// them into the string table is one cut-and-paste. +// --------------------------------------------------------------------------- + +const T = { + artDone: "Immagine pronta.", + artDraftNote: "È una prova veloce: serve a decidere l'aspetto, non a stampare.", + artNoLettering: "Nell'immagine non c'è nessuna scritta: le parole le compongo dopo, così restano perfette.", + specSaved: "Ho salvato le scelte di stile.", + specMissing: "In questa cartella non c'è nessuna scheda di stile (spec.json), quindi non so cosa comporre.", + specMissingFix: "Passa una scheda completa a typeset_spec, oppure rifai il lavoro con /poster.", + artMissing: "In questa cartella manca l'immagine di sfondo (art.png), quindi non posso comporre niente.", + artMissingFix: "Fai prima l'immagine con generate_art, oppure rifai il lavoro con /poster.", + jobUnknown: "Non ho nessun lavoro che si chiami «{slug}».", + jobUnknownFix: "Chiedimi list_jobs per vedere quelli che ci sono.", + nothingWritten: "Non è uscito nessun file.", + noJobs: "Non c'è ancora nessun lavoro: la cartella è vuota.", + jobsHeader: "Lavori più recenti:", + reEdition: "Ho fatto una copia nuova in «{cartella}»: quella di prima resta dov'è.", + vectorDone: "Ricalco finito: l'SVG lo puoi ingrandire quanto vuoi senza che si sgrani.", + captionAsk: "Il testo per i social lo scrivo solo se me lo chiedi.", + labels: { + art: "L'immagine di sfondo", + draft: "La prova veloce", + svg: "Il marchio ricalcato (SVG)", + }, +} as const; + +// --------------------------------------------------------------------------- +// Dependencies — index.ts owns construction, this file owns adaptation only +// --------------------------------------------------------------------------- + +/** The one CPU entry point that has no home in `/retouch`. Defaults to backends/cpu.ts. */ +export interface VectorizePort { + vectorize(input: string, out: string, opts?: VectorizeOptions): Promise; +} + +/** + * Everything the tool surface needs, as the three dependency bundles the commands already + * define. Nothing is re-derived here: index.ts builds `PosterDeps`, `SocialDeps` and + * `RetouchDeps` once and hands the same objects to the commands and to `registerTools()`, + * which is what guarantees the two lanes cannot drift apart. + */ +export interface ToolsDeps { + /** The live config. A function when /presets can rewrite it under us. */ + config: ImgenConfig | (() => ImgenConfig); + /** Supplies the backend, the Typst renderer, the director and the optional preflight. */ + poster: PosterDeps; + /** Supplies the social renderer and the caption writer. */ + social: SocialDeps; + /** Supplies the CPU ops, the smart cropper and the generative-edit path. */ + retouch: RetouchDeps; + /** Defaults to `backends/cpu.ts`. */ + cpu?: VectorizePort; + /** Defaults to job.ts's macOS-only Finder reveal. Injected for tests. */ + openFolder?: (dir: string) => boolean; + now?: () => number; +} + +function cfgOf(deps: ToolsDeps): ImgenConfig { + return typeof deps.config === "function" ? deps.config() : deps.config; +} + +function backendOf(deps: ToolsDeps): Backend { + return deps.poster.backend; +} + +function vectorizerOf(deps: ToolsDeps): VectorizePort { + return deps.cpu ?? { vectorize: vectorizeImpl }; +} + +// --------------------------------------------------------------------------- +// Shared plumbing +// --------------------------------------------------------------------------- + +/** Draft artwork long edge, matching /poster's draft loop so a draft costs the same there. */ +const DRAFT_LONG_EDGE = 512; +const DRAFT_STEPS = 4; +const FINAL_STEPS = 8; +/** A terminal cell is not 300 dpi: the fast loop renders small on purpose. */ +const DRAFT_PPI = 72; + +type Content = (TextContent | ImageContent)[]; + +/** An `onUpdate` we can call with plain Italian, whatever the tool's details type is. */ +type Streamer = (message: string) => void; + +function streamer( + onUpdate: ((partial: { content: Content; details: T }) => void) | undefined, +): Streamer { + if (!onUpdate) return () => {}; + return (message: string) => { + onUpdate({ content: [{ type: "text", text: message }], details: undefined as unknown as T }); + }; +} + +function isAbort(e: unknown): boolean { + return e instanceof Error && (e.name === "AbortError" || e.name === "TimeoutError"); +} + +/** + * The only way a failure leaves this file. Prefers the Italian message the lower layers + * already carry (`BackendError.italian`, `RetouchError.italian`, `DirectorError.italian`), + * falls back to the string table, and always keeps the technical detail as `cause`. + */ +function fail(e: unknown, fallback: ErrorMessage): never { + if (isAbort(e)) throw e; + const italian = (e as { italian?: unknown } | null | undefined)?.italian; + const message = typeof italian === "string" && italian.trim() ? italian : errorText(fallback); + const detail = e instanceof Error ? e.message : String(e); + throw new Error(detail && detail !== message ? `${message}\n(${detail})` : message, { cause: e }); +} + +/** A refusal that is ours, not a lower layer's. Same shape, so callers cannot tell. */ +function refuse(message: string, fix?: string): never { + throw new Error(errorText(fix ? { message, fix } : { message })); +} + +function directorContextOf(ctx: ExtensionContext, signal?: AbortSignal): DirectorContext { + return { + modelRegistry: ctx.modelRegistry, + model: ctx.model, + ...(signal ? { signal } : {}), + }; +} + +/** Resolve a path the model handed us: `~`, `file://`, relative-to-cwd, escaped spaces. */ +function inputPath(raw: string, ctx: ExtensionContext): string { + return normaliseImagePath(raw, ctx.cwd); +} + +/** Resolve an OUTPUT path: absolute wins, otherwise relative to the job folder. */ +function outPath(raw: string | undefined, dir: string, fallbackName: string): string { + if (!raw) return join(dir, fallbackName); + const p = normaliseImagePath(raw, dir); + return isAbsolute(p) ? p : resolve(dir, p); +} + +/** Opens an existing job, or refuses in Italian naming the slug the model invented. */ +function requireJob(deps: ToolsDeps, slug: string): Job { + const key = normaliseSlug(slug); + const job = key ? openJob(cfgOf(deps), key) : undefined; + if (!job || !existsSync(job.dir)) { + refuse(fill(T.jobUnknown, { slug }), T.jobUnknownFix); + } + return job; +} + +/** The artwork a typeset pass should place: an explicit override, a photo, or art.png. */ +function artworkFor(job: Job, spec: DesignSpec, override?: string): string { + if (override) { + if (!existsSync(override)) fail(new Error(override), S.errors.photoMissing(override)); + return override; + } + if (spec.art.source === "photo" && spec.art.photoPath) { + if (!existsSync(spec.art.photoPath)) fail(new Error(spec.art.photoPath), S.errors.photoMissing(spec.art.photoPath)); + return spec.art.photoPath; + } + if (!existsSync(job.artPath)) refuse(T.artMissing, T.artMissingFix); + return job.artPath; +} + +/** spec.json is the one file this module writes directly; everything else is the renderer's. */ +async function persistSpec(job: Job, spec: DesignSpec): Promise { + const path = jobFile(job, "spec.json"); + await withFileMutationQueue(path, async () => saveSpec(job, spec)); + return spec; +} + +/** An inline preview plus the machine-readable path list, in that order. */ +async function present(paths: readonly string[], text: string): Promise { + const content: Content = []; + const first = paths.find((p) => /\.(png|jpe?g|webp|svg)$/i.test(p) && existsSync(p)); + if (first) { + const image = await previewImage(first); + if (image) content.push(image); + } + content.push({ type: "text", text }); + return content; +} + +/** Italian for him, then the absolute paths on their own lines for the model. */ +function report(italian: string, paths: readonly string[], extra?: string): string { + const parts = [italian]; + if (paths.length) parts.push("", bullets(paths.map((p) => `${basename(p)} — ${fileLabel(basename(p))}`))); + if (extra) parts.push("", extra); + if (paths.length) parts.push("", ...paths); + return parts.join("\n"); +} + +function isPrint(format: Format): boolean { + return format === "a3-portrait" || format === "a4-portrait"; +} + +function sizeLabel(size: PixelSize): string { + return `${size.width}x${size.height}`; +} + +// --------------------------------------------------------------------------- +// Shared parameter fragments +// --------------------------------------------------------------------------- + +const FormatParam = StringEnum([...FORMATS], { + description: "Output format. Print sizes are a3-portrait / a4-portrait; the rest are screen sizes.", +}); + +const SlugParam = Type.String({ + description: "Job folder name under the output directory, as returned by list_jobs.", +}); + +/** + * The text blocks, exhaustive over BLOCK_ROLES by construction. An empty string REMOVES + * the block — there is no null in a tool schema the providers all agree on. + */ +const BlocksParam = Type.Object( + { + title: Type.Optional(Type.String()), + subtitle: Type.Optional(Type.String()), + date: Type.Optional(Type.String({ description: "Verbatim as it should be typeset, e.g. «12 settembre 2026»." })), + venue: Type.Optional(Type.String()), + details: Type.Optional(Type.String()), + price: Type.Optional(Type.String()), + footer: Type.Optional(Type.String()), + }, + { + description: + "Italian text, verbatim. Only the roles you pass are touched; an empty string removes that block. " + + "This is DATA, never pixels: accents and dates are always typeset correctly.", + }, +); + +/** Compile-time proof that BlocksParam covers every role in the fixed contract. */ +type _BlocksCoverAllRoles = + Exclude<(typeof BLOCK_ROLES)[number], keyof (typeof BlocksParam)["properties"]> extends never ? true : never; + +/** + * The whole DesignSpec as a tool parameter — the fixed contract, reused verbatim rather + * than restated. `$id` is dropped: a nested `$id` inside a tool schema confuses providers. + */ +const SpecParam = Type.Object( + { ...DesignSpecSchema.properties }, + { + description: + "A complete DesignSpec. The art prompt must be ENGLISH and describe ARTWORK ONLY — " + + "never ask the model for text, words or lettering.", + }, +); + +// ═══════════════════════════════════════════════════════════════════════════ +// generate_art — the SLOW lane. Artwork only, never a letter. +// ═══════════════════════════════════════════════════════════════════════════ + +const GenerateArtParams = Type.Object({ + prompt: Type.String({ + description: + "ENGLISH description of the ARTWORK ONLY: subject, style, light, palette, composition. " + + "Never ask for text, a title, a date, a poster layout or any lettering — the words are " + + "typeset afterwards by typeset_spec.", + }), + negative: Type.Optional(Type.String({ + description: "Extra things to keep out. The mandatory anti-lettering terms are always added.", + })), + format: Type.Optional(FormatParam), + tier: Type.Optional(StringEnum(["draft", "final"], { + description: "draft = small and fast, to judge composition. final = full local resolution. Default draft.", + default: "draft", + })), + seed: Type.Optional(Type.Integer({ + minimum: 0, + maximum: 2_147_483_647, + description: "Pin it to reproduce a draft exactly at final quality. Omit for a new image.", + })), + steps: Type.Optional(Type.Integer({ minimum: 1, maximum: 50, description: "Sampling steps. Defaults per tier." })), + slug: Type.Optional(Type.String({ description: "Existing job folder to paint into. Omit to make a new one." })), + title: Type.Optional(Type.String({ description: "Used to name a new job folder when slug is omitted." })), + upscale_for_print: Type.Optional(Type.Boolean({ + description: + "Enlarge the finished artwork to the print target. Only meaningful with tier=final on a print format. " + + "Default: true for final print jobs, false otherwise.", + })), +}); + +export interface GenerateArtDetails { + path: string; + slug: string; + dir: string; + width: number; + height: number; + seed: number; + model: string; + tier: "draft" | "final"; + format: Format; + elapsedMs: number; + notes: string[]; +} + +export function createGenerateArtTool(deps: ToolsDeps) { + return defineTool({ + name: "generate_art", + label: S.toolLabels.generateArt, + description: + "Paint the ARTWORK for a poster, cover or background with the local diffusion model. " + + "generate_art writes art.png (or art-draft.png) into a job folder and does nothing else: " + + "it never draws a single letter, because every word is typeset afterwards by typeset_spec. " + + "Runs locally; a draft takes seconds, a final render takes minutes.", + promptSnippet: + "generate_art — paint lettering-free artwork with the local diffusion model into a job folder.", + promptGuidelines: [ + "Use generate_art whenever an image has to be painted from a description. Write its `prompt` in ENGLISH and describe only the picture — subject, style, light, colour, composition.", + "Never ask generate_art for a title, a date, a headline, a logo lockup or 'a poster with text on it'. Words are DATA: put them in typeset_spec's `blocks`, which is why accents and dates always come out right.", + "Call generate_art with tier=draft first when the user is still deciding. Once he approves, call it again with the SAME `seed` and tier=final to get the identical image at full quality.", + "generate_art is expensive and holds the whole machine: never call it twice in one batch, and never call it at all for a change of colour, font, layout or wording — typeset_spec does those for free.", + ], + parameters: GenerateArtParams, + // Two concurrent diffusion runs OOM a 16GB machine. Never parallel. + executionMode: "sequential", + async execute(_id, params, signal, onUpdate, ctx) { + const cfg = cfgOf(deps); + const say = streamer(onUpdate); + const started = deps.now ? deps.now() : Date.now(); + const notes: string[] = []; + + const tier: "draft" | "final" = params.tier === "final" ? "final" : "draft"; + const format = (params.format ?? "a3-portrait") as Format; + + // The prompt is scrubbed with the DIRECTOR's own function, not a local regex: there + // is exactly one definition of "this prompt is asking for lettering". + const prompt = scrubArtPrompt(params.prompt, undefined, []); + const negative = ensureNegative(params.negative ?? "", []); + + const requested = params.slug ? normaliseSlug(params.slug) : ""; + const job = createJob(cfg, requested || slugify(params.title ?? "immagine"), { reuse: true }); + + const print = { bleedMm: cfg.print.bleedMm, dpi: cfg.print.dpi }; + const size = + tier === "draft" ? generationSize(format, print, DRAFT_LONG_EDGE) : generationSize(format, print); + const seed = params.seed ?? Math.floor(Math.random() * 2_147_483_647); + const target = jobFile(job, tier === "draft" ? "art-draft.png" : ART_FILENAME); + + say(tier === "draft" ? S.progress.draft : S.progress.finalArt); + + let produced; + try { + produced = await withFileMutationQueue(target, () => + backendOf(deps).generate( + { + prompt, + negative, + seed, + width: size.width, + height: size.height, + steps: params.steps ?? (tier === "draft" ? DRAFT_STEPS : FINAL_STEPS), + tier, + outPath: target, + }, + (msg) => say(msg), + signal, + ), + ); + } catch (e) { + fail(e, S.errors.generationFailed(e instanceof Error ? e.message : undefined)); + } + + let path = produced.path; + let width = produced.width; + let height = produced.height; + + // Print needs far more pixels than 16GB can generate in one pass. The enlargement + // goes through runRetouch() so it uses the SAME Real-ESRGAN-first policy (and the + // same fallbacks) as /retouch — there is no second upscaler in this codebase. + const wantUpscale = params.upscale_for_print ?? (tier === "final" && isPrint(format)); + if (wantUpscale && needsUpscale(format, { width, height }, print)) { + say(S.progress.upscaling); + const factor = upscaleFactor(format, print, { width, height }); + try { + const up = await runRetouch( + deps.retouch, + { action: "upscale", input: path, outDir: job.dir, factor }, + { onProgress: (msg) => say(msg), ...(signal ? { signal } : {}) }, + ); + const first = up.outputs[0]; + if (first) { + await withFileMutationQueue(job.artPath, async () => { + renameSync(first.path, job.artPath); + }); + path = job.artPath; + width = first.width; + height = first.height; + } + notes.push(...up.notes); + } catch (e) { + if (isAbort(e)) throw e; + // A slightly soft poster beats no poster: keep what we painted and say so. + notes.push(errorText(S.errors.upscaleFailed)); + } + } + + const details: GenerateArtDetails = { + path, + slug: job.slug, + dir: job.dir, + width, + height, + seed: produced.seed, + model: produced.model, + tier, + format, + elapsedMs: (deps.now ? deps.now() : Date.now()) - started, + notes, + }; + + const italian = [ + T.artDone, + tier === "draft" ? T.artDraftNote : T.artNoLettering, + ...(notes.length ? ["", ...notes] : []), + "", + fill(S.progress.elapsed, { tempo: duration(details.elapsedMs) }), + ].join("\n"); + + const text = report( + italian, + [path], + `slug=${job.slug} seed=${produced.seed} size=${sizeLabel({ width, height })} tier=${tier} ` + + `format=${format}\nNext: typeset_spec with slug=${job.slug} to put the real text on it.`, + ); + + return { content: await present([path], text), details }; + }, + }); +} + +// ═══════════════════════════════════════════════════════════════════════════ +// typeset_spec — the INSTANT lane. No model, ever. +// ═══════════════════════════════════════════════════════════════════════════ + +const TypesetSpecParams = Type.Object({ + slug: Type.Optional(Type.String({ + description: "Job folder to edit. Defaults to spec.slug when a full `spec` is supplied.", + })), + spec: Type.Optional(SpecParam), + palette: Type.Optional(StringEnum(PALETTE_NAMES, { + description: "Named palette to switch to. Colours locked by a brand preset are kept.", + })), + preset: Type.Optional(Type.String({ description: "Brand preset key whose locked colours must survive the change." })), + display_font: Type.Optional(StringEnum(displayFamilies(), { description: "Headline family." })), + body_font: Type.Optional(StringEnum(bodyFamilies(), { description: "Body family." })), + template: Type.Optional(StringEnum([...TEMPLATES], { description: "Layout template." })), + format: Type.Optional(FormatParam), + blocks: Type.Optional(BlocksParam), + art_path: Type.Optional(Type.String({ description: "Artwork to place instead of the job's art.png." })), + pdf: Type.Optional(Type.Boolean({ description: "Also write the print PDF. Default false — this is the fast loop." })), + draft: Type.Optional(Type.Boolean({ description: "Render small and fast for on-screen judging. Default true." })), +}); + +export interface TypesetSpecDetails { + slug: string; + dir: string; + specPath: string; + files: string[]; + warnings: string[]; + changed: string[]; + usedModel: false; + elapsedMs: number; +} + +export function createTypesetSpecTool(deps: ToolsDeps) { + return defineTool({ + name: "typeset_spec", + label: S.toolLabels.typeset, + description: + "Set or change the design of a job — palette, fonts, layout template, format, and the actual " + + "Italian text — and typeset it with Typst over the existing artwork. typeset_spec NEVER runs the " + + "diffusion model: it is a pure edit of spec.json plus one sub-second Typst pass, so changing a " + + "colour, a font or a word is free. Supply a full `spec` to establish the design, or individual " + + "fields to adjust an existing one.", + promptSnippet: + "typeset_spec — change colours, fonts, layout or wording and re-typeset instantly; no model runs.", + promptGuidelines: [ + "Use typeset_spec for EVERY change that is not the artwork itself: colour, font, layout, format, and any wording. It costs nothing and takes under a second, so prefer it freely and iterate.", + "Never call generate_art to fix a typo, a date, a colour or a font — call typeset_spec. The artwork carries no lettering, so the words are always a re-typeset away.", + "Put the user's Italian text into typeset_spec's `blocks` verbatim, accents included. Do not translate it, do not rewrite it, and do not invent a subtitle or a price he did not give you.", + "After generate_art has painted a new job, call typeset_spec once with a full `spec` to establish palette, fonts, template and blocks; afterwards pass only the fields that change.", + "typeset_spec renders a fast on-screen proof by default. When the user wants the finished, printable files, follow it with render_spec.", + ], + parameters: TypesetSpecParams, + // Pure CPU: a Typst pass may safely run beside anything else. + executionMode: "parallel", + async execute(_id, params, signal, onUpdate, ctx) { + const cfg = cfgOf(deps); + const say = streamer(onUpdate); + const started = deps.now ? deps.now() : Date.now(); + const changed: string[] = []; + + const slug = normaliseSlug(params.slug ?? params.spec?.slug ?? ""); + if (!slug) refuse(T.specMissing, T.specMissingFix); + + // A supplied spec establishes the design; otherwise we edit the saved one. + const job = params.spec ? createJob(cfg, slug, { reuse: true }) : requireJob(deps, slug); + let spec = params.spec ? ({ ...(params.spec as DesignSpec), slug: job.slug }) : tryLoadSpec(job); + if (!spec) refuse(T.specMissing, T.specMissingFix); + if (params.spec) changed.push("spec"); + + const preset = params.preset ? cfgOf(deps).presets[params.preset] : undefined; + if (params.preset && !preset) fail(new Error(params.preset), S.errors.presetUnknown(params.preset)); + + // Each edit goes through the command's own `applyInstantRefinement`, which is typed + // against InstantAction — the compiler is what stops "art" leaking into this lane. + if (params.palette) { + spec = applyInstantRefinement(spec, "colours", { palette: params.palette }, preset); + changed.push("colours"); + } + if (params.display_font || params.body_font) { + spec = applyInstantRefinement(spec, "fonts", { + fonts: { + display: params.display_font ?? spec.fonts.display, + body: params.body_font ?? spec.fonts.body, + }, + }); + changed.push("fonts"); + } + if (params.template) { + spec = applyInstantRefinement(spec, "layout", { template: params.template as TemplateName }); + changed.push("layout"); + } + if (params.blocks && Object.keys(params.blocks).length > 0) { + // "" means "drop this block"; job.ts's updateBlocks takes null for that. + const patch: Record = {}; + for (const [role, value] of Object.entries(params.blocks)) { + if (typeof value !== "string") continue; + patch[role] = value.trim() === "" ? null : value; + } + spec = applyInstantRefinement(spec, "text", { blocks: patch }); + changed.push("text"); + } + if (params.format && params.format !== spec.format) { + spec = { ...spec, format: params.format as Format }; + changed.push("format"); + } + + await persistSpec(job, spec); + + const artPath = artworkFor(job, spec, params.art_path ? inputPath(params.art_path, ctx) : undefined); + const draft = params.draft ?? true; + + say(S.progress.typesetting); + const req: RenderRequest = { + spec, + artPath, + outDir: job.dir, + formats: [spec.format], + config: cfg, + pdf: params.pdf ?? false, + ...(draft ? { ppi: DRAFT_PPI } : {}), + ...(signal ? { signal } : {}), + onProgress: (msg) => say(msg), + }; + + let outcome: RenderOutcome; + try { + outcome = await deps.poster.renderer.render(req); + } catch (e) { + fail(e, S.errors.renderFailed(e instanceof Error ? e.message : undefined)); + } + if (!outcome.files.length) refuse(T.nothingWritten); + + const details: TypesetSpecDetails = { + slug: job.slug, + dir: job.dir, + specPath: jobFile(job, "spec.json"), + files: [...outcome.files], + warnings: [...(outcome.warnings ?? [])], + changed, + usedModel: false, + elapsedMs: (deps.now ? deps.now() : Date.now()) - started, + }; + + const italian = [ + T.specSaved, + ...(draft ? [S.warnings.draftOnly] : []), + ...(details.warnings.length ? ["", ...details.warnings] : []), + ].join("\n"); + + const text = report( + italian, + details.files, + `slug=${job.slug} spec=${details.specPath} changed=${changed.join(",") || "nothing"}\n` + + `No model ran. Call render_spec for the finished, printable set.`, + ); + + return { content: await present(details.files, text), details }; + }, + }); +} + +// ═══════════════════════════════════════════════════════════════════════════ +// render_spec — the finished deliverables of a saved job. +// ═══════════════════════════════════════════════════════════════════════════ + +const RenderSpecParams = Type.Object({ + slug: SlugParam, + formats: Type.Optional(Type.Array(FormatParam, { + description: "Which formats to write. Default: the job's own format plus every social size.", + })), + new_title: Type.Optional(Type.String({ description: "Re-edition: the new title, verbatim Italian." })), + new_date: Type.Optional(Type.String({ description: "Re-edition: the new date, verbatim as it should be typeset." })), + fork: Type.Optional(Type.Boolean({ + description: "Re-editions write into a NEW folder and leave the old one untouched. Default true.", + })), + caption: Type.Optional(Type.Boolean({ + description: + "Write caption.txt with Italian social copy. Set true ONLY after the user has explicitly asked for it.", + })), + pdf: Type.Optional(Type.Boolean({ description: "Write the print PDF for print formats. Default true." })), + open_folder: Type.Optional(Type.Boolean({ description: "Reveal the folder in Finder when done. Default false." })), +}); + +export interface RenderSpecDetails { + slug: string; + dir: string; + forked: boolean; + formats: Format[]; + files: string[]; + warnings: string[]; + captionPath?: string; + usedModel: boolean; + elapsedMs: number; +} + +export function createRenderSpecTool(deps: ToolsDeps) { + return defineTool({ + name: "render_spec", + label: S.toolLabels.export, + description: + "Produce the finished files for a saved job: the print PDF and PNG plus every social size, " + + "typeset from spec.json over the existing artwork. render_spec runs no diffusion model. " + + "Pass new_title / new_date to make a re-edition of last year's poster — same artwork, new words, " + + "written into a brand-new folder so the original survives.", + promptSnippet: + "render_spec — export a saved job to print PDF + all social sizes, or fork it into a re-edition.", + promptGuidelines: [ + "Use render_spec when the user wants the actual deliverables — the A3 PDF, the Instagram sizes, the Facebook cover — from a job that already has a spec and artwork.", + "Use render_spec with new_title / new_date for 'same poster, new date'. It reuses the artwork, forks into a new folder by default, and costs no model time; do not call generate_art for this.", + "Never set render_spec's `caption` to true on your own initiative. Ask the user first whether he also wants social copy, and only pass caption=true after he says yes — his own wording is never rewritten silently.", + "render_spec is the last step. If the design still needs a change, call typeset_spec first and render_spec afterwards.", + ], + parameters: RenderSpecParams, + // Typst plus optionally one caption call. No diffusion: safe to run in parallel. + executionMode: "parallel", + async execute(_id, params, signal, onUpdate, ctx) { + const cfg = cfgOf(deps); + const say = streamer(onUpdate); + const started = deps.now ? deps.now() : Date.now(); + const dctx = directorContextOf(ctx, signal); + + const job = requireJob(deps, params.slug); + const spec = tryLoadSpec(job); + if (!spec) refuse(T.specMissing, T.specMissingFix); + + const reEdition = Boolean(params.new_title?.trim() || params.new_date?.trim()); + let details: RenderSpecDetails; + + if (reEdition) { + // A re-edition is /social's job, verbatim: it forks the folder, rewrites the two + // blocks and re-renders. One implementation, called from both lanes. + const requested = (params.formats as Format[] | undefined)?.filter((f) => FORMATS.includes(f)); + const result = await runSocial( + { + slug: job.slug, + formats: requested?.length ? requested : SOCIAL_FORMATS, + ...(params.new_title?.trim() ? { newTitle: params.new_title.trim() } : {}), + ...(params.new_date?.trim() ? { newDate: params.new_date.trim() } : {}), + fork: params.fork ?? true, + caption: params.caption === true, + onProgress: (msg) => say(msg), + }, + deps.social, + dctx, + ).catch((e: unknown) => fail(e, S.errors.renderFailed(e instanceof Error ? e.message : undefined))); + + details = { + slug: result.slug, + dir: result.dir, + forked: result.forked, + formats: result.formats, + files: result.files, + warnings: result.warnings, + ...(result.captionPath ? { captionPath: result.captionPath } : {}), + usedModel: result.usedModel, + elapsedMs: result.elapsedMs, + }; + } else { + const formats = ((params.formats as Format[] | undefined)?.filter((f) => FORMATS.includes(f)) ?? []) + .length + ? (params.formats as Format[]) + : exportFormats(spec.format); + const artPath = artworkFor(job, spec); + + say(S.progress.exporting); + let outcome: RenderOutcome; + try { + outcome = await deps.poster.renderer.renderAllFormats({ + spec, + artPath, + outDir: job.dir, + formats, + config: cfg, + pdf: params.pdf ?? true, + ...(signal ? { signal } : {}), + onProgress: (msg) => say(msg), + }); + } catch (e) { + fail(e, S.errors.renderFailed(e instanceof Error ? e.message : undefined)); + } + if (!outcome.files.length) refuse(T.nothingWritten); + + const warnings = [...(outcome.warnings ?? [])]; + + // The PDF is the one file a tipografia will reject silently. Check it when we can. + if (deps.poster.preflight && isPrint(spec.format)) { + const pdf = jobFile(job, outputFile(spec.format, "pdf")); + if (existsSync(pdf)) { + say(S.progress.checkingPdf); + const check = await deps.poster.preflight(pdf).catch(() => ({ ok: true, problems: [] })); + if (!check.ok) warnings.push(errorText(S.errors.preflightFailed(check.problems))); + } + } + + details = { + slug: job.slug, + dir: job.dir, + forked: false, + formats: [...formats], + files: [...outcome.files], + warnings, + usedModel: false, + elapsedMs: (deps.now ? deps.now() : Date.now()) - started, + }; + + // Captions are opt-in, always. See the guideline on this tool. + if (params.caption === true) { + say(S.progress.caption); + try { + const written = await writeSocialCaption(job.slug, deps.social, dctx); + details.captionPath = written.path; + details.files = [...details.files, written.path]; + details.usedModel = true; + } catch (e) { + if (isAbort(e)) throw e; + warnings.push(errorText(S.errors.directorUnavailable)); + } + } + } + + if (params.open_folder === true) { + say(S.progress.openingFolder); + (deps.openFolder ?? openInFinder)(details.dir); + } + + const italian = [ + S.done.header, + fill(S.done.folder, { cartella: details.dir }), + ...(details.forked ? [fill(T.reEdition, { cartella: details.dir })] : []), + ...(isPrint(spec.format) && (params.pdf ?? true) + ? [S.done.printNote, ...(cfg.print.cropMarks ? [] : [S.done.printNoCropMarks])] + : []), + ...(details.captionPath ? [] : [T.captionAsk]), + ...(details.warnings.length ? ["", ...details.warnings] : []), + "", + fill(S.done.elapsed, { tempo: duration(details.elapsedMs) }), + ].join("\n"); + + const text = report( + italian, + details.files, + `slug=${details.slug} dir=${details.dir} forked=${details.forked} ` + + `formats=${details.formats.join(",")} usedModel=${details.usedModel}`, + ); + + return { content: await present(details.files, text), details }; + }, + }); +} + +// ═══════════════════════════════════════════════════════════════════════════ +// Image operations — every one of them a thin adapter over runRetouch() +// ═══════════════════════════════════════════════════════════════════════════ + +/** Shared shape of a /retouch-backed tool result. */ +async function retouchResult(res: RetouchResult, italian: string, extra?: string): Promise<{ + content: Content; + details: RetouchResult; +}> { + if (!res.outputs.length) refuse(T.nothingWritten); + const paths = res.outputs.map((o) => o.path); + const body = [ + italian, + ...(res.notes.length ? ["", ...res.notes] : []), + ].join("\n"); + const machine = [ + ...res.outputs.map((o) => `${o.path} (${o.width}x${o.height})`), + `dir=${res.dir} tool=${res.tool}`, + ...(extra ? [extra] : []), + ].join("\n"); + return { content: await present(paths, report(body, [], machine)), details: res }; +} + +// --- edit_image ------------------------------------------------------------- + +const EditImageParams = Type.Object({ + image_path: Type.String({ description: "Absolute path (or one relative to the cwd) of the image to edit." }), + instruction: Type.String({ + description: "What to change, in the user's own words. It is translated into an English art prompt first.", + }), + strength: Type.Optional(Type.Number({ + minimum: 0.05, + maximum: 1, + description: "How far from the original, 0..1. Default 0.55.", + })), + out_dir: Type.Optional(Type.String({ description: "Where to write. Defaults to a job folder under the output directory." })), +}); + +export function createEditImageTool(deps: ToolsDeps) { + return defineTool({ + name: "edit_image", + label: S.toolLabels.retouch, + description: + "Edit an existing image generatively with the local diffusion model (img2img). edit_image is the " + + "FRAGILE one: Draw Things upstream issue #121 makes img2img crash on 16GB Macs, and this is a 16GB " + + "Mac. When it fails it says so as a known bug of the program. remove_background, upscale_image and " + + "vectorize_image never touch the model and always work.", + promptSnippet: + "edit_image — generative img2img edit of an existing image (fragile on this machine; prefer CPU tools).", + promptGuidelines: [ + "Use edit_image only when the user genuinely wants the CONTENT of an existing photo changed. For removing a background use remove_background, for enlarging use upscale_image, for a crisp logo outline use vectorize_image — those are CPU-only and reliable.", + "If edit_image reports the known img2img defect, do not retry it. Offer to paint a fresh image with generate_art instead, which is more reliable and usually better.", + "edit_image runs the diffusion model: never call it alongside generate_art in the same batch.", + ], + parameters: EditImageParams, + // img2img is a full diffusion run. Never parallel. + executionMode: "sequential", + async execute(_id, params, signal, onUpdate, ctx) { + const say = streamer(onUpdate); + const res = await runRetouch( + deps.retouch, + { + action: "edit", + input: inputPath(params.image_path, ctx), + instruction: params.instruction, + ...(params.strength !== undefined ? { strength: params.strength } : {}), + ...(params.out_dir ? { outDir: inputPath(params.out_dir, ctx) } : {}), + }, + { + onProgress: (msg) => say(msg), + ...(signal ? { signal } : {}), + directorContext: directorContextOf(ctx, signal), + }, + ).catch((e: unknown) => fail(e, S.errors.generationFailed(e instanceof Error ? e.message : undefined))); + + return retouchResult(res, "Immagine modificata."); + }, + }); +} + +// --- remove_background ------------------------------------------------------ + +const RemoveBackgroundParams = Type.Object({ + image_path: Type.String({ description: "Absolute path (or one relative to the cwd) of the image." }), + crop_to_subject: Type.Optional(Type.Boolean({ + description: "Also crop to the subject's bounding box. Right for a logo, wrong for compositing. Default false.", + })), + out_dir: Type.Optional(Type.String({ description: "Where to write. Defaults to a job folder under the output directory." })), +}); + +export function createRemoveBackgroundTool(deps: ToolsDeps) { + return defineTool({ + name: "remove_background", + label: S.toolLabels.retouch, + description: + "Cut the subject out of an image and write a transparent RGBA PNG. remove_background uses Apple " + + "Vision first and rembg as a fallback: no diffusion model, no download, always available even when " + + "the image backend is broken.", + promptSnippet: "remove_background — cut out the subject and write a transparent PNG (CPU only).", + promptGuidelines: [ + "Use remove_background whenever the user wants a logo, a face or an object lifted off its background, or wants a PNG he can lay over any colour.", + "Pass crop_to_subject=true to remove_background when the result is a logo or an icon that should sit tight in its box; leave it off when the cut-out has to be composited back at its original position.", + "remove_background outputs a PNG whatever extension is asked for — a JPEG cannot hold the alpha channel that is the whole point.", + ], + parameters: RemoveBackgroundParams, + // Pure CPU (Vision / rembg): safe beside other work. + executionMode: "parallel", + async execute(_id, params, signal, onUpdate, ctx) { + const say = streamer(onUpdate); + const res = await runRetouch( + deps.retouch, + { + action: "background", + input: inputPath(params.image_path, ctx), + ...(params.crop_to_subject !== undefined ? { cropToSubject: params.crop_to_subject } : {}), + ...(params.out_dir ? { outDir: inputPath(params.out_dir, ctx) } : {}), + }, + { onProgress: (msg) => say(msg), ...(signal ? { signal } : {}) }, + ).catch((e: unknown) => fail(e, S.errors.backgroundRemovalFailed)); + + return retouchResult(res, "Sfondo tolto: il PNG è trasparente."); + }, + }); +} + +// --- upscale_image ---------------------------------------------------------- + +const UpscaleImageParams = Type.Object({ + image_path: Type.String({ description: "Absolute path (or one relative to the cwd) of the image." }), + factor: Type.Optional(Type.Number({ + minimum: 1.1, + maximum: 8, + description: "How much bigger. Default 2. Use 4 for an A3 print from a small original.", + })), + out_dir: Type.Optional(Type.String({ description: "Where to write. Defaults to a job folder under the output directory." })), +}); + +export function createUpscaleImageTool(deps: ToolsDeps) { + return defineTool({ + name: "upscale_image", + label: S.toolLabels.retouch, + description: + "Enlarge an image without it going soft. upscale_image runs the ncnn/Vulkan Real-ESRGAN binary — " + + "no diffusion model — and only falls back to the image backend when that binary is missing.", + promptSnippet: "upscale_image — enlarge an image with Real-ESRGAN (CPU/GPU, no diffusion model).", + promptGuidelines: [ + "Use upscale_image when a photo is too small for print, or when a draft has to reach A3 resolution. Factor 2 is the sane default; 4 is for a genuinely small original.", + "upscale_image can fall back to the diffusion backend when the Real-ESRGAN binary is absent, so treat it as heavy work: do not batch it with generate_art or edit_image.", + "A failed upscale is never fatal — if upscale_image reports it could not enlarge, the original is still perfectly good for social sizes.", + ], + parameters: UpscaleImageParams, + // Real-ESRGAN is a second heavy process, and its fallback path is the diffusion + // backend. backends/serialize.ts already queues it; this keeps the agent honest too. + executionMode: "sequential", + async execute(_id, params, signal, onUpdate, ctx) { + const say = streamer(onUpdate); + const res = await runRetouch( + deps.retouch, + { + action: "upscale", + input: inputPath(params.image_path, ctx), + ...(params.factor !== undefined ? { factor: params.factor } : {}), + ...(params.out_dir ? { outDir: inputPath(params.out_dir, ctx) } : {}), + }, + { onProgress: (msg) => say(msg), ...(signal ? { signal } : {}) }, + ).catch((e: unknown) => fail(e, S.errors.upscaleFailed)); + + return retouchResult(res, "Immagine ingrandita."); + }, + }); +} + +// --- vectorize_image -------------------------------------------------------- + +const VectorizeImageParams = Type.Object({ + image_path: Type.String({ description: "Absolute path (or one relative to the cwd) of the raster to trace." }), + out_path: Type.Optional(Type.String({ description: "Where to write the .svg. Defaults to a job folder." })), + mode: Type.Optional(StringEnum(["color", "bw"], { + description: "bw gives the flat silhouette a logo usually wants. Default color.", + default: "color", + })), + max_colors: Type.Optional(Type.Integer({ minimum: 1, maximum: 256, description: "Hard cap on colours. The logo lever." })), + simplify: Type.Optional(Type.Number({ minimum: 0, maximum: 10, description: "Curve simplification tolerance in px, try 1-2.5." })), + filter_speckle: Type.Optional(Type.Integer({ minimum: 0, maximum: 128, description: "Discard patches smaller than N px. Default 4." })), + palette: Type.Optional(Type.Array(Type.String({ pattern: "^#[0-9a-fA-F]{6}$" }), { + description: "Force a fixed palette, e.g. the brand colours.", + })), + background: Type.Optional(Type.String({ + pattern: "^#[0-9a-fA-F]{6}$", + description: "Flatten transparency onto this colour before tracing.", + })), +}); + +export interface VectorizeDetails extends CpuOpResult { + svgPath: string; + source: string; +} + +export function createVectorizeImageTool(deps: ToolsDeps) { + return defineTool({ + name: "vectorize_image", + label: S.toolLabels.retouch, + description: + "Trace a raster image into a clean SVG with vtracer. vectorize_image is what turns a generated logo " + + "into the file a printer can enlarge to any size without pixels; it uses no diffusion model. " + + "Failure is never fatal — the raster stays perfectly usable on screen.", + promptSnippet: "vectorize_image — trace a raster logo into a scalable SVG with vtracer (CPU only).", + promptGuidelines: [ + "Use vectorize_image after remove_background when the user wants a logo he can hand to a printer, put on a T-shirt, or blow up to any size.", + "Pass mode=bw and a small max_colors to vectorize_image for a logo: a mark wants a flat silhouette, not a hundred gradient layers.", + "If vectorize_image reports vtracer is missing, do not improvise another tracer — say the raster PNG is still fine for screen use and name the one-line install the error gives.", + ], + parameters: VectorizeImageParams, + // Pure CPU: safe beside other work. + executionMode: "parallel", + async execute(_id, params, signal, onUpdate, ctx) { + const cfg = cfgOf(deps); + const say = streamer(onUpdate); + const source = inputPath(params.image_path, ctx); + if (!existsSync(source)) fail(new Error(source), S.errors.photoMissing(source)); + + const base = basename(source, extname(source)); + const dir = params.out_path + ? cfg.outputDir + : createJob(cfg, slugify(`ricalco ${base}`), { reuse: true }).dir; + const target = outPath(params.out_path, dir, `${base}.svg`); + + say(S.progress.vectorizing); + const opts: VectorizeOptions = { + config: cfg, + onProgress: (msg) => say(msg), + ...(signal ? { signal } : {}), + ...(params.mode ? { mode: params.mode as "color" | "bw" } : {}), + ...(params.max_colors !== undefined ? { maxColors: params.max_colors } : {}), + ...(params.simplify !== undefined ? { simplify: params.simplify } : {}), + ...(params.filter_speckle !== undefined ? { filterSpeckle: params.filter_speckle } : {}), + ...(params.palette?.length ? { palette: [...params.palette] } : {}), + ...(params.background ? { background: params.background } : {}), + }; + + const res = await withFileMutationQueue(target, () => vectorizerOf(deps).vectorize(source, target, opts)) + .catch((e: unknown) => fail(e, S.errors.vectorizeFailed)); + + const details: VectorizeDetails = { ...res, svgPath: res.path, source }; + const italian = [T.vectorDone, ...(res.notes.length ? ["", ...res.notes] : [])].join("\n"); + const text = report(italian, [res.path], `source=${source} tool=${res.tool}`); + + // sharp rasterises the SVG for the inline preview; if it cannot, present() falls + // back to the text block alone, which still carries the path. + return { content: await present([res.path, source], text), details }; + }, + }); +} + +// ═══════════════════════════════════════════════════════════════════════════ +// list_jobs — what is already on disk +// ═══════════════════════════════════════════════════════════════════════════ + +const ListJobsParams = Type.Object({ + slug: Type.Optional(Type.String({ description: "Show one job in full instead of listing them." })), + query: Type.Optional(Type.String({ description: "Filter by title or slug, accent-insensitive." })), + limit: Type.Optional(Type.Integer({ minimum: 1, maximum: 100, description: "How many to list. Default 25." })), + renderable_only: Type.Optional(Type.Boolean({ + description: "Only jobs with a readable spec.json, i.e. the ones that can be re-rendered. Default false.", + })), +}); + +export interface ListJobsDetails { + jobs: JobSummary[]; + outputDir: string; +} + +export function createListJobsTool(deps: ToolsDeps) { + return defineTool({ + name: "list_jobs", + label: S.toolLabels.brief, + description: + "List the poster and logo jobs already on disk: slug, title, date, format, and which files each " + + "folder holds. list_jobs is how you find the slug that render_spec and typeset_spec need, and how " + + "you check whether a job still has its artwork before promising a re-render.", + promptSnippet: "list_jobs — list existing jobs on disk with their slugs, titles and produced files.", + promptGuidelines: [ + "Call list_jobs before render_spec or typeset_spec whenever you do not already know the exact slug. Never guess a slug — the folder names are Italian and accent-folded.", + "Use list_jobs to answer 'what did I make last time?' and to find the poster a re-edition should fork from.", + "list_jobs reports hasArt: a job without art.png cannot be re-typeset, so say so instead of calling typeset_spec and failing.", + ], + parameters: ListJobsParams, + executionMode: "parallel", + async execute(_id, params, _signal, _onUpdate, _ctx) { + const cfg = cfgOf(deps); + const all = listJobs(cfg, { + limit: params.slug ? 200 : (params.limit ?? 25), + ...(params.renderable_only ? { withSpecOnly: true } : {}), + }); + + const wanted = params.slug ? normaliseSlug(params.slug) : ""; + const needle = params.query?.trim().toLowerCase() ?? ""; + const jobs = all.filter((j) => { + if (wanted) return j.slug === wanted; + if (!needle) return true; + return `${j.slug} ${j.title ?? ""}`.toLowerCase().includes(needle); + }); + + const details: ListJobsDetails = { jobs, outputDir: cfg.outputDir }; + + if (!jobs.length) { + const italian = wanted ? fill(T.jobUnknown, { slug: params.slug ?? "" }) : T.noJobs; + return { + content: [{ type: "text", text: `${italian}\noutputDir=${cfg.outputDir} jobs=0` }] as Content, + details, + }; + } + + const lines = jobs.map((j) => { + const flags = [ + j.hasSpec ? "spec" : "no-spec", + j.hasArt ? "art" : "no-art", + ...(j.hasCaption ? ["caption"] : []), + ].join("/"); + return `${describeJob(j)} — ${j.slug} [${flags}] ${j.outputs.length} file`; + }); + + const machine = jobs + .map((j) => + `slug=${j.slug} dir=${j.dir} format=${j.format ?? "?"} template=${j.template ?? "?"} ` + + `hasSpec=${j.hasSpec} hasArt=${j.hasArt} hasCaption=${j.hasCaption} ` + + `outputs=${j.outputs.join(",") || "-"}`, + ) + .join("\n"); + + const text = [T.jobsHeader, bullets(lines), "", machine].join("\n"); + + // One job asked for by name: show it, since he almost certainly means "that one". + const preview = + jobs.length === 1 && jobs[0] + ? jobs[0].outputs.filter((f) => f.toLowerCase().endsWith(".png")).map((f) => join(jobs[0]!.dir, f)) + : []; + + return { content: await present(preview, text), details }; + }, + }); +} + +// --------------------------------------------------------------------------- +// Registration +// --------------------------------------------------------------------------- + +/** Every tool, in the order the pipeline uses them. index.ts may register a subset. */ +export function createTools(deps: ToolsDeps) { + return [ + createGenerateArtTool(deps), + createTypesetSpecTool(deps), + createRenderSpecTool(deps), + createEditImageTool(deps), + createRemoveBackgroundTool(deps), + createUpscaleImageTool(deps), + createVectorizeImageTool(deps), + createListJobsTool(deps), + ]; +} + +/** + * Registers the whole agent-callable surface. Safe to call from the extension factory — + * nothing here starts a process, a socket, a watcher or a timer. + */ +export function registerTools(pi: ExtensionAPI, deps: ToolsDeps): void { + for (const tool of createTools(deps)) pi.registerTool(tool); +} + +export default registerTools; diff --git a/templates/centred-stack.typ b/templates/centred-stack.typ index b5722b3..7a1255e 100644 --- a/templates/centred-stack.typ +++ b/templates/centred-stack.typ @@ -279,7 +279,10 @@ // wants. With no footer and no logo it is nothing at all. #let footer-band = calc.max(footer-content + footer-lift, logo-reserve) -#let avail = sa.height - 2 * lift - footer-band - top-reserve +// What is left for the stack. Clamped: a logo at the schema's maximum scale on the +// shortest format could otherwise claim more than the sheet has, and a negative box +// height is a broken render rather than a cramped one. +#let avail = calc.max(sa.height * 0.25, sa.height - 2 * lift - footer-band - top-reserve) // A rule is drawn only where it means something: between the title and whatever follows. #let has-rule = { diff --git a/templates/split.typ b/templates/split.typ index 70fe689..255aabd 100644 --- a/templates/split.typ +++ b/templates/split.typ @@ -433,7 +433,17 @@ /// blocks are dropped first. It then spills symmetrically over the seam and the colophon /// rather than overlapping itself, which is the failure worth having: visibly wrong, but /// with nothing silently deleted from an event poster. -#let type-stack = block(width: panel.width, height: flow-h, align(horizon + left, stack-body)) +/// `breakable: false` is not decoration. A breakable block with a fixed height whose +/// content does not fit is split across REGIONS, and a page background offers exactly +/// one — so the remainder is re-laid at the same origin and the blocks print on top of +/// each other. Measured with `safeMm: 80`: subtitle, date and venue all piled onto the +/// same three lines. Unbreakable, the same spec simply overflows in one piece. +#let type-stack = block( + width: panel.width, + height: flow-h, + breakable: false, + align(horizon + left, stack-body), +) /// The colophon, pinned to the foot of the panel and pushed clear of the logo when they /// share that edge. It is aligned AWAY from the logo, so the two read as one line with a