docs: platform notes - macOS 13 vs 14 split, backend assumptions, issue #121

This commit is contained in:
mozempk
2026-08-27 09:52:24 +02:00
parent 756dd9994e
commit 4191489995
6 changed files with 2148 additions and 4 deletions
+6 -2
View File
@@ -35,7 +35,10 @@ veri. Due conseguenze pratiche:
## Requisiti
- **macOS su Apple Silicon.** Sviluppato per un Mac mini M4 base con 16 GB.
- **macOS su Apple Silicon** (obbligatorio: esistono solo build arm64). Sviluppato per un
Mac mini M4 base con 16 GB. **macOS 13** è il minimo; con **macOS 13** funziona tutto
tranne lo scontorno in un passaggio, che richiede macOS 14 e in mancanza ripiega su
un'alternativa — vedi [`docs/platform-notes.md`](docs/platform-notes.md).
- [`draw-things-cli`](https://github.com/drawthingsai/draw-things-community) e
[`typst`](https://typst.app) — installati da `install.sh`.
- Spazio per i modelli (~10 GB). Possono stare su un **disco esterno**: il percorso è
@@ -79,7 +82,8 @@ configurazione.
| Impaginazione | Typst 0.15 — un solo binario, PDF di stampa nativo, sillabazione italiana |
| Caratteri | 22 famiglie OFL/Apache incluse nel pacchetto ([THIRD-PARTY-FONTS.md](THIRD-PARTY-FONTS.md)) |
Dettagli verificati sperimentalmente in [`docs/typst-verified.md`](docs/typst-verified.md).
Dettagli verificati sperimentalmente in [`docs/typst-verified.md`](docs/typst-verified.md);
requisiti di sistema e assunzioni non verificate in [`docs/platform-notes.md`](docs/platform-notes.md).
## Licenza
+64
View File
@@ -0,0 +1,64 @@
# Platform requirements and unverified assumptions
Written down because these bite on someone else's machine, not on the one where the code
was authored. Everything below was established by reading source or vendor docs; items
marked **UNVERIFIED** need the actual Mac to settle and are flagged in the code too.
## macOS version matrix
The stack does **not** have one minimum version — it has three, and the highest one only
affects an optional feature.
| Component | Minimum | If unavailable |
|---|---|---|
| `draw-things-cli` | **macOS 13** (Homebrew Core, `arm64` bottles only) | Nothing works — this is the floor |
| `typst` | macOS 11 | Nothing renders |
| `native/cutout.swift` (Apple Vision) | **macOS 14** | Exits **3**; `cpu.ts` reports it in Italian and falls back to `rembg` |
⚠️ **The cutout helper needs a newer macOS than everything else.** It uses
`VNGenerateForegroundInstanceMaskRequest`, which is macOS 14+, while the rest of the
stack runs on 13. So on macOS 13 everything works *except* one-step background removal,
which degrades rather than failing:
- `/logo` still generates and vectorises; the cutout falls back to `rembg` if installed.
- ⚠️ If falling back, **never accept rembg's default model**: `bria-rmbg` is
**CC BY-NC 4.0** and not usable commercially. Pass `isnet` or `u2net` explicitly.
- rembg also **never touches the Neural Engine** unless given the undocumented
`-x '{"providers":["CoreMLExecutionProvider"]}'` — silently slow otherwise.
Apple Silicon is required outright: `draw-things-cli` ships `arm64` bottles only, and
there is no Intel build. `install.sh` refuses non-arm64 with a clear message.
`native/cutout.swift` is **not compile-tested** — this repo was authored on Linux, with
no Swift toolchain and no Vision framework. It is written in the single-file `swiftc`
pattern, but its first real compile will happen on the Mac.
## Unverified assumptions in the backend
| Assumption | Risk if wrong |
|---|---|
| **Units of `decodingTileWidth`/`decodingTileHeight`** — pixels assumed, matching the 512/512/64 defaults | If they are *latent* units, each tile covers 8× the area and the memory saving is smaller than intended. Flagged on `tilingPayload()`. |
| **No `--strength` CLI flag exists**, so img2img strength travels inside `--config-json` as the `strength` key | Strength silently ignored; edits come back too weak or too strong |
| **Which lever an upscale model expects** — the configured model is driven as the main `--model` over `--image` at low strength; ESRGAN-family names *also* set `upscaler`/`upscalerScaleFactor` | Upscale falls back to a Lanczos3 resample via sharp, announced in Italian, with `model: "lanczos(sharp)"` in the result so the caller can tell |
| **Daemon argv** assumed `<modelsDir> --no-tls --port N [--cpu-offload]`, models dir positional and first | Daemon fails to start; `doctor` reports it |
| **Readiness = a TCP accept** on the port. Without FlatBuffers we cannot ask the daemon anything over gRPC | "Porta aperta" means *reachable*, **not warm**. The first generation may still pay a cold model load. |
| **A Node `fetch` download does not carry `com.apple.quarantine`** | `xattr -d` runs best-effort and its failure is non-fatal |
| **Download integrity is structural, not cryptographic** — size ≥ 1 MiB plus a Mach-O magic number; no published checksum exists to pin | A tampered-but-valid Mach-O would pass |
| **Timeouts** (60 s ready budget, 5 s SIGTERM grace) chosen without a machine to measure on | The ready budget is the one most likely to need raising on a cold first model load |
Also deliberate: **`-w/--weights-cache` is never passed.** Its 0 GiB default is correct on
16 GB — raising it steals exactly the RAM the VAE decode needs.
Draw Things works in **64 px units**, so requested sizes are snapped **up** to that grid
(never down: art slightly larger than the frame can be cropped, art slightly smaller
would have to be upscaled). Real dimensions are read back from the produced file, so a
caller is never told a size the image does not have.
## Still open upstream
**Draw Things issue #121** — img2img crashes with `EXC_BREAKPOINT` on 16 GB Macs on the
current build, via both the API and the UI, unresolved. txt2img is unaffected. Detection
is graded rather than binary: an abnormal signal or an explicit Mach-exception fingerprint
gives confident Italian wording, a bare dead-transport marker gives hedged wording
("con ogni probabilità"), and the diagnosis is only ever offered for runs that passed
`--image`. See `docs/OPEN-DEFECTS.md` for what still works when it bites.
+798
View File
@@ -0,0 +1,798 @@
/**
* index.ts — the extension entry point, and the ONLY place where the pieces meet.
*
* Everything below is wiring. There is no business logic here on purpose: each command
* owns its own flow, each backend owns its own process, and this file's single job is to
* build one config, one backend, one renderer and one director, hand the same objects to
* every command and tool, and start/stop the resident daemon at the right moment.
*
* THE ONE RULE THIS FILE EXISTS TO OBEY:
* an extension factory may run in invocations that never start a session (`pi --help`,
* package listing, a config dump). So the factory below starts NOTHING — no process,
* no socket, no watcher, no timer. It reads a JSON file and registers callbacks.
* The daemon starts in `session_start`, in the background, and is torn down — once,
* idempotently — in `session_shutdown`.
*
* The second rule is graceful degradation: an unplugged model disk, a missing binary or
* a render module that is not there yet must surface as one Italian sentence at the point
* of use. Nothing in here may throw at load time.
*/
import { basename } from "node:path";
import { defineTool, getAgentDir } from "@earendil-works/pi-coding-agent";
import type {
AgentToolResult,
AgentToolUpdateCallback,
ExtensionAPI,
ExtensionCommandContext,
ExtensionContext,
} from "@earendil-works/pi-coding-agent";
import { StringEnum } from "@earendil-works/pi-ai";
import type { ImageContent, TextContent } from "@earendil-works/pi-ai";
import { Type, type Static } from "typebox";
import { loadConfig, type ImgenConfig, type Preset } from "./config.ts";
import type { Backend, ProgressFn } from "./backends/types.ts";
import { createDrawThingsBackend } from "./backends/drawthings.ts";
import { startForSession, type DrawThingsServer } from "./backends/server.ts";
import { removeBackground, upscaleRealesrgan, vectorize } from "./backends/cpu.ts";
import * as director from "./design/director.ts";
import type { Brief, DirectorContext } from "./design/director.ts";
import { FORMATS, TEMPLATES, type Format } from "./design/spec.ts";
import { formatReport, runDoctor, sessionBanner } from "./doctor.ts";
import { openInFinder, outputFile } from "./job.ts";
import { S, duration, errorText, fileLabel, fill } from "./ui/strings.ts";
import {
registerPoster,
runPoster,
type Director,
type PosterDeps,
type PosterResult,
type RenderOutcome,
type RenderRequest,
type Renderer,
} from "./commands/poster.ts";
import { registerLogo, type LogoDeps } from "./commands/logo.ts";
import {
SOCIAL_FORMATS,
registerSocial,
runSocial,
type SocialDeps,
type SocialRenderOutcome,
type SocialRenderRequest,
type SocialRenderedFile,
type SocialRenderer,
type SocialResult,
} from "./commands/social.ts";
import {
previewImage,
registerRetouch,
type PosterHandoff,
type RetouchCropper,
type RetouchDeps,
} from "./commands/retouch.ts";
import { registerPresets, savePreset, type PresetsDeps } from "./commands/presets.ts";
// ---------------------------------------------------------------------------
// Strings this file needs that ui/strings.ts does not have yet
// ---------------------------------------------------------------------------
/**
* TODO: these belong in `ui/strings.ts` (as `S.commands.doctor` and `S.errors.*`).
* They live here only because strings.ts is owned by another module and editing it in
* parallel would conflict. Everything already in `S` is reused rather than duplicated.
*/
const T = {
doctorDescription: "Controlla che ci sia tutto",
doctorUsage: "Scrivi /doctor per sapere cosa manca e come sistemarlo.",
rendererMissing:
"Manca il pezzo che compone le scritte, quindi non posso preparare i file.",
rendererMissingFix: "Reinstalla l'estensione, poi scrivi /doctor per ricontrollare.",
cropperMissing: "Manca il pezzo che ritaglia le immagini per i social.",
handoffAskTitle: "Di che evento si tratta?",
nothingProduced: "Non è uscito nessun file.",
} as const;
const MESSAGE_TYPE = "imgen";
/** How long session_start waits for the daemon before giving up and going cold. */
const SERVER_READY_MS = 90_000;
// ---------------------------------------------------------------------------
// The factory
// ---------------------------------------------------------------------------
export default function imgen(pi: ExtensionAPI): void {
// ---- config: one object, shared by reference ------------------------------
// /presets MUTATES this object in place so that every command holding the same
// reference — and the backend, which keeps it too — sees a new preset at once.
const piConfigDir = getAgentDir();
const loaded = loadConfig(piConfigDir);
const config: ImgenConfig = loaded.config;
const configProblems: string[] = loaded.problems;
// ---- resident daemon: declared here, started in session_start -------------
let server: DrawThingsServer | null = null;
let serverStart: Promise<{ ok: boolean; message: string }> | null = null;
let serverReady = false;
let teardown: Promise<void> | null = null;
/**
* Idempotent, memoised, never throws. Called from session_start (fire and forget) and
* from the first generation (which awaits it), so a tool invoked in a session that
* never fired session_start still gets a warm model.
*
* A failed start stays memoised: retrying the spawn on every generation would hammer a
* broken install. `/doctor` is how he finds out why.
*/
const startServer = (): Promise<{ ok: boolean; message: string }> => {
if (!config.server.enabled) return Promise.resolve({ ok: false, message: "" });
serverStart ??= startForSession(config, piConfigDir, { readyTimeoutMs: SERVER_READY_MS })
.then((res) => {
server = res.server;
serverReady = res.ok;
return { ok: res.ok, message: res.message };
})
// startForSession is documented never to throw; this is belt and braces, because a
// rejected promise here would surface as an unhandled rejection at session start.
.catch((e: unknown) => ({
ok: false,
message: errorText(S.errors.serverStartFailed((e as Error)?.message)),
}));
return serverStart;
};
/**
* Called before every model run. The daemon is an optimisation, never a requirement:
* `drawthings.ts` probes the port itself and runs cold when nothing answers, so all we
* do here is avoid racing a start that is already under way.
*/
const waitForServer = async (onProgress?: ProgressFn): Promise<void> => {
if (!config.server.enabled) return;
if (!serverReady) onProgress?.(S.progress.warmup);
const res = await startServer();
if (!res.ok && res.message) onProgress?.(res.message);
};
const stopServer = async (): Promise<void> => {
const running = server;
server = null;
serverStart = null;
serverReady = false;
if (!running) return;
try {
// `stop()` leaves a daemon it did not spawn alone: killing his own Draw Things
// session out from under him would be rude.
await running.stop();
} catch {
/* a teardown that throws would take the whole shutdown down with it */
}
};
// ---- the backend ---------------------------------------------------------
// Constructing it is pure: no probe, no spawn, no filesystem. The wrapper's only job
// is to let the first generation wait for the daemon that session_start kicked off.
const rawBackend = createDrawThingsBackend(config);
const backend: Backend = {
name: rawBackend.name,
probe: () => rawBackend.probe(),
async generate(opts, onProgress, signal) {
await waitForServer(onProgress);
return rawBackend.generate(opts, onProgress, signal);
},
async upscale(input, outPath, factor, onProgress, signal) {
await waitForServer(onProgress);
return rawBackend.upscale(input, outPath, factor, onProgress, signal);
},
};
// ---- the renderer --------------------------------------------------------
//
// TODO — ASSUMED SIGNATURES. `render/typst.ts` is being written in parallel and owns
// resolveSpec() / render() / renderAllFormats(). This file is the single adapter, so a
// mismatch is fixed here and nowhere else. What is assumed:
// - `render(req)` and `renderAllFormats(req)` take the poster `RenderRequest` shape
// (spec, artPath, outDir, formats, config, pdf, ppi, signal, onProgress);
// - they return `{ files, warnings? }`, where `files` is either absolute paths or
// `{ format, path }` records — both are accepted below;
// - the renderer resolves the spec itself (contrast, geometry, font paths) and writes
// under job.ts's OUTPUT_BASENAMES.
// The module is imported lazily so that a missing or broken renderer is one Italian
// sentence at the point of use rather than a crash while pi is still booting.
type TypstFile = string | { format?: Format; path: string; warnings?: readonly string[] };
interface TypstOutcome {
files?: readonly TypstFile[];
warnings?: readonly string[];
}
interface TypstApi {
render(req: RenderRequest): Promise<TypstOutcome>;
renderAllFormats(req: RenderRequest): Promise<TypstOutcome>;
}
let typstApi: Promise<TypstApi> | null = null;
const typst = (): Promise<TypstApi> => {
typstApi ??= import("./render/typst.ts")
.then((mod) => {
const api = mod as unknown as Partial<TypstApi>;
if (typeof api.render !== "function" || typeof api.renderAllFormats !== "function") {
throw new Error("render/typst.ts: render()/renderAllFormats() mancanti");
}
return api as TypstApi;
})
.catch((e: unknown) => {
typstApi = null; // let a later call try again, e.g. after a reinstall
throw new Error(
errorText({
message: T.rendererMissing,
fix: T.rendererMissingFix,
detail: (e as Error)?.message,
}),
);
});
return typstApi;
};
const pathOf = (f: TypstFile): string => (typeof f === "string" ? f : f.path);
/** basename -> format, so a returned path can be labelled without the renderer's help. */
const FORMAT_BY_FILE = new Map<string, Format>();
for (const f of FORMATS) {
FORMAT_BY_FILE.set(outputFile(f, "png"), f);
FORMAT_BY_FILE.set(outputFile(f, "pdf"), f);
}
const collectWarnings = (out: TypstOutcome): string[] => {
const warnings = [...(out.warnings ?? [])];
for (const f of out.files ?? []) {
if (typeof f !== "string" && f.warnings) warnings.push(...f.warnings);
}
return warnings;
};
const toRenderOutcome = (out: TypstOutcome): RenderOutcome => {
const files = (out.files ?? []).map(pathOf);
const warnings = collectWarnings(out);
return warnings.length > 0 ? { files, warnings } : { files };
};
const renderer: Renderer = {
async render(req) {
return toRenderOutcome(await (await typst()).render(req));
},
async renderAllFormats(req) {
return toRenderOutcome(await (await typst()).renderAllFormats(req));
},
};
/**
* The same renderer seen through /social's eyes: it wants one record per format and
* carries `print` overrides instead of a whole config. Never asks for a PDF — /social
* only ever produces screen sizes.
*/
const socialRenderer: SocialRenderer = {
async renderAllFormats(req: SocialRenderRequest): Promise<SocialRenderOutcome> {
const out = await (await typst()).renderAllFormats({
spec: req.spec,
artPath: req.artPath,
outDir: req.outDir,
formats: req.formats,
config: req.print ? { ...config, print: { ...config.print, ...req.print } } : config,
pdf: false,
signal: req.signal,
onProgress: req.onProgress,
});
const files: SocialRenderedFile[] = [];
const returned = out.files ?? [];
returned.forEach((f, i) => {
const path = pathOf(f);
const named = typeof f === "string" ? undefined : f.format;
// Fall back to the filename, then to the request order: the renderer may not
// label its output, but the names are ours and the order is the one we asked for.
const format = named ?? FORMAT_BY_FILE.get(basename(path)) ?? req.formats[i];
if (!format) return;
const perFile = typeof f === "string" ? undefined : f.warnings;
files.push(perFile && perFile.length > 0 ? { format, path, warnings: perFile } : { format, path });
});
const warnings = out.warnings ?? [];
return warnings.length > 0 ? { files, warnings } : { files };
},
};
/**
* PDF preflight. TODO — ASSUMED SIGNATURE: `render/preflight.ts` exports
* `checkPdf(pdfPath, config?) -> { ok, problems }`. A preflight that cannot run is not
* a broken PDF: the files are still produced, they are simply not inspected.
*/
const preflight = async (pdfPath: string): Promise<{ ok: boolean; problems: string[] }> => {
try {
const mod = (await import("./render/preflight.ts")) as unknown as {
checkPdf?: (path: string, cfg?: ImgenConfig) => Promise<{ ok: boolean; problems: string[] }>;
};
if (typeof mod.checkPdf !== "function") return { ok: true, problems: [] };
return await mod.checkPdf(pdfPath, config);
} catch {
return { ok: true, problems: [] };
}
};
/**
* Saliency-aware crop for /retouch. TODO — ASSUMED SIGNATURE: `render/contrast.ts`
* exports `smartCrop(input, out, {width,height}, opts?)`.
*/
const crop: RetouchCropper = {
async smartCrop(input, out, size, opts) {
let mod: {
smartCrop?: RetouchCropper["smartCrop"];
};
try {
mod = (await import("./render/contrast.ts")) as unknown as { smartCrop?: RetouchCropper["smartCrop"] };
} catch (e) {
throw new Error(
errorText({ message: T.cropperMissing, fix: T.rendererMissingFix, detail: (e as Error)?.message }),
);
}
if (typeof mod.smartCrop !== "function") {
throw new Error(errorText({ message: T.cropperMissing, fix: T.rendererMissingFix }));
}
return mod.smartCrop(input, out, size, opts);
},
};
// ---- shared deps ---------------------------------------------------------
const posterDeps: PosterDeps = {
config,
backend,
renderer,
director: director as Director,
preflight,
openFolder: openInFinder,
};
const presetsDeps: PresetsDeps = {
config,
configDir: piConfigDir,
backend,
renderer,
director,
onConfigChange: (updated) => {
// Keep the ONE shared object: half the extension holds this reference.
if (updated !== config) Object.assign(config, updated);
// /presets only ever writes palette, fonts, logo, tone and defaultPreset — it never
// touches `server`, so the running daemon stays valid.
},
};
const logoDeps: LogoDeps = {
config,
backend,
renderer,
director,
cpu: { removeBackground, vectorize },
piConfigDir,
// Wired to the /presets writer so the two can never disagree about the file format.
savePreset: (key: string, preset: Preset) => savePreset(presetsDeps, key, preset),
};
const socialDeps: SocialDeps = {
config,
renderer: socialRenderer,
director,
// "Parti da zero" is the only branch of /social that may reach a model, and it goes
// through the /poster pipeline rather than growing a second one.
createJobFromBrief: async (brief, ctx, signal) => {
const result = await runPoster(posterDeps, brief, directorContext(ctx, signal), { signal });
return { job: result.job, spec: result.spec };
},
openFolder: openInFinder,
};
/** /retouch → /poster: the photo becomes the artwork, the pipeline does the rest. */
const startPoster: PosterHandoff = async (opts, ctx) => {
let text = opts.freeText?.trim();
if (!text && ctx.hasUI) {
text = (await ctx.ui.input(T.handoffAskTitle, S.commands.poster.usage))?.trim();
}
if (!text) {
ctx.ui.notify(S.common.cancelled, "info");
return;
}
const parsed = await director.parseFreeText(text, directorContext(ctx), config).catch(() => undefined);
const fields = parsed?.fields ?? {};
const brief: Brief = {
kind: "poster",
title: fields.title,
subtitle: fields.subtitle,
date: fields.date,
venue: fields.venue,
details: fields.details,
price: fields.price,
footer: fields.footer,
freeText: parsed?.freeText || text,
format: parsed?.format,
photoPath: opts.photoPath,
};
ctx.ui.setStatus(MESSAGE_TYPE, S.progress.thinking);
try {
const result = await runPoster(posterDeps, brief, directorContext(ctx), {
onProgress: (m) => ctx.ui.setStatus(MESSAGE_TYPE, m),
});
say(pi, describePoster(result));
ctx.ui.notify(S.notify.finished, "info");
} catch (e) {
say(pi, italianError(e));
ctx.ui.notify(S.notify.failed, "error");
} finally {
ctx.ui.setStatus(MESSAGE_TYPE, undefined);
}
};
const retouchDeps: RetouchDeps = {
config,
backend,
cpu: { removeBackground, upscaleRealesrgan },
crop,
director,
startPoster,
};
// ---- commands ------------------------------------------------------------
registerPoster(pi, posterDeps);
registerLogo(pi, logoDeps);
registerSocial(pi, socialDeps);
registerRetouch(pi, retouchDeps);
registerPresets(pi, presetsDeps);
/**
* /doctor. Half the error messages in the string table end with "poi scrivi /doctor per
* ricontrollare", so the command has to exist for those sentences to be true.
*/
pi.registerCommand("doctor", {
description: `${T.doctorDescription}. ${T.doctorUsage}`,
handler: async (args: string, ctx: ExtensionCommandContext): Promise<void> => {
ctx.ui.setStatus(MESSAGE_TYPE, S.doctor.running);
try {
const report = await runDoctor({ piConfigDir, config, configProblems });
say(pi, formatReport(report, { verbose: /(-v|--verbose|dettagl)/i.test(args) }));
} catch (e) {
say(pi, italianError(e));
} finally {
ctx.ui.setStatus(MESSAGE_TYPE, undefined);
}
},
});
// ---- tools ---------------------------------------------------------------
//
// /logo and /retouch register their own (imgen_logo, imgen_retouch) inside their
// register functions. The three below are the ones whose implementation lives in a
// command module but which have no tool of their own yet.
//
// Every tool that can reach the diffusion model is `sequential`: two concurrent runs
// would OOM a 16GB machine.
const PosterParams = Type.Object({
title: Type.Optional(Type.String({ description: "Event title, exactly as it must be typeset." })),
subtitle: Type.Optional(Type.String()),
date: Type.Optional(Type.String({ description: "Italian date as it should appear, e.g. «12 settembre»." })),
venue: Type.Optional(Type.String()),
details: Type.Optional(Type.String({ description: "Extra line: line-up, opening time, booking." })),
price: Type.Optional(Type.String()),
footer: Type.Optional(Type.String({ description: "Organiser, sponsors, small print." })),
free_text: Type.Optional(
Type.String({ description: "Anything else: tone, audience, what must NOT appear." }),
),
format: Type.Optional(StringEnum(FORMATS, { description: "Main format. Defaults to A3 portrait." })),
template: Type.Optional(StringEnum(TEMPLATES, { description: "Layout. Omit to let the art director choose." })),
preset: Type.Optional(Type.String({ description: "Key of a saved brand preset in the user's config." })),
photo_path: Type.Optional(
Type.String({ description: "Absolute path to an existing photo to use instead of generated artwork." }),
),
formats: Type.Optional(
Type.Array(StringEnum(FORMATS), { description: "Every format to export. Defaults to A3 plus the social sizes." }),
),
});
interface PosterToolDetails {
status: string;
slug?: string;
dir?: string;
files?: string[];
warnings?: string[];
elapsedMs?: number;
}
pi.registerTool(
defineTool({
name: "imgen_poster",
label: S.toolLabels.generateArt,
description:
"Create a finished event poster: the art director turns the brief into a design spec, the local " +
"diffusion model paints ARTWORK ONLY (never a letter), and Typst typesets the real Italian text " +
"on top, exporting a print-ready A3 PDF plus the social sizes. Text is data, so accents and dates " +
"are always correct. Slow (minutes) and holds the machine: call it once, never in parallel. " +
"Use imgen_social to re-render an existing job — that path costs seconds and touches no model.",
parameters: PosterParams,
executionMode: "sequential",
async execute(
_toolCallId: string,
params: Static<typeof PosterParams>,
signal: AbortSignal | undefined,
onUpdate: AgentToolUpdateCallback<PosterToolDetails> | undefined,
ctx: ExtensionContext,
): Promise<AgentToolResult<PosterToolDetails>> {
const brief: Brief = {
kind: "poster",
title: params.title,
subtitle: params.subtitle,
date: params.date,
venue: params.venue,
details: params.details,
price: params.price,
footer: params.footer,
freeText: params.free_text,
format: params.format,
template: params.template,
preset: params.preset,
photoPath: params.photo_path,
};
const result = await runPoster(posterDeps, brief, directorContext(ctx, signal), {
formats: params.formats,
signal,
onProgress: (message) =>
onUpdate?.({ content: [{ type: "text", text: message }], details: { status: message } }),
});
return {
content: await presentation(describePoster(result), result.files),
details: {
status: "done",
slug: result.job.slug,
dir: result.job.dir,
files: result.files,
warnings: result.warnings,
elapsedMs: result.elapsedMs,
},
};
},
}),
);
const SocialParams = Type.Object({
slug: Type.String({ description: "Folder name of an existing job under the output directory." }),
formats: Type.Optional(
Type.Array(StringEnum(FORMATS), { description: "Screen formats to write. Defaults to all of them." }),
),
new_title: Type.Optional(Type.String({ description: "Replacement title, verbatim as it must be typeset." })),
new_date: Type.Optional(Type.String({ description: "Replacement date, verbatim, e.g. «11 settembre»." })),
fork: Type.Optional(
Type.Boolean({
description:
"When the text changes, write into a NEW folder (default true). False overwrites a job that may " +
"already have gone to the print shop.",
}),
),
caption: Type.Optional(
Type.Boolean({
description:
"Write caption.txt with Italian social copy and hashtags. Only ever true after the user was asked.",
}),
),
});
interface SocialToolDetails {
status: string;
slug?: string;
dir?: string;
forked?: boolean;
files?: string[];
warnings?: string[];
usedModel?: boolean;
elapsedMs?: number;
}
pi.registerTool(
defineTool({
name: "imgen_social",
label: S.toolLabels.export,
description:
"Re-render an existing job into the social formats (Instagram post and story, Facebook cover, " +
"YouTube thumbnail), optionally with a new title or date. This is a pure Typst pass over the saved " +
"spec: no diffusion model runs, it costs seconds, and changing the text forks a new folder so last " +
"year's file survives. Prefer this over imgen_poster whenever the artwork already exists.",
parameters: SocialParams,
executionMode: "sequential",
async execute(
_toolCallId: string,
params: Static<typeof SocialParams>,
signal: AbortSignal | undefined,
onUpdate: AgentToolUpdateCallback<SocialToolDetails> | undefined,
ctx: ExtensionContext,
): Promise<AgentToolResult<SocialToolDetails>> {
const result: SocialResult = await runSocial(
{
slug: params.slug,
formats: params.formats ?? SOCIAL_FORMATS,
newTitle: params.new_title,
newDate: params.new_date,
fork: params.fork,
caption: params.caption,
onProgress: (message) =>
onUpdate?.({ content: [{ type: "text", text: message }], details: { status: message } }),
},
socialDeps,
{ ...directorContext(ctx, signal), signal },
);
return {
content: await presentation(describeFiles(result.dir, result.files, result.warnings, result.elapsedMs), result.files),
details: {
status: "done",
slug: result.slug,
dir: result.dir,
forked: result.forked,
files: result.files,
warnings: result.warnings,
usedModel: result.usedModel,
elapsedMs: result.elapsedMs,
},
};
},
}),
);
const DoctorParams = Type.Object({
verbose: Type.Optional(Type.Boolean({ description: "List every item, including the healthy ones." })),
});
interface DoctorToolDetails {
ok: boolean;
warnings: boolean;
problems: string[];
checks: { id: string; level: string; message: string; fix?: string }[];
}
pi.registerTool(
defineTool({
name: "imgen_doctor",
label: S.toolLabels.doctor,
description:
"Check that everything the poster pipeline needs is in place: config, model disk, model files, " +
"Draw Things, the resident server, Typst, fonts and the output folder. Read-only, fast, never " +
"throws. Run it first whenever a generation failed, and repeat what it says — the report is in " +
"Italian and already contains the remedy.",
parameters: DoctorParams,
executionMode: "parallel",
async execute(
_toolCallId: string,
params: Static<typeof DoctorParams>,
): Promise<AgentToolResult<DoctorToolDetails>> {
const report = await runDoctor({ piConfigDir, config, configProblems });
return {
content: [{ type: "text", text: formatReport(report, { verbose: params.verbose === true }) }],
details: {
ok: report.ok,
warnings: report.warnings,
problems: report.problems,
checks: report.checks.map((c) => ({
id: c.id,
level: c.level,
message: c.message,
...(c.fix ? { fix: c.fix } : {}),
})),
},
};
},
}),
);
// ---- lifecycle -----------------------------------------------------------
pi.on("session_start", (_event, ctx) => {
// A previous session in this same process may have torn everything down.
teardown = null;
// Deliberately NOT awaited: session start must not wait on a disk check, let alone
// on a model load. Both the report and the daemon land when they land.
void (async () => {
try {
// `fast: true` skips the external --version calls, so this stays instant.
const report = await runDoctor({ piConfigDir, config, configProblems, fast: true });
// sessionBanner() returns null when everything is fine: a health check that
// greets him every morning is a health check he stops reading.
const banner = sessionBanner(report);
if (banner && ctx.hasUI) ctx.ui.notify(banner, report.ok ? "warning" : "error");
// Only warm the daemon when the machine can actually paint: spawning
// gRPCServerCLI against an unplugged model disk just adds a second failure.
if (report.ok && config.server.enabled) void startServer();
} catch {
// A health check that crashes the session is worse than no health check.
}
})();
});
pi.on("session_shutdown", () => {
teardown ??= stopServer();
return teardown;
});
// ---- small helpers -------------------------------------------------------
function describePoster(result: PosterResult): string {
return describeFiles(result.job.dir, result.files, result.warnings, result.elapsedMs);
}
}
// ---------------------------------------------------------------------------
// Free functions — no state, nothing to inject
// ---------------------------------------------------------------------------
function directorContext(ctx: ExtensionContext, signal?: AbortSignal): DirectorContext {
return { modelRegistry: ctx.modelRegistry, model: ctx.model, signal: signal ?? ctx.signal };
}
function say(pi: ExtensionAPI, text: string): void {
pi.sendMessage({ customType: MESSAGE_TYPE, content: text, display: true });
}
/** The closing report: what was made, where it is, what went less than perfectly. */
function describeFiles(
dir: string,
files: readonly string[],
warnings: readonly string[],
elapsedMs: number,
): string {
if (files.length === 0) return T.nothingProduced;
const lines = [S.done.header, fill(S.done.folder, { cartella: dir }), "", S.done.filesHeader];
for (const f of files) lines.push(` • ${fileLabel(basename(f))}`);
if (warnings.length > 0) lines.push("", ...warnings.map((w) => ` ! ${w}`));
lines.push("", fill(S.done.elapsed, { tempo: duration(elapsedMs) }));
return lines.join("\n");
}
/**
* Both halves of a tool result: the image so he SEES it in the terminal, and the text so
* the model gets the paths. A preview that cannot be built is survivable — the text still
* names every file.
*/
async function presentation(
text: string,
files: readonly string[],
): Promise<(TextContent | ImageContent)[]> {
const preview = pickPreview(files);
const image = preview ? await previewImage(preview) : undefined;
return image ? [image, { type: "text", text }] : [{ type: "text", text }];
}
/** The most representative PNG: the square-ish social post first, then the poster. */
function pickPreview(files: readonly string[]): string | undefined {
const byName = (name: string): string | undefined => files.find((f) => basename(f) === name);
return (
byName(outputFile("ig-post")) ??
byName(outputFile("a3-portrait")) ??
byName(outputFile("a4-portrait")) ??
files.find((f) => f.toLowerCase().endsWith(".png"))
);
}
/**
* Every module in this extension raises errors whose `italian` field is what the user
* should read; anything else gets the generic sentence rather than a stack trace.
*/
function italianError(e: unknown): string {
const err = e as { italian?: unknown; message?: unknown };
if (typeof err?.italian === "string" && err.italian) return err.italian;
const message = typeof err?.message === "string" ? err.message : String(e);
// Commands already format their own errors with errorText(); those arrive pre-Italian.
return message.includes("\n") || /[àèéìòù]/i.test(message)
? message
: errorText(S.errors.unknown(message));
}
File diff suppressed because it is too large Load Diff
+4 -1
View File
@@ -279,7 +279,10 @@
// wants. With no footer and no logo it is nothing at all.
#let footer-band = calc.max(footer-content + footer-lift, logo-reserve)
#let avail = sa.height - 2 * lift - footer-band - top-reserve
// What is left for the stack. Clamped: a logo at the schema's maximum scale on the
// shortest format could otherwise claim more than the sheet has, and a negative box
// height is a broken render rather than a cramped one.
#let avail = calc.max(sa.height * 0.25, sa.height - 2 * lift - footer-band - top-reserve)
// A rule is drawn only where it means something: between the title and whatever follows.
#let has-rule = {
+11 -1
View File
@@ -433,7 +433,17 @@
/// blocks are dropped first. It then spills symmetrically over the seam and the colophon
/// rather than overlapping itself, which is the failure worth having: visibly wrong, but
/// with nothing silently deleted from an event poster.
#let type-stack = block(width: panel.width, height: flow-h, align(horizon + left, stack-body))
/// `breakable: false` is not decoration. A breakable block with a fixed height whose
/// content does not fit is split across REGIONS, and a page background offers exactly
/// one — so the remainder is re-laid at the same origin and the blocks print on top of
/// each other. Measured with `safeMm: 80`: subtitle, date and venue all piled onto the
/// same three lines. Unbreakable, the same spec simply overflows in one piece.
#let type-stack = block(
width: panel.width,
height: flow-h,
breakable: false,
align(horizon + left, stack-body),
)
/// The colophon, pinned to the foot of the panel and pushed clear of the logo when they
/// share that edge. It is aligned AWAY from the logo, so the two read as one line with a