docs: platform notes - macOS 13 vs 14 split, backend assumptions, issue #121
This commit is contained in:
@@ -35,7 +35,10 @@ veri. Due conseguenze pratiche:
|
||||
|
||||
## Requisiti
|
||||
|
||||
- **macOS su Apple Silicon.** Sviluppato per un Mac mini M4 base con 16 GB.
|
||||
- **macOS su Apple Silicon** (obbligatorio: esistono solo build arm64). Sviluppato per un
|
||||
Mac mini M4 base con 16 GB. **macOS 13** è il minimo; con **macOS 13** funziona tutto
|
||||
tranne lo scontorno in un passaggio, che richiede macOS 14 e in mancanza ripiega su
|
||||
un'alternativa — vedi [`docs/platform-notes.md`](docs/platform-notes.md).
|
||||
- [`draw-things-cli`](https://github.com/drawthingsai/draw-things-community) e
|
||||
[`typst`](https://typst.app) — installati da `install.sh`.
|
||||
- Spazio per i modelli (~10 GB). Possono stare su un **disco esterno**: il percorso è
|
||||
@@ -79,7 +82,8 @@ configurazione.
|
||||
| Impaginazione | Typst 0.15 — un solo binario, PDF di stampa nativo, sillabazione italiana |
|
||||
| Caratteri | 22 famiglie OFL/Apache incluse nel pacchetto ([THIRD-PARTY-FONTS.md](THIRD-PARTY-FONTS.md)) |
|
||||
|
||||
Dettagli verificati sperimentalmente in [`docs/typst-verified.md`](docs/typst-verified.md).
|
||||
Dettagli verificati sperimentalmente in [`docs/typst-verified.md`](docs/typst-verified.md);
|
||||
requisiti di sistema e assunzioni non verificate in [`docs/platform-notes.md`](docs/platform-notes.md).
|
||||
|
||||
## Licenza
|
||||
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
# Platform requirements and unverified assumptions
|
||||
|
||||
Written down because these bite on someone else's machine, not on the one where the code
|
||||
was authored. Everything below was established by reading source or vendor docs; items
|
||||
marked **UNVERIFIED** need the actual Mac to settle and are flagged in the code too.
|
||||
|
||||
## macOS version matrix
|
||||
|
||||
The stack does **not** have one minimum version — it has three, and the highest one only
|
||||
affects an optional feature.
|
||||
|
||||
| Component | Minimum | If unavailable |
|
||||
|---|---|---|
|
||||
| `draw-things-cli` | **macOS 13** (Homebrew Core, `arm64` bottles only) | Nothing works — this is the floor |
|
||||
| `typst` | macOS 11 | Nothing renders |
|
||||
| `native/cutout.swift` (Apple Vision) | **macOS 14** | Exits **3**; `cpu.ts` reports it in Italian and falls back to `rembg` |
|
||||
|
||||
⚠️ **The cutout helper needs a newer macOS than everything else.** It uses
|
||||
`VNGenerateForegroundInstanceMaskRequest`, which is macOS 14+, while the rest of the
|
||||
stack runs on 13. So on macOS 13 everything works *except* one-step background removal,
|
||||
which degrades rather than failing:
|
||||
|
||||
- `/logo` still generates and vectorises; the cutout falls back to `rembg` if installed.
|
||||
- ⚠️ If falling back, **never accept rembg's default model**: `bria-rmbg` is
|
||||
**CC BY-NC 4.0** and not usable commercially. Pass `isnet` or `u2net` explicitly.
|
||||
- rembg also **never touches the Neural Engine** unless given the undocumented
|
||||
`-x '{"providers":["CoreMLExecutionProvider"]}'` — silently slow otherwise.
|
||||
|
||||
Apple Silicon is required outright: `draw-things-cli` ships `arm64` bottles only, and
|
||||
there is no Intel build. `install.sh` refuses non-arm64 with a clear message.
|
||||
|
||||
`native/cutout.swift` is **not compile-tested** — this repo was authored on Linux, with
|
||||
no Swift toolchain and no Vision framework. It is written in the single-file `swiftc`
|
||||
pattern, but its first real compile will happen on the Mac.
|
||||
|
||||
## Unverified assumptions in the backend
|
||||
|
||||
| Assumption | Risk if wrong |
|
||||
|---|---|
|
||||
| **Units of `decodingTileWidth`/`decodingTileHeight`** — pixels assumed, matching the 512/512/64 defaults | If they are *latent* units, each tile covers 8× the area and the memory saving is smaller than intended. Flagged on `tilingPayload()`. |
|
||||
| **No `--strength` CLI flag exists**, so img2img strength travels inside `--config-json` as the `strength` key | Strength silently ignored; edits come back too weak or too strong |
|
||||
| **Which lever an upscale model expects** — the configured model is driven as the main `--model` over `--image` at low strength; ESRGAN-family names *also* set `upscaler`/`upscalerScaleFactor` | Upscale falls back to a Lanczos3 resample via sharp, announced in Italian, with `model: "lanczos(sharp)"` in the result so the caller can tell |
|
||||
| **Daemon argv** assumed `<modelsDir> --no-tls --port N [--cpu-offload]`, models dir positional and first | Daemon fails to start; `doctor` reports it |
|
||||
| **Readiness = a TCP accept** on the port. Without FlatBuffers we cannot ask the daemon anything over gRPC | "Porta aperta" means *reachable*, **not warm**. The first generation may still pay a cold model load. |
|
||||
| **A Node `fetch` download does not carry `com.apple.quarantine`** | `xattr -d` runs best-effort and its failure is non-fatal |
|
||||
| **Download integrity is structural, not cryptographic** — size ≥ 1 MiB plus a Mach-O magic number; no published checksum exists to pin | A tampered-but-valid Mach-O would pass |
|
||||
| **Timeouts** (60 s ready budget, 5 s SIGTERM grace) chosen without a machine to measure on | The ready budget is the one most likely to need raising on a cold first model load |
|
||||
|
||||
Also deliberate: **`-w/--weights-cache` is never passed.** Its 0 GiB default is correct on
|
||||
16 GB — raising it steals exactly the RAM the VAE decode needs.
|
||||
|
||||
Draw Things works in **64 px units**, so requested sizes are snapped **up** to that grid
|
||||
(never down: art slightly larger than the frame can be cropped, art slightly smaller
|
||||
would have to be upscaled). Real dimensions are read back from the produced file, so a
|
||||
caller is never told a size the image does not have.
|
||||
|
||||
## Still open upstream
|
||||
|
||||
**Draw Things issue #121** — img2img crashes with `EXC_BREAKPOINT` on 16 GB Macs on the
|
||||
current build, via both the API and the UI, unresolved. txt2img is unaffected. Detection
|
||||
is graded rather than binary: an abnormal signal or an explicit Mach-exception fingerprint
|
||||
gives confident Italian wording, a bare dead-transport marker gives hedged wording
|
||||
("con ogni probabilità"), and the diagnosis is only ever offered for runs that passed
|
||||
`--image`. See `docs/OPEN-DEFECTS.md` for what still works when it bites.
|
||||
@@ -0,0 +1,798 @@
|
||||
/**
|
||||
* index.ts — the extension entry point, and the ONLY place where the pieces meet.
|
||||
*
|
||||
* Everything below is wiring. There is no business logic here on purpose: each command
|
||||
* owns its own flow, each backend owns its own process, and this file's single job is to
|
||||
* build one config, one backend, one renderer and one director, hand the same objects to
|
||||
* every command and tool, and start/stop the resident daemon at the right moment.
|
||||
*
|
||||
* THE ONE RULE THIS FILE EXISTS TO OBEY:
|
||||
* an extension factory may run in invocations that never start a session (`pi --help`,
|
||||
* package listing, a config dump). So the factory below starts NOTHING — no process,
|
||||
* no socket, no watcher, no timer. It reads a JSON file and registers callbacks.
|
||||
* The daemon starts in `session_start`, in the background, and is torn down — once,
|
||||
* idempotently — in `session_shutdown`.
|
||||
*
|
||||
* The second rule is graceful degradation: an unplugged model disk, a missing binary or
|
||||
* a render module that is not there yet must surface as one Italian sentence at the point
|
||||
* of use. Nothing in here may throw at load time.
|
||||
*/
|
||||
|
||||
import { basename } from "node:path";
|
||||
|
||||
import { defineTool, getAgentDir } from "@earendil-works/pi-coding-agent";
|
||||
import type {
|
||||
AgentToolResult,
|
||||
AgentToolUpdateCallback,
|
||||
ExtensionAPI,
|
||||
ExtensionCommandContext,
|
||||
ExtensionContext,
|
||||
} from "@earendil-works/pi-coding-agent";
|
||||
import { StringEnum } from "@earendil-works/pi-ai";
|
||||
import type { ImageContent, TextContent } from "@earendil-works/pi-ai";
|
||||
import { Type, type Static } from "typebox";
|
||||
|
||||
import { loadConfig, type ImgenConfig, type Preset } from "./config.ts";
|
||||
import type { Backend, ProgressFn } from "./backends/types.ts";
|
||||
import { createDrawThingsBackend } from "./backends/drawthings.ts";
|
||||
import { startForSession, type DrawThingsServer } from "./backends/server.ts";
|
||||
import { removeBackground, upscaleRealesrgan, vectorize } from "./backends/cpu.ts";
|
||||
import * as director from "./design/director.ts";
|
||||
import type { Brief, DirectorContext } from "./design/director.ts";
|
||||
import { FORMATS, TEMPLATES, type Format } from "./design/spec.ts";
|
||||
import { formatReport, runDoctor, sessionBanner } from "./doctor.ts";
|
||||
import { openInFinder, outputFile } from "./job.ts";
|
||||
import { S, duration, errorText, fileLabel, fill } from "./ui/strings.ts";
|
||||
|
||||
import {
|
||||
registerPoster,
|
||||
runPoster,
|
||||
type Director,
|
||||
type PosterDeps,
|
||||
type PosterResult,
|
||||
type RenderOutcome,
|
||||
type RenderRequest,
|
||||
type Renderer,
|
||||
} from "./commands/poster.ts";
|
||||
import { registerLogo, type LogoDeps } from "./commands/logo.ts";
|
||||
import {
|
||||
SOCIAL_FORMATS,
|
||||
registerSocial,
|
||||
runSocial,
|
||||
type SocialDeps,
|
||||
type SocialRenderOutcome,
|
||||
type SocialRenderRequest,
|
||||
type SocialRenderedFile,
|
||||
type SocialRenderer,
|
||||
type SocialResult,
|
||||
} from "./commands/social.ts";
|
||||
import {
|
||||
previewImage,
|
||||
registerRetouch,
|
||||
type PosterHandoff,
|
||||
type RetouchCropper,
|
||||
type RetouchDeps,
|
||||
} from "./commands/retouch.ts";
|
||||
import { registerPresets, savePreset, type PresetsDeps } from "./commands/presets.ts";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Strings this file needs that ui/strings.ts does not have yet
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* TODO: these belong in `ui/strings.ts` (as `S.commands.doctor` and `S.errors.*`).
|
||||
* They live here only because strings.ts is owned by another module and editing it in
|
||||
* parallel would conflict. Everything already in `S` is reused rather than duplicated.
|
||||
*/
|
||||
const T = {
|
||||
doctorDescription: "Controlla che ci sia tutto",
|
||||
doctorUsage: "Scrivi /doctor per sapere cosa manca e come sistemarlo.",
|
||||
rendererMissing:
|
||||
"Manca il pezzo che compone le scritte, quindi non posso preparare i file.",
|
||||
rendererMissingFix: "Reinstalla l'estensione, poi scrivi /doctor per ricontrollare.",
|
||||
cropperMissing: "Manca il pezzo che ritaglia le immagini per i social.",
|
||||
handoffAskTitle: "Di che evento si tratta?",
|
||||
nothingProduced: "Non è uscito nessun file.",
|
||||
} as const;
|
||||
|
||||
const MESSAGE_TYPE = "imgen";
|
||||
|
||||
/** How long session_start waits for the daemon before giving up and going cold. */
|
||||
const SERVER_READY_MS = 90_000;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// The factory
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export default function imgen(pi: ExtensionAPI): void {
|
||||
// ---- config: one object, shared by reference ------------------------------
|
||||
// /presets MUTATES this object in place so that every command holding the same
|
||||
// reference — and the backend, which keeps it too — sees a new preset at once.
|
||||
const piConfigDir = getAgentDir();
|
||||
const loaded = loadConfig(piConfigDir);
|
||||
const config: ImgenConfig = loaded.config;
|
||||
const configProblems: string[] = loaded.problems;
|
||||
|
||||
// ---- resident daemon: declared here, started in session_start -------------
|
||||
let server: DrawThingsServer | null = null;
|
||||
let serverStart: Promise<{ ok: boolean; message: string }> | null = null;
|
||||
let serverReady = false;
|
||||
let teardown: Promise<void> | null = null;
|
||||
|
||||
/**
|
||||
* Idempotent, memoised, never throws. Called from session_start (fire and forget) and
|
||||
* from the first generation (which awaits it), so a tool invoked in a session that
|
||||
* never fired session_start still gets a warm model.
|
||||
*
|
||||
* A failed start stays memoised: retrying the spawn on every generation would hammer a
|
||||
* broken install. `/doctor` is how he finds out why.
|
||||
*/
|
||||
const startServer = (): Promise<{ ok: boolean; message: string }> => {
|
||||
if (!config.server.enabled) return Promise.resolve({ ok: false, message: "" });
|
||||
serverStart ??= startForSession(config, piConfigDir, { readyTimeoutMs: SERVER_READY_MS })
|
||||
.then((res) => {
|
||||
server = res.server;
|
||||
serverReady = res.ok;
|
||||
return { ok: res.ok, message: res.message };
|
||||
})
|
||||
// startForSession is documented never to throw; this is belt and braces, because a
|
||||
// rejected promise here would surface as an unhandled rejection at session start.
|
||||
.catch((e: unknown) => ({
|
||||
ok: false,
|
||||
message: errorText(S.errors.serverStartFailed((e as Error)?.message)),
|
||||
}));
|
||||
return serverStart;
|
||||
};
|
||||
|
||||
/**
|
||||
* Called before every model run. The daemon is an optimisation, never a requirement:
|
||||
* `drawthings.ts` probes the port itself and runs cold when nothing answers, so all we
|
||||
* do here is avoid racing a start that is already under way.
|
||||
*/
|
||||
const waitForServer = async (onProgress?: ProgressFn): Promise<void> => {
|
||||
if (!config.server.enabled) return;
|
||||
if (!serverReady) onProgress?.(S.progress.warmup);
|
||||
const res = await startServer();
|
||||
if (!res.ok && res.message) onProgress?.(res.message);
|
||||
};
|
||||
|
||||
const stopServer = async (): Promise<void> => {
|
||||
const running = server;
|
||||
server = null;
|
||||
serverStart = null;
|
||||
serverReady = false;
|
||||
if (!running) return;
|
||||
try {
|
||||
// `stop()` leaves a daemon it did not spawn alone: killing his own Draw Things
|
||||
// session out from under him would be rude.
|
||||
await running.stop();
|
||||
} catch {
|
||||
/* a teardown that throws would take the whole shutdown down with it */
|
||||
}
|
||||
};
|
||||
|
||||
// ---- the backend ---------------------------------------------------------
|
||||
// Constructing it is pure: no probe, no spawn, no filesystem. The wrapper's only job
|
||||
// is to let the first generation wait for the daemon that session_start kicked off.
|
||||
const rawBackend = createDrawThingsBackend(config);
|
||||
const backend: Backend = {
|
||||
name: rawBackend.name,
|
||||
probe: () => rawBackend.probe(),
|
||||
async generate(opts, onProgress, signal) {
|
||||
await waitForServer(onProgress);
|
||||
return rawBackend.generate(opts, onProgress, signal);
|
||||
},
|
||||
async upscale(input, outPath, factor, onProgress, signal) {
|
||||
await waitForServer(onProgress);
|
||||
return rawBackend.upscale(input, outPath, factor, onProgress, signal);
|
||||
},
|
||||
};
|
||||
|
||||
// ---- the renderer --------------------------------------------------------
|
||||
//
|
||||
// TODO — ASSUMED SIGNATURES. `render/typst.ts` is being written in parallel and owns
|
||||
// resolveSpec() / render() / renderAllFormats(). This file is the single adapter, so a
|
||||
// mismatch is fixed here and nowhere else. What is assumed:
|
||||
// - `render(req)` and `renderAllFormats(req)` take the poster `RenderRequest` shape
|
||||
// (spec, artPath, outDir, formats, config, pdf, ppi, signal, onProgress);
|
||||
// - they return `{ files, warnings? }`, where `files` is either absolute paths or
|
||||
// `{ format, path }` records — both are accepted below;
|
||||
// - the renderer resolves the spec itself (contrast, geometry, font paths) and writes
|
||||
// under job.ts's OUTPUT_BASENAMES.
|
||||
// The module is imported lazily so that a missing or broken renderer is one Italian
|
||||
// sentence at the point of use rather than a crash while pi is still booting.
|
||||
type TypstFile = string | { format?: Format; path: string; warnings?: readonly string[] };
|
||||
interface TypstOutcome {
|
||||
files?: readonly TypstFile[];
|
||||
warnings?: readonly string[];
|
||||
}
|
||||
interface TypstApi {
|
||||
render(req: RenderRequest): Promise<TypstOutcome>;
|
||||
renderAllFormats(req: RenderRequest): Promise<TypstOutcome>;
|
||||
}
|
||||
|
||||
let typstApi: Promise<TypstApi> | null = null;
|
||||
const typst = (): Promise<TypstApi> => {
|
||||
typstApi ??= import("./render/typst.ts")
|
||||
.then((mod) => {
|
||||
const api = mod as unknown as Partial<TypstApi>;
|
||||
if (typeof api.render !== "function" || typeof api.renderAllFormats !== "function") {
|
||||
throw new Error("render/typst.ts: render()/renderAllFormats() mancanti");
|
||||
}
|
||||
return api as TypstApi;
|
||||
})
|
||||
.catch((e: unknown) => {
|
||||
typstApi = null; // let a later call try again, e.g. after a reinstall
|
||||
throw new Error(
|
||||
errorText({
|
||||
message: T.rendererMissing,
|
||||
fix: T.rendererMissingFix,
|
||||
detail: (e as Error)?.message,
|
||||
}),
|
||||
);
|
||||
});
|
||||
return typstApi;
|
||||
};
|
||||
|
||||
const pathOf = (f: TypstFile): string => (typeof f === "string" ? f : f.path);
|
||||
|
||||
/** basename -> format, so a returned path can be labelled without the renderer's help. */
|
||||
const FORMAT_BY_FILE = new Map<string, Format>();
|
||||
for (const f of FORMATS) {
|
||||
FORMAT_BY_FILE.set(outputFile(f, "png"), f);
|
||||
FORMAT_BY_FILE.set(outputFile(f, "pdf"), f);
|
||||
}
|
||||
|
||||
const collectWarnings = (out: TypstOutcome): string[] => {
|
||||
const warnings = [...(out.warnings ?? [])];
|
||||
for (const f of out.files ?? []) {
|
||||
if (typeof f !== "string" && f.warnings) warnings.push(...f.warnings);
|
||||
}
|
||||
return warnings;
|
||||
};
|
||||
|
||||
const toRenderOutcome = (out: TypstOutcome): RenderOutcome => {
|
||||
const files = (out.files ?? []).map(pathOf);
|
||||
const warnings = collectWarnings(out);
|
||||
return warnings.length > 0 ? { files, warnings } : { files };
|
||||
};
|
||||
|
||||
const renderer: Renderer = {
|
||||
async render(req) {
|
||||
return toRenderOutcome(await (await typst()).render(req));
|
||||
},
|
||||
async renderAllFormats(req) {
|
||||
return toRenderOutcome(await (await typst()).renderAllFormats(req));
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* The same renderer seen through /social's eyes: it wants one record per format and
|
||||
* carries `print` overrides instead of a whole config. Never asks for a PDF — /social
|
||||
* only ever produces screen sizes.
|
||||
*/
|
||||
const socialRenderer: SocialRenderer = {
|
||||
async renderAllFormats(req: SocialRenderRequest): Promise<SocialRenderOutcome> {
|
||||
const out = await (await typst()).renderAllFormats({
|
||||
spec: req.spec,
|
||||
artPath: req.artPath,
|
||||
outDir: req.outDir,
|
||||
formats: req.formats,
|
||||
config: req.print ? { ...config, print: { ...config.print, ...req.print } } : config,
|
||||
pdf: false,
|
||||
signal: req.signal,
|
||||
onProgress: req.onProgress,
|
||||
});
|
||||
|
||||
const files: SocialRenderedFile[] = [];
|
||||
const returned = out.files ?? [];
|
||||
returned.forEach((f, i) => {
|
||||
const path = pathOf(f);
|
||||
const named = typeof f === "string" ? undefined : f.format;
|
||||
// Fall back to the filename, then to the request order: the renderer may not
|
||||
// label its output, but the names are ours and the order is the one we asked for.
|
||||
const format = named ?? FORMAT_BY_FILE.get(basename(path)) ?? req.formats[i];
|
||||
if (!format) return;
|
||||
const perFile = typeof f === "string" ? undefined : f.warnings;
|
||||
files.push(perFile && perFile.length > 0 ? { format, path, warnings: perFile } : { format, path });
|
||||
});
|
||||
|
||||
const warnings = out.warnings ?? [];
|
||||
return warnings.length > 0 ? { files, warnings } : { files };
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* PDF preflight. TODO — ASSUMED SIGNATURE: `render/preflight.ts` exports
|
||||
* `checkPdf(pdfPath, config?) -> { ok, problems }`. A preflight that cannot run is not
|
||||
* a broken PDF: the files are still produced, they are simply not inspected.
|
||||
*/
|
||||
const preflight = async (pdfPath: string): Promise<{ ok: boolean; problems: string[] }> => {
|
||||
try {
|
||||
const mod = (await import("./render/preflight.ts")) as unknown as {
|
||||
checkPdf?: (path: string, cfg?: ImgenConfig) => Promise<{ ok: boolean; problems: string[] }>;
|
||||
};
|
||||
if (typeof mod.checkPdf !== "function") return { ok: true, problems: [] };
|
||||
return await mod.checkPdf(pdfPath, config);
|
||||
} catch {
|
||||
return { ok: true, problems: [] };
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Saliency-aware crop for /retouch. TODO — ASSUMED SIGNATURE: `render/contrast.ts`
|
||||
* exports `smartCrop(input, out, {width,height}, opts?)`.
|
||||
*/
|
||||
const crop: RetouchCropper = {
|
||||
async smartCrop(input, out, size, opts) {
|
||||
let mod: {
|
||||
smartCrop?: RetouchCropper["smartCrop"];
|
||||
};
|
||||
try {
|
||||
mod = (await import("./render/contrast.ts")) as unknown as { smartCrop?: RetouchCropper["smartCrop"] };
|
||||
} catch (e) {
|
||||
throw new Error(
|
||||
errorText({ message: T.cropperMissing, fix: T.rendererMissingFix, detail: (e as Error)?.message }),
|
||||
);
|
||||
}
|
||||
if (typeof mod.smartCrop !== "function") {
|
||||
throw new Error(errorText({ message: T.cropperMissing, fix: T.rendererMissingFix }));
|
||||
}
|
||||
return mod.smartCrop(input, out, size, opts);
|
||||
},
|
||||
};
|
||||
|
||||
// ---- shared deps ---------------------------------------------------------
|
||||
|
||||
const posterDeps: PosterDeps = {
|
||||
config,
|
||||
backend,
|
||||
renderer,
|
||||
director: director as Director,
|
||||
preflight,
|
||||
openFolder: openInFinder,
|
||||
};
|
||||
|
||||
const presetsDeps: PresetsDeps = {
|
||||
config,
|
||||
configDir: piConfigDir,
|
||||
backend,
|
||||
renderer,
|
||||
director,
|
||||
onConfigChange: (updated) => {
|
||||
// Keep the ONE shared object: half the extension holds this reference.
|
||||
if (updated !== config) Object.assign(config, updated);
|
||||
// /presets only ever writes palette, fonts, logo, tone and defaultPreset — it never
|
||||
// touches `server`, so the running daemon stays valid.
|
||||
},
|
||||
};
|
||||
|
||||
const logoDeps: LogoDeps = {
|
||||
config,
|
||||
backend,
|
||||
renderer,
|
||||
director,
|
||||
cpu: { removeBackground, vectorize },
|
||||
piConfigDir,
|
||||
// Wired to the /presets writer so the two can never disagree about the file format.
|
||||
savePreset: (key: string, preset: Preset) => savePreset(presetsDeps, key, preset),
|
||||
};
|
||||
|
||||
const socialDeps: SocialDeps = {
|
||||
config,
|
||||
renderer: socialRenderer,
|
||||
director,
|
||||
// "Parti da zero" is the only branch of /social that may reach a model, and it goes
|
||||
// through the /poster pipeline rather than growing a second one.
|
||||
createJobFromBrief: async (brief, ctx, signal) => {
|
||||
const result = await runPoster(posterDeps, brief, directorContext(ctx, signal), { signal });
|
||||
return { job: result.job, spec: result.spec };
|
||||
},
|
||||
openFolder: openInFinder,
|
||||
};
|
||||
|
||||
/** /retouch → /poster: the photo becomes the artwork, the pipeline does the rest. */
|
||||
const startPoster: PosterHandoff = async (opts, ctx) => {
|
||||
let text = opts.freeText?.trim();
|
||||
if (!text && ctx.hasUI) {
|
||||
text = (await ctx.ui.input(T.handoffAskTitle, S.commands.poster.usage))?.trim();
|
||||
}
|
||||
if (!text) {
|
||||
ctx.ui.notify(S.common.cancelled, "info");
|
||||
return;
|
||||
}
|
||||
const parsed = await director.parseFreeText(text, directorContext(ctx), config).catch(() => undefined);
|
||||
const fields = parsed?.fields ?? {};
|
||||
const brief: Brief = {
|
||||
kind: "poster",
|
||||
title: fields.title,
|
||||
subtitle: fields.subtitle,
|
||||
date: fields.date,
|
||||
venue: fields.venue,
|
||||
details: fields.details,
|
||||
price: fields.price,
|
||||
footer: fields.footer,
|
||||
freeText: parsed?.freeText || text,
|
||||
format: parsed?.format,
|
||||
photoPath: opts.photoPath,
|
||||
};
|
||||
ctx.ui.setStatus(MESSAGE_TYPE, S.progress.thinking);
|
||||
try {
|
||||
const result = await runPoster(posterDeps, brief, directorContext(ctx), {
|
||||
onProgress: (m) => ctx.ui.setStatus(MESSAGE_TYPE, m),
|
||||
});
|
||||
say(pi, describePoster(result));
|
||||
ctx.ui.notify(S.notify.finished, "info");
|
||||
} catch (e) {
|
||||
say(pi, italianError(e));
|
||||
ctx.ui.notify(S.notify.failed, "error");
|
||||
} finally {
|
||||
ctx.ui.setStatus(MESSAGE_TYPE, undefined);
|
||||
}
|
||||
};
|
||||
|
||||
const retouchDeps: RetouchDeps = {
|
||||
config,
|
||||
backend,
|
||||
cpu: { removeBackground, upscaleRealesrgan },
|
||||
crop,
|
||||
director,
|
||||
startPoster,
|
||||
};
|
||||
|
||||
// ---- commands ------------------------------------------------------------
|
||||
|
||||
registerPoster(pi, posterDeps);
|
||||
registerLogo(pi, logoDeps);
|
||||
registerSocial(pi, socialDeps);
|
||||
registerRetouch(pi, retouchDeps);
|
||||
registerPresets(pi, presetsDeps);
|
||||
|
||||
/**
|
||||
* /doctor. Half the error messages in the string table end with "poi scrivi /doctor per
|
||||
* ricontrollare", so the command has to exist for those sentences to be true.
|
||||
*/
|
||||
pi.registerCommand("doctor", {
|
||||
description: `${T.doctorDescription}. ${T.doctorUsage}`,
|
||||
handler: async (args: string, ctx: ExtensionCommandContext): Promise<void> => {
|
||||
ctx.ui.setStatus(MESSAGE_TYPE, S.doctor.running);
|
||||
try {
|
||||
const report = await runDoctor({ piConfigDir, config, configProblems });
|
||||
say(pi, formatReport(report, { verbose: /(-v|--verbose|dettagl)/i.test(args) }));
|
||||
} catch (e) {
|
||||
say(pi, italianError(e));
|
||||
} finally {
|
||||
ctx.ui.setStatus(MESSAGE_TYPE, undefined);
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
// ---- tools ---------------------------------------------------------------
|
||||
//
|
||||
// /logo and /retouch register their own (imgen_logo, imgen_retouch) inside their
|
||||
// register functions. The three below are the ones whose implementation lives in a
|
||||
// command module but which have no tool of their own yet.
|
||||
//
|
||||
// Every tool that can reach the diffusion model is `sequential`: two concurrent runs
|
||||
// would OOM a 16GB machine.
|
||||
|
||||
const PosterParams = Type.Object({
|
||||
title: Type.Optional(Type.String({ description: "Event title, exactly as it must be typeset." })),
|
||||
subtitle: Type.Optional(Type.String()),
|
||||
date: Type.Optional(Type.String({ description: "Italian date as it should appear, e.g. «12 settembre»." })),
|
||||
venue: Type.Optional(Type.String()),
|
||||
details: Type.Optional(Type.String({ description: "Extra line: line-up, opening time, booking." })),
|
||||
price: Type.Optional(Type.String()),
|
||||
footer: Type.Optional(Type.String({ description: "Organiser, sponsors, small print." })),
|
||||
free_text: Type.Optional(
|
||||
Type.String({ description: "Anything else: tone, audience, what must NOT appear." }),
|
||||
),
|
||||
format: Type.Optional(StringEnum(FORMATS, { description: "Main format. Defaults to A3 portrait." })),
|
||||
template: Type.Optional(StringEnum(TEMPLATES, { description: "Layout. Omit to let the art director choose." })),
|
||||
preset: Type.Optional(Type.String({ description: "Key of a saved brand preset in the user's config." })),
|
||||
photo_path: Type.Optional(
|
||||
Type.String({ description: "Absolute path to an existing photo to use instead of generated artwork." }),
|
||||
),
|
||||
formats: Type.Optional(
|
||||
Type.Array(StringEnum(FORMATS), { description: "Every format to export. Defaults to A3 plus the social sizes." }),
|
||||
),
|
||||
});
|
||||
|
||||
interface PosterToolDetails {
|
||||
status: string;
|
||||
slug?: string;
|
||||
dir?: string;
|
||||
files?: string[];
|
||||
warnings?: string[];
|
||||
elapsedMs?: number;
|
||||
}
|
||||
|
||||
pi.registerTool(
|
||||
defineTool({
|
||||
name: "imgen_poster",
|
||||
label: S.toolLabels.generateArt,
|
||||
description:
|
||||
"Create a finished event poster: the art director turns the brief into a design spec, the local " +
|
||||
"diffusion model paints ARTWORK ONLY (never a letter), and Typst typesets the real Italian text " +
|
||||
"on top, exporting a print-ready A3 PDF plus the social sizes. Text is data, so accents and dates " +
|
||||
"are always correct. Slow (minutes) and holds the machine: call it once, never in parallel. " +
|
||||
"Use imgen_social to re-render an existing job — that path costs seconds and touches no model.",
|
||||
parameters: PosterParams,
|
||||
executionMode: "sequential",
|
||||
async execute(
|
||||
_toolCallId: string,
|
||||
params: Static<typeof PosterParams>,
|
||||
signal: AbortSignal | undefined,
|
||||
onUpdate: AgentToolUpdateCallback<PosterToolDetails> | undefined,
|
||||
ctx: ExtensionContext,
|
||||
): Promise<AgentToolResult<PosterToolDetails>> {
|
||||
const brief: Brief = {
|
||||
kind: "poster",
|
||||
title: params.title,
|
||||
subtitle: params.subtitle,
|
||||
date: params.date,
|
||||
venue: params.venue,
|
||||
details: params.details,
|
||||
price: params.price,
|
||||
footer: params.footer,
|
||||
freeText: params.free_text,
|
||||
format: params.format,
|
||||
template: params.template,
|
||||
preset: params.preset,
|
||||
photoPath: params.photo_path,
|
||||
};
|
||||
|
||||
const result = await runPoster(posterDeps, brief, directorContext(ctx, signal), {
|
||||
formats: params.formats,
|
||||
signal,
|
||||
onProgress: (message) =>
|
||||
onUpdate?.({ content: [{ type: "text", text: message }], details: { status: message } }),
|
||||
});
|
||||
|
||||
return {
|
||||
content: await presentation(describePoster(result), result.files),
|
||||
details: {
|
||||
status: "done",
|
||||
slug: result.job.slug,
|
||||
dir: result.job.dir,
|
||||
files: result.files,
|
||||
warnings: result.warnings,
|
||||
elapsedMs: result.elapsedMs,
|
||||
},
|
||||
};
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const SocialParams = Type.Object({
|
||||
slug: Type.String({ description: "Folder name of an existing job under the output directory." }),
|
||||
formats: Type.Optional(
|
||||
Type.Array(StringEnum(FORMATS), { description: "Screen formats to write. Defaults to all of them." }),
|
||||
),
|
||||
new_title: Type.Optional(Type.String({ description: "Replacement title, verbatim as it must be typeset." })),
|
||||
new_date: Type.Optional(Type.String({ description: "Replacement date, verbatim, e.g. «11 settembre»." })),
|
||||
fork: Type.Optional(
|
||||
Type.Boolean({
|
||||
description:
|
||||
"When the text changes, write into a NEW folder (default true). False overwrites a job that may " +
|
||||
"already have gone to the print shop.",
|
||||
}),
|
||||
),
|
||||
caption: Type.Optional(
|
||||
Type.Boolean({
|
||||
description:
|
||||
"Write caption.txt with Italian social copy and hashtags. Only ever true after the user was asked.",
|
||||
}),
|
||||
),
|
||||
});
|
||||
|
||||
interface SocialToolDetails {
|
||||
status: string;
|
||||
slug?: string;
|
||||
dir?: string;
|
||||
forked?: boolean;
|
||||
files?: string[];
|
||||
warnings?: string[];
|
||||
usedModel?: boolean;
|
||||
elapsedMs?: number;
|
||||
}
|
||||
|
||||
pi.registerTool(
|
||||
defineTool({
|
||||
name: "imgen_social",
|
||||
label: S.toolLabels.export,
|
||||
description:
|
||||
"Re-render an existing job into the social formats (Instagram post and story, Facebook cover, " +
|
||||
"YouTube thumbnail), optionally with a new title or date. This is a pure Typst pass over the saved " +
|
||||
"spec: no diffusion model runs, it costs seconds, and changing the text forks a new folder so last " +
|
||||
"year's file survives. Prefer this over imgen_poster whenever the artwork already exists.",
|
||||
parameters: SocialParams,
|
||||
executionMode: "sequential",
|
||||
async execute(
|
||||
_toolCallId: string,
|
||||
params: Static<typeof SocialParams>,
|
||||
signal: AbortSignal | undefined,
|
||||
onUpdate: AgentToolUpdateCallback<SocialToolDetails> | undefined,
|
||||
ctx: ExtensionContext,
|
||||
): Promise<AgentToolResult<SocialToolDetails>> {
|
||||
const result: SocialResult = await runSocial(
|
||||
{
|
||||
slug: params.slug,
|
||||
formats: params.formats ?? SOCIAL_FORMATS,
|
||||
newTitle: params.new_title,
|
||||
newDate: params.new_date,
|
||||
fork: params.fork,
|
||||
caption: params.caption,
|
||||
onProgress: (message) =>
|
||||
onUpdate?.({ content: [{ type: "text", text: message }], details: { status: message } }),
|
||||
},
|
||||
socialDeps,
|
||||
{ ...directorContext(ctx, signal), signal },
|
||||
);
|
||||
|
||||
return {
|
||||
content: await presentation(describeFiles(result.dir, result.files, result.warnings, result.elapsedMs), result.files),
|
||||
details: {
|
||||
status: "done",
|
||||
slug: result.slug,
|
||||
dir: result.dir,
|
||||
forked: result.forked,
|
||||
files: result.files,
|
||||
warnings: result.warnings,
|
||||
usedModel: result.usedModel,
|
||||
elapsedMs: result.elapsedMs,
|
||||
},
|
||||
};
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
const DoctorParams = Type.Object({
|
||||
verbose: Type.Optional(Type.Boolean({ description: "List every item, including the healthy ones." })),
|
||||
});
|
||||
|
||||
interface DoctorToolDetails {
|
||||
ok: boolean;
|
||||
warnings: boolean;
|
||||
problems: string[];
|
||||
checks: { id: string; level: string; message: string; fix?: string }[];
|
||||
}
|
||||
|
||||
pi.registerTool(
|
||||
defineTool({
|
||||
name: "imgen_doctor",
|
||||
label: S.toolLabels.doctor,
|
||||
description:
|
||||
"Check that everything the poster pipeline needs is in place: config, model disk, model files, " +
|
||||
"Draw Things, the resident server, Typst, fonts and the output folder. Read-only, fast, never " +
|
||||
"throws. Run it first whenever a generation failed, and repeat what it says — the report is in " +
|
||||
"Italian and already contains the remedy.",
|
||||
parameters: DoctorParams,
|
||||
executionMode: "parallel",
|
||||
async execute(
|
||||
_toolCallId: string,
|
||||
params: Static<typeof DoctorParams>,
|
||||
): Promise<AgentToolResult<DoctorToolDetails>> {
|
||||
const report = await runDoctor({ piConfigDir, config, configProblems });
|
||||
return {
|
||||
content: [{ type: "text", text: formatReport(report, { verbose: params.verbose === true }) }],
|
||||
details: {
|
||||
ok: report.ok,
|
||||
warnings: report.warnings,
|
||||
problems: report.problems,
|
||||
checks: report.checks.map((c) => ({
|
||||
id: c.id,
|
||||
level: c.level,
|
||||
message: c.message,
|
||||
...(c.fix ? { fix: c.fix } : {}),
|
||||
})),
|
||||
},
|
||||
};
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
// ---- lifecycle -----------------------------------------------------------
|
||||
|
||||
pi.on("session_start", (_event, ctx) => {
|
||||
// A previous session in this same process may have torn everything down.
|
||||
teardown = null;
|
||||
|
||||
// Deliberately NOT awaited: session start must not wait on a disk check, let alone
|
||||
// on a model load. Both the report and the daemon land when they land.
|
||||
void (async () => {
|
||||
try {
|
||||
// `fast: true` skips the external --version calls, so this stays instant.
|
||||
const report = await runDoctor({ piConfigDir, config, configProblems, fast: true });
|
||||
|
||||
// sessionBanner() returns null when everything is fine: a health check that
|
||||
// greets him every morning is a health check he stops reading.
|
||||
const banner = sessionBanner(report);
|
||||
if (banner && ctx.hasUI) ctx.ui.notify(banner, report.ok ? "warning" : "error");
|
||||
|
||||
// Only warm the daemon when the machine can actually paint: spawning
|
||||
// gRPCServerCLI against an unplugged model disk just adds a second failure.
|
||||
if (report.ok && config.server.enabled) void startServer();
|
||||
} catch {
|
||||
// A health check that crashes the session is worse than no health check.
|
||||
}
|
||||
})();
|
||||
});
|
||||
|
||||
pi.on("session_shutdown", () => {
|
||||
teardown ??= stopServer();
|
||||
return teardown;
|
||||
});
|
||||
|
||||
// ---- small helpers -------------------------------------------------------
|
||||
|
||||
function describePoster(result: PosterResult): string {
|
||||
return describeFiles(result.job.dir, result.files, result.warnings, result.elapsedMs);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Free functions — no state, nothing to inject
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function directorContext(ctx: ExtensionContext, signal?: AbortSignal): DirectorContext {
|
||||
return { modelRegistry: ctx.modelRegistry, model: ctx.model, signal: signal ?? ctx.signal };
|
||||
}
|
||||
|
||||
function say(pi: ExtensionAPI, text: string): void {
|
||||
pi.sendMessage({ customType: MESSAGE_TYPE, content: text, display: true });
|
||||
}
|
||||
|
||||
/** The closing report: what was made, where it is, what went less than perfectly. */
|
||||
function describeFiles(
|
||||
dir: string,
|
||||
files: readonly string[],
|
||||
warnings: readonly string[],
|
||||
elapsedMs: number,
|
||||
): string {
|
||||
if (files.length === 0) return T.nothingProduced;
|
||||
const lines = [S.done.header, fill(S.done.folder, { cartella: dir }), "", S.done.filesHeader];
|
||||
for (const f of files) lines.push(` • ${fileLabel(basename(f))}`);
|
||||
if (warnings.length > 0) lines.push("", ...warnings.map((w) => ` ! ${w}`));
|
||||
lines.push("", fill(S.done.elapsed, { tempo: duration(elapsedMs) }));
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Both halves of a tool result: the image so he SEES it in the terminal, and the text so
|
||||
* the model gets the paths. A preview that cannot be built is survivable — the text still
|
||||
* names every file.
|
||||
*/
|
||||
async function presentation(
|
||||
text: string,
|
||||
files: readonly string[],
|
||||
): Promise<(TextContent | ImageContent)[]> {
|
||||
const preview = pickPreview(files);
|
||||
const image = preview ? await previewImage(preview) : undefined;
|
||||
return image ? [image, { type: "text", text }] : [{ type: "text", text }];
|
||||
}
|
||||
|
||||
/** The most representative PNG: the square-ish social post first, then the poster. */
|
||||
function pickPreview(files: readonly string[]): string | undefined {
|
||||
const byName = (name: string): string | undefined => files.find((f) => basename(f) === name);
|
||||
return (
|
||||
byName(outputFile("ig-post")) ??
|
||||
byName(outputFile("a3-portrait")) ??
|
||||
byName(outputFile("a4-portrait")) ??
|
||||
files.find((f) => f.toLowerCase().endsWith(".png"))
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Every module in this extension raises errors whose `italian` field is what the user
|
||||
* should read; anything else gets the generic sentence rather than a stack trace.
|
||||
*/
|
||||
function italianError(e: unknown): string {
|
||||
const err = e as { italian?: unknown; message?: unknown };
|
||||
if (typeof err?.italian === "string" && err.italian) return err.italian;
|
||||
const message = typeof err?.message === "string" ? err.message : String(e);
|
||||
// Commands already format their own errors with errorText(); those arrive pre-Italian.
|
||||
return message.includes("\n") || /[àèéìòù]/i.test(message)
|
||||
? message
|
||||
: errorText(S.errors.unknown(message));
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -279,7 +279,10 @@
|
||||
// wants. With no footer and no logo it is nothing at all.
|
||||
#let footer-band = calc.max(footer-content + footer-lift, logo-reserve)
|
||||
|
||||
#let avail = sa.height - 2 * lift - footer-band - top-reserve
|
||||
// What is left for the stack. Clamped: a logo at the schema's maximum scale on the
|
||||
// shortest format could otherwise claim more than the sheet has, and a negative box
|
||||
// height is a broken render rather than a cramped one.
|
||||
#let avail = calc.max(sa.height * 0.25, sa.height - 2 * lift - footer-band - top-reserve)
|
||||
|
||||
// A rule is drawn only where it means something: between the title and whatever follows.
|
||||
#let has-rule = {
|
||||
|
||||
+11
-1
@@ -433,7 +433,17 @@
|
||||
/// blocks are dropped first. It then spills symmetrically over the seam and the colophon
|
||||
/// rather than overlapping itself, which is the failure worth having: visibly wrong, but
|
||||
/// with nothing silently deleted from an event poster.
|
||||
#let type-stack = block(width: panel.width, height: flow-h, align(horizon + left, stack-body))
|
||||
/// `breakable: false` is not decoration. A breakable block with a fixed height whose
|
||||
/// content does not fit is split across REGIONS, and a page background offers exactly
|
||||
/// one — so the remainder is re-laid at the same origin and the blocks print on top of
|
||||
/// each other. Measured with `safeMm: 80`: subtitle, date and venue all piled onto the
|
||||
/// same three lines. Unbreakable, the same spec simply overflows in one piece.
|
||||
#let type-stack = block(
|
||||
width: panel.width,
|
||||
height: flow-h,
|
||||
breakable: false,
|
||||
align(horizon + left, stack-body),
|
||||
)
|
||||
|
||||
/// The colophon, pinned to the foot of the panel and pushed clear of the logo when they
|
||||
/// share that edge. It is aligned AWAY from the logo, so the two read as one line with a
|
||||
|
||||
Reference in New Issue
Block a user