mozempk
756dd9994e
render/typst.ts stages each job as a self-contained bundle (<job>/.typst/ lib.typ + template + resolved.json) and roots Typst there, so artwork living outside the package is reachable and old jobs stay re-renderable. Paths are root-relative throughout -- Typst resolves a leading / against --root, not the filesystem. render/contrast.ts measures the region behind the text with sharp and forces a scrim on high variance even when mean luminance passes; verified with a checkerboard that scores 5.45:1 contrast yet stdev 119.9. render/preflight.ts checks TrimBox, embedded fonts, image dpi and structure, degrading to 'non installato' rather than failing when a tool is absent. E2E verified via jiti: TrimBox 297.0x420.0mm, font axes injected, bundle staged. 0 type errors.
765 lines
30 KiB
Bash
Executable File
765 lines
30 KiB
Bash
Executable File
#!/usr/bin/env bash
|
||
#
|
||
# fetch-fonts.sh — scarica le 22 famiglie di caratteri incluse in pi-imgen.
|
||
#
|
||
# ---------------------------------------------------------------------------
|
||
# Target shell: bash 3.2 (the one macOS actually ships). No associative arrays,
|
||
# no `mapfile`, no `${var,,}`. BSD sed/grep/awk only: no `grep -P`, no GNU-only
|
||
# flags. Tools required: curl, unzip, awk, sed, od.
|
||
#
|
||
# WHERE THE FILES COME FROM, AND WHY (measured 2026-08-27, not assumed):
|
||
#
|
||
# 1. PRIMARY for the binaries — raw.githubusercontent.com/google/fonts.
|
||
# This is the ONLY source that yields VARIABLE TTFs, which is what we want
|
||
# because Typst 0.15 sets axes (wght/wdth/opsz/SOFT/WONK) at render time.
|
||
# Exact file names are read from METADATA.pb, never guessed: the bracketed
|
||
# variable name is spelled differently per family (Archivo[wdth,wght].ttf
|
||
# but Fraunces[SOFT,WONK,opsz,wght].ttf) and the axis order is not
|
||
# alphabetical by luck. Brackets are percent-encoded for the URL.
|
||
#
|
||
# NOTE — this inverts the source order given in the brief, deliberately.
|
||
# The brief assumed the Fontsource zip ships plain TTFs. It does not:
|
||
# `api.fontsource.org/v1/download/archivo` returns 54 TTFs that are all
|
||
# SUBSET BY UNICODE-RANGE (…-latin-400-normal.ttf, …-latin-ext-…,
|
||
# …-vietnamese-…) plus a `variable/` folder that is WOFF2 ONLY. Bundling
|
||
# those would give Typst a dozen faces sharing one family name and no
|
||
# variable axes at all. So Fontsource stays in the chain as the keyless
|
||
# safety net (tier 2), not as the primary. Set FONT_SOURCE=fontsource (or
|
||
# --source=fontsource) to force the brief's original order.
|
||
#
|
||
# 2. FALLBACK — the Fontsource zip. Keyless, ships a LICENSE, always up.
|
||
# We take its `static/*-latin-*.ttf` slice only: the Google "latin" subset
|
||
# covers the whole audited Italian set (accented vowels, « », ° € – — ‘ ’
|
||
# “ ”), and taking latin-ext as well would install duplicate faces under
|
||
# the same family name. Degraded mode — recorded as such in the manifest
|
||
# and in THIRD-PARTY-FONTS.md.
|
||
#
|
||
# 3. LAST RESORT — the Google css2 endpoint with a NON-BROWSER User-Agent,
|
||
# which hands back full unsubsetted static TTFs keyless. css2 ALWAYS
|
||
# instantiates: it can never return a variable font, so a family that
|
||
# lands here loses its axes.
|
||
#
|
||
# THE THREE RULES THAT PREVENT REAL MISTAKES (each implemented below):
|
||
#
|
||
# RULE 1 — the licence is read from the DIRECTORY, never from the binary.
|
||
# Roboto Condensed's shipped binary still declares Apache-2.0 in name ID 13
|
||
# while the actual grant is OFL. The bucket a family lives in under
|
||
# google/fonts is the grant; see license_id_for_bucket().
|
||
#
|
||
# RULE 2 — the bucket is NEVER hardcoded. Roboto and Open Sans have already
|
||
# migrated out of apache/. resolve_bucket() probes ofl/ apache/ ufl/
|
||
# cc-by-sa/ and uses whichever answers.
|
||
#
|
||
# RULE 3 — a 200 proves a path exists, NOT that it is the family you asked
|
||
# for. There are five distinct "Big Shoulders" families. resolve_bucket()
|
||
# accepts a bucket only when METADATA.pb's `name:` matches the family in
|
||
# fonts.ts EXACTLY. Downloaded bytes are checked for a real sfnt signature
|
||
# too, so an HTML error page can never be saved as a .ttf.
|
||
#
|
||
# The family list is PARSED from extensions/imgen/design/fonts.ts. It is never
|
||
# duplicated here — adding a family there is the only edit needed.
|
||
#
|
||
# Idempotent: a family is skipped only when every file its manifest lists is
|
||
# still on disk (see manifest_complete), so a partial install is repaired
|
||
# instead of being skipped for ever. --force re-fetches regardless.
|
||
#
|
||
# Uso:
|
||
# scripts/fetch-fonts.sh [--force] [--only=<id|famiglia>]
|
||
# [--source=gf|fontsource] [--check] [--help]
|
||
# ---------------------------------------------------------------------------
|
||
|
||
set -euo pipefail
|
||
|
||
SCRIPT_DIR=$(cd "$(dirname "$0")" && pwd)
|
||
ROOT=$(cd "$SCRIPT_DIR/.." && pwd)
|
||
FONTS_TS="$ROOT/extensions/imgen/design/fonts.ts"
|
||
DEST="$ROOT/vendor/fonts"
|
||
REPORT="$ROOT/THIRD-PARTY-FONTS.md"
|
||
|
||
RAW="https://raw.githubusercontent.com/google/fonts/main"
|
||
TREE="https://github.com/google/fonts/tree/main"
|
||
FONTSOURCE="https://api.fontsource.org/v1/download"
|
||
CSS2="https://fonts.googleapis.com/css2"
|
||
|
||
# Non-browser UA: this is what makes css2 return static TTFs instead of woff2.
|
||
UA="pi-imgen/1.0"
|
||
|
||
# RULE 2: the bucket is probed, in this order, never assumed.
|
||
BUCKETS="ofl apache ufl cc-by-sa"
|
||
|
||
FORCE=0
|
||
ONLY=""
|
||
CHECK_ONLY=0
|
||
SOURCE_PREF="${FONT_SOURCE:-gf}"
|
||
|
||
# How many files the fetcher that last ran expected to install. Set by every
|
||
# fetcher, recorded in the manifest as `expected=`, and compared against the
|
||
# installed file list on later runs so a truncated install is never mistaken
|
||
# for a complete one.
|
||
FETCH_EXPECTED=0
|
||
|
||
TMP=$(mktemp -d "${TMPDIR:-/tmp}/pi-imgen-fonts.XXXXXX")
|
||
trap 'rm -rf "$TMP"' EXIT INT TERM
|
||
|
||
# --- output -----------------------------------------------------------------
|
||
if [ -t 1 ]; then C_OK=$'\033[32m'; C_WARN=$'\033[33m'; C_ERR=$'\033[31m'; C_DIM=$'\033[2m'; C_OFF=$'\033[0m'
|
||
else C_OK=""; C_WARN=""; C_ERR=""; C_DIM=""; C_OFF=""; fi
|
||
|
||
say() { printf '%s\n' "$*"; }
|
||
ok() { printf '%s✓%s %s\n' "$C_OK" "$C_OFF" "$*"; }
|
||
info() { printf '%s %s%s\n' "$C_DIM" "$*" "$C_OFF"; }
|
||
warn() { printf '%s!%s %s\n' "$C_WARN" "$C_OFF" "$*" >&2; }
|
||
fail() { printf '%s✗%s %s\n' "$C_ERR" "$C_OFF" "$*" >&2; }
|
||
die() { fail "$*"; exit 1; }
|
||
|
||
usage() {
|
||
cat <<'USAGE'
|
||
Scarica le famiglie di caratteri incluse in pi-imgen dentro vendor/fonts/.
|
||
|
||
--force riscarica anche le famiglie già complete
|
||
--only=<id|nome> lavora su una sola famiglia (es. --only=anton)
|
||
--source=gf|fontsource
|
||
sorgente preferita per i binari (predefinita: gf,
|
||
cioè i TTF variabili di google/fonts)
|
||
--check nessuna rete: verifica quanto è già su disco e
|
||
rigenera THIRD-PARTY-FONTS.md
|
||
--help questo messaggio
|
||
|
||
Variabili d'ambiente: FONT_SOURCE=gf|fontsource
|
||
USAGE
|
||
}
|
||
|
||
for arg in "$@"; do
|
||
case "$arg" in
|
||
--force) FORCE=1 ;;
|
||
--check) CHECK_ONLY=1 ;;
|
||
--only=*) ONLY=$(printf '%s' "${arg#--only=}" | tr '[:upper:]' '[:lower:]') ;;
|
||
--source=*) SOURCE_PREF="${arg#--source=}" ;;
|
||
--help|-h) usage; exit 0 ;;
|
||
*) die "Opzione sconosciuta: $arg (usa --help)" ;;
|
||
esac
|
||
done
|
||
|
||
case "$SOURCE_PREF" in
|
||
gf|fontsource) ;;
|
||
*) die "--source deve essere 'gf' oppure 'fontsource', ricevuto '$SOURCE_PREF'." ;;
|
||
esac
|
||
|
||
for bin in curl unzip awk sed od; do
|
||
command -v "$bin" >/dev/null 2>&1 || die "Manca il comando richiesto: $bin"
|
||
done
|
||
[ -f "$FONTS_TS" ] || die "Non trovo l'elenco delle famiglie: $FONTS_TS"
|
||
|
||
# --- helpers ----------------------------------------------------------------
|
||
|
||
# One place for every network read: retries, timeouts, the non-browser UA, and
|
||
# a hard requirement of HTTP 200 + non-empty body. curl leaves a truncated file
|
||
# behind on failure, so we remove it ourselves rather than trust a partial.
|
||
http_get() { # url dest
|
||
local url="$1" dest="$2" code
|
||
code=$(curl -sS -L -A "$UA" \
|
||
--connect-timeout 15 --max-time 300 \
|
||
--retry 3 --retry-delay 2 \
|
||
-o "$dest" -w '%{http_code}' "$url" </dev/null 2>/dev/null) || { rm -f "$dest"; return 1; }
|
||
[ "$code" = "200" ] || { rm -f "$dest"; return 1; }
|
||
[ -s "$dest" ] || { rm -f "$dest"; return 1; }
|
||
return 0
|
||
}
|
||
|
||
# RULE 3, byte level: a 200 can still be an HTML error page. Require a real
|
||
# sfnt signature (0x00010000 | 'true' | 'OTTO' | 'ttcf').
|
||
is_sfnt() { # path
|
||
local sig
|
||
sig=$(od -An -tx1 -N4 "$1" 2>/dev/null | tr -d ' \n')
|
||
case "$sig" in
|
||
00010000|74727565|4f54544f|74746366) return 0 ;;
|
||
*) return 1 ;;
|
||
esac
|
||
}
|
||
|
||
# Variable TTFs live at Family[axes].ttf; the brackets must be percent-encoded
|
||
# or raw.githubusercontent answers 404. Commas are legal in a path, leave them.
|
||
urlenc() { printf '%s' "$1" | sed -e 's/\[/%5B/g' -e 's/\]/%5D/g'; }
|
||
|
||
lower() { printf '%s' "$1" | tr '[:upper:]' '[:lower:]'; }
|
||
|
||
# RULE 1: the grant comes from the directory the family lives in, never from
|
||
# name ID 13 inside the binary (Roboto Condensed still says Apache-2.0 there
|
||
# while the real grant is OFL).
|
||
license_id_for_bucket() {
|
||
case "$1" in
|
||
ofl) printf 'OFL-1.1' ;;
|
||
apache) printf 'Apache-2.0' ;;
|
||
ufl) printf 'UFL-1.0' ;;
|
||
cc-by-sa) printf 'CC-BY-SA-4.0' ;;
|
||
*) printf 'SCONOSCIUTA' ;;
|
||
esac
|
||
}
|
||
|
||
# Candidate licence file names inside the bucket directory, most likely first.
|
||
license_files_for_bucket() {
|
||
case "$1" in
|
||
ofl) printf 'OFL.txt LICENSE.txt LICENSE' ;;
|
||
apache) printf 'LICENSE.txt LICENSE' ;;
|
||
ufl) printf 'UFL.txt LICENSE.txt LICENSE' ;;
|
||
cc-by-sa) printf 'LICENSE.txt LICENSE' ;;
|
||
*) printf 'LICENSE.txt OFL.txt LICENSE' ;;
|
||
esac
|
||
}
|
||
|
||
manifest_get() { # dir key
|
||
[ -f "$1/.fetch-manifest" ] || return 1
|
||
sed -n "s/^$2=//p" "$1/.fetch-manifest" | head -1
|
||
}
|
||
|
||
# A family counts as complete only when the manifest lists files AND every one
|
||
# of them is still on disk AND the count matches what the fetcher expected.
|
||
# Checking only "LICENSE + at least one *.ttf" is what let a half-downloaded
|
||
# family (say the italic face alone) be skipped for ever: the idempotency gate
|
||
# saw a .ttf, said "già presente", and Typst then rendered everything italic
|
||
# with no error anywhere. Compare against `files=`/`expected=` instead.
|
||
manifest_complete() { # dir
|
||
local dir="$1" files expected listed=0 f
|
||
[ -s "$dir/LICENSE" ] || return 1
|
||
[ -n "$(manifest_get "$dir" license || true)" ] || return 1
|
||
files=$(manifest_get "$dir" files || true)
|
||
[ -n "$files" ] || return 1
|
||
while IFS= read -r f; do
|
||
[ -n "$f" ] || continue
|
||
listed=$((listed + 1))
|
||
[ -f "$dir/$f" ] || return 1
|
||
done <<EOF
|
||
$(printf '%s' "$files" | tr ';' '\n')
|
||
EOF
|
||
[ "$listed" -gt 0 ] || return 1
|
||
# `expected=` is absent in manifests written before this check existed; an
|
||
# old manifest is trusted for its file list only.
|
||
expected=$(manifest_get "$dir" expected || true)
|
||
case "$expected" in
|
||
''|*[!0-9]*) ;;
|
||
*) [ "$listed" -ge "$expected" ] || return 1 ;;
|
||
esac
|
||
# Nothing may have been added or removed behind the manifest's back either.
|
||
[ "$(ls "$dir"/*.ttf 2>/dev/null | wc -l | tr -d ' ')" -ge "$listed" ] || return 1
|
||
return 0
|
||
}
|
||
|
||
# --- the family list, parsed from fonts.ts ---------------------------------
|
||
# Emits TSV: family \t id \t slug \t declared-licence \t axes ("-" if static).
|
||
# Records are gathered by brace balancing, so reformatting fonts.ts does not
|
||
# break this. Never hand-duplicate the list.
|
||
parse_fonts_ts() {
|
||
awk '
|
||
function fld(rec, key, re, s) {
|
||
re = "(^|[^A-Za-z_])" key ":[ \t]*\"[^\"]*\""
|
||
if (!match(rec, re)) return ""
|
||
s = substr(rec, RSTART, RLENGTH)
|
||
sub(/^[^"]*"/, "", s)
|
||
sub(/"$/, "", s)
|
||
return s
|
||
}
|
||
function axesof(rec, s, out, t) {
|
||
if (!match(rec, /axes:[ \t]*\{[^}]*\}/)) return "-"
|
||
s = substr(rec, RSTART, RLENGTH)
|
||
sub(/^axes:[ \t]*\{/, "", s)
|
||
sub(/\}$/, "", s)
|
||
out = ""
|
||
while (match(s, /[A-Za-z]+:[ \t]*\[[ \t]*[0-9.]+[ \t]*,[ \t]*[0-9.]+[ \t]*\]/)) {
|
||
t = substr(s, RSTART, RLENGTH)
|
||
s = substr(s, RSTART + RLENGTH)
|
||
gsub(/[ \t\[\]]/, "", t)
|
||
sub(/,/, "-", t)
|
||
out = (out == "" ? t : out "," t)
|
||
}
|
||
return (out == "" ? "-" : out)
|
||
}
|
||
/export const FONTS/ { inarr = 1; next }
|
||
!inarr { next }
|
||
/^\];/ { inarr = 0; next }
|
||
{
|
||
line = $0
|
||
if (rec == "" && line !~ /\{[ \t]*family:/) next
|
||
rec = rec " " line
|
||
depth += gsub(/\{/, "{", line) - gsub(/\}/, "}", line)
|
||
if (depth <= 0) {
|
||
printf "%s\t%s\t%s\t%s\t%s\n", fld(rec, "family"), fld(rec, "id"), \
|
||
fld(rec, "slug"), fld(rec, "license"), axesof(rec)
|
||
rec = ""; depth = 0
|
||
}
|
||
}
|
||
' "$FONTS_TS"
|
||
}
|
||
|
||
LIST="$TMP/families.tsv"
|
||
parse_fonts_ts > "$LIST"
|
||
TOTAL=$(wc -l < "$LIST" | tr -d ' ')
|
||
# Sanity check the parse against a dumb count of the entries in the array.
|
||
DECLARED=$(awk '/export const FONTS/{i=1;next} /^\];/{i=0} i && /\{[ \t]*family:/{n++} END{print n+0}' "$FONTS_TS")
|
||
[ "$TOTAL" -gt 0 ] || die "Nessuna famiglia estratta da $FONTS_TS — il parser va aggiornato."
|
||
[ "$TOTAL" = "$DECLARED" ] || die "Parser incoerente: $TOTAL famiglie lette, $DECLARED dichiarate in fonts.ts."
|
||
awk -F'\t' '$1=="" || $2=="" || $3=="" {exit 1}' "$LIST" || die "Una voce di fonts.ts non ha family/id/slug."
|
||
|
||
say "pi-imgen — caratteri: $TOTAL famiglie da $FONTS_TS"
|
||
[ "$CHECK_ONLY" = 1 ] && say "Modalità --check: nessun download."
|
||
say ""
|
||
|
||
mkdir -p "$DEST"
|
||
|
||
# --- per-source fetchers ----------------------------------------------------
|
||
# Each prints the names of the files it installed into $stage, one per line, on
|
||
# fd 3 (stdout is reserved for progress). ALL OR NOTHING: a fetcher returns
|
||
# non-zero unless every file it expected arrived intact, so a half-family is a
|
||
# failure the next tier gets a shot at, never something we install. Each also
|
||
# sets FETCH_EXPECTED to the number of files it expected.
|
||
|
||
# Tier 1 — google/fonts raw. Variable TTFs, full character set, exact names
|
||
# taken from METADATA.pb.
|
||
fetch_gf() { # stage meta bucket slug
|
||
local stage="$1" meta="$2" bucket="$3" slug="$4"
|
||
local got=0 expected=0 f url names="$1/.gf-names"
|
||
|
||
# Names straight from METADATA.pb: never reconstructed from the family name,
|
||
# never guessed at the axis order.
|
||
sed -n 's/^[ ]*filename:[ ]*"\(.*\)"/\1/p' "$meta" | sort -u > "$names"
|
||
expected=$(grep -c . "$names" || true)
|
||
FETCH_EXPECTED="$expected"
|
||
|
||
for f in $(cat "$names"); do
|
||
url="$RAW/$bucket/$slug/$(urlenc "$f")"
|
||
if http_get "$url" "$stage/$f"; then
|
||
if is_sfnt "$stage/$f"; then
|
||
got=$((got + 1))
|
||
printf '%s\n' "$f" >&3
|
||
else
|
||
# 200 but not a font: an error page, or the path moved.
|
||
rm -f "$stage/$f"
|
||
warn " $slug: $f non è un font valido, scartato."
|
||
fi
|
||
else
|
||
# Every network failure is reported: a family missing one of the two
|
||
# faces named in METADATA.pb (roman + italic) must never pass silently.
|
||
warn " $slug: $f non scaricato da google/fonts."
|
||
fi
|
||
done
|
||
rm -f "$names"
|
||
|
||
# All-or-nothing: a partial family is a failure, so the caller falls through
|
||
# to the next source instead of installing an italic-only Archivo.
|
||
if [ "$expected" -eq 0 ]; then
|
||
warn " $slug: METADATA.pb non elenca nessun file, niente da scaricare."
|
||
return 1
|
||
fi
|
||
if [ "$got" -ne "$expected" ]; then
|
||
warn " $slug: $got file su $expected da google/fonts — famiglia incompleta, provo un'altra sorgente."
|
||
return 1
|
||
fi
|
||
return 0
|
||
}
|
||
|
||
# Tier 2 — the Fontsource zip. Keyless and always ships a LICENSE, but its TTFs
|
||
# are unicode-range subsets and its variable folder is woff2 only. We keep the
|
||
# `latin` subset, which covers the whole audited Italian character set.
|
||
fetch_fontsource() { # stage id
|
||
local stage="$1" id="$2" zip="$stage/.fontsource.zip" n expected f
|
||
if ! http_get "$FONTSOURCE/$id" "$zip"; then
|
||
warn " $id: zip Fontsource non scaricato."
|
||
return 1
|
||
fi
|
||
unzip -o -q -j "$zip" 'static/*-latin-[0-9]*.ttf' -d "$stage" 2>/dev/null || true
|
||
# LICENSE from the zip is only a fallback for the bucket copy (RULE 1).
|
||
unzip -o -q -j "$zip" 'LICENSE' -d "$stage" 2>/dev/null || true
|
||
[ -f "$stage/LICENSE" ] && mv -f "$stage/LICENSE" "$stage/.fontsource-LICENSE"
|
||
rm -f "$zip"
|
||
# Everything the zip actually yielded is expected; if any of it is not a real
|
||
# font the extraction is treated as failed rather than partially installed.
|
||
n=0; expected=0
|
||
for f in "$stage"/*.ttf; do
|
||
[ -e "$f" ] || continue
|
||
expected=$((expected + 1))
|
||
if is_sfnt "$f"; then n=$((n + 1)); printf '%s\n' "$(basename "$f")" >&3
|
||
else
|
||
rm -f "$f"
|
||
warn " $id: $(basename "$f") non è un font valido, scartato."
|
||
fi
|
||
done
|
||
FETCH_EXPECTED="$expected"
|
||
if [ "$expected" -eq 0 ]; then
|
||
warn " $id: lo zip Fontsource non contiene TTF static/*-latin-*."
|
||
return 1
|
||
fi
|
||
if [ "$n" -ne "$expected" ]; then
|
||
warn " $id: $n file validi su $expected dallo zip Fontsource — incompleto."
|
||
return 1
|
||
fi
|
||
return 0
|
||
}
|
||
|
||
# css2 is strict about the weight list: it MUST be ascending and duplicate-free,
|
||
# and every value must sit inside the family's own wght range — otherwise the
|
||
# endpoint answers 400 with an HTML page instead of CSS. Verified 2026-08-27:
|
||
# `Oswald:wght@200;400;700;700` -> 400, `…@200;400;700` -> 200.
|
||
css2_weights() { # min max
|
||
local lo="$1" hi="$2" v
|
||
{
|
||
printf '%s\n' "$lo" "$hi"
|
||
for v in 400 700; do
|
||
if [ "$v" -lt "$lo" ]; then printf '%s\n' "$lo"
|
||
elif [ "$v" -gt "$hi" ]; then printf '%s\n' "$hi"
|
||
else printf '%s\n' "$v"; fi
|
||
done
|
||
} | sort -n -u | tr '\n' ';' | sed 's/;$//'
|
||
}
|
||
|
||
# Tier 3 — css2 with a non-browser UA. Full unsubsetted statics, but css2 always
|
||
# instantiates: a family that lands here has NO variable axes.
|
||
fetch_css2() { # stage family axes
|
||
local stage="$1" family="$2" axes="$3" q wmin wmax n=0 expected=0 line w url
|
||
q=$(printf '%s' "$family" | sed 's/ /+/g')
|
||
case "$axes" in
|
||
*wght:*)
|
||
wmin=$(printf '%s' "$axes" | sed -n 's/.*wght:\([0-9.]*\)-.*/\1/p')
|
||
wmax=$(printf '%s' "$axes" | sed -n 's/.*wght:[0-9.]*-\([0-9.]*\).*/\1/p')
|
||
q="$q:wght@$(css2_weights "${wmin%.*}" "${wmax%.*}")" ;;
|
||
esac
|
||
if ! http_get "$CSS2?family=$q" "$stage/.css2.css"; then
|
||
warn " $family: css2 non ha risposto per «$q»."
|
||
return 1
|
||
fi
|
||
# Pair each @font-face's weight/style with its .ttf url.
|
||
awk '
|
||
/font-style:/ { st=$2; sub(/;/,"",st) }
|
||
/font-weight:/ { w=$2; sub(/;/,"",w) }
|
||
/url\(/ { if (match($0, /https:[^)]*\.ttf/)) print w "-" st " " substr($0, RSTART, RLENGTH) }
|
||
' "$stage/.css2.css" | sort -u > "$stage/.css2.list"
|
||
rm -f "$stage/.css2.css"
|
||
# Every @font-face the CSS names is expected: css2 lists exactly the faces it
|
||
# will serve, so a missing one is a hole in the family, not a variant we can
|
||
# do without.
|
||
expected=$(grep -c . "$stage/.css2.list" || true)
|
||
FETCH_EXPECTED="$expected"
|
||
while read -r line; do
|
||
[ -n "$line" ] || continue
|
||
w=${line%% *}; url=${line##* }
|
||
local out
|
||
out=$(printf '%s' "$family" | tr -d ' ')"-$w.ttf"
|
||
if http_get "$url" "$stage/$out" && is_sfnt "$stage/$out"; then
|
||
n=$((n + 1)); printf '%s\n' "$out" >&3
|
||
else
|
||
rm -f "$stage/$out"
|
||
warn " $family: faccia $w non scaricata da css2."
|
||
fi
|
||
done < "$stage/.css2.list"
|
||
rm -f "$stage/.css2.list"
|
||
if [ "$expected" -eq 0 ]; then
|
||
warn " $family: css2 non ha elencato nessun .ttf (risposta non CSS?)."
|
||
return 1
|
||
fi
|
||
if [ "$n" -ne "$expected" ]; then
|
||
warn " $family: $n facce su $expected da css2 — famiglia incompleta."
|
||
return 1
|
||
fi
|
||
return 0
|
||
}
|
||
|
||
# Wipe the binaries a failed tier staged, and the names it wrote on fd 3, so
|
||
# the next tier starts from an empty slate. The Fontsource LICENSE copy is
|
||
# deliberately kept: it is a licence fallback, not a binary.
|
||
reset_binaries() { # stage filelist
|
||
rm -f "$1"/*.ttf
|
||
: > "$2"
|
||
}
|
||
|
||
# --- bucket resolution ------------------------------------------------------
|
||
# RULE 2 (probe, never hardcode) + RULE 3 (a 200 is not proof of identity).
|
||
resolve_bucket() { # family slug meta-out -> echoes bucket
|
||
local family="$1" slug="$2" meta="$3" b name
|
||
for b in $BUCKETS; do
|
||
http_get "$RAW/$b/$slug/METADATA.pb" "$meta" || continue
|
||
name=$(awk -F'"' '/^name:/ { print $2; exit }' "$meta")
|
||
if [ "$name" = "$family" ]; then
|
||
printf '%s' "$b"
|
||
return 0
|
||
fi
|
||
# e.g. bigshoulders vs the four other "Big Shoulders …" families.
|
||
warn " $slug: $b/ esiste ma contiene «$name», non «$family» — ignorato."
|
||
rm -f "$meta"
|
||
done
|
||
return 1
|
||
}
|
||
|
||
# Cross-check the axes declared in fonts.ts against the ones the upstream
|
||
# variable font actually has. Typst asks for these at render time, so a stale
|
||
# range in fonts.ts would silently render at a clamped weight.
|
||
check_axes() { # meta declared slug
|
||
local meta="$1" declared="$2" slug="$3" upstream a tag lo hi
|
||
[ "$declared" = "-" ] && { printf '%s' "-"; return 0; }
|
||
upstream=$(awk '
|
||
/^axes \{/ { inax=1; tag=""; lo=""; hi=""; next }
|
||
inax && /tag:/ { t=$2; gsub(/"/,"",t); tag=t }
|
||
inax && /min_value:/ { lo=$2; sub(/\.0$/,"",lo) }
|
||
inax && /max_value:/ { hi=$2; sub(/\.0$/,"",hi) }
|
||
inax && /^\}/ { inax=0; printf "%s%s:%s-%s", (n++ ? "," : ""), tag, lo, hi }
|
||
END { print "" }
|
||
' "$meta")
|
||
[ -z "$upstream" ] && upstream="-"
|
||
for a in $(printf '%s' "$declared" | tr ',' ' '); do
|
||
tag=${a%%:*}
|
||
case ",$upstream," in
|
||
*",$a,"*) ;;
|
||
*) warn " $slug: asse $a dichiarato in fonts.ts, upstream ha «$upstream» — controlla." ;;
|
||
esac
|
||
done
|
||
printf '%s' "$upstream"
|
||
}
|
||
|
||
# --- main loop --------------------------------------------------------------
|
||
FAILED=""
|
||
COUNT_OK=0; COUNT_SKIP=0; COUNT_FAIL=0
|
||
|
||
while IFS=$(printf '\t') read -r family id slug declic axes; do
|
||
[ -n "$family" ] || continue
|
||
if [ -n "$ONLY" ] && [ "$(lower "$id")" != "$ONLY" ] && [ "$(lower "$family")" != "$ONLY" ]; then
|
||
continue
|
||
fi
|
||
|
||
dir="$DEST/$id"
|
||
|
||
# Idempotency: complete family + no --force -> no network at all.
|
||
# "Complete" is measured against the manifest's own file list, so a family
|
||
# installed with only part of its faces is re-fetched instead of skipped.
|
||
if [ "$FORCE" = 0 ] || [ "$CHECK_ONLY" = 1 ]; then
|
||
if manifest_complete "$dir"; then
|
||
info "$family — già presente, salto."
|
||
COUNT_SKIP=$((COUNT_SKIP + 1))
|
||
continue
|
||
fi
|
||
if [ "$CHECK_ONLY" = 0 ] && [ -d "$dir" ]; then
|
||
warn " $family: installazione incompleta rispetto al manifest, riscarico."
|
||
fi
|
||
fi
|
||
if [ "$CHECK_ONLY" = 1 ]; then
|
||
fail "$family — incompleta o assente (modalità --check, non scarico)."
|
||
FAILED="$FAILED $family"
|
||
COUNT_FAIL=$((COUNT_FAIL + 1))
|
||
continue
|
||
fi
|
||
|
||
say "$family ($id)"
|
||
stage="$TMP/stage-$id"
|
||
rm -rf "$stage"; mkdir -p "$stage"
|
||
meta="$stage/METADATA.pb"
|
||
filelist="$stage/.files"
|
||
: > "$filelist"
|
||
|
||
bucket=""
|
||
if bucket=$(resolve_bucket "$family" "$slug" "$meta"); then
|
||
info "bucket risolto: $bucket/ (licenza dalla directory: $(license_id_for_bucket "$bucket"))"
|
||
else
|
||
bucket=""
|
||
warn " $family: nessun bucket di google/fonts corrisponde a «$family» (slug $slug)."
|
||
fi
|
||
|
||
# --- binaries, in the configured order -----------------------------------
|
||
# A tier that fails now leaves partial downloads behind (that is how it knows
|
||
# it is partial), so the staging area and the file list are cleared before
|
||
# every attempt: the next tier must never inherit the previous one's leftovers.
|
||
source_used=""
|
||
FETCH_EXPECTED=0
|
||
if [ "$SOURCE_PREF" = "gf" ] && [ -n "$bucket" ]; then
|
||
reset_binaries "$stage" "$filelist"
|
||
if fetch_gf "$stage" "$meta" "$bucket" "$slug" 3>>"$filelist"; then source_used="google-fonts-raw"; fi
|
||
fi
|
||
if [ -z "$source_used" ]; then
|
||
reset_binaries "$stage" "$filelist"
|
||
if fetch_fontsource "$stage" "$id" 3>>"$filelist"; then source_used="fontsource-latin-subset"; fi
|
||
fi
|
||
if [ -z "$source_used" ] && [ "$SOURCE_PREF" = "fontsource" ] && [ -n "$bucket" ]; then
|
||
reset_binaries "$stage" "$filelist"
|
||
if fetch_gf "$stage" "$meta" "$bucket" "$slug" 3>>"$filelist"; then source_used="google-fonts-raw"; fi
|
||
fi
|
||
if [ -z "$source_used" ]; then
|
||
reset_binaries "$stage" "$filelist"
|
||
if fetch_css2 "$stage" "$family" "$axes" 3>>"$filelist"; then source_used="google-css2-static"; fi
|
||
fi
|
||
if [ -z "$source_used" ]; then reset_binaries "$stage" "$filelist"; fi
|
||
|
||
if [ -z "$source_used" ]; then
|
||
fail "$family: nessuna sorgente ha fornito un TTF valido."
|
||
FAILED="$FAILED $family"
|
||
COUNT_FAIL=$((COUNT_FAIL + 1))
|
||
continue
|
||
fi
|
||
|
||
# --- licence: from the DIRECTORY first (RULE 1) --------------------------
|
||
license_url=""
|
||
if [ -n "$bucket" ]; then
|
||
for lf in $(license_files_for_bucket "$bucket"); do
|
||
if http_get "$RAW/$bucket/$slug/$lf" "$stage/LICENSE"; then
|
||
license_url="$RAW/$bucket/$slug/$lf"
|
||
break
|
||
fi
|
||
done
|
||
fi
|
||
if [ ! -s "$stage/LICENSE" ] && [ -s "$stage/.fontsource-LICENSE" ]; then
|
||
mv -f "$stage/.fontsource-LICENSE" "$stage/LICENSE"
|
||
license_url="$FONTSOURCE/$id (LICENSE nello zip Fontsource)"
|
||
warn " $family: licenza presa dallo zip Fontsource, non dalla directory upstream."
|
||
fi
|
||
rm -f "$stage/.fontsource-LICENSE"
|
||
|
||
if [ ! -s "$stage/LICENSE" ]; then
|
||
fail "$family: nessun file LICENSE trovato — la famiglia NON viene installata."
|
||
FAILED="$FAILED $family"
|
||
COUNT_FAIL=$((COUNT_FAIL + 1))
|
||
continue
|
||
fi
|
||
|
||
license_id=$(license_id_for_bucket "${bucket:-ofl}")
|
||
[ -z "$bucket" ] && license_id="$declic(dichiarata)"
|
||
if [ -n "$bucket" ] && [ "$license_id" != "$declic" ]; then
|
||
warn " $family: fonts.ts dichiara $declic, la directory upstream dice $license_id."
|
||
fi
|
||
|
||
axes_upstream="-"
|
||
[ -f "$meta" ] && axes_upstream=$(check_axes "$meta" "$axes" "$slug")
|
||
|
||
variable="no"
|
||
grep -q '\[' "$filelist" 2>/dev/null && variable="sì"
|
||
|
||
# --- install atomically ---------------------------------------------------
|
||
files_csv=$(tr '\n' ';' < "$filelist" | sed 's/;$//')
|
||
nfiles=$(grep -c . "$filelist" || true)
|
||
{
|
||
printf 'family=%s\n' "$family"
|
||
printf 'id=%s\n' "$id"
|
||
printf 'slug=%s\n' "$slug"
|
||
printf 'bucket=%s\n' "${bucket:-sconosciuto}"
|
||
printf 'license=%s\n' "$license_id"
|
||
printf 'license_declared=%s\n' "$declic"
|
||
printf 'license_url=%s\n' "$license_url"
|
||
printf 'upstream=%s\n' "${bucket:+$TREE/$bucket/$slug}"
|
||
printf 'source=%s\n' "$source_used"
|
||
printf 'variable=%s\n' "$variable"
|
||
printf 'axes_declared=%s\n' "$axes"
|
||
printf 'axes_upstream=%s\n' "$axes_upstream"
|
||
printf 'files=%s\n' "$files_csv"
|
||
printf 'expected=%s\n' "$FETCH_EXPECTED"
|
||
printf 'fetched=%s\n' "$(date -u '+%Y-%m-%dT%H:%M:%SZ')"
|
||
} > "$stage/.fetch-manifest"
|
||
rm -f "$filelist"
|
||
|
||
rm -rf "$dir"
|
||
mkdir -p "$(dirname "$dir")"
|
||
mv "$stage" "$dir"
|
||
|
||
ok "$family — $nfiles file, $license_id, sorgente: $source_used, variabile: $variable"
|
||
COUNT_OK=$((COUNT_OK + 1))
|
||
done < "$LIST"
|
||
|
||
if [ -n "$ONLY" ] && [ $((COUNT_OK + COUNT_SKIP + COUNT_FAIL)) -eq 0 ]; then
|
||
die "--only=$ONLY non corrisponde a nessuna famiglia di fonts.ts."
|
||
fi
|
||
|
||
# --- report -----------------------------------------------------------------
|
||
# Rebuilt from the manifests on disk, so it always describes what is actually
|
||
# installed — including families skipped as already-present in this run.
|
||
build_report() {
|
||
{
|
||
printf '# Caratteri di terze parti inclusi in pi-imgen\n\n'
|
||
printf 'Generato da `scripts/fetch-fonts.sh` il %s. Non modificare a mano.\n\n' "$(date -u '+%Y-%m-%d %H:%M UTC')"
|
||
printf 'I file dei caratteri stanno in `vendor/fonts/<id>/` e **non** sono versionati.\n'
|
||
printf 'Ogni cartella contiene il file `LICENSE` originale.\n\n'
|
||
printf 'La licenza è letta dalla **directory upstream** in `google/fonts` (il bucket\n'
|
||
printf '`ofl/`, `apache/`, `ufl/`, `cc-by-sa/`), mai dai metadati dentro il binario:\n'
|
||
printf 'per esempio il binario di Roboto Condensed dichiara ancora Apache-2.0 nel name\n'
|
||
printf 'ID 13 mentre la concessione reale è OFL. Il bucket viene risolto a ogni\n'
|
||
printf 'esecuzione, mai scritto a mano, perché le famiglie migrano fra bucket.\n\n'
|
||
printf '| Famiglia | Licenza | Origine upstream | File | Variabile | Sorgente |\n'
|
||
printf '|---|---|---|---|---|---|\n'
|
||
|
||
while IFS=$(printf '\t') read -r family id slug declic axes; do
|
||
[ -n "$family" ] || continue
|
||
d="$DEST/$id"
|
||
if [ ! -f "$d/.fetch-manifest" ]; then
|
||
printf '| %s | %s (dichiarata) | — | **non scaricata** | — | — |\n' "$family" "$declic"
|
||
continue
|
||
fi
|
||
if [ ! -s "$d/LICENSE" ]; then
|
||
printf '| %s | **LICENSE MANCANTE** | — | **non ridistribuibile** | — | — |\n' "$family"
|
||
continue
|
||
fi
|
||
lic=$(manifest_get "$d" license)
|
||
up=$(manifest_get "$d" upstream)
|
||
lurl=$(manifest_get "$d" license_url)
|
||
src=$(manifest_get "$d" source)
|
||
var=$(manifest_get "$d" variable)
|
||
nf=$(ls "$d"/*.ttf 2>/dev/null | wc -l | tr -d ' ')
|
||
[ -n "$up" ] || up="—"
|
||
case "$src" in
|
||
google-fonts-raw) srclabel='google/fonts (TTF completi)' ;;
|
||
fontsource-latin-subset) srclabel='Fontsource (**subset `latin`**, niente assi)' ;;
|
||
google-css2-static) srclabel='css2 (**statici**, niente assi)' ;;
|
||
*) srclabel="$src" ;;
|
||
esac
|
||
printf '| %s | [%s](%s) | [%s](%s) | %s | %s | %s |\n' \
|
||
"$family" "$lic" "${lurl:-$up}" "${up##*/main/}" "$up" "$nf" "$var" "$srclabel"
|
||
done < "$LIST"
|
||
|
||
printf '\n## Note\n\n'
|
||
printf -- '- **OFL-1.1**: ridistribuzione libera, anche commerciale, purché i caratteri\n'
|
||
printf -- ' restino accompagnati dalla licenza e non vengano venduti da soli. Le famiglie\n'
|
||
printf -- ' con Reserved Font Name (Playfair Display, DM Serif Display, Alfa Slab One)\n'
|
||
printf -- ' vanno incluse **non modificate**: rinominare il file non basta, va rinominato\n'
|
||
printf -- ' il font se lo si altera.\n'
|
||
printf -- '- **Apache-2.0**: nessun obbligo di attribuzione nel prodotto finale, ma il file\n'
|
||
printf -- ' `LICENSE` va conservato accanto ai binari.\n'
|
||
printf -- '- I TTF variabili arrivano solo da `raw.githubusercontent.com/google/fonts`.\n'
|
||
printf -- ' L'"'"'endpoint css2 istanzia sempre e non può restituire un font variabile;\n'
|
||
printf -- ' lo zip di Fontsource contiene TTF statici divisi per unicode-range e i\n'
|
||
printf -- ' variabili solo in WOFF2, inutili per Typst.\n'
|
||
printf -- '- Rigenera questo file con `scripts/fetch-fonts.sh --check`.\n'
|
||
} > "$REPORT"
|
||
}
|
||
|
||
build_report
|
||
ok "Rigenerato $REPORT"
|
||
|
||
# --- final verification: no family may end up without a LICENSE -------------
|
||
MISSING=""
|
||
while IFS=$(printf '\t') read -r family id slug declic axes; do
|
||
[ -n "$family" ] || continue
|
||
if [ -n "$ONLY" ] && [ "$(lower "$id")" != "$ONLY" ] && [ "$(lower "$family")" != "$ONLY" ]; then
|
||
continue
|
||
fi
|
||
d="$DEST/$id"
|
||
if [ ! -s "$d/LICENSE" ]; then
|
||
MISSING="$MISSING $family(LICENSE)"
|
||
elif ! ls "$d"/*.ttf >/dev/null 2>&1; then
|
||
MISSING="$MISSING $family(TTF)"
|
||
elif ! manifest_complete "$d"; then
|
||
MISSING="$MISSING $family(incompleta)"
|
||
fi
|
||
done < "$LIST"
|
||
|
||
say ""
|
||
say "Scaricate: $COUNT_OK · già presenti: $COUNT_SKIP · fallite: $COUNT_FAIL"
|
||
|
||
if [ -n "$MISSING" ] || [ -n "$FAILED" ]; then
|
||
say ""
|
||
if [ -n "$FAILED" ]; then
|
||
fail "Non scaricate:$FAILED"
|
||
fi
|
||
if [ -n "$MISSING" ]; then
|
||
fail "Incomplete su disco:$MISSING"
|
||
# The licence warning only when a LICENSE is what is actually missing:
|
||
# «(incompleta)» means faces are missing, which is a different problem.
|
||
case "$MISSING" in
|
||
*"(LICENSE)"*) fail "Una famiglia senza LICENSE non è ridistribuibile: non pubblicare questa build." ;;
|
||
esac
|
||
fi
|
||
fail "Riprova con: scripts/fetch-fonts.sh --force --only=<id>"
|
||
exit 1
|
||
fi
|
||
|
||
ok "Tutte le famiglie richieste hanno TTF e LICENSE."
|