Compare commits

..

3 Commits

Author SHA1 Message Date
Stéphan Sainléger
002d8fe196 fix: [carbone-ee] enable Studio and template management
The charm had no ``hooks/init``, so Carbone started in its default
stateless mode: no Studio, no template management, and ``CARBONE_BIND``
defaulting to ``127.0.0.1`` (unreachable behind the web-proxy).

Add ``hooks/init`` to:
- bind on ``0.0.0.0`` so the ``web-proxy`` can reach the service ;
- serve the Studio and ``/carbone-studio.js`` (``CARBONE_STUDIO``) ;
- enable stateful template management (``CARBONE_TEMPLATE_MANAGEMENT``):
  stable 64-bit template IDs, versioning and the ``/template`` endpoints ;
- align the datastore/configstore ownership on the unprivileged
  ``carbone`` image user, otherwise the container crash-loops with
  ``EACCES`` on ``/app/config/config.json``.

Persist ``/app/config`` as a ``config-resources`` and set a
``docker-compose.stop_grace_period`` of 200s: Carbone flushes its
template metadata on graceful shutdown, and Docker's 10s default would
SIGKILL it and lose templates on redeploy.  Flush metadata every
5 minutes to shrink the hard-crash window.

Bump the image to ``full-5.15.1`` and add a local ``basic-deploy`` test.

Verified locally: ``/status`` 200, ``/carbone-studio.js`` 200,
``GET /templates`` 200, template upload with versioning plus render
from the stable template ID, and metadata survival across a container
recreation.
2026-09-25 19:06:38 +02:00
Stéphan Sainléger
fda96565ad fix: [outline] align datastore ownership with the application user
Outline runs as an unprivileged user in the image (``nodejs`` since
1.10.0, ``root`` up to 1.6.1) while the datastore is provisioned by
``root``.  Without realignment the application cannot write its
``uploads``, ``public`` and ``avatars`` buckets, so every attachment
upload fails with "Permission denied writing to ... Check the host
machine file system permissions".  This was seen on elabore.coop
after the 1.6.1 to 1.10.0 upgrade.

The ``init`` hook now reads the image's ``Config.User`` and chowns the
datastore to that user, skipping root-based images.  It is idempotent
and version-agnostic: the realignment also covers buckets added by
future Outline versions.
2026-09-16 23:43:01 +02:00
Stéphan Sainléger
9947900389 fix: [outline] align database ownership before migrations
The ``pre_deploy`` hook reassigns every object of the outline database
(tables, sequences, views, materialized views, standalone types,
functions, procedures) to the application role before the container
runs its migrations.

Historical provisioning or restores running as the ``postgres``
superuser leave objects owned by ``postgres``, so any later ``ALTER``
on them fails with "must be owner of ..." and puts outline in a
crash-loop at migration time.  Seen on elabore.coop when upgrading
1.6.1 -> 1.10.0: migration
``20260714000000-add-mcp-to-search-queries-source.js`` failed on
``enum_search_queries_source``.

Extensions are excluded from the realignment (they are managed by the
``postgres`` charm).  The hook is idempotent, silent when there is no
drift, and blocks the deployment (``exit 1``) if any drift remains, so
the problem surfaces at deploy time instead of as a cryptic
crash-loop.
2026-09-16 23:41:42 +02:00
9 changed files with 421 additions and 6 deletions

View File

@@ -1,17 +1,66 @@
# -*- ispell-local-dictionary: "english" -*-
* Info
* Info
From: Carbone https://hub.docker.com/r/carbone/carbone-ee#running-carbone-community-edition-forever-free
## Usage :
Upstream documentation:
- On-premise configuration: https://carbone.io/documentation/developer/on-premise-installation/configuration.html
- Template management: https://carbone.io/documentation/developer/on-premise-installation/template-management.html
- Studio Web Component: https://carbone.io/documentation/developer/embedding/studio-web-component.html
* What this charm enables
By default Carbone starts in the *stateless* mode: it only renders the
template sent with each request, exposes no Studio and keeps no template
metadata. The =init= hook turns it into the *stateful* service, which is
what the charm is meant to provide:
- =CARBONE_STUDIO=true= serves the Studio web interface and the
=/carbone-studio.js= Web Component.
- =CARBONE_TEMPLATE_MANAGEMENT=true= enables stable 64-bit template IDs,
versioning, the =GET/POST/PATCH /template= endpoints and the
=embedded-versioning= Studio mode.
- =CARBONE_BIND=0.0.0.0= makes Carbone listen on all interfaces so the
=web-proxy= can reach it (the upstream default is =127.0.0.1=).
* Usage
ex :
```
#+begin_src yaml
carbone:
relations:
web-proxy:
frontend:
domain: carbone.dev1.elabore.coop
```
#+end_src
* Persistence
| Path | Resource type | Content |
|-----------------+------------------+------------------------------------------------------|
| =/app/template= | =data-resources= | Templates and the metadata store (metadata.db) |
| =/app/config= | =config-resources= | Optional license, authentication public key (key.pub) |
Templates and their versioning metadata both live under =/app/template=,
so they survive redeploys and are covered by the =backup= relation.
* Enterprise license
Carbone runs fine without a license (Community Edition, forever free).
Advanced features (dynamic images/colors, barcodes, charts, HTML
aggregation, PDF operations, ...) require an Enterprise license.
To enable it, drop your =*.carbone-license= file into the config store
(=/app/config= on the host side, i.e.
=$SERVICE_CONFIGSTORE/app/config=) and redeploy the service.
* API authentication
Authentication is *disabled by default*: the API is reachable as soon as
the service is up, which is the expected setup behind the =web-proxy=.
If you ever need an API key (JWT), enable =CARBONE_AUTHENTICATION=true=,
generate an EC keypair with =generate-keys=, expose the public key as
=/app/config/key.pub= and generate a token with =generate-token=. See
the upstream "Deploy with Docker" documentation for the exact commands.

47
carbone/hooks/init Executable file
View File

@@ -0,0 +1,47 @@
#!/bin/bash
## Init is run on host
## For now it is run every time the script is launched, but
## it should be launched only once after build.
## Accessible variables are:
## - SERVICE_NAME Name of current service
## - DOCKER_BASE_IMAGE Base image from which this service might be built if any
## - SERVICE_DATASTORE Location on host of the DATASTORE of this service
## - SERVICE_CONFIGSTORE Location on host of the CONFIGSTORE of this service
set -e
## The image runs as the unprivileged "carbone" user, while the datastore
## and configstore bind mounts are created by root. Both /app/template
## (templates + metadata.db) and /app/config (config.json, keys, license)
## must be writable by that user, otherwise the container crash-loops with
## "EACCES: permission denied, open '/app/config/config.json'".
uid_gid=($(docker_get_uid_gid "$SERVICE_NAME" "carbone" "carbone")) || exit 1
uid="${uid_gid[0]}"
gid="${uid_gid[1]}"
mkdir -p "$SERVICE_DATASTORE/app/template" "$SERVICE_CONFIGSTORE/app/config"
chown "$uid:$gid" "$SERVICE_DATASTORE/app/template" "$SERVICE_CONFIGSTORE/app/config"
## Carbone starts in stateless mode (document generation only) and binds
## to 127.0.0.1 by default (see `carbone webserver --help`, v5.15.1).
## Inside Docker it must:
## - listen on all interfaces so the web-proxy can reach it (CARBONE_BIND) ;
## - serve the Studio web interface and /carbone-studio.js (CARBONE_STUDIO) ;
## - enable stateful template management: stable 64-bit template IDs,
## versioning and the GET/POST/PATCH /template endpoints
## (CARBONE_TEMPLATE_MANAGEMENT).
##
## CARBONE_TEMPLATE_METADATA_FLUSH_CRON defaults to once a day; flushing
## every 5 minutes shrinks the window where a hard crash (SIGKILL, OOM)
## would lose template metadata. The graceful-shutdown flush remains the
## safety net for redeploys (see docker-compose.stop_grace_period).
init-config-add "
$SERVICE_NAME:
environment:
CARBONE_BIND: \"0.0.0.0\"
CARBONE_STUDIO: \"true\"
CARBONE_TEMPLATE_MANAGEMENT: \"true\"
CARBONE_TEMPLATE_METADATA_FLUSH_CRON: \"*/5 * * * *\"
"

View File

@@ -1,7 +1,20 @@
## From carbone/carbone-ee:full-5.4.2
docker-image: docker.0k.io/carbone-ee:5.4.2
## From carbone/carbone-ee:full-5.15.1
docker-image: docker.0k.io/carbone-ee:5.15.1
docker-compose:
## Carbone flushes its template metadata to disk on graceful shutdown.
## Docker's default 10s stop timeout is shorter than Carbone's shutdown
## sequence (exitQuietPeriod + flush), so a plain redeploy would SIGKILL
## it and lose the templates/versioning. Keep this above Carbone's
## exitDeadline (default 3 min).
stop_grace_period: 200s
data-resources:
## Templates and the template-management metadata store (metadata.db)
## both live here, so they survive redeploys and are covered by backup.
- /app/template
config-resources:
## Holds the optional Enterprise license (*.carbone-license) and the
## authentication public key (key.pub). See README.org.
- /app/config
uses:
web-proxy:

View File

@@ -0,0 +1,2 @@
carbone-test:
charm: carbone

View File

@@ -42,6 +42,44 @@ outline:
We monkey-patch odoo in order to make it work, be sure to use latest version in 14.0 of galicea openIDConnection module
* Database ownership alignment
The =pre_deploy= hook ensures that every object of the database
(tables, sequences, views, materialized views, standalone types,
functions, procedures) is owned by the application role before the
container starts and runs its migrations.
Historical provisioning or restores executed as the =postgres=
superuser leave objects owned by =postgres=, which makes any later
=ALTER= on these objects fail with "must be owner of ..." and puts
outline in a crash-loop at migration time. This was seen on
2026-09-11 when upgrading elabore.coop from 1.6.1 to 1.10.0:
migration =20260714000000-add-mcp-to-search-queries-source.js=
failed on =enum_search_queries_source=. The same drift was found
on every managed server (lokavaluto.fr, lagemme.org, moneko.org).
Extensions are excluded from the realignment (they are managed by
the =postgres= charm). The hook is idempotent and silent when
there is no drift, and blocks the deployment (=exit 1=) if the
realignment fails, so the problem is visible at deploy time instead
of as a cryptic crash-loop.
* Datastore ownership alignment
The =init= hook aligns the ownership of the service datastore with
the user the Outline container runs as. Since 1.10.0 the image runs
as the unprivileged =nodejs= user (older images, up to 1.6.1, ran as
=root=), while the datastore is provisioned by =root=. Without
realignment the application cannot write its =uploads=, =public= and
=avatars= buckets and every attachment upload fails with "Permission
denied writing to ... Check the host machine file system
permissions". This was seen on 2026-09-11 on elabore.coop after the
1.6.1 to 1.10.0 upgrade, on every existing datastore.
The hook reads the user from the image's =Config.User=, so it stays
version-agnostic: images running as =root= are left untouched, and
re-running the hook on an already aligned datastore is a no-op.
* Building a new image
We use the official image with an added patch due to 2 bugs:

View File

@@ -71,4 +71,42 @@ $SERVICE_NAME:
#DEBUG: \"http\"
"
## The datastore is bind-mounted into the container. Outline runs as
## an unprivileged user (image Config.User: root up to 1.6.1, "nodejs"
## since 1.10.0) and must write its uploads, public and avatars
## buckets. Provisioned by root, the datastore is not writable by
## that user and every upload fails with "Permission denied writing
## to ... Check the host machine file system permissions". Align the
## datastore ownership with the image user; skip images running as
## root. See README.org, "Datastore ownership alignment".
app_user=
if [ -n "$DOCKER_BASE_IMAGE" ]; then
app_user=$(docker image inspect "$DOCKER_BASE_IMAGE" \
--format '{{.Config.User}}') || exit 1
fi
case "$app_user" in
""|0|0:0|root)
## image runs as root: nothing to align
;;
*:*)
uid="${app_user%%:*}"
gid="${app_user#*:}"
;;
*)
uid_gid=($(docker_get_uid_gid "$SERVICE_NAME" "$app_user" "$app_user")) || exit 1
uid="${uid_gid[0]}"
gid="${uid_gid[1]}"
;;
esac
if [ -n "${uid:-}" ]; then
mkdir -p "$SERVICE_DATASTORE"
chown -R "$uid:$gid" "$SERVICE_DATASTORE" || {
err "Failed to align datastore ownership on '$uid:$gid'."
exit 1
}
info "Datastore ownership aligned on '$uid:$gid'."
fi

150
outline/hooks/pre_deploy Executable file
View File

@@ -0,0 +1,150 @@
#!/bin/bash
## Should be executable N time in a row with same result.
##
## Ensure the outline application role owns every object of its
## database before the container boots and runs its migrations.
##
## Historical provisioning or restores executed as the "postgres"
## superuser leave objects owned by "postgres", which makes any
## later ALTER on these objects fail with "must be owner of ..."
## and puts outline in a crash-loop at migration time. See
## README.org, section "Database ownership alignment".
. lib/common
set -e
relation="postgres-database"
db_role=$(outline:named-relation-get "$relation" user) || {
err "Couldn't get ${WHITE}user${NORMAL} value" \
"in ${DARKCYAN}$relation${NORMAL} relation's data."
exit 1
}
## List every object of schema "public" not owned by the application
## role: tables, sequences, views, materialized views, standalone
## types and functions. Extensions are excluded (managed by the
## postgres charm).
audit_query="SET app.role = '$db_role';
SELECT obj FROM (
SELECT CASE c.relkind
WHEN 'S' THEN 'sequence '
WHEN 'v' THEN 'view '
WHEN 'm' THEN 'matview '
ELSE 'table '
END || c.relname AS obj
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE n.nspname = 'public'
AND c.relkind IN ('r','p','S','v','m')
AND pg_get_userbyid(c.relowner) <> current_setting('app.role')
UNION ALL
SELECT 'type ' || t.typname AS obj
FROM pg_type t
JOIN pg_namespace n ON n.oid = t.typnamespace
WHERE n.nspname = 'public'
AND t.typtype IN ('e','d','c')
AND NOT EXISTS (SELECT 1 FROM pg_class c WHERE c.reltype = t.oid)
AND pg_get_userbyid(t.typowner) <> current_setting('app.role')
UNION ALL
SELECT CASE p.prokind
WHEN 'p' THEN 'procedure '
ELSE 'function '
END || p.proname AS obj
FROM pg_proc p
JOIN pg_namespace n ON n.oid = p.pronamespace
WHERE n.nspname = 'public'
AND p.prokind IN ('f','p')
AND pg_get_userbyid(p.proowner) <> current_setting('app.role')
) drift
ORDER BY obj;"
drift=$(sql < <(e "$audit_query")) || {
err "Failed to audit database ownership for ${WHITE}$db_role${NORMAL}."
exit 1
}
if [ -z "$drift" ]; then
## fast-path: nothing to do
exit 0
fi
info "Found database objects not owned by '${db_role}', reassigning ownership:"
e "$drift" | prefix " ${GRAY}|${NORMAL} " >&2
dbname=$(outline:named-relation-get "$relation" dbname) || {
err "Couldn't get ${WHITE}dbname${NORMAL} value" \
"in ${DARKCYAN}$relation${NORMAL} relation's data."
exit 1
}
## Reassign ownership of every drifted object, then the database
## itself. Role name is passed through a session GUC and quoted
## with format('%I') in every generated statement.
repair_query="SET app.role = '$db_role';
DO \$\$
DECLARE
r record;
app_role text := current_setting('app.role');
BEGIN
FOR r IN
SELECT CASE c.relkind
WHEN 'S' THEN format('ALTER SEQUENCE public.%I OWNER TO %I', c.relname, app_role)
WHEN 'v' THEN format('ALTER VIEW public.%I OWNER TO %I', c.relname, app_role)
WHEN 'm' THEN format('ALTER MATERIALIZED VIEW public.%I OWNER TO %I', c.relname, app_role)
ELSE format('ALTER TABLE public.%I OWNER TO %I', c.relname, app_role)
END AS stmt
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE n.nspname = 'public'
AND c.relkind IN ('r','p','S','v','m')
AND pg_get_userbyid(c.relowner) <> app_role
UNION ALL
SELECT format('ALTER TYPE public.%I OWNER TO %I', t.typname, app_role)
FROM pg_type t
JOIN pg_namespace n ON n.oid = t.typnamespace
WHERE n.nspname = 'public'
AND t.typtype IN ('e','d','c')
AND NOT EXISTS (SELECT 1 FROM pg_class c WHERE c.reltype = t.oid)
AND pg_get_userbyid(t.typowner) <> app_role
UNION ALL
SELECT CASE p.prokind
WHEN 'p' THEN format('ALTER PROCEDURE public.%I(%s) OWNER TO %I',
p.proname, pg_get_function_identity_arguments(p.oid), app_role)
ELSE format('ALTER FUNCTION public.%I(%s) OWNER TO %I',
p.proname, pg_get_function_identity_arguments(p.oid), app_role)
END AS stmt
FROM pg_proc p
JOIN pg_namespace n ON n.oid = p.pronamespace
WHERE n.nspname = 'public'
AND p.prokind IN ('f','p')
AND pg_get_userbyid(p.proowner) <> app_role
LOOP
EXECUTE r.stmt;
END LOOP;
END \$\$;
ALTER DATABASE \"$dbname\" OWNER TO \"$db_role\";"
sql < <(e "$repair_query") || {
err "Failed to reassign ownership to '${db_role}'."
exit 1
}
## Fail-hard: verify the realignment actually worked.
remaining_drift=$(sql < <(e "$audit_query")) || {
err "Failed to re-audit database ownership for ${WHITE}$db_role${NORMAL}."
exit 1
}
if [ -n "$remaining_drift" ]; then
err "Some database objects are still not owned by '${db_role}':"
e "$remaining_drift" | prefix " ${GRAY}|${NORMAL} " >&2
err "Deployment halted. Fix the ownership manually and retry, e.g.:"
err " docker exec <postgres-container> psql -U postgres -d \"$dbname\" \\" >&2
err " -c 'ALTER TYPE public.<name> OWNER TO \"$db_role\"'" >&2
exit 1
fi
info "Database ownership aligned on '${db_role}'."

65
outline/lib/common Normal file
View File

@@ -0,0 +1,65 @@
# -*- mode: shell-script -*-
##
## Database access helpers (from cyclos/immich pattern in 0k-charms)
##
## Get target service name for a named relation
outline:relation-get-target-service() {
local relation="$1" ts
if ! read-0 ts _ _ < <(get_service_relation "$SERVICE_NAME" "$relation"); then
err "Couldn't find relation ${DARKCYAN}$relation${NORMAL}."
return 1
fi
e "$ts"
}
## Get the raw data of a named relation
outline:relation-get-config() {
local relation="$1" ts relation_dir
ts=$(outline:relation-get-target-service "$relation") || return 1
relation_dir=$(get_relation_data_dir "$SERVICE_NAME" "$ts" "$relation") || return 1
cat "${relation_dir}/data"
}
## Get a key from the relation data
outline:named-relation-get() {
local relation="$1" key="$2" config
config=$(outline:relation-get-config "$relation") || return 1
e "$config" | shyaml get-value "$key" || {
err "Couldn't get ${WHITE}$key${NORMAL} value" \
"in ${DARKCYAN}$relation${NORMAL} relation's data."
return 1
}
}
## Run SQL as the postgres superuser on the database related to this
## service through the "postgres-database" relation.
## Usage: sql < <(echo "SELECT ...")
## echo "SELECT ..." | sql
sql() {
(
local dbname ts target_charm target_charm_path
dbname="$(outline:named-relation-get "postgres-database" dbname)" || exit 1
ts=$(outline:relation-get-target-service "postgres-database") || exit 1
export SERVICE_NAME="$ts"
export SERVICE_DATASTORE="$DATASTORE/$SERVICE_NAME"
DOCKER_BASE_IMAGE=$(service_ensure_image_ready "$SERVICE_NAME") || exit 1
export DOCKER_BASE_IMAGE
target_charm=$(get_service_charm "$ts") || exit 1
target_charm_path=$(charm.get_dir "$target_charm") || exit 1
set +e
. "$target_charm_path/lib/common"
set -e
ensure_db_docker_running
ddb -d "$dbname" -v ON_ERROR_STOP=1
)
}

View File

@@ -0,0 +1,13 @@
outline:
options:
sender-email: outline@example.com
oidc-client-id: test-client
oidc-client-secret: test-secret
oidc-auth-uri: https://example.com/auth
oidc-token-uri: https://example.com/token
oidc-user-info-uri: https://example.com/userinfo
oidc-logout-uri: https://example.com/logout
smtp-stub:
options:
host: smtp.example.com