The purge ran one correlated ``NOT EXISTS`` scan of ``ir_ui_view`` per
candidate attachment (``ILIKE '%web/image/<id>%'`` on ``arch_db::text``),
taking hours on large databases. It now scans ``ir_ui_view`` once with
``regexp_matches``, builds a deduplicated set of referenced ids, and
deletes orphans via hash lookups.
This also fixes substring false positives: ``/web/image/12`` was kept
because it matched ``/web/image/123``. Integer comparison (``ref_id =
a.id``) now requires an exact match.
The table ``theme_ir_ui_view`` only exists when the ``theme`` (website)
module was installed in the source database. If it was never installed,
the cleanup block crashes with "relation does not exist". Add a
``to_regclass`` guard to skip silently instead.
Delete image attachments linked to deleted ``ir.ui.view`` records only when
no other view's HTML references them via ``/web/image/<id>`` or
``/web/content/<id>``. This avoids the aggressive approach of deleting all
``res_id = 0`` attachments, which can break website images that survived
their parent view's deletion but are still embedded in live pages.
The ``NOT EXISTS`` subquery scans all ``arch_db`` columns and is expensive
but runs once at migration time.
Add ``lib/python/file_gc.py``, a minimal Odoo-shell script that calls
``ir.attachment._gc_file_store_unsafe()`` to remove filestore files
no longer referenced by any ``ir_attachment`` record.
Invoke it in ``finalize_db.sh`` after the SQL attachment purges so
that the deleted CSS/JS asset bundles, resized partner thumbnails,
and orphaned ``ir_attachment`` rows also have their backing files
cleaned from disk.
Delete the four thumbnail sizes (``image_128``, ``image_256``,
``image_512``, ``image_1024``) from ``ir_attachment`` for
``res.partner`` records. Odoo 13+ regenerates all thumbnails
automatically from ``image_1920`` on the next partner write.
Also delete attachments whose ``res_model`` is ``NULL`` (orphaned
``factur-x.xml``, SCSS, ICS files) — these have no associated record
and consume ~48 MB of filestore space.
Truncate the ``ir_logging`` table which accumulates Python logger
messages (63 MB of ``INFO``-level ``ir.actions.server`` traces) with
no business value after migration.
``account_factoring`` (depends on ``account_payment_partner`` +
``account_payment_mode``) is replaced by ``account_factoring_oca``
(depends on ``account_payment_base_oca`` +
``account_payment_base_oca_sale``) in v18. Both modules share rigorously
identical stored fields on ``account_journal``, ``account_move`` and
``res_partner``, so a rename (not an uninstall) preserves the 4910
invoice factor statuses, 311 partner credit limits and journal factor
config; OpenUpgrade then treats ``account_factoring_oca`` as already
installed and adopts the existing columns via ORM ``_auto_init``.
Redirects dependencies pointing to the old name, drops the obsolete
``account_payment_partner``/``account_payment_mode`` dependencies and
inserts the new OCA equivalents. Must run AFTER the bank-payment
renaming block which creates ``account_payment_base_oca`` in the DB.
The orphaned ``base.module_account_factoring`` xmlid is renamed so
OpenUpgrade can rediscover the physical ``account_factoring`` addon dir
without conflicting on ``(base, module_account_factoring)``.
``contract_payment_mode`` v18 still depends on ``account_payment_partner``
(removed by the bank-payment renaming) and uses the obsolete
``account.payment.mode`` model; no v18 replacement provides contract
integration, and only 2 contracts carried a ``payment_mode_id``.
Must run BEFORE the bank-payment renaming and BEFORE OpenUpgrade's
``button_upgrade()`` which would otherwise crash on the missing
dependency.
The module is DELETED (module row + ``ir_model_data`` + dependencies)
rather than marked 'to remove', because Odoo's "Transient module states
were reset" in ``load_modules()`` converts 'to remove' -> 'installed',
making ``button_upgrade()`` re-parse the missing
``account_payment_partner`` dependency and abort the registry load. The
orphaned ``base.module_contract_payment_mode`` xmlid is renamed so
``update_list()`` rediscovers the physical addon as a fresh uninstalled
module that ``button_upgrade()`` never touches.
The migrated 16.0 database still holds ``theme_ir_ui_view`` rows whose
``inherit_id`` points to ``ir.ui.view`` ids removed by the 17.0 asset
bundle rework (``web._assets_utils``, ``website.assets_editor``,
``website._assets_frontend_helpers`` replaced by the ``ir.asset`` model).
When ``website._theme_load`` runs during the registry load,
``ThemeView._convert_to_base_model`` accesses ``inherit.website_id`` on
the ghost record and raises ``MissingError``, aborting the whole
registry build.
Delete the ``ir_model_data`` + ``theme_ir_ui_view`` rows whose
``inherit_id`` targets a missing ``ir.ui.view``. Generic (covers any
theme), idempotent, scoped to ``theme_ir_ui_view`` only; no
``ir.ui.view`` copies exist for the affected templates.
The core script ``l10n_fr_account/migrations/2.1/end-migrate_update_taxes.py``
calls ``account.chart.template.try_loading('fr', company)``. During
``_post_load_data``, Odoo 18 resolves the French template's default accounts via
``self.ref()`` WITHOUT ``raise_if_not_found=False`` (chart_template.py:711/714
and the ``_get_property_accounts`` loop at :746). If any expected
``account.<company>_<key>`` xml-id is missing, it raises
``ValueError: External ID not found`` and aborts the registry load.
This happens when a database customized its chart of accounts: the generic PCG
account was deleted (e.g. 607000) or recreated without its xml-id (e.g. 707000).
Add a generic ORM ``post-migration`` script that, for every company with a
``chart_template``, reads the expected account xml-ids from the template API
(``_get_property_accounts`` / ``_get_chart_template_data`` /
``_get_account_account``) and recreates any missing ``account.<company>_<key>``
``ir.model.data`` pointing to the best matching account: exact code, else the
company account sharing the longest common code prefix, else a non-blocking
warning. It runs at stage ``post`` (before the native ``end`` script) and only
creates missing xml-ids — no account is modified. Idempotent, generic across
all migrated FR databases.
Remove the previous hardcoded 607/707 SQL block from
``versions/18.0/pre_upgrade.sh`` now superseded by this script.
``project_list`` (OCA) is not ported to 17.0 and is merged into core: in 17.0
the core ``project`` module natively adds the kanban/tree views to the
"Projects" actions (``act_window_id`` 149 & 760), and 17.0 introduces the
constraint ``act_window_view_unique_mode_per_action = unique(act_window_id,
view_mode)``.
When core reloads ``project_project_views.xml`` it INSERTs its own
``(act_window_id, view_mode='kanban')`` rows, colliding with the rows still
owned by ``project_list`` and aborting the registry load with a
``UniqueViolation`` on ``act_window_view_unique_mode_per_action``.
Delete ``project_list``'s ``ir.actions.act_window.view`` records (and their
``ir_model_data``) in ``versions/17.0/pre_upgrade.sh`` so core can recreate its
canonical rows. Verified against the 17.0 OpenUpgrade scripts: none reference
``project_list`` xml-ids nor remap these records, so this is safe. Scoped to
``ir.actions.act_window.view`` only, idempotent.
Custom migration scripts for a core module used to require copying the
whole module into ``versions/18.0/addons/`` so its ``migrations/`` folder
would be picked up. Because the migration container mounts that copy with
priority (``--addons-path=mig,addons``) and ``get_module_path`` returns the
first match, the copy fully shadowed the core module — dragging along its
entire data/models just to carry one script.
Instead, inject the scripts through a dedicated OpenUpgrade ``upgrade_path``:
- New ``versions/18.0/scripts/l10n_fr_account/18.0.2.2/`` holding
``pre-migration-0k-fix-orphan-expressions.py`` (converted to the
``@openupgrade.migrate()`` / ``migrate(env, version)`` convention) and
``post-migration-0k-vat-box25.py`` + ``noupdate_changes.xml`` that reapply
the box 25 VAT-report customization via ``openupgrade.load_data``.
- ``config/compose.yml``: mount it as ``/opt/odoo/auto/upgrade`` for ou18.
- ``versions/18.0/upgrade.sh``: pass
``--upgrade-path=/opt/odoo/auto/upgrade,/opt/odoo/auto/addons/openupgrade_scripts/scripts``.
Odoo appends every path to ``odoo.upgrade.__path__`` and concatenates the
per-version scripts, so our scripts run IN ADDITION to the native ones;
the native path is listed explicitly because ``--upgrade-path`` overrides
``config['upgrade_path']`` which ``openupgrade_framework`` only sets when empty.
Files are named with a ``-0k-`` marker so they never collide with the native
``pre-migration.py`` / ``post-migration.py``. This removes the need for the
shadowing module copy entirely.
Also ignore ``__pycache__``/``*.pyc``.
Some target versions add a ``UNIQUE`` constraint on a field that allowed
duplicates in the source version (e.g. ``utm.source.name`` in 16.0). The
unique index creation then aborts the whole registry load with a
``psycopg2.errors.UniqueViolation``.
Declare such constraints per version in ``known_changes.yaml`` under the
``new_unique_constraints`` key (only ``model`` and ``fields`` needed).
During ``prepare_db.sh``, ``dedup_unique_constraints.py`` aggregates the
declarations of every traversed version and, on the source database,
renames duplicates by suffixing `` [<id>]`` while preserving foreign keys.
The actual PostgreSQL column type is used (not the ORM ``translate``
attribute) so both ``varchar`` and already-converted ``jsonb`` columns are
handled: ``jsonb`` duplicates are compared and renamed on every language
key (indexed ``en_US`` value). Renamed records are reported in a
timestamped JSON file and summarized at the end of ``migration.log``.
OpenUpgrade migrations can delete and recreate ``ir_actions`` records with
new ids, leaving ``ir_filters.action_id`` (custom user filters) pointing to
a now-missing action. Opening the custom filters list then fails with
"Record does not exist or has been deleted (ir.actions.actions(<id>,))"
when Odoo resolves the ``action_id`` many2one ``display_name``.
Detach such filters (set ``action_id = NULL``) as a final, version-agnostic
cleanup step: the filter stays usable, only the broken action link is
dropped. Placed in ``finalize_db.sh`` so the check runs whatever the target
version, with an integrated re-count that warns if orphans remain.
The pre-shell state reset in ``scripts/finalize_db.sh`` forced both
``to upgrade`` and ``to install`` modules to ``installed``. Forcing
``to install`` to ``installed`` makes Odoo skip their install scripts
entirely (tables, ``noupdate`` data, init hooks never run), leaving
ghost "installed but empty" modules that the final ``-u all`` cannot
recover.
Restrict the reset to ``to upgrade`` only, whose real update is honored
by the controlled ``-u all`` at the end of the script. Also drop the
dead ``NOT IN (... state = 'uninstalled')`` sub-query: ``name`` is
unique so a module never has two states, making the filter a no-op.
A commented-out ``SELECT`` is added to trace pending-upgrade modules if
the trailing ``-u all`` is ever removed.
The 0k dev-pack's compose script doesn't handle absolute paths correctly.
It passes HOST_COMPOSE_YML_FILE to the container, which tries to open
it directly instead of using the mounted path.
Add run_compose() wrapper that changes to PROJECT_ROOT before calling
compose with a relative path, ensuring consistent behavior regardless
of the current working directory.
Add complete documentation in French including:
- Table of contents for easy navigation
- Prerequisites section (0k dev-pack, Docker, rsync, sudo)
- Project structure explanation with directory tree
- Detailed workflow explanation with step-by-step breakdown
- ASCII diagram showing migration flow
- Usage examples with command-line syntax
- Customization guide for version-specific scripts
- Troubleshooting section with common issues and solutions
Replace the previous minimal README that only contained basic
installation and configuration notes.
Replace cd into version directories with absolute path execution:
Before:
cd "${version}.0"
./pre_upgrade.sh
./upgrade.sh
./post_upgrade.sh
cd ..
After:
"${SCRIPT_DIR}/${version}.0/pre_upgrade.sh"
"${SCRIPT_DIR}/${version}.0/upgrade.sh"
"${SCRIPT_DIR}/${version}.0/post_upgrade.sh"
Benefits:
- No working directory state to track
- More robust: script works regardless of where it's called from
- Easier debugging: no need to remember current directory
- Avoids potential issues if a subscript changes directory
Replace $FINALE_DB_MODEL_NAME with $FINALE_DB_NAME in the call to
prepare_db.sh.
FINALE_DB_MODEL_NAME was never defined anywhere in the codebase,
causing the script to fail immediately with 'set -u' (unbound variable
error). The intended variable is FINALE_DB_NAME which contains the
target database name (e.g., 'ou16').
Add confirm_or_exit() function to lib/common.sh to eliminate duplicated
confirmation dialog code in prepare_db.sh.
Before: Two 10-line case statements with identical logic
After: Two single-line function calls
The function provides consistent behavior:
- Displays the question with Y/N options
- Returns 0 on Y/y (continue execution)
- Exits with error on any other input
This follows DRY principle and ensures all confirmation prompts
behave identically across the codebase.
Replace mkdir + rm -rf + cp -a sequence with rsync --delete:
Before (3 commands):
sudo mkdir -p "$dst_path"
sudo rm -rf "$dst_path"
sudo cp -a "$src_path" "$dst_path"
After (2 commands):
sudo mkdir -p "$(dirname "$dst_path")"
sudo rsync -a --delete "${src_path}/" "${dst_path}/"
Benefits:
- Incremental copy: only transfers changed files on re-run
- Atomic delete + copy: --delete removes extra files in destination
- Preserves all attributes like cp -a
- Faster for large filestores when re-running migration
Added rsync to required commands check.
Merge three separate SQL queries into one for better performance:
- 1 database connection instead of 3
- Atomic execution of all cleanup operations
Added detailed SQL comments explaining each operation:
- DROP SEQUENCE: Why stale sequences prevent Odoo startup
- UPDATE ir_ui_view: Why website templates are reset except pages
- DELETE ir_attachment: Why compiled assets must be purged
Also changed DROP SEQUENCE to DROP SEQUENCE IF EXISTS to avoid
errors if sequences don't exist.
Replace manual loop building version array with seq + readarray:
Before (4 lines):
declare -a versions
nb_migrations=$((FINAL_VERSION - ORIGIN_VERSION))
for ((i = 0; i < nb_migrations; i++)); do
versions[i]=$((ORIGIN_VERSION + 1 + i))
done
After (1 line):
readarray -t versions < <(seq $((ORIGIN_VERSION + 1)) "$FINAL_VERSION")
The seq command is purpose-built for generating number sequences,
making the intent clearer and the code more concise.
Replace double grep pattern with readarray for cleaner container detection:
- Single grep call instead of two
- Native bash array instead of string manipulation
- Array length check instead of grep -c
- Proper formatting when listing multiple containers
The readarray approach is more idiomatic and avoids edge cases with
empty strings and newline handling.
The SQL query already filters on module_origin.state = 'installed',
so the second query to get installed addons and the grep intersection
were completely redundant.
Before: 2 SQL queries + grep + 3 temp files
After: 1 SQL query + variable
This simplifies the code and reduces database round-trips.
Add check_required_commands() function to verify that all required
external tools are available before the script begins execution:
- docker: Container runtime
- compose: Docker compose wrapper (0k-scripts)
- sudo: Required for filestore operations
Benefits:
- Fails fast with a clear error message listing missing commands
- Prevents cryptic 'command not found' errors mid-execution
- Documents script dependencies explicitly
- Called immediately after argument validation in upgrade.sh
Apply consistent naming conventions throughout upgrade.sh:
- UPPERCASE + readonly for script-level constants (immutable values)
- lowercase for temporary/local variables within the script flow
Constants marked readonly:
- ORIGIN_VERSION, FINAL_VERSION, ORIGIN_DB_NAME, ORIGIN_SERVICE_NAME
- COPY_DB_NAME, FINALE_DB_NAME, FINALE_SERVICE_NAME
- POSTGRES_SERVICE_NAME
Local variables renamed to lowercase:
- postgres_containers, postgres_count (detection phase)
- db_exists, filestore_path (validation phase)
This convention makes it immediately clear which variables are
configuration constants vs runtime values, and prevents accidental
modification of critical values.
Replace hardcoded temporary file paths with mktemp -d for secure
temporary directory creation, and add a trap to automatically clean
up on script exit (success, failure, or interruption).
Benefits:
- Automatic cleanup even on Ctrl+C or script errors
- No leftover temporary files in the working directory
- Secure temporary directory creation (proper permissions)
- Files isolated in dedicated temp directory
Added '|| true' to grep command since it returns exit code 1 when
no matches are found, which would trigger set -e otherwise.
Convert the SQL_404_ADDONS_LIST query from a quoted string to a heredoc
without quotes (<<EOF instead of <<'EOF') to make variable expansion
explicit and consistent with other SQL blocks in the codebase.
Key difference between heredoc variants:
- <<'EOF': Literal content, no variable expansion (use for static SQL)
- <<EOF: Variables like ${FINALE_DB_NAME} are expanded (use when needed)
Also improved SQL formatting for better readability.
Add logging functions to lib/common.sh for consistent output formatting:
- log_info(): Standard informational messages with [INFO] prefix
- log_warn(): Warning messages to stderr with [WARN] prefix
- log_error(): Error messages to stderr with [ERROR] prefix
- log_step(): Section headers with visual separators
Update upgrade.sh to use these functions throughout, replacing ad-hoc
echo statements. This provides:
- Consistent visual formatting across all scripts
- Clear distinction between info, warnings and errors
- Errors properly sent to stderr
- Easier log parsing and filtering
Also removed redundant '|| exit 1' statements since set -e handles
command failures automatically.
Extract shared utility functions into a dedicated library file:
- query_postgres_container: Execute SQL queries in postgres container
- copy_database: Copy database using pgm
- copy_filestore: Copy Odoo filestore directory
- exec_python_script_in_odoo_shell: Run Python scripts in Odoo shell
Benefits:
- Single source of truth for utility functions
- Easier maintenance and testing
- Consistent behavior across all scripts
- Reduced code duplication
Also introduces readonly constants DATASTORE_PATH and FILESTORE_SUBPATH
to avoid hardcoded paths scattered throughout the codebase.
Replace $DB_CONTAINER_NAME with $POSTGRES_SERVICE_NAME which is the
correct variable exported from the parent script (upgrade.sh).
DB_CONTAINER_NAME was never defined, causing the script to fail
immediately with 'set -u' enabled (unbound variable error). The
intended variable is POSTGRES_SERVICE_NAME which contains the name
of the PostgreSQL container detected at runtime.
Replace 'return 1' with 'exit 1' in prepare_db.sh.
The 'return' statement is only valid inside functions. When used at
the script's top level, it behaves unpredictably - in some shells it
exits the script, in others it's an error. Using 'exit 1' explicitly
terminates the script with an error status, which is the intended
behavior when the PostgreSQL container is not running.
Replace single bracket [ ] with double bracket [[ ]] for all test
conditionals in the main scripts.
Benefits of [[ over [:
- No need to quote variables (though we still do for consistency)
- Supports regex matching with =~
- Supports pattern matching with == and !=
- && and || work inside [[ ]] without escaping
- More predictable behavior with empty strings
- Is a bash keyword, not an external command
Note: posbox scripts are left unchanged as they appear to be
third-party code imported into the repository.
