3. Reinitialization, upgrade, and removal

By the end of this chapter: You create new data or restore the previous data, upgrade to a new version or roll back, and remove GritDB.

• Run the commands in this chapter in the directory where you extracted the installation package. The installation script is not in the installation directory.

3.1 Reinitialization (--reinit-data)

Reinitialization creates new data. It does not delete the existing data. It keeps the data under a new name.

1. For an analytics setup, first remove the subscription (steps 1 to 3 in 6.4.4). If you do not remove it, the slot on the operational database keeps holding WAL.

2. Stop GritDB.

   ~/gritdb1/bin/gritdb-stop
   

   To check, run the stop check in QuickStart 4.5. It shows no output. If you run the installation script while GritDB is running, it usually stops at ▶ 2.1. It cannot overwrite files that are in use (Text file busy). In that case, stop GritDB, then run the same command again. If a PostgreSQL from another path is using this data, the script refuses with A PostgreSQL instance is running on this data directory (PID: <pid>).

3. If you relocated the data (2.4), check that the service account can write to the parent directory of the real path. If the data volume is a mount point, also check the free space of the parent file system (warning below).

   test -w "$(dirname "$(readlink -f ~/gritdb1/datafiles)")" && echo OK
   

   • It is normal if OK appears. If you relocated with --pgdata, run the same check on ~/gritdb1/datafiles/pgdata too.
   • Otherwise, root hands that directory over to the service account, or you reinitialize as root with --user (2.3). If the script cannot rename the data, it stops with Rename failed; nothing was lost, rolled back. The data stays as it is.

4. Reinitialize. If you do not use the standard path, also pass the --prefix from the first installation. Also pass --user as root, and --no-config if you installed with it (2.3, 2.6).

   cd <extracted directory of the installed version>
   jq -r .product_version manifest.json ~/gritdb1/share/gritdb/manifest.json
   

   If the two lines are the same, this is the installed version, so run the command below. If they differ, do not run it. In that case, find the installation package of the installed version, or bring it in again (QuickStart 2.3).

   ./install_gritdb.sh --reinit-data
   

   Check that the screen shows the line Previous data preserved with suffix .old-<timestamp> and that step ▶ 3.1 is OK. The first line [INFO] Upgrading existing 1.x install in place: <installation path> also appears for a reinitialization. This is normal.

   • Do not enter the directory with the ~/pkg/gritdb-*/ pattern. After an upgrade (3.5), there are two extracted directories, so cd fails and the next line runs in the current directory.
   • If you relocated with --pgdata, run the installation script once more before you start GritDB. Use the same --pgdata value, without --reinit-data. This is needed because reinitialization moves the datafiles/pgdata link together with the old kept copy. In this run, ▶ 3.1 shows SKIP (existing data kept; ...). To check, run readlink ~/gritdb1/datafiles/pgdata. It returns <pgdata real path>.

5. Add the operator settings again (warning below).

6. Start GritDB and check it with QuickStart 4.2 and 4.3. Next, set up security again. Revoke function execute privileges, starting with template1 and postgres (11.5). The new datafiles real path starts as 775, so run chmod 700 (11.7). After that, re-create the databases, tables, and registrations (chapter 6), and check with the checklist in 11.10. An analytics setup receives its data from the beginning (6.4.1).

   • In the new data, function execute privileges follow the installation default, so all accounts have them (11.5). If you revoke them in template1 first, databases that you create after that also start with the privileges revoked.

• The old data is only renamed, to <datafiles real path>.old-<YYYYmmddHHMMSS> (with the standard path, ~/gritdb1/datafiles.old-<timestamp>). If you relocated with --pgdata, <pgdata real path>.old-<timestamp> is also created. To restore it, see 3.2. To delete it, see 3.3.
• The port carries over.
• config/ (GritDB engine configuration) and log/ stay as they are.

⚠ The new data does not have the settings that the operator added

• Reason: pg_hba.conf, the lines outside the block in postgresql.conf, and ALTER SYSTEM values remain only in the renamed old directory. If you start GritDB as it is, authentication goes back to the installation default, trust (11.2).
• Instead: Before you start GritDB, compare pg_hba.conf with the old file and fix it (11.3). For postgresql.conf, move only the differences outside the managed block to the new file. The installation rewrote the block (5.2). For ALTER SYSTEM values, look at the old postgresql.auto.conf and add them again after the start (5.3). Check the values you added again with the check query in 5.3. If pending_restart is t, restart.

OLD=$(ls -d "$(readlink -f ~/gritdb1/datafiles)".old-* | tail -1)
P=$(readlink -f ~/gritdb1/datafiles/pgdata)
OLDP=$(ls -d "$P".old-* 2>/dev/null | tail -1)
OLDP=${OLDP:-$OLD/pgdata}
diff "$OLDP/pg_hba.conf" "$P/pg_hba.conf"
diff "$OLDP/postgresql.conf" "$P/postgresql.conf"
cat "$OLDP/postgresql.auto.conf"

• If you relocated with --pgdata, the commands above pick <pgdata real path>.old-<timestamp>/. In that case, run them after the extra run in step 4. The pgdata link inside the old kept copy points to the new data.

• ⚠ If the data volume is a mount point, reinitialization moves the old data to <mount point>.old-<timestamp> on the parent file system (for example, /). In that case, if there is not enough free space, that file system fills up. So check the free space first with df -h <parent directory>.

3.2 Restore previous data

Rename *.old-<timestamp> back to the original name. Restore it to the same real path (1.5).

1. Stop GritDB, then run the stop check in QuickStart 4.5.

2. Choose the kept copy to restore.

   D=$(readlink -f ~/gritdb1/datafiles)
   ls -d "$D".old-*
   

3. Keep the current data under a new name, and rename the kept copy back to the original name.

   mv "$D" "$D.cur-$(date +%Y%m%d%H%M%S)"
   mv "$D.old-<timestamp>" "$D"
   

   • If you relocated with --pgdata, restore <pgdata real path> the same way. Use the same <timestamp>. The datafiles.old-<timestamp>/pgdata link in the kept copy points to <pgdata real path>. So the old data fits together only when both places use the same <timestamp>.
   • If the data volume is a mount point, move the contents, not the directory itself. First move the current contents to another location, then move the contents of <mount point>.old-<timestamp>/ to <mount point>/. For free space on the parent file system, see 3.1.

4. If the normal stop marker exists, rename it. Then the next start runs recovery.

   mv ~/gritdb1/run/grit_admin_clean_shutdown ~/gritdb1/run/grit_admin_clean_shutdown.cur-$(date +%Y%m%d%H%M%S)
   

   An error that the file does not exist is normal. In that case, skip this step.

5. Start GritDB, then check with 8.5.

• In an analytics setup, changes on the operational database after the point of the restored data do not come again. So after the restore, take the tables again with 8.6 (warning in 8.4).
• Delete .cur-<timestamp> after the checks are complete. Follow the same rules as the secure deletion in 3.3.

3.3 Clean up previous data (--purge-old)

• What it deletes: <datafiles real path>.old-*, <pgdata real path>.old-* (--pgdata relocation), and run/grit_admin_clean_shutdown.old-*
• What it does not delete: .cur-* (3.2), .failed-* (3.7), and .before_restore_* (8.4). Delete these three yourself after the checks are complete.
• It deletes without asking for confirmation. If the same run also has --reinit-data, it deletes before the reinitialization.
• Even with only --purge-old, the installation script reinstalls the same version in place. So stop GritDB, and run the script in the extracted directory of the same installation package as the installed version. Pass the options from the first installation with it (--prefix, --no-config, and --user for root).

⚠ --purge-old deletes without confirmation, and you cannot undo it

• Reason: The old kept copy is the only data that can undo a reinitialization (3.2).
• Instead: First look at the list below, back up what you want to keep (8.2), and then run it.

1. Look at what will be deleted.

   D=$(readlink -f ~/gritdb1/datafiles)
   P=$(readlink -f ~/gritdb1/datafiles/pgdata)
   ls -d "$D".old-* "$P".old-* ~/gritdb1/run/grit_admin_clean_shutdown.old-* 2>/dev/null
   

2. Stop GritDB, then delete in the extracted directory of the installed version. Compare the versions first, as in 3.1 step 4.

   cd <extracted directory of the installed version>
   ./install_gritdb.sh --purge-old
   

   To check, run the command from step 1 again. It shows no output. The screen shows [INFO] Purging <path> for each deleted item, and ▶ 3.1 shows SKIP (existing data kept; ...). The data stays.

   • If you did not relocate with --pgdata, the old pgdata is inside $D.old-*. In that case, no separate $P.old-* is created.

• If you pass --reinit-data --purge-old together, the script first deletes the old kept copies. Then it keeps the current data as a new kept copy. So only one kept copy remains. ⚠ It deletes without confirmation.
• --purge-old is a regular deletion. For secure deletion, as in financial institutions, delete the targets above with the tool that your organization’s rules require (shred, and so on).

3.4 Pre-upgrade checks

• If the gritdb-* pattern in QuickStart 2.3 matches both versions, the new version is not extracted, and the old version is verified instead. That is why item 2 moves the old version’s files. Rollback (3.7) uses the copy from 3.4, so it does not use the old installation package.
• For an analytics setup, first check that the time from the stop in item 8 to the start in 3.5 step 6 is within “how long it can stay stopped” in 4.7. If it goes over, take the tables again from the start after the upgrade (8.6). In that case, include that time in the planned work time. For the copy time in ④, see 8.2.

① Registered tables, row counts, and status functions (do ① to ④ in the same shell, where $UB is set)

UB=<backup location>/gritdb_before_upgrade_$(date +%Y%m%d_%H%M%S)
mkdir -p "$UB"
~/gritdb1/bin/psql -p 5432 -d <db> -Atc "SELECT schema_name, table_name FROM get_gritsurfer_tables() ORDER BY 1, 2" > "$UB/tables_<db>.txt"
~/gritdb1/bin/psql -p 5432 -d <db> -Atc "SELECT count(*), sum(<numeric column>) FROM <s>.<t>" > "$UB/count_<s>.<t>.txt"
~/gritdb1/bin/psql -p 5432 -d <db> -c '\df get_gritsurfer_*' > "$UB/functions_<db>.txt"

• Run the first two lines only once. Run the third and fifth lines for each database, and the fourth line for each sample table. Samples follow the criterion in 7.10 step 1, and the file names are the same as in 8.2 step 2.
• <backup location> is a disk other than the data volume. Put the copy from ④ in this location too. The comparison after the upgrade (3.5 step 8) and the rollback (3.7) use these files.

② Operator settings

~/gritdb1/bin/psql -p 5432 -d postgres -Atc "SELECT name, setting, source, sourcefile FROM pg_settings WHERE source NOT IN ('default', 'override') ORDER BY 1;" > "$UB/settings_before.txt"

• Tuning block values are rewritten with the new version’s defaults. The three memory values are calculated again from the server memory at that time (5.5). ALTER SYSTEM values remain (5.2 and 5.3). The column meanings are described in pg_settings.
• sourcefile is the file the value came from. With postgresql.conf, the value comes from the tuning block or lines outside the block. With postgresql.auto.conf, it comes from ALTER SYSTEM. Compare after the upgrade (3.5 step 7).

③ GritDB engine configuration keys — Find keys added in the new version. The installation script does not overwrite existing configuration files. If even one key is missing, GritDB does not start (5.6).

cd <newly extracted directory>
for f in etc/templates/grit_*.config; do
  diff <(grep -oE '^[a-z0-9_.]+' "$f" | sort -u) <(grep -oE '^[a-z0-9_.]+' ~/gritdb1/config/"$(basename "$f")" | sort -u)
done

• If there is no output, the keys are the same.
• Lines that start with < are keys added in the new version. After the upgrade, add the line from the new version’s template to that file (3.5 step 5).

④ Backup — After the stop in item 8, copy the whole installation directory (in the same shell as ①, where $UB is set). Rollback (3.7) uses this copy.

readlink -f ~/gritdb1/datafiles ~/gritdb1/datafiles/pgdata > "$UB/realpaths.txt"
tar --sparse -cf "$UB/gritdb1.tar" -C ~ gritdb1 ; echo "gritdb1.tar = $?"

• If the first character in the output of ls -ld ~/gritdb1/datafiles ~/gritdb1/datafiles/pgdata is l, it is a symlink. For a symlink, tar stores only the link, so store its real path separately.
• If datafiles is a symlink (--datafiles, 2.4), run tar --sparse -cf "$UB/datafiles.tar" -C "$(readlink -f ~/gritdb1/datafiles)" . ; echo "datafiles.tar = $?".
• If datafiles/pgdata is a symlink (--pgdata, 2.4), run tar --sparse -cf "$UB/pgdata.tar" -C "$(readlink -f ~/gritdb1/datafiles/pgdata)" . ; echo "pgdata.tar = $?".
• It is normal if every line shows = 0. If a line is not 0, you cannot roll back with that copy, so fix the problem and do it again.
• Do not compress with -z. Compression runs on one CPU, so the downtime grows. For time and space, see 8.2.

3.5 Patch and minor upgrade

• A patch is a new installation package of the same version (1.1). In a minor upgrade, the version goes up within the same major. Both install in place in the same installation directory.
• Data, the port, config/, lines outside the block in postgresql.conf, and ALTER SYSTEM values are kept. Among the lines outside the block, a setting that is also in the block is overridden by the block value (2.5).
• Installed files (bin/, lib/, share/ …) and the managed block are rewritten. The tuning values in the managed block become the new defaults.

1. Finish the checks in 3.4.

2. Confirm that GritDB is stopped (QuickStart 4.5). If it is running, see 3.1 step 2.

3. Install from the newly extracted directory. Pass the --prefix, --no-config, and (for root) --user options from the first installation. Leave out the relocation options, or pass the same values (2.4).

   cd <newly extracted directory>
   ./install_gritdb.sh
   

   • The screen shows Upgrading existing 1.x install in place: <installation directory>, and step ▶ 3.1 shows SKIP (existing data kept; ...). The last line is Install complete.

4. Confirm the installation.

   jq -r .product_version ~/gritdb1/share/gritdb/manifest.json
   

   It is normal if the new version number appears.

5. Add the new keys that you found in ③ of 3.4 to the matching files in ~/gritdb1/config/ (5.6).

6. Start GritDB and check it with QuickStart 4.2 and 4.3.

7. Compare the operator settings. $UB is the backup location in 3.4. If the shell changed, set it again with UB=<that location>.

   ~/gritdb1/bin/psql -p 5432 -d postgres -Atc "SELECT name, setting, source, sourcefile FROM pg_settings WHERE source NOT IN ('default', 'override') ORDER BY 1;" > "$UB/settings_after.txt"
   diff "$UB/settings_before.txt" "$UB/settings_after.txt"
   

   • Lines that differ are values that the new version changed (new defaults in the tuning block). Decide whether to keep them.
   • If you had changed that value in the tuning block, set it again with ALTER SYSTEM (5.3). For values that need a restart, see 5.4.

8. Compare as in 8.5 steps 2 to 5. Read <backup location>/<backup name> there as $UB.

9. Check the status functions in each database. The extension SQL is not updated in place, so the function signatures can differ from the new version.

   ~/gritdb1/bin/psql -p 5432 -d <db> -c '\df get_gritsurfer_*'
   

   • It is normal if the signatures match A.3 of the new version.
   • If they differ, do not use those functions, and contact support (chapter 0).
   • ALTER EXTENSION gritsurfer UPDATE does not change them. The version number in \dx gritsurfer stays the same even when the version goes up. So do not run this command.

10. If steps 8 and 9 are correct, reopen the applications, reports, BI tools, CDC tools, and ETL tools. Decide before that whether to roll back (3.7). If you roll back, writes that came in after you reopened are lost.

3.6 Major upgrade (for example, 1.x → 2.x)

• Install a version with a new major in a new path. The default path is ~/gritdb<major> (for example, ~/gritdb2). Installation in the same path is rejected with Prefix <path> holds major version <n>; this dist is major <m>. The old installation stays as it is.
• For installation in a new path, see QuickStart, chapter 3. Install with another port (--pgport, 2.1, 2.9). Two versions on one server overcommit memory (2.9). So keep both only during the move.
• For an analytics setup, install the new version in a new path and take the tables again from the operational database (6.4.1 and 8.6). In that case, you do not need to move the data. Meanwhile, the operational database has one more slot (6.4.3). After the tables are taken again, remove the old version’s subscription (6.4.4).
• For a single-database setup, move to the new version in the order schema → registration → data (8.3). Do not use PostgreSQL’s pg_upgrade (A.1). See chapter 3 of the Admin guide for that version, and contact support (chapter 0).

3.7 Rollback

If the upgrade fails, go back with the copy from 3.4.

⚠ Rollback takes you back to the time of the backup in 3.4

• Reason: Rollback restores the whole copy. In a single-database setup, writes that came in after you made the copy (after you reopened the applications in 3.5 step 10) are lost.
• Instead: If you already reopened the applications, contact support (chapter 0) before you roll back. For an analytics setup, follow step 5 below (take the tables again).

1. Stop GritDB, then run the stop check in QuickStart 4.5.

2. Keep the current installation directory under a new name. Get the real paths first. After the rename, you cannot find them through the links.

   D=$(readlink -f ~/gritdb1/datafiles)
   P=$(readlink -f ~/gritdb1/datafiles/pgdata)
   T=$(date +%Y%m%d%H%M%S)
   mv ~/gritdb1 ~/gritdb1.failed-$T
   

   • To check, run printf '%s\n' "$D" "$P" | diff - "$UB/realpaths.txt". It shows no output. $UB is the backup location in 3.4.
   • If datafiles was a symlink, keep $D too. In that case, run mv "$D" "$D.failed-$T", mkdir "$D", and chmod 700 "$D" in that order. For a mount point, move the contents (3.2).
   • If datafiles/pgdata was a symlink, keep $P the same way.

3. Restore the copy from 3.4 to the same real paths.

   tar -xf "$UB/gritdb1.tar" -C ~
   

   • If datafiles was a symlink, run tar -xf "$UB/datafiles.tar" -C "$D".
   • If datafiles/pgdata was a symlink, run tar -xf "$UB/pgdata.tar" -C "$P".
   • To check, run readlink -f ~/gritdb1/datafiles ~/gritdb1/datafiles/pgdata | diff - "$UB/realpaths.txt". It shows no output.

4. Start GritDB, then run jq -r .product_version ~/gritdb1/share/gritdb/manifest.json. It returns the old version number. Then check with 8.5. Read <backup location>/<backup name> there as $UB.

5. For an analytics setup, if you started the new version even once (3.5 step 6), take the tables again with 8.6. When GritDB starts, the subscription immediately starts to receive from the slot on the operational database. That slot does not move backward (warning in 8.4).

• Do not reinstall the old version’s installation package in the same path. An in-place installation leaves the files that exist only in the new version, so the files of the two versions mix. Restore the whole copy instead.

⚠ A downgrade that overwrites with a lower-version installation package is not supported

• Reason: The installation script does not compare version numbers, so it does not block a downgrade. If you overwrite, the files of the two versions mix.
• Instead: Go back with the copy from 3.4 (procedure above).

3.8 Removal and disposal

⚠ You cannot undo a removal

• Reason: If you delete the installation directory and the data volume, all data, configuration, and logs are gone.
• Instead: First decide whether to keep the data. To keep it, finish an offline backup (8.2) before you delete.

1. For an analytics setup, first remove the subscription (6.4.4). Do not leave a slot on the operational database.

2. Stop GritDB, then run the stop check in QuickStart 4.5.

3. To keep the data, back it up (8.2).

4. Write down the locations to delete. Before you delete the installation directory, get the instance ID and the real paths.

   ~/gritdb1/bin/grit_instance_id ~/gritdb1
   readlink -f ~/gritdb1/datafiles
   readlink -f ~/gritdb1/datafiles/pgdata
   cat /proc/sys/kernel/core_pattern
   

   • The second and third lines are the real paths of datafiles and pgdata. If you did not relocate, the third line is <second line>/pgdata, so there is nothing extra to delete.

5. Follow the fix in QuickStart GDB-SHUTDOWN-001 to delete the remaining shared memory. Delete only the segments that end with the instance ID from step 4.

6. Delete the installation directory, the relocated locations, and the kept copies (.old-*, and so on).

   rm -rf ~/gritdb1 ~/gritdb1.failed-*
   

   • For relocated locations, delete the two real paths from step 4 and everything next to them: .old-* (3.1), .cur-* (3.2), .failed-* (3.7), and .before_restore_* (8.4). rm -rf ~/gritdb1 deletes only the symlinks and leaves the data that the symlinks point to.
   • If a real path is a mount point (mountpoint -q <real path> && echo mount), rm -rf deletes only the contents and ends with Device or resource busy. In that case, root runs umount <real path>, deletes that line from /etc/fstab, and removes the empty directory with rmdir. For mount points, see 3.1 and 3.2.
   • If the parent directory of the installation path belongs to root (such as /opt; see 2.2 and 2.3), rm -rf by the service account deletes only the contents. It ends with Permission denied. In that case, root removes the empty directory with rmdir <installation path>.
   • Also delete the extracted installation package (~/pkg).

7. Clean up the other locations. Delete the core files (the core_pattern location from step 4), __tkidx_spill_*.tmp in the TMPDIR of the starting shell (1.4), the file for start at server boot (4.6), the lines in /etc/security/limits.conf (1.4), firewall rules (1.6), and /etc/logrotate.d/gritdb (9.4). Delete /etc/systemd/logind.conf.d/gritdb.conf (1.4 ⑤) only when no other service on the same server needs this setting. After you delete it, run systemctl restart systemd-logind. Under /etc, root does the cleanup.

8. Check the result. Run ls -d ~/gritdb1*. It returns an error that it does not exist. This pattern also matches .failed-* from 3.7. Run ls -d <datafiles real path>* <pgdata real path>* 2>/dev/null (values from step 4) and ls /dev/shm | grep "_<instance ID>$". They show no output.

9. Run userdel -r gritdb as root to remove the service account. The home directory is deleted with it.

• If disposal rules apply, as in financial institutions, do not delete the targets above with a regular deletion (rm). In that case, delete them with your organization’s secure deletion tool (shred, and so on) or by disposing of whole volumes. Logs, core files, and datafiles/query_text_archive.jsonl contain original SQL text and data values (11.1).
• The copies outside the installation directory are backups (8.2, <backup location>/gritdb_*) and upgrade copies (3.4, <backup location>/gritdb_before_upgrade_*). Decide whether to keep them. If you do not keep them, delete them under the same rules.
• userdel -r in step 9 is also a regular deletion. So under disposal rules, delete the copies in the home directory with the same tool before step 9. Targets: ~/.pgpass, ~/.psql_history (11.7), ~/gritdb_support_* (9.7, QuickStart 7.4), ~/gritdb_baseline_* (7.7), dumps left over from a move, and <t>.csv (8.3)

If a problem occurs

• See the GDB-INSTALL entries in QuickStart, chapter 7, and TroubleShooting.

Learn more

• For backup and restore, see chapter 8.
• PostgreSQL 17 documentation: pg_hba.conf, ALTER SYSTEM, ALTER EXTENSION, and pg_upgrade