10. Limitations
By the end of this chapter: You know the design limitations of GritDB 1.1 and the boundaries of its support scope. You also know the impact of each limitation and what to do about it.
- This chapter lists only the boundaries of the design and the support scope. Changes by version are in the release notes. Conditions that apply only inside a procedure (setting values, registration, analytics setup steps) are in that section.
- Entry format: Numeric limits (10.2) use
Item | Limit type | Limit value | Adjustment. Platform, compatibility, and storage (10.1, 10.3, 10.4) useLimitation | Impact | What to do | How to check. Operating rules (10.5) useLimitation | Impact | What to do. Features not in 1.1 (10.6) useFeature | What to do.
10.1 Platform
| Limitation | Impact | What to do | How to check |
|---|---|---|---|
x86-64 with cx16 and sse4_2 |
On a server that falls short, GritDB does not start. The installation script only warns | Check when you select the server | QuickStart, chapter 2, ① |
glibc ≥ min_glibc |
If it falls short, GritDB cannot run | Check when you select the OS | QuickStart, chapter 2, ② |
Total memory that the OS sees (total in free -m) 8192 or more |
If it falls short, the GritDB engine does not start. A server sold as 8 GB usually falls short | Check when you select the server | free -m |
| OS: Ubuntu 24.04 LTS | Other OSs and containers are outside the supported environment | When you select the server | cat /etc/os-release |
| Single server | No redundant or multi-node setup | 10.6, and 8.7 for a standby server | — |
10.2 Numeric limits
| Item | Limit type | Limit value | Adjustment |
|---|---|---|---|
| Installation directory length | Fixed (unix socket limit) | 80 characters | Not possible |
| Number of columns, name length, and value size | Fixed (PostgreSQL) | 1600 columns, 63 bytes, and 1 GB | Not possible |
| Number of registered tables | — | No limit | — |
| Query text in the analytic query history | Fixed | First 1024 characters (the full text is in datafiles/query_text_archive.jsonl) |
Not possible |
| Concurrent executions in analytics sessions | Derived | Number of vCPUs × 4 | vCPU |
| Number of connections | Adjustable (PostgreSQL) | The number that regular accounts can use is max_connections − superuser_reserved_connections (default 3) − reserved_connections (default 0) − GritDB internal connections (grit_surfer.catalog.read_threads + 2, default 6, 7.1) |
ALTER SYSTEM. If you change it, see the shm_limit_mb warning in 5.6 |
max_worker_processes |
Fixed (GritDB) | 256 or less (the upper bound is fixed). If it is higher, GritDB does not start | ALTER SYSTEM, within 256 (6.4.1 step 5) |
| Rows in one transaction | Derived (server memory, no setting changes it) | max= on the Tier-3 (RowVersion line of the engine log (6.8) |
Commit in batches below half of it. For initial copies or operational database batches that exceed it, contact support |
| Validated vCPU count | — | Pending measurement | — |
| Open file limit | OS-dependent | 65536 or more | OS settings (1.4) |
10.3 PostgreSQL compatibility scope
| Limitation | Impact | What to do | How to check |
|---|---|---|---|
The installation package does not include PostgreSQL additional modules (contrib, for example pg_stat_statements) |
You cannot do monitoring or diagnosis that uses those modules | Checks in chapter 7, A.6 diagnostic queries, and analytic query history (6.5) | Not in the list that SELECT name FROM pg_available_extensions ORDER BY 1; returns |
| Setups with other preloaded extensions are not tested in 1.1 | Behavior is not guaranteed | If needed, contact support (chapter 0). Do not change shared_preload_libraries in the required block (5.2) |
SHOW shared_preload_libraries; returns gritsurfer |
| Collation is fixed to C (set by the installation) | Sort results differ from ko_KR and en_US collations |
Do not create databases with another collation through TEMPLATE template0 (1.8) |
SELECT datname, datcollate, datctype FROM pg_database; returns C and C.UTF-8 |
Time zone: Analytics session queries do not use the session time zone (SET timezone) |
timestamptz output can differ from regular sessions |
Match the application’s time zone to the server. Convert with AT TIME ZONE (1.8 and 6.6) |
Compare SHOW timezone; in the application and on the server |
numeric: Columns without a declared precision or with a precision above 38 give approximate values in analytics sessions |
Sums of amounts and quantities can differ from regular sessions | Create them as numeric(p, s) (p ≤ 38) (6.2) |
The precision check query in 6.2 returns 0 rows |
Analytics session results can differ from PostgreSQL in avg precision, numeric digits, sum of real columns, octet_length of char(n), concat, and timestamptz output |
The same SELECT gives different results in the two sessions | If you need exact values, use a regular session or /*+ engine=pg */ (6.6) |
Run the same SELECT in both sessions and compare |
restart_after_crash is off and cannot be changed |
If one backend terminates abnormally, all of GritDB stops | Start again with the procedure in 4.3 | SHOW restart_after_crash; returns off |
pg_upgrade and pg_basebackup backups are not used with GritDB |
They do not handle GritDB data | Chapter 3 for upgrades, chapter 8 for backups (A.1 and 8.3) | — |
| Logical replication: GritDB is only the receiving side (analytics setup, subscription) | Setups that use GritDB as the publication source are not covered. GritDB also does not receive data through physical replication of the operational database (streaming standby server) or through pg_createsubscriber (1.1 and 6.4) |
6.4 for an analytics setup | — |
Only a connection option turns on gritsurfer_session_mode |
You cannot turn it on with account or database defaults or with SET |
The driver’s connection option (6.6) | SHOW gritsurfer_session_mode; |
A transaction that ran DROP on a registered table or database cannot use PREPARE TRANSACTION |
In tools that use two-phase commit, that transaction fails | Run DROP outside two-phase commit (6.3) |
Error message (6.3) |
10.4 GritDB storage structure
| Limitation | Impact | What to do | How to check |
|---|---|---|---|
Registered tables: REPLICA IDENTITY FULL |
If it is not set, registration is rejected | Set it before you register | \d+ <table> |
| Registration does not move rows that existed before registration into the GritDB engine | If you register a table that has data, analytics session results miss the rows from before registration | Register the table while it is empty. When you attach an analytics setup, register before the subscription (CREATE SUBSCRIPTION) |
Compare the same count(*) in a regular session and an analytics session (in the analytics session, after the apply check and with the GritDB engine forced, 6.6.1) |
| For registration status, each connection uses the value it checked first until the connection ends | A connection that wrote to the table before register or unregister does not see the changed registration status (its changes are not passed on, or keep being passed on) | After you register or unregister, reopen the connections and replication workers that use that table (6.3) | Before and after reconnecting, compare the same count(*) in a regular session and an analytics session (in the analytics session, after the apply check and with the GritDB engine forced, 6.6.1) |
| Registration: supported column types only (15 types, 6.2) | If a column has an unsupported type, registration is rejected (ERROR) | Check the column types before you register | Error message of the register function |
| Registration: only tables that never had a column dropped | If a column was ever dropped (DROP COLUMN), registration is rejected (ERROR) |
Re-create the table, then register it (6.7 and 6.2) | Count of attisdropped rows in pg_attribute (6.2) |
| Analytics session queries read the data applied to cold files | A change you just committed may not show in analytics session query results right away | Run queries that must see the latest values in a regular session (pg) (6.6). In an analytics setup, run them on the operational database |
Run the same SELECT in a regular session and an analytics session and compare the results |
TRUNCATE and structure changes (DDL) are not applied to the GritDB engine |
Analytics session results become different from regular session results | 6.7 “What to do instead”. In an analytics setup, take that table again (6.4.2) | Compare the same count(*) (6.6) |
- The apply lag is usually about 12 seconds, and within about 22 seconds 99 times out of 100 (in-house tests). In an analytics setup, the replication time from the operational database adds to it. To confirm that changes are applied, see 6.6.1.
10.5 Operating rules
| Limitation | Impact | What to do |
|---|---|---|
Start and stop: gritdb-start and gritdb-stop only |
If you start PostgreSQL by itself with pg_ctl, it shuts down after about 60 seconds |
QuickStart, chapter 6, prohibition 1 |
Data: datafiles/ as one unit |
If you replace or delete only part of it, GritDB cannot start | QuickStart, chapter 6, prohibition 2, and the backup unit in chapter 8 |
| Data: the same real path | If you change the real path of datafiles/ after installation (including the relocation target and the mount point), GritDB loses the cold files |
Do not change the path. Restore to the same path too (8.4) |
Keep gritsurfer_slot |
If you delete it, GritDB data can become inconsistent and GritDB may not start | QuickStart, chapter 6, prohibition 3 |
Do not set max_slot_wal_keep_size on the GritDB server |
If the recovery replication slot becomes invalid, GritDB may not start | Keep the default (-1). Plan disk space for pg_wal (1.3). For an analytics setup, put the limit on the operational database side (6.4.3) |
wal_level = logical is fixed |
If you change it, GritDB does not work | Required block in chapter 5 |
| Analytics setup publication: only tables with a replica identity, added by name | If you add a table without a replica identity, UPDATE and DELETE on that table stop with an error on the operational database | 6.4.1 steps 2 and 3 |
| Analytics setup tables: the same primary key as the operational database | Without it, applying updates and deletes stops and WAL accumulates on the operational database | 6.4.1 step 6 |
Analytics setup subscription: streaming = off |
on and parallel are not used in 1.1 |
6.4.1 step 8 |
| An analytics setup takes the tables again instead of restoring from a backup | If you restore from a backup, the operational database changes made after the backup are missing | 8.6 take the tables again (8.4) |
| One transaction: below half the limit (6.8), including the analytics setup initial copy and operational database batches | If it goes over, GritDB stops (PostgreSQL included), and an analytics setup stops again even after you start it again | Commit in batches. For tables or batches that exceed it, contact support (6.4 and 6.8) |
Do not set restart_after_crash |
If you write it in postgresql.conf, the start fails |
5.3 |
| New databases are copies of template1 | Databases copied from template0 have no extension | 6.1 of chapter 6 |
Keep the postgres database |
If you drop or rename it, GritDB cannot start | 6.1 |
Do not modify or lock GritDB internal tables (the gritdb schema in the postgres database), even as superuser. Do not unregister or drop gritdb.grit_surfer_heartbeat in the registration list either |
GritDB can stop | Query them only with the functions in 6.5 |
| Logs can record original SQL text | Data exposure when you export them | 9.7 of chapter 9 |
| Quick stop: core files | They can use tens of GB of disk | 4.2 and 9.6 |
Turn off RemoveIPC in the OS (when the service account is a regular account) |
When the service account logs out, the GritDB shared memory is removed | 1.4 ⑤ |
10.6 Features not in 1.1
| Feature | What to do |
|---|---|
| GritDB redundancy (standby, HA, DR) and multiple nodes | Single server and backup (chapter 8). An analytics setup takes the tables again from the operational database (8.6). A standby server is manual (8.7) |
| Setting the GritDB engine memory size | It is set as a ratio of physical memory (1.3 and 5.5) |
| Automatic selection of tables to register | The operator registers tables (6.2) |
| Workload management and resource groups | The concurrency limit is fixed (6.9) |
| Password policy (complexity and lockout after failed logins) | PostgreSQL default (VALID UNTIL) and 11.10 |
| Dedicated audit extension | PostgreSQL log settings (11.9) |
| Data masking (redaction) | Mask data when you export it (9.7) |
| Diagnostic data collection tool | Manual collection in QuickStart 7.4 |
| Automatic restart and restart without downtime | 4.3 and planned shutdown (4.7) |
| Adding GritDB to an existing PostgreSQL | Build an analytics setup with a newly installed GritDB (6.4) |
| Online backup, file system snapshot backup, and point-in-time recovery with archived WAL (PITR) | Offline backup (8.2). The point you can restore to is the last backup (8.3) |
| Connection encryption (TLS), external authentication (LDAP, PAM, GSSAPI), and encryption of stored data | 11.8 |
| Status command and file to start GritDB at server boot | 4.5 and 4.6 |
| Metrics collection endpoint | Checks in chapter 7 and A.6 diagnostic queries |
If a problem occurs
- For symptoms when a limitation is violated, see TroubleShooting.
Learn more
- For security limits, see 11.8. For operations not to perform and what to do instead, see 6.7. For an analytics setup, see 6.4.
- PostgreSQL 17 documentation: PostgreSQL limits (number of columns, name length, and value size in 10.2), Connection settings (
max_connectionsand reserved connections), Error handling settings (restart_after_crash), Collation, and Logical replication restrictions