BumpAllocator: bringing memory that grew outside the engine under its control
Instead of calling malloc for each value, the GritDB analytics engine now uses bump allocation to carve row memory out of large blocks that the engine counts. Memory outside the engine (the glibc heap) fell 63 %, and long-run memory growth dropped to one sixth.
GritDB is a dual-engine database that adds a columnar analytics engine to the PostgreSQL engine. This post is about how we changed the way the analytics engine handles memory.
Memory that the engine could not count (the glibc heap) fell 63 %, and long-run memory growth dropped to one sixth. Even after 200 rounds without a restart, peak memory stays below 83 GiB.
| At a glance | Before | After | Change |
|---|---|---|---|
| Allocations outside the engine (glibc heap) | 86.7 GiB | 31.9 GiB | -63 % |
| Growth of peak memory over long runs | +0.64 GiB per run | +0.10 GiB per round | 1/6 |
| Share the engine manages within its limit | 15 % of the engine memory limit | 96 % | Wider |
| Peak memory (VmHWM) | 118.3 GiB | 74.1 GiB | -37 % |
Situation: memory the engine could not count kept growing
Storing one row in memory took 2 to 8 calls to malloc. UNIFY in the analytics engine keeps the inserts, updates and deletes of registered tables in memory as rows, then moves them to columnar files. A row has several variable-length values, so it called malloc in the C library (glibc) separately for each value.
That memory sat outside the engine’s memory limit. The engine counts and manages its memory within a set limit. But the engine could neither count the row memory taken from the glibc heap nor stop it with the limit.
Freed memory did not go back to the operating system. The thread that creates a row and the thread that removes it are different. glibc keeps a separate heap per thread and does not return freed memory to the operating system right away. So the longer the engine ran, the higher the process memory climbed.
| Problem | Result |
|---|---|
| Several malloc calls per row | Allocations and frees happen several times per row |
| Memory outside the engine limit | The engine can neither count it nor cap it |
| Memory that does not go back | Process memory climbs the longer the engine runs |
Task: the engine counts all row memory and keeps it within its limit
We set three goals.
- One allocation per row: gather a row’s values into one piece.
- Memory the engine counts: take memory from the engine’s memory manager, count it by kind and keep it within the limit.
- Flat over long runs: memory does not climb even when the same workload repeats hundreds of times.
Solution: carve from large blocks and give back whole blocks
Bump allocation hands out memory by moving a cursor forward inside a large block. Each allocation takes only one addition and one comparison. The trade-off is that a single piece in the middle cannot be given back on its own, so data with similar lifetimes goes into the same block. PostgreSQL uses the same approach (the Bump memory context) for the memory that holds rows while sorting.
We split blocks into two kinds by lifetime.
| Block | What it holds | When it is given back |
|---|---|---|
| Row block | Rows kept in memory | Piece by piece as rows move to columnar files and are removed, and the whole block once all its pieces are back |
| Batch block | A batch of rows being moved to columnar files | The whole block at once when the batch is done |
Blocks come from the engine’s memory manager. So the engine counts blocks by kind and keeps them within the memory limit.
We also cut the memory reserved for each table. Registering a table used to reserve 40 MiB of index memory up front. Now it starts small, at about 1 KB, and grows as files are added. The memory of a dropped table also comes back without waiting for file cleanup.
Memory cleanup keeps going while the engine waits for the disk. Even while a columnar file is being written to disk and the engine waits for it, the writer no longer blocks memory cleanup.
| One row | Before | After |
|---|---|---|
| Memory allocation | 2 to 8 malloc calls on the glibc heap | 1 inside a block from the engine (long string values are still separate) |
| Does the engine count it | No | Counted by kind and kept within the limit |
| Unit of giving back | free for each value | By piece and by block |
Results: memory moved inside the engine’s limit
On the same server, we repeated the same workloads for a long time on the engine before and after the change. The after test used rounds with extra workloads, a heavier condition than before.
| Test condition | Details |
|---|---|
| Server | AMD Ryzen 7 9800X3D (8 cores, 16 threads) · 186.3 GiB of memory · same server before and after |
| Tests | ① Long write workload: repeated bulk inserts, updates and deletes ② Create and drop: create and drop 500 tables per run ③ Round: ① and ② plus two other workloads · 200 times without a restart |
| Dates | Before 2026-10-06 to 14 · after 2026-10-15 to 23 (development builds) |
| Measured values | Analytics engine process · RSS: memory in use now · VmHWM: highest memory used so far · allocations outside the engine: process memory minus the memory the engine manages |
| Compared values in the top table | Before: an engine that ran ① 10 times and then the full regression suite · after: an engine that ran 100 rounds (the 200-round values are listed separately) |
1. Peak memory no longer climbs over long runs
Before, peak memory reached 127 GiB in 33 runs; after, it stays below 83 GiB even after 200 rounds.
| Value | Before | After |
|---|---|---|
| First record | 109.2 GiB at run 4 | 63.2 GiB at round 1 |
| Last | 127.2 GiB at run 33 | 74.1 GiB at round 100 · 82.6 GiB at round 200 |
| Growth | +0.64 GiB per run | +0.10 GiB per round |
| Outcome of repeating | Stopped at run 39 on a timeout | 100 of 100 passed · then 200 rounds |
2. Memory comes back when tables are created and dropped
Before, memory grew by 12.3 GiB per run and the test stopped at run 8; after, it stays at 21 to 25 GiB over 100 runs.
| Value | Before | After |
|---|---|---|
| Start | 73.4 GiB | 20.8 GiB |
| Growth per run | +12.3 GiB | +0.013 GiB (runs 4 to 100) |
| Outcome of repeating | Stopped at run 8 at 175 GiB when server memory ran out | 100 of 100 passed · 24.0 GiB |
3. The growth was all outside the engine
Before, all of the growth was in allocations outside the engine (the glibc heap). The memory the engine manages stayed flat at 41 GiB the whole time.
| Value | Before (run 0 → 5) | After (run 0 → 100) |
|---|---|---|
| Allocations outside the engine (glibc heap) | 31.8 → 100.1 GiB | 0.1 → 2.9 GiB |
| Memory the engine manages | 41.0 → 41.2 GiB | 20.7 → 21.0 GiB |
4. Other improvements
| Item | Before | After | Change |
|---|---|---|---|
| Allocations per row | 2 to 8 malloc calls | 1 | 1/2 to 1/8 |
| Memory reserved when a table is registered | 40 MiB | about 1 KB | 1/40,000 |
| Cache evicted because memory ran short | 5.8 TB per hour | 6 to 11 GB per hour | -99 % |
| Wait for the analytics engine to catch up after a load (longest) | 180.9 s (timeout) | 23.1 s | -87 % |
| Processing time that grew with each repeat | +8.0 min over 17 runs | +2.0 min over 100 runs | about 1/20 of the slope |
Costs and what remains
What got worse
- The default engine memory limit rose from 30 % to 50 % of server memory. This is because the engine now counts more of the memory it uses. As before, the limit is the only memory setting. On a server with little memory (16 GiB), look at the engine share and PostgreSQL memory together.
- The share outside the engine still grows slowly. Part of the key index of columnar files is still outside the engine limit, so it grows by about 6 GiB every 100 rounds (11.6 GiB at 200 rounds). Most of the long-run growth is here.
What stayed the same
- No data was lost during the tests. No test failed in the full regression suite.
- The time to process one run of the same workload did not change (average of the first 10 runs: 12.6 min → 12.8 min).
- No new memory settings were added.
Next
- Move the key index memory that is still outside the engine into the memory the engine manages.
- Move allocations that are still separate, such as long string values, into the same blocks.
- Set default memory limits that fit the server memory size.
How UNIFY moves rows in memory to columnar files is on the architecture page. What GritDB does is on the product home, and results measured on real work are on the Cases page.
Terms in this post
- Bump allocation: handing out memory by moving a cursor inside a large block by the requested size. A single piece cannot be given back on its own; memory goes back block by block.
- Allocations outside the engine (glibc heap): memory used by malloc in the C library (glibc). It is outside the engine’s memory limit.
- Engine memory limit: the upper bound of memory the analytics engine can use. It is set as a share of server memory.
- RSS · VmHWM: RSS is the memory a process uses now, and VmHWM is the highest memory it has used so far. Linux records both for each process.
- Round: in the tests in this post, one pass through four workloads in order.
- Cursor: in this post, the mark inside a block that points to the next place to hand out. It is not the SQL cursor.
- UNIFY: the part of the analytics engine that gathers changes to registered tables in memory and moves them to columnar files.
Revision history
- 2026-10-29: Added three measurement charts (peak memory over long runs · creating and dropping tables · allocations outside the engine), other improvements, and the two kinds of blocks in the solution.
- 2026-10-28: The first version used numbers measured during development (2026-10-12 to 14). We replaced them with the numbers and test conditions of the final measurement, and added what got worse and what stayed the same.
← All blog posts · Subscribe with RSS (in Korean)