Shared Memory

All blockchain state in a VIZ node is stored in a single memory-mapped file (shared_memory.bin) managed by the chainbase library. The node cannot operate without this file.

---

Architecture

vizd process
├── block_log / dlt_block_log  — raw block bytes (disk)
└── shared_memory.bin (mmap)   — all chain state (chainbase)
    ├── account_index
    ├── witness_index
    ├── transaction_index
    └── ... (all other object indices)

API threads (webserver thread pool) acquire shared read locks; block application holds an exclusive write lock. Multiple readers can coexist; a writer blocks all readers.

---

Configuration

All options go in config.ini.

Size Options

Option	Default	Description
shared-file-dir	state	Directory for shared_memory.bin (relative to data dir or absolute)
shared-file-size	2G	Initial allocation. If file exists and this is larger, file grows. Does not trigger replay.
inc-shared-file-size	2G	Auto-growth step when free space falls below threshold
min-free-shared-file-size	500M	Free-space threshold that triggers auto-growth

Rule: min-free-shared-file-size must be less than inc-shared-file-size, otherwise cascading resizes occur.

Lock Timeout Options

Option	Default	Description
read-wait-micro	500000 (500 ms)	Timeout per read lock attempt
max-read-wait-retries	3	Max read attempts before error
write-wait-micro	500000 (500 ms)	Timeout per write lock attempt
max-write-wait-retries	3	Max write attempts before error

Performance Options

Option	Default	Description
single-write-thread	false	Serialize all block/transaction pushes. Recommended for production.
block-num-check-free-size	1000	Check free space every N blocks
flush-state-interval	—	Flush shared memory to disk every N blocks
clear-votes-before-block	0	Drop votes older than this block (0 = keep all). Reduces memory.
skip-virtual-ops	false	Skip virtual operation notifications. Saves CPU during replay.

---

Recommended Configurations

Validator node (production):

shared-file-size = 4G
inc-shared-file-size = 2G
min-free-shared-file-size = 500M
single-write-thread = true

API node (high read throughput):

shared-file-size = 8G
inc-shared-file-size = 2G
min-free-shared-file-size = 500M
single-write-thread = true
read-wait-micro = 1000000
max-read-wait-retries = 10
webserver-thread-pool-size = 256

Replay / initial sync:

shared-file-size = 8G
inc-shared-file-size = 4G
min-free-shared-file-size = 500M
block-num-check-free-size = 10
skip-virtual-ops = true

---

Auto-Resize

The database auto-grows when free space drops below min-free-shared-file-size. Each resize:

1. Writes a resize_in_progress crash marker file.
2. Flushes all dirty pages to disk (flush()).
3. Pauses all operations (including block production and API requests).
4. Destroys the current memory mapping.
5. Extends the file by inc-shared-file-size.
6. Re-maps the file and rebuilds all index pointers.
7. Validates key objects (e.g., dynamic_global_property_object) survived the remap.
8. Removes the crash marker.

Safety Mechanisms

• Flush-before-resize: Dirty pages are written to disk before the mapping is destroyed, ensuring the on-disk file is consistent if anything fails during grow.
• Crash marker: A resize_in_progress file is written before the destructive remap and removed after success. If the process crashes mid-resize, the marker survives and triggers automatic recovery on the next startup.
• Post-resize validation: After the remap, the node verifies that max_memory() matches the expected size and that critical objects (e.g., dynamic_global_property_object) are intact. Corruption is detected early instead of causing confusing downstream failures.
• bad_alloc safety: If shared memory is exhausted during block application, the undo session is safely discarded (rather than attempting a doomed undo that would crash the process via std::terminate). A deferred resize is scheduled for the next block.

Pre-allocate shared-file-size generously to minimize resize frequency. Each resize causes a latency spike.

---

Size Planning

Approximate usage for a VIZ mainnet full node:

Component	Estimated Size
Account index (~14 K accounts)	~50 MB
Validator index	~5 MB
Operation history (operation_history plugin)	200–500 MB
Account history (account_history plugin)	100–300 MB
Other indexes	100–200 MB
Recommended starting size	4–8 GB

---

Startup Sequence

1. Open shared_memory.bin (grow if shared-file-size is larger)
2. Acquire exclusive file lock
3. Check for resize_in_progress crash marker → trigger recovery if found
4. Initialize indices
5. If genesis missing → init_genesis()
6. Open block_log or dlt_block_log
7. undo_all() → rewind to last irreversible block
8. Verify head block matches block log

---

Recovery

Symptom	Action
CRITICAL: validator X account object MISSING	Corruption — use --replay-from-snapshot --snapshot-auto-latest
Could not modify object, uniqueness constraint violated	Corruption — use --replay-from-snapshot --snapshot-auto-latest
Unable to acquire READ lock	Lock contention — increase read-wait-micro / enable single-write-thread
Shared memory corrupted: previous resize() crashed	Interrupted resize — use --replay-from-snapshot --snapshot-auto-latest
dynamic_global_property_object missing after resize	Resize corruption — use --replay-from-snapshot --snapshot-auto-latest
Node crashes in a loop on startup	Corrupted file — --replay-from-snapshot --snapshot-auto-latest

Recovery options:

• --replay-from-snapshot --snapshot-auto-latest — delete shared memory, import latest snapshot, replay dlt_block_log.
• --resync-blockchain — delete shared memory and block log, sync from peers.
• --snapshot <path> — load from specific snapshot, replay dlt_block_log on top.

---

See also: Chain Plugin, Snapshot Plugin, Block Log.