Huge Pages Not Configured: TLB Pressure on Large Servers

Scenario

A 128 GB PostgreSQL server has shared_buffers = 32GB and huge_pages = off. Performance profiling shows high TLB (Translation Lookaside Buffer) miss rates. The OS is managing 32GB of shared memory with standard 4KB pages, requiring over 8 million page table entries. Enabling huge pages (2MB per page) reduces that to ~16,000 entries, cutting TLB pressure and CPU overhead by approximately 15%. The OS must be configured first — PostgreSQL’s huge_pages = 'on' will fail to start if vm.nr_hugepages = 0.

How to Identify

Conditions:

• SHOW huge_pages returns off
• shared_buffers is large (>8 GB) on a server with significant RAM
• perf stat or oprofile shows high TLB miss rates (dTLB-load-misses)
• /proc/meminfo shows HugePages_Total: 0
• CPU usage is elevated relative to query throughput — no I/O bottleneck

Analysis Steps

-- Check current huge_pages setting and shared_buffers size:
SHOW huge_pages;
SHOW shared_buffers;

-- View all memory-related parameters together:
SELECT name, setting, unit
FROM pg_settings
WHERE name IN ('huge_pages', 'shared_buffers', 'work_mem', 'maintenance_work_mem')
ORDER BY name;

-- Check the context — huge_pages requires a restart:
SELECT name, setting, context
FROM pg_settings
WHERE name = 'huge_pages';
-- context='postmaster' → restart required

-- Estimate how many huge pages are needed:
-- nr_hugepages = ceil(shared_buffers_bytes / 2MB)
-- 32GB shared_buffers → ceil(32*1024 / 2) = 16384 huge pages
SELECT
    current_setting('shared_buffers')                    AS shared_buffers,
    pg_size_bytes(current_setting('shared_buffers'))     AS shared_buffers_bytes,
    ceil(pg_size_bytes(current_setting('shared_buffers'))
         / (2.0 * 1024 * 1024))::int                    AS recommended_nr_hugepages;

Pitfalls

• huge_pages = 'on' causes PostgreSQL to fail to start if vm.nr_hugepages is 0 or insufficient. Use huge_pages = 'try' during initial rollout — it falls back to standard pages gracefully.
• Huge pages must be configured at the OS level (vm.nr_hugepages) before starting PostgreSQL. The pages are reserved at boot time; if memory is fragmented, the kernel may not be able to allocate them.
• Add extra margin to nr_hugepages (10–20%) — PostgreSQL’s huge page allocation includes shared memory overhead beyond just shared_buffers.
• Transparent Huge Pages (THP) is a different feature (/sys/kernel/mm/transparent_hugepage/enabled) and should be set to madvise or never for PostgreSQL — THP causes latency spikes from background compaction.
• Huge pages require vm.nr_hugepages in /etc/sysctl.conf for persistence across reboots. A one-time sysctl -w does not survive reboot.
• huge_pages requires a PostgreSQL restart — it is not reload-safe.

Resolution Approach

Calculate required huge pages (ceil(shared_buffers / 2MB) plus 10% margin). Configure vm.nr_hugepages persistently in /etc/sysctl.conf. Set huge_pages = 'try' initially, then switch to 'on' once confirmed working. Verify via /proc/meminfo and pg_settings.