For this experiment I am using Postgres v11 on Linux 3.10 kernel. The goal was to see what gains can be made from using hugepages. I use the “built in” benchmark pgbench to run a simple set of queries.
Since I am interested in only the gains from hugepages I chose to use the “-S” parameter to pgbench which means perform only the “select” statements. Obviously this masks any costs that might be seen when dirtying hugepages – but it kept the experiment from having to be concerned with writing to the filesystem.
Experiment
The workstation has 32GB of memory
Postgres is given 16GB of memory using the parameter
|
1 |
shared_buffers = 16384MB |
pgbench creates a ~7.4gb database using a scale-factor of 500
|
1 |
pgbench -i -s 500 |
Run the experiment like this
|
1 |
$ pgbench -c 10 -S -T 600 -P 1 p gbench |
Result
Default : No hugepages :
tps = 62190.452850 (excluding connections establishing)
2MB Hugepages
tps = 66864.410968 (excluding connections establishing)
+7.5% over default
1GB Hugepages
tps = 69702.358303 (excluding connections establishing)
+12% over default
Enabling hugepages
Getting the default hugepages is as easy as entering a value into /etc/sysctl.conf. To allow for 16GB of hugepages I used the value of 8400, followed by “sysctl -p”
|
1 |
[root@arches gary]# grep huge /etc/sysctl.conf <br>vm.nr_hugepages = 8400<br>[root@arches gary]# sysctl -p |
To get 1GB hugepages, the kernel has to have it configured during boot e.g.
|
1 |
[root@arches boot]# grep CMDLINE /etc/default/grub<br>GRUB_CMDLINE_LINUX="rd.lvm.lv=centos/swap vconsole.font=latarcyrheb-sun16 rd.lvm.lv=centos/root crashkernel=auto vconsole.keymap=us rhgb quiet rdblacklist=nouveau default_hugepagesz=1G hugepagesz=1G |
Then reboot the kernel
I used these excellent resources
How to modify the kernel command line
How to enable hugepages
and this great video on Linux virtual memory