Home Original page

hakavlad/le9-patch

le9*.patch: Protect file pages during low memory

The current le9 patches are based on patches that were originally created by Mandeep Singh Baines (2010) and Marcus Linsner (2018-2019). The current le9 patches provide two sysctl knobs for soft and hard protection of file pages. Let's give the floor to the original founders:

On ChromiumOS, we do not use swap. When memory is low, the only way to free memory is to reclaim pages from the file list. This results in a lot of thrashing under low memory conditions. We see the system become unresponsive for minutes before it eventually OOMs. We also see very slow browser tab switching under low memory. Instead of an unresponsive system, we'd really like the kernel to OOM as soon as it starts to thrash. If it can't keep the working set in memory, then OOM. Losing one of many tabs is a better behaviour for the user than an unresponsive system.

This patch create a new sysctl, min_filelist_kbytes, which disables reclaim of file-backed pages when when there are less than min_filelist_bytes worth of such pages in the cache. This tunable is handy for low memory systems using solid-state storage where interactive response is more important than not OOMing.

With this patch and min_filelist_kbytes set to 50000, I see very little block layer activity during low memory. The system stays responsive under low memory and browser tab switching is fast. Eventually, a process a gets killed by OOM. Without this patch, the system gets wedged for minutes before it eventually OOMs.

— https://lore.kernel.org/patchwork/patch/222042/

The attached kernel patch (applied on top of 4.18.5) that I've tried, almost completely eliminates the disk thrashing(the constant reading of executable(and .so) files on every context switch) associated with freezing the OS and so, with this patch, the OOM-killer is triggered within a maxium of 1 second when it is needed, rather than, without this patch, freeze the OS for minutes(or just a long time, it may even auto reboot depending on your kernel .config options set to panic(reboot) on hang after xx seconds) with constant disk reading well before OOM-killer gets triggered.

— https://bugs.launchpad.net/ubuntu/+source/linux/+bug/159356/comments/89

le9bb

le9bb*-5.10 patches may be correctly applied to Linux 5.10—5.11.

le9bb0 just provides two sysctl knobs with 0 values and does not protect file pages by default.

le9bb1 provides only soft protection (vm.file_low_kbytes=200000). This patch may be safly used by default.

le9bb2 provides hard protection of file pages (vm.file_min_kbytes=200000).

Effects

  • Improving system responsiveness under low-memory conditions;
  • Improving performans in I/O bound tasks under memory pressure;
  • OOM killer comes faster (with hard protection);
  • Fast system reclaiming after OOM.

Testing

These tools may be used to monitor memory and PSI metrics during stress tests:

  • mem2log may be used to log memory metrics from /proc/meminfo;
  • psi2log from nohang package may be used to log PSI metrics during stress tests.

Demo

  • https://youtu.be/c5bAOJkX_uc - Linux 5.9 + le9i-5.9.patch, playing supertuxkart with 7 tail /dev/zero in background. vm.unevictable_activefile_kbytes=1000000, vm.unevictable_inactivefile_kbytes=0.

Warnings

  • These patches were written by an amateur. Use at your own risk;
  • MemAvailable may be calculated incorrectly (reserved vm.file_min_kbytes value cannot be reclaimed);
  • Hard protection of file pages may invoke Fatal IO error 11 #5 with DRM/i915 driver. Disabling DRM/i915 GEM shrinker can prevent this.

Need to review

These patches need to be reviewed by linux-mm peoples.

How to get it

Resources