Performance regression on Linux Kernels 5.15
Ubuntu 20.04 on Anthos for vSphere
We've been observing less performance after upgrading into kernel 5.15 from 5.4. It is something related with Portworx?
This is actually an identified problem with kernel 5.15.
We were able to reproduce such performance regression without Portworx installed, measuring disk raw performance, thus the culprit is not Portworx.
This performance regression can be indeed noticeable under certain circumstances, up to 20% to 30% impact.
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2042564
Here is the explanation from the bug report:
| Joseph Salisbury (jsalisbury) wrote : | #14 |
I performed a bisect between the 5.4 and 5.15 kernels. The performance regression was introduced by a stable update in 5.15.57 by the following commit:
62b4db57eefec ("x86/entry: Add kernel IBRS implementation")
This commit applies IBRS kernel mitigation for Spectre_v2. IBRS is: Indirect Branch Restricted Speculation.
This commit was also applied up upstream stable 5.4 with the following SHA1:
a3111faed5c1d ("x86/entry: Add kernel IBRS implementation")
However, the backport to 5.4 did not introduce as much as a performance regression as the backport to 5.15. There are many difference between the 5.4 and 5.15 backports of the commit. Much of the assembly logic in 5.15 does not exist in 5.4, since it was not needed.
There are several commits that are later applied to 5.15 stable that depend on this patch, so it would not be easily reverted.
One option is to use a boot option to disable IBRS mitigation. However, the security versus performance trade-off must be considered carefully. IBRS can be disabled with the boot parameter "noibrs"
There is a wiki page that describes the various boot parameters here:
https:/