Overview
About vulnerability
In the Linux kernel, the following vulnerability has been resolved: IB/mlx5: Fix potential deadlock in MR deregistration The issue arises when kzalloc() is invoked while holding umem_mutex or any other lock acquired under umem_mutex. This is problematic because kzalloc() can trigger fs_reclaim_aqcuire(), which may, in turn, invoke mmu_notifier_invalidate_range_start(). This function can lead to mlx5_ib_invalidate_range(), which attempts to acquire umem_mutex again, resulting in a deadlock. The problematic flow:
| CPU0 | CPU1 |
|---|---|
| mlx5_ib_dereg_mr() | |
| → revoke_mr() | |
| → mutex_lock(&umem_odp->umem_mutex) | |
| mlx5_mkey_cache_init() | |
| → mutex_lock(&dev->cache.rb_lock) | |
| → mlx5r_cache_create_ent_locked() | |
| → kzalloc(GFP_KERNEL) | |
| → fs_reclaim() | |
| → mmu_notifier_invalidate_range_start() | |
| → mlx5_ib_invalidate_range() | |
| → mutex_lock(&umem_odp->umem_mutex) | |
| → cache_ent_find_and_store() | |
| → mutex_lock(&dev->cache.rb_lock) | |
| Additionally, when kzalloc() is called from within | |
| cache_ent_find_and_store(), we encounter the same deadlock due to | |
| re-acquisition of umem_mutex. | |
| Solve by releasing umem_mutex in dereg_mr() after umr_revoke_mr() | |
| and before acquiring rb_lock. This ensures that we don’t hold | |
| umem_mutex while performing memory allocations that could trigger | |
| the reclaim path. | |
| This change prevents the deadlock by ensuring proper lock ordering and | |
| avoiding holding locks during memory allocation operations that could | |
| trigger the reclaim path. | |
| The following lockdep warning demonstrates the deadlock: | |
| python3/20557 is trying to acquire lock: | |
| ffff888387542128 (&umem_odp->umem_mutex){+.+.}-{4:4}, at: | |
| mlx5_ib_invalidate_range+0x5b/0x550 [mlx5_ib] | |
| but task is already holding lock: | |
| ffffffff82f6b840 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: | |
| unmap_vmas+0x7b/0x1a0 | |
| which lock already depends on the new lock. | |
| the existing dependency chain (in reverse order) is: | |
| -> #3 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}: | |
| fs_reclaim_acquire+0x60/0xd0 | |
| mem_cgroup_css_alloc+0x6f/0x9b0 | |
| cgroup_init_subsys+0xa4/0x240 | |
| cgroup_init+0x1c8/0x510 | |
| start_kernel+0x747/0x760 | |
| x86_64_start_reservations+0x25/0x30 | |
| x86_64_start_kernel+0x73/0x80 | |
| common_startup_64+0x129/0x138 | |
| -> #2 (fs_reclaim){+.+.}-{0:0}: | |
| fs_reclaim_acquire+0x91/0xd0 | |
| __kmalloc_cache_noprof+0x4d/0x4c0 | |
| mlx5r_cache_create_ent_locked+0x75/0x620 [mlx5_ib] | |
| mlx5_mkey_cache_init+0x186/0x360 [mlx5_ib] | |
| mlx5_ib_stage_post_ib_reg_umr_init+0x3c/0x60 [mlx5_ib] | |
| __mlx5_ib_add+0x4b/0x190 [mlx5_ib] | |
| mlx5r_probe+0xd9/0x320 [mlx5_ib] | |
| auxiliary_bus_probe+0x42/0x70 | |
| really_probe+0xdb/0x360 | |
| __driver_probe_device+0x8f/0x130 | |
| driver_probe_device+0x1f/0xb0 | |
| __driver_attach+0xd4/0x1f0 | |
| bus_for_each_dev+0x79/0xd0 | |
| bus_add_driver+0xf0/0x200 | |
| driver_register+0x6e/0xc0 | |
| __auxiliary_driver_register+0x6a/0xc0 | |
| do_one_initcall+0x5e/0x390 | |
| do_init_module+0x88/0x240 | |
| init_module_from_file+0x85/0xc0 | |
| idempotent_init_module+0x104/0x300 | |
| __x64_sys_finit_module+0x68/0xc0 | |
| do_syscall_64+0x6d/0x140 | |
| entry_SYSCALL_64_after_hwframe+0x4b/0x53 | |
| -> #1 (&dev->cache.rb_lock){+.+.}-{4:4}: | |
| __mutex_lock+0x98/0xf10 | |
| __mlx5_ib_dereg_mr+0x6f2/0x890 [mlx5_ib] | |
| mlx5_ib_dereg_mr+0x21/0x110 [mlx5_ib] | |
| ib_dereg_mr_user+0x85/0x1f0 [ib_core] | |
| —truncated— | |
| A flaw was found in the Linux kernel’s IB/mlx5 driver. A local attacker could trigger a deadlock by invoking memory allocation functions while a mutex (mutual exclusion) is held. This can lead to a situation where the system attempts to acquire the same mutex again, resulting in a system freeze or unresponsiveness, effectively causing a Denial of Service (DoS). |
Details
KernelCare state
Live-patch status from KernelCare for each operating system.
| Operating system | Status | Covered kernels |
|---|---|---|
| Ubuntu 24.04 | Released |
12 kernels
|