Skip to content

SSV-25461 High latency due to ZFSin driver#111

Merged
datacore-PankajSharma merged 18 commits into
datacore-windowsfrom
SSV-25461-HighLatencyDueToZFSin
May 22, 2026
Merged

SSV-25461 High latency due to ZFSin driver#111
datacore-PankajSharma merged 18 commits into
datacore-windowsfrom
SSV-25461-HighLatencyDueToZFSin

Conversation

@datacore-PankajSharma
Copy link
Copy Markdown
Collaborator

@datacore-PankajSharma datacore-PankajSharma commented Apr 29, 2026

Motivation and Context

This change addresses the SSV-25461 ZFSin high latency issue after restart

Description

Here is the detailed list of changes that were added to address the issue

  1. spl_abd_prealloc_thread which will preallocate memory upto arc_max in abd_cache during boot time to warm cache.
  2. spl_free_wrapper and spl_free_manual_pressure_wrapper logic changed to tell system not to account free warm memory in abd_cache as used, thereby not creating a low on memory panic just after reboot.
  3. Dynamic dirty max adjustment logic incorporated in txg_sync_thread, which will dynamically adjust zfs_dirty_data_max based on spa_sync flush time (Although this works well without mirror special devices, future changes will modify this code to work for MSD as well).
  4. Relevant kstat data added regarding dynamic dirty code for better debugging.

How Has This Been Tested?

Testing has been done in two phases

  1. For high latency, running FIO with 1mb block size has been performed and compared the same test results with old driver.
  2. For performance testing, Running vdbench on 4 vdisks (ILD,ILC,ILDC,RAW) parallely with different workloads has been performed

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

Comment thread module/zfs/txg.c Outdated
Comment thread module/zfs/txg.c Outdated
@datacore-PankajSharma datacore-PankajSharma force-pushed the SSV-25461-HighLatencyDueToZFSin branch from b175f78 to df12a2a Compare May 21, 2026 02:11
@datacore-senthil
Copy link
Copy Markdown
Collaborator

Approved

@datacore-senthil datacore-senthil marked this pull request as ready for review May 22, 2026 08:07
@datacore-senthil datacore-senthil self-requested a review May 22, 2026 08:08
@datacore-PankajSharma datacore-PankajSharma merged commit 452e064 into datacore-windows May 22, 2026
0 of 8 checks passed
datacore-PankajSharma added a commit that referenced this pull request May 22, 2026
* Added support to prealloc the abd cache
* Registry 'zfs_abd_prealloc_percent' can be used to set the percent of prealloc wrt zfs_arc_max
* Disabled memory pressure thread
* Changed abd_cache alloc size
* skip source free when the reserved memory is not crossing arc_max
* skip source free when the reserved memory is not crossing arc_max
* skip source free when the reserved memory is not crossing arc_max
* Changed the registry name from zfs_abd_prealloc_percent to zfs_prealloc_percent
Setting zfs_prealloc_percent=0 will disable the prealloc feature
* Reverted zfs_abd_chunk_size from 65536 to 4096 which was causing read performace issue
* Fix spl_thread_create not setting thread priority
* Change memory available logic and manual pressure
* Add dynamic zfs_dirty_data_max adjustments
* Memory consumption crossing zfs_total_memory_limit
* change spl_free_wrapper and spl_free_manual_pressure_wrapper logic for prealloc

---------

Co-authored-by: Arun KV <arun.kv@datacore.com>
datacore-PankajSharma added a commit that referenced this pull request May 22, 2026
* Added support to prealloc the abd cache
* Registry 'zfs_abd_prealloc_percent' can be used to set the percent of prealloc wrt zfs_arc_max
* Disabled memory pressure thread
* Changed abd_cache alloc size
* skip source free when the reserved memory is not crossing arc_max
* skip source free when the reserved memory is not crossing arc_max
* skip source free when the reserved memory is not crossing arc_max
* Changed the registry name from zfs_abd_prealloc_percent to zfs_prealloc_percent
Setting zfs_prealloc_percent=0 will disable the prealloc feature
* Reverted zfs_abd_chunk_size from 65536 to 4096 which was causing read performace issue
* Fix spl_thread_create not setting thread priority
* Change memory available logic and manual pressure
* Add dynamic zfs_dirty_data_max adjustments
* Memory consumption crossing zfs_total_memory_limit
* change spl_free_wrapper and spl_free_manual_pressure_wrapper logic for prealloc

---------

Co-authored-by: Arun KV <arun.kv@datacore.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants