🌐
OpenZFS
openzfs.github.io › openzfs-docs › Basic Concepts › Pool Structure › Special vdev.html
Special vdev — OpenZFS documentation
July 22, 2026 - Deduplication tables go to a dedup vdev if the pool has one; otherwise to the special vdev. Setting the zfs_ddt_data_is_special module parameter to 0 keeps them out of the special class.
🌐
Big Iron
bigiron.cc › big iron › guides › zfs special vdevs: for metadata and small blocks
ZFS Special Vdevs: For Metadata and Small Blocks - Big Iron
May 11, 2026 - For a 200TB pool: 600GB-1TB of special vdev. Pair of 1TB NVMe mirrored. You can ask the existing pool how much metadata it has: zpool get all tank | grep -i metadata # Or: zdb -bb tank | grep -E "Metadata|Total|^[[:space:]]*ZFS plain"
Discussions

ZFS Metadata Special Device: Z - L1 Articles & Video-related - Level1Techs Forums
Introduction ZFS Allocation Classes: It isn’t storage tiers or caching, but gosh darn it, you can really REALLY speed up your zfs pool. From the manual: Special Allocation Class The allocations in the special class are dedicated to specific block types. By default this includes all metadata, ... More on forum.level1techs.com
🌐 forum.level1techs.com
35
May 2, 2024
Advice on the special VDEV in my ZFS setup
If you want to be certain that your thumbnails and whatnot render faster in Explorer windows, you'll want a special with special_small_blocks set large enough to capture the thumbnails (and you'll want to double-check to see how large those thumbnails are). L2ARC might or might not do that great a job at keeping those cached. Then again, L2ARC is safe whereas the special is an additional element of risk. I'd probably start out with L2ARC (which can be safely removed later if necessary) and see if that did what I wanted it to do well enough. More on reddit.com
🌐 r/zfs
38
7
March 6, 2023
How do I make a metadata special device vdev?
I would highly recommend using something more robust than raidz1 on a 100TB pool. The likelihood of an error during a rebuild of a pool of that magnitude is not inconsequential. If this is just a backup pool, then so be it, but if this is to be primary storage then you're gambling with your data. More on reddit.com
🌐 r/zfs
16
3
January 31, 2023
What does the ZFS Metadata Special Device do?
zpool create pool sdb special sdc This command create a pool with a metadata device sdc. I want to know what role with this device sdc.Under what circumstances will it be used? More on github.com
🌐 github.com
4
2
February 27, 2023
People also ask

What metadata actually is in ZFS
Every read or write in ZFS touches several layers of structure: Dnode : per-file/dataset structure with attributes. Inode-equivalent block pointer trees : indirect blocks pointing to data. DDT entries (if dedup): hash table mapping data hashes to physical addresses. Space maps : per-vdev free-space tracking. ZIL records (sync writes only). Snapshot/clone delta structures . On a non-special-vdev pool, all of this lives intermixed with data on the same drives.
🌐
bigiron.cc
bigiron.cc › big iron › guides › zfs special vdevs: for metadata and small blocks
ZFS Special Vdevs: For Metadata and Small Blocks - Big Iron
Who needs one - workload patterns that benefit
Strongly benefits: Photo libraries (Immich, PhotoPrism, NextCloud Photos) - millions of small files, frequent metadata operations. Mail server backing storage - /var/spool/mail/ with millions of small messages. Container image stores - Docker layer storage, Podman registries. find / rsync over the pool - backup tools love metadata access. Source code / repository hosting ( Gitea / Forgejo ) - millions of small files in .git/ .
🌐
bigiron.cc
bigiron.cc › big iron › guides › zfs special vdevs: for metadata and small blocks
ZFS Special Vdevs: For Metadata and Small Blocks - Big Iron
🌐
Cr0x
cr0x.net › tuning › zfs special vdev sizing: how big it should be (so you don’t regret it)
ZFS Special VDEV Sizing: How Big It Should Be (So You Don’t Regret It) – cr0x.net
February 3, 2026 - cr0x@server:~$ zfs list -o name,used,avail,refer,mountpoint tank NAME USED AVAIL REFER MOUNTPOINT tank 120T 48T 256K /tank · Output meaning: Pool space is not the same as special vdev space. This command doesn’t show special usage directly.
🌐
Techno Tim
technotim.com › posts › special-vdev-truenas
Boost ZFS Performance with a Special VDEV in TrueNAS | Techno Tim
January 7, 2026 - Curious about how a special metadata VDEV can boost ZFS performance, especially with spinning disks? In this video, I walk through what it is, why it matters, and how to set it up in TrueNAS and some from the terminal. I’ll also share real-world benchmarks comparing pools with and without ...
🌐
Level1Techs
forum.level1techs.com › l1 articles & video-related
ZFS Metadata Special Device: Z - L1 Articles & Video-related - Level1Techs Forums
May 2, 2024 - Introduction ZFS Allocation Classes: It isn’t storage tiers or caching, but gosh darn it, you can really REALLY speed up your zfs pool. From the manual: Special Allocation Class The allocations in the special class are dedicated to specific block types. By default this includes all metadata, the indirect blocks of user data, and any deduplication tables.
🌐
Reddit
reddit.com › r/zfs › advice on the special vdev in my zfs setup
r/zfs on Reddit: Advice on the special VDEV in my ZFS setup
March 6, 2023 -

I am planning out my ZFS setup as I'm moving from snapraid. I bought a few sticks of 118gb optane to play around with and am considering using them or some high end SSD's mirrored in a special vdev. I'm considering using some 2tb sn850x's instead of optane to be able to store small blocks on the special vdev. I store mostly video and photos in my server and plan on having a 5x20tb raidz2 and a 3x8tb mirror vdev in my pool. I have 64gb of non ecc ram and my server is on a 1gbit nic. The performance improvement I want to see is faster loading of my folders as it currently takes 10-20 seconds to load the file structure and thumbnails in the worse case. Would a special vdev suit my needs or would arc and l2arc be fine enough for my needs? I would appreciate any advice on my setup.

🌐
GitHub
openzfs.github.io › openzfs-docs › man › master › 7 › zpoolconcepts.7.html
zpoolconcepts.7 — OpenZFS documentation
A pool can have any number of virtual devices at the top of the configuration (known as "root vdevs"). Data is dynamically distributed across all top-level devices to balance data among devices. As new virtual devices are added, ZFS automatically places data on the newly available devices. Virtual devices are specified one at a time on the command line, separated by whitespace.
🌐
GitHub
openzfs.github.io › openzfs-docs › man › v0.8 › 8 › zpool.8.html
zpool.8 — OpenZFS documentation
A pool must always have at least one normal (non-dedup/special) vdev before other devices can be assigned to the special class. If the special class becomes full, then allocations intended for it will spill back into the normal class. Deduplication tables can be excluded from the special class by setting the zfs_ddt_data_is_special zfs module parameter to false (0).
Find elsewhere
🌐
Cr0x
cr0x.net › fundamentals & design › zfs: when to add a special vdev (and when it’s a bad idea)
ZFS: When to Add a Special VDEV (And When It’s a Bad Idea) – cr0x.net
February 9, 2026 - In ZFS, a special vdev is a dedicated vdev class where ZFS can store metadata (and optionally small data blocks) on faster devices—usually SSDs or NVMe—while the main pool data stays on slower, higher-capacity disks.
🌐
Reddit
reddit.com › r/zfs › how do i make a metadata special device vdev?
r/zfs on Reddit: How do I make a metadata special device vdev?
January 31, 2023 -

I have 100 TB raidz1 pool that I am about to create, and I want to store the metadata for it on mirrored SSDs.

How do I create the metadata storage for only the 100TB pool (not the OS)?

zpool create -f -o ashift=12  -m /media storage \
			-o recordsize=1M \
		    -o primarycache=metadata -o secondarycache=none \
               raidz \
                  ata-ST3000DM001-9YN166_HWID \
                  ata-ST3000DM001-9YN166_HWID \
                  ata-ST3000DM001-9YN166_HWID \
                  ata-ST3000DM001-9YN166_HWID
zpool add storage -o ashift=12 special mirror /dev/ssd0n1 /dev/ssd1n1
zfs set special_small_blocks=128K storage 
🌐
Proxmox
forum.proxmox.com › home › forums › proxmox backup server › proxmox backup: installation and configuration
PBS and ZFS Special Allocation Class VDEV ... aka Fusion Drive | Proxmox Support Forum
June 14, 2024 - ===================================================== Add a special vdev. zpool add rpool -f -o ashift=12 special mirror scsi-<>-part3 scsi-<>-part3 scsi-<>-part3 Configure it. zfs set recordsize=1M rpool zfs set special_small_blocks=512K rpool ===================================================== Test results
🌐
OpenZFS
openzfs.github.io › openzfs-docs › Basic Concepts › Pool Structure › VDEVs.html
VDEVs — OpenZFS documentation
July 22, 2026 - Administrators can zpool-replace(8) a failed leaf vdev with a new one and ZFS will automatically resilver (rebuild) the data onto it to return the pool to a healthy state. See Scrub and Resilver. Vdevs are managed using the zpool(8) command-line utility.
🌐
Proxmox
forum.proxmox.com › home › forums › proxmox virtual environment › proxmox ve: installation and configuration
ZFS Metadata Special Device | Proxmox Support Forum
June 16, 2023 - Thanks in advance! Click to expand... It is more or less not much more than: zpool add POOLNAME special mirror /dev/sdX /dev/sdY and for the blocksize: zfs set special_small_blocks=1M POOLNAME
🌐
DATAZONE
datazone.de › home › news › zfs slog and special vdev: accelerate sy...
ZFS SLOG and Special VDEV: Accelerate Sync Writes and Optimize Metadata | DATAZONE
April 30, 2026 - ZFS stores metadata (directory structures, file attributes, deduplication tables) on the pool disks by default. On large pools with millions of files, metadata lookups become the bottleneck. ... # Add Special VDEV (mirror recommended) zpool add tank special mirror /dev/nvme2n1 /dev/nvme3n1 # Set threshold for small blocks (e.g., 128 KB) zfs set special_small_blocks=128k tank # Check pool status zpool status tank
Price: $$
Address: Neuhofstraße 228, 86633, Neuburg an der Donau
🌐
Cr0x
cr0x.net › tuning › zfs special vdevs on sas ssds: the pro move for metadata
ZFS Special VDEVs on SAS SSDs: The Pro Move for Metadata – cr0x.net
February 3, 2026 - Use SAS SSDs as ZFS special vdevs to accelerate metadata and small I/O safely. Learn sizing, commands, failure modes, and ops playbooks.
Top answer
1 of 2
20

In order to view the current value of a specific setting like ashift, you will need to use the zdb command instead of the zpool command.

Running zdb on its own with no arguments will give you a view of any pools found on the system, and their vdevs, and disks within the vdevs.

root@pve1:/home/tim# zdb
pm1:
    version: 5000
    name: 'pm1'
    state: 0
    txg: 801772
    pool_guid: 13783858310243843123
    errata: 0
    hostid: 2831164162
    hostname: 'pve1'
    vdev_children: 1
    vdev_tree:
        type: 'root'
        id: 0
        guid: 13783858310243843123
        children[0]:
            type: 'raidz'
            id: 0
            guid: 13677153442601001142
            nparity: 2
            metaslab_array: 34
            metaslab_shift: 33
            ashift: 9
            asize: 1600296845312
            is_log: 0
            create_txg: 4
            children[0]:
                type: 'disk'
                id: 0
                guid: 4356695485691064080
                path: '/dev/disk/by-id/ata-DENRSTE251M45-0400.C_A181B011241000542-part1'
                whole_disk: 1
                not_present: 1
                DTL: 64
                create_txg: 4
            children[1]:
                type: 'disk'
                id: 1
                guid: 14648277375932894482
                path: '/dev/disk/by-id/ata-DENRSTE251M45-0400.C_A181B011241000521-part1'
                whole_disk: 1
                DTL: 82
                create_txg: 4
            children[2]:
                type: 'disk'
                id: 2
                guid: 11362800770521042303
                path: '/dev/disk/by-id/ata-DENRSTE251M45-0400.C_A181B011241000080-part1'
                whole_disk: 1
                DTL: 59
                create_txg: 4
            children[3]:
                type: 'disk'
                id: 3
                guid: 10494331395233532833
                path: '/dev/disk/by-id/ata-DENRSTE251M45-0400.C_A181B011241000517-part1'
                whole_disk: 1
                DTL: 58
                create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data

or, for just ashift with some context:

root@pve1:/home/tim#  sudo zdb | egrep 'ashift|vdev|type' | grep -v disk
    vdev_children: 1
    vdev_tree:
        type: 'root'
            type: 'raidz'
            ashift: 9

Here is an old blog post about zdb that is still very informative about the origins and intent, and the information that comes out of zdb. A quick google also reveals many posts that may be more specifically relevant to ZFS on Linux.

2 of 2
3

As of today (ZFS 2.2.4), it is possible to get and set vdev properties:

zpool get <property> <pool> <vdev>

The keyword all-vdevs can be used to specify all vdevs. For example, to get ashift for all vdevs:

zpool get ashift rpool all-vdevs
🌐
Reddit
reddit.com › r/zfs › guidance on how the special vdev performs.
r/zfs on Reddit: Guidance on how the special vdev performs.
January 31, 2022 -

Racking my brain trying to figure out it's actual behavior. No documentation anywhere I can find actually comments on this.

The special vdev is used to store metadata from the pool, so operations like directory listing are at the speed of an NVME drive, which will also improve the performance of spinning rust by reducing the load of small IO required to lookup the pool metadata. This makes sense.

What confuses me is the behavior of the small block allocation class when the special vdev is at capacity. There seems to be a few scenarios that aren't talked about anywhere in the documentation that would be important for performance in various scenarios.

From how i've seen it talked about my understanding is that it acts as a write back cache for small IO based on the special_small_blocks value of the dataset, and when the special vdev is at capacity or needs more space for metadata, small block allocation is offloaded back to the data vdevs.

However i've never actually seen this mentioned anywhere. From the closest thing to a confirmation is in the TrueNas documentation on a fusion pool saying

If the special class becomes full, then allocations spill back into the normal class.

By spill, does that mean, removed from the special class so new incoming small block writes are added to the cache. Or does that mean when the special class is full small IO is sent directly back to the normal class bypassing the special vdev.

Top answer
1 of 2
12
I have answered my own question eventually, partially from some old forum posts and practical testing. Once the special VDEV is full it stays full. The standard small block allocation is 75% of the drive space(default but adjustable), after that further blocks will be sent directly to the backing storage. It can be expanded after the fact, and also striped to another vdev if space becomes constrained, but there is no way to rebalance. Furthermore, the special vdev hits it's allocation limit and later expanded, the small IO will have to be re-written to move it back to it's allocation class. As long as you're using all striped mirrors in your pool with the same ashift(I am), you can "flush" the metadata and small IO back to your main pool by removing the special vdev. I.E zpool remove pool mirror- with x being the mirrored SSD's vdev name This will write all metadata and small blocks back to the main pool. But it also prevents the special vdev from being useful again if re-added without re-writing all your data as new metadata is only added on write.
2 of 2
3
What confuses me is the behavior of the small block allocation class when the special vdev is at capacity. There seems to be a few scenarios that aren't talked about anywhere in the documentation that would be important for performance in various scenarios. When it reaches 75% full, all further small blocks writes go to regular vdevs like normal. Current blocks stay where they are. The remaining space will be reserved for just metadata. Adjust the percentage here: /etc/modprobe.d/zfs.conf by adding zfs_special_class_metadata_reserve_pct=10% for example and rebooting. There might be a way to do it live (only lasts until reboot) I think, but I forget right now. You can search the source code here for "zfs_special_class_metadata_reserve_pct" and find what it touches. Sometimes there are comments you might find helpful. Searching the github issue and pull requests can also be helpful. You're probably thinking that all of this is horribly documented and scattered around like marbles. You're right. As long as you're using all striped mirrors in your pool with the same ashift(I am), you can "flush" the metadata and small IO back to your main pool by removing the special vdev Make absolutely certain you have things backed up and confirmed good (scrubbed) before you try this. This has rarely resulted in problems, mercenary_sysadmin saw corruption I believe when he did some testing when special vdevs first came out. The issue was never resolved and I don't know if he's revisited it. I personally won't trust vdev removal for a long time. Also I do highly recommend a triple mirror as the smallest you consider. Preferably on old enterprise ssds with real PLP (visible capacitors), which are cheap enough to find on ebay. Here's some other references for those curious about special vdevs. Various findings: https://forum.level1techs.com/t/zfs-metadata-special-device-z/159954 Generate a histogram of block sizes Calculate data from ZDB output Clarifies behavior of what goes where when setting small block size: https://github.com/openzfs/zfs/issues/9131#issuecomment-528562601 Small blocks can now be set to anything you can set recordsize to (512B-1M) and even bigger if you enable larger recordsizes (up to 16M): https://github.com/openzfs/zfs/pull/9355
🌐
GitHub
github.com › openzfs › zfs › discussions › 17798
Special vdev for metadata and small blocks on raidz not just mirror · openzfs/zfs · Discussion #17798 · GitHub
If I do that, will the special vdev simply inherit/adopt the pool’s settings (like encryption)? ... zpool add <pool_name> special raidz2 <ssd_device1> <ssd_device2> …<ssd_deviceN> zfs set special_small_blocks=16M <pool_name> zfs rewrite -rvx /mnt/pool_name
🌐
Proxmox
forum.proxmox.com › home › forums › proxmox virtual environment › proxmox ve: installation and configuration
ZFS and special device how to use unused free space ? | Proxmox Support Forum
May 18, 2020 - In this case, the zpool remove command initiates the removal and returns, while the evacuation continues in the background. The removal progress can be monitored with zpool status. If an IO error is encountered during the removal process it will be cancelled. The device_removal feature flag must be enabled to remove a top-level vdev, see zpool-features(5).