May 9, 2024 - It looks to me like ZFS Deduplication is not really needed for PBS, because PBS does already the deduplication. I have on my Pool with a lot of backups a dedup ratio of 1,1. So dedup happens a bit on ZFS side, but seems to me not worth it. Same for ZFS-Compression, runs here with ZSTD as Compression algorithm.

January 11, 2024 - No, you won't get any benefits of deduplicating data again. You may profit from ZFS compression, but that's enabled by default. You can look up the compression ratio by running zfs get compressratio ... Best regards, Folke Do you already have a Commercial Support Subscription? - If not, Buy now and read the documentation ... Just to add: de-duplication advantage on PBS is beneficial because the second and following backups will not use space if the chunks already exists in the datastore, not just because it is able to de-duplicate data in the same backup.

Therefore I can either: dedicate all of my SSDs and HDDs to one large PBS that is remote from my PVEs, or split my resources: locally (with the PVEs) place... ... There are a large amounts of improvements in ZFS 2.3.0 if you use NVME and deduplication: https://github.com/openzfs/zfs/releas...

August 31, 2025 - It also requires more ram for the dedup table. Click to expand... i definitely noticed that one, i got the most out of it at 4k but the slower file transfers, ram usage and all that came with it and nullifying compression kind of nuked any value. ... i have been considering PBS, but i havent got around to it and i only have a spare HP SFF pc with 2x500GB drives i could toss in, ive been debating if its even worth it at all to run a whole PC just for 500GB mirrored zfs as a backup, it doesn't really seem like it will be.

March 8, 2023 - Die schnellen inkrementellen Sicherungen und hohe Platzersparnis liegen nur an der Deduplizierung. ... Da der PBS aber für seine Datastores eine eigene Konsistenzprüfung mitringt, ist die Selfhealing-Funktion von zfs eine doppelte Absicherung, ...

October 21, 2020 - ZFS only works as long as you have access to a block device. If you are using NFS or other network file systems you are once again limited by this. I don't really see a disadvantage with PBS doing deduplication itself, it keeps it a lot more ...

March 19, 2024 - Once you've enabled dedup on a dataset, it's not trivial to disable it: the data needs to be moved out of the dedup-enabled dataset and that dataset needs to be destroyed. If your workload needs even a moderate level of performance, dedup will likely result in unacceptably low performance.

January 14, 2024 - PBS also already does deduplication and ZFS deduplication is · strongly discouraged - so nothing to be gained there as well. The biggest benefit would be that, with a redundant disk setup, you would gain self-healing - which is something PBS ...

February 3, 2026 - ZFS is, I think, the best way to make leftover drives into one or more pools, even without redundancy. PBS includes dedup chunking and other cool benefits.

December 8, 2023 - PBS relies heavily on cryptography, so does ZFS - checksums everywhere As the backups chunks are encrypted and hashed at the client, your backup speed probably don't improve a lot, depending on the load of the current machines (i mean, its an Opteron), but Verify Jobs for example will greatly benefit from a modern and strong CPU.

March 18, 2023 - This is internal in ZFS with the tooling of ZFS, so while it takes steps over it happening with a simple command or entirely automatically, it's "internal with steps". This can be done without touching the pre-expansion data until everything is done and over with, so you have all opportunities to test the data before swapping it out. So rebalance should not be "no", but "internal with steps". Pretty much same deal with "disable deduplication feature in the file system possible".

May 11, 2019 - VMA itself is not compressed.. but the content inside will be out of order, so the alignment between ZFS records and VMA extents must be there for deduplication to work (this is also the reason why just overwriting the vma file in place and relying on ZFS snapshots/CoW doesn't work). VMA internally uses 64k-sized clusters, and 59 of those + a 512 byte header make up one extent. there's also a header up front that is variable size (containing the metadata of the backup and things like contained config file data). now you can probably guess why deduplication doesn't work very well with your approach if you want to deduplicate, you probably want to do it on the original data (like PBS does -> use PBS!

July 26, 2026 - Hi, The standing advice for years has been to leave dedup=off on ZFS pools under Proxmox: the DDT lives in RAM, the write path suffers, and you can't turn it back off for data already written. OpenZFS 2.3 ships fast dedup, which is supposed to address most of that — quota'd DDT, pruning, dedup logs.

November 9, 2015 - Click to expand... deduplication is a block level feature. It knows nothing about file systems or images. ... Click to expand... I know but how is going to save me space when hard drives of VM-s are fixed in space without thin provision ? ... If the checksum of one or more blocks are identical only one block will be physical stored while the rest of the blocks is replaced with a reference. Since ZFS is copy-on-write there is no maintenance overhead with this since a write will either result in a block or a reference to an existing block.

May 25, 2023 - Solange der gleiche Key verwendet wird, sollte gut dedupliziert werden können. Andere Keys liefern andere Daten, die am PBS gespeichert werden.

July 13, 2019 - Only after setting it, all previously written blocks are still in it. I have no ballpark number if this improves the read/write time a lot, because every I/O is still routed through the dedup table. ... thanks LnxBil, that's what I thought. i can zfs send/receive or just move it in and out of ZFS.

November 12, 2022 - Hi there, some time ago I had to change our PBS backup server from spinnning disks to SSDs because the verify jobs on all the data stores needed too much time, so they overlaped and ended in a never ending queue. After running some tests with ZFS special devices, this could maybe a solution to...

December 3, 2021 - So, whats the benefit of using zfs as datastore for pbs? IMHO it just makes things slower. The only reason for using ZFS is when you use multiple disks / devices for one pool, or did i miss something?

July 26, 2026 - Und auch mit PBS und VMA/VZDump kommt man um Restore-Tests nicht herum. EDIT: Alternativer RAW Modus hinzugefügt. RAW benötigt am wenigsten Platz auf dem Zieldatenträger, da es hervorragend dedupliziert wird ... RAW spart Schreib I/O ggü. VZDump, aber nur wenig Zeit · RAW spart keine Lese I/O ggü. VZDump · zfs_send benötigt erheblich weniger Lese und Schreib I/O

May 9, 2024 - ZSTD -> 1GB/s limit LZO -> Didn't tested, but i don't expect a differnce. GZIP -> Around 500MB/s none -> 4,5-5 GB/s PBS: (PVE is always the Genoa 9374 Node)