Reddit
reddit.com › r/proxmox › how long before deduplication on pbs kicks in?
r/Proxmox on Reddit: How long before deduplication on PBS kicks in?
September 29, 2023 -
I just started a pbs server to check out how it works. I have 8 vm's at 50GB each, and each has 5 snapshots. Each snapshot appears to be the full 50GB and the dashboard is giving me a dedupe factor of 1. I have the backup retention set to last 14. How long before I should see the deduplication saving space?
Top answer 1 of 3
13
Run a garbage collection. After that you see the dedup in Datastores => Your Datastore Or Just read it at the last 5-10 Lines of the Garbage collect logs. For me its from 6 TB Data to 800 GB on disk And its activated by default
2 of 3
5
Deduplication in PBS is managed externally, much like in Seafile. This means that the underlying file system used for PBS storage doesn't need to support deduplication. In fact, enabling deduplication at the file system level would be redundant and could waste resources.
NimbusBackup
nimbus.rdem-systems.com › rdem systems › backups › home › blog › pbs deduplication
PBS deduplication: how it works, real ratios
3 weeks ago - An encrypted chunk cannot be identified by the fingerprint of its encrypted content, otherwise nothing would deduplicate. PBS computes the fingerprint on the plain data, concatenated with a key derived from your encryption key (in the code: SHA-256(data ‖ id_key), with an id_key derived by PBKDF2, pbs-tools/src/crypt_config.rs).
06:22
Proxmox Backup Server Magic: 11TB to 84GB with Block-Level ...
10:38
You should be using Proxmox Backup Server - YouTube
21:57
How to setup PBS to backup your Proxmox VMs and Containers - YouTube
43:38
Getting the most out of Proxmox Backup Server: Backing up other ...
Does PBS deduplication work with compressed vzdumps?
Yes. PBS performs client-side chunk-level deduplication regardless of compression settings. Each backup is chunked and hashed before storage, so compression settings on the Proxmox host don't affect deduplication ratios.
industrialmonitordirect.com
industrialmonitordirect.com › industrial monitor direct › knowledge base › proxmox backup server vs restic: vm backup strategy comparison
Proxmox Backup Server: Solve VZDump Deduplication vs Compression ...
Does deduplication work across namespaces in the same datastore?
Yes. Namespaces are a logical separation for access control and organization. The chunk store is shared across all namespaces in a datastore, so identical chunks from different namespaces are stored once. If you need strict storage isolation between tenants, use separate datastores, not separate namespaces.
remote-backups.com
remote-backups.com › home › blog › inside the pbs chunk store: how dedup works
Inside the PBS Chunk Store: How Dedup Works
Does client-side encryption prevent deduplication?
No, as long as the key is the same. The fingerprint of an encrypted chunk is computed on the plain data and a key derived from the encryption key: the same data with the same key gives the same fingerprint, and deduplicates. With two different keys, or between an encrypted and an unencrypted backup, there is no deduplication at all.
nimbus.rdem-systems.com
nimbus.rdem-systems.com › rdem systems › backups › home › blog › pbs deduplication
PBS deduplication: how it works, real ratios
Remote-backups
remote-backups.com › home › blog › inside the pbs chunk store: how dedup works
Inside the PBS Chunk Store: How Dedup Works
May 25, 2026 - Typical dedup ratios: 1.2-2x for a single VM, 3-5x for a small homelab, 8-12x for an MSP with similar OS stacks. This is post 3 in the How Proxmox Backup Server Works Under the Hood series.
Proxmox
pbs.proxmox.com › docs › technical-overview.html
Technical Overview — Proxmox Backup 4.2.8-1 documentation
The deduplication of datastores is based on reusing chunks, which are referenced by the indexes in a backup snapshot.
Stackscale
stackscale.com › home › business continuity › proxmox backup server in depth: deduplication, verification and retention in a private cloud
Proxmox Backup Server in depth: deduplication, verification, retention
September 16, 2026 - Here lies the most common misunderstanding about PBS, and the one that fills repositories: prune does not free space. Prune applies the retention policy and decides which snapshots are no longer needed. It removes the reference to the backup, but it does not touch the blocks: they are deduplicated and very likely still in use by another copy.
Reddit
reddit.com › r/proxmox › what affects dedupe factor on pbs backups??
r/Proxmox on Reddit: What affects dedupe factor on PBS backups??
July 5, 2025 -
Just got this up and running yesterday. Set it for daily backups. It did the first full backup and this morning did the first incremental. Shows only a value of 3.04. TechnoTim had a value like 64. Not sure what he was doing for that but I do think he was doing like hourly backups so that maybe is why. Anyhow just curious what things I could do to possibly increase this value?
On that note, does the garbage collection schedule make any difference? Right now I am doing on PBS pruning for 7 dailys and 2 weeklys (pve job retention is off) and garbage collection every 6hrs. Not sure if this impacts anything but wanted to mention.
I guess what is odd (more with the incremental aspect). I haven't changed a single thing on any of my 10 CTs yet it got an incremental backup and took an hour. Still better than the almost 3hrs it too for a full backup but confused why it was so much if it's just the difference. Since nothing changes shouldn't it be close to nothing?
Top answer 1 of 6
19
> The deduplication of datastores is based on reusing chunks, which are referenced by the indexes in a backup snapshot. This means that multiple indexes can reference the same chunks, reducing the amount of space needed to contain the data (even across backup snapshots). https://pbs.proxmox.com/docs/technical-overview.html#datastores If you want to understand the in and outs behind the curtain I suggest reading the technical docs, they are great to understand everything
2 of 6
4
depuplication allows for only one copy of a file to be backed up when the might be might multiple copies across many backups. So if you've got 5 x Windows 11 VMs, 3 x Ubuntu 25.10, 3 x Debian 12 you've gonna have a high depub factor because there will be lots of files common to each so only it only needs to be backup once. On the otherhand if you 2 x Win 11, 1 x Win10, 1 x Win Server, 2 x Ubuntu 25.10, 1 x Debian 12, 1 x Arch you're going to be much lower on the dedupe cos there's not as many common files. When you run a garbage collection, the file aren't actually deleted for another 24hrs by default. There's a setting you can change that was mentioned in here recently, that I changed on my PBS but damned it I can remember where it was :(
Proxmox
forum.proxmox.com › home › forums › proxmox backup server › proxmox backup: installation and configuration
Proxmox & deduplication? | Proxmox Support Forum
January 23, 2024 - Click to expand... In theory yes, but in practice its not that easy. For VMs, PVE will chop that virtual disk in 4MB chunks, compress them, encrypt them and send them to the PBS. Only if every bit of those 4MB is identical it will be deduplicated. So won't help that much for small files.
YouTube
youtube.com › shorts › EJHGI7wpk0c
Proxmox 101 #16 — PBS Deduplication: 30 Backups, 1× the Space - YouTube
Proxmox 101, episode 16: Proxmox Backup Server (PBS) deduplication — why 30 nightly backups need barely more space than one. Content-addressed chunks, increm...
Published: September 7, 2026
Industrialmonitordirect
industrialmonitordirect.com › industrial monitor direct › knowledge base › proxmox backup server vs restic: vm backup strategy comparison
Proxmox Backup Server: Solve VZDump Deduplication vs Compression – Industrial Monitor Direct
May 23, 2026 - Restic Alternative: If PBS integration is not feasible, configure a pipeline: compressed vzdump → extract to named pipe → restic backup from pipe. This preserves deduplication without persistent uncompressed storage but requires custom scripting and monitoring.
Remote-backups
remote-backups.com › home › blog › pbs vs veeam: when open source wins
PBS vs Veeam: When Open Source Wins
April 4, 2026 - PBS uses content-addressed, chunk-level deduplication with variable-length chunks split at content-defined boundaries. This means similar files — same OS base, same application binaries — produce identical chunks regardless of offset shifts ...
Reddit
reddit.com › r/proxmox › pbs deduplication zfs only?
r/Proxmox on Reddit: PBS deduplication zfs only?
June 8, 2022 -
Hi,
so im currently testing the proxmox backup server für my vms. I was wondering if the deduplication / dirtybit only works with zfs or if it also works on "normal" ext4 formated disks (mounted directory).
Best
Top answer 1 of 3
2
The normal mode of operation for PBS is to run on the hypervisor host and carry out a block-level backup of the VM/LXC virtual disks. Since it's a block level backup the filesystem on the virtual disk is irrelevant at the time it is captured. However a key feature of PBS is that you can browse and extract individual files. That's only going to be possible if the PBS server can read the filesystem inside the backup. The backups created by PBS are de-duplicated. ZFS provides for de-duplication of the primary storage (i.e. where the live vmdk lives when the VM is running). Also, as Qemu is able to track the dirt blocks, when PBS creates a VM backup, it only needs to read/write the blocks changed since the last backup. This does not work for lxc's - where PBS creates an image of the whole device then compares this with the most recent backup to get the changed blocks. There's also proxmox-backup-client which reads the local devices and sends them to a PBS server. I've not used this myself, but I believe it is currently only available for Linux and in the absence of a mechanism for tracking dirty blocks, will behave similarly to an LXC backup. But since the host where the backup client is running cannot be quiesced you'd be looking at crash-recovery on restore.
2 of 3
1
As far as I know yes, ZFS dedup is a part from the ZFS FS so it makes sense, the entire server is build for you to use ZFS
NodeSpace
help.nodespace.com › knowledgebase.php
What is the Deduplication Factor in Proxmox Backup Server?
After you run some backups you will start to see the Deduplication Factor increase. For example, you might see 10.34. This means that without deduplication, you would need 10.34x more storage space to backup the same data.
Derek Seaman
derekseaman.com › home › how to: setup synology nfs for proxmox backup server datastore
How To: Setup Synology NFS for Proxmox Backup Server Datastore
April 21, 2023 - After a few weeks of usage my PBS dedupe ratio was an amazing 64:1 and keeps climbing.
Proxmox
forum.proxmox.com › home › forums › proxmox backup server › proxmox backup: installation and configuration
[SOLVED] - Deduplication? | Proxmox Support Forum
October 21, 2020 - Yes it needs to cache it in memory to be fast. But you can store the dedup table also on a special device (i.e. a ssd) > Many setups to not use ZFS Could't you just let pbs create one or more big files on any storage backend and create a zpool->zfs over it (all done on the fly by pbs).
Proxmox
forum.proxmox.com › home › forums › proxmox backup server › proxmox backup: installation and configuration
ZFS Datastore on PBS - ZFS Recordsize/Dedup... | Proxmox Support Forum
May 9, 2024 - It looks to me like ZFS Deduplication is not really needed for PBS, because PBS does already the deduplication. I have on my Pool with a lot of backups a dedup ratio of 1,1. So dedup happens a bit on ZFS side, but seems to me not worth it. Same for ZFS-Compression, runs here with ZSTD as Compression algorithm.
NimbusBackup
nimbus.rdem-systems.com › rdem systems › backups › blog › proxmox backup guide
Proxmox Backup Best Practices 2026: PBS verify, 3-2-1-1-0
May 13, 2026 - Use PBS, not vzdump alone — block-level deduplication stores each identical block only once; a measured example: on our MariaDB bench, 97.82% of a dump's data was already present 24 h after the previous one (30/09/2026).
YouTube
youtube.com › watch
Save Up to 90% Disk Space: Mastering Deduplication with PBS - YouTube
"If you’re still backing up your Proxmox Virtual Environment to a basic NAS, you might be wasting time, storage, and efficiency.In this video, we break down ...
Published: March 26, 2026
HorizonIQ
horizoniq.com › home › blog › proxmox backup server: what it is and why you should use it
Proxmox Backup Server: What It Is and Why You Should Use It | HorizonIQ
August 28, 2025 - The introduction of the PBS marked a significant improvement in backup strategies for Proxmox VE users, offering incremental backups, deduplication, and a more modern approach to data management.
Proxmox
forum.proxmox.com › home › tags
deduplication | Proxmox Support Forum
Hey there, I've recently set up PBS to serve as a central backup server in my network, but I've run into quite an inconvenience: Whenever I make a backup of e.g. my home directory, it's taking quite a while since it has to transfer like 200+ gigs over the network. With about 120MBps (seemingly... ... Hello, Proxmox community! I have a query that I've been unable to resolve for the past weeks. My deduplication ratio on Proxmox Backup Server (PBS) has significantly decreased.
Cloud-PBS
cloud-pbs.com › home › tools › pbs storage and cost calculator
PBS Storage & Cost Calculator — Proxmox Backup Sizing Tool | Cloud-PBS
PBS content-defined chunking deduplicates across backups.