You can't extend the existing raidz1 vdev by adding another disk but you can add another vdev to the pool to increase the pool's capacity. You'll need more than one additional disk if you want to retain redundancy. For example, you could use two disks to add a mirror vdev or 3+ disks for another raidz1 vdev.
Answer from Mike Fitzpatrick on Stack ExchangeYou can't extend the existing raidz1 vdev by adding another disk but you can add another vdev to the pool to increase the pool's capacity. You'll need more than one additional disk if you want to retain redundancy. For example, you could use two disks to add a mirror vdev or 3+ disks for another raidz1 vdev.
I know this question is old, but I just came across it. It looks like the ability to extend a RAIDZ one drive at a time is in the works! No ETA that I've seen though.
https://www.freebsdfoundation.org/blog/openzfs-raid-z-online-expansion-project-announcement/
Hi, i have a zfs mirror of 4TB drives and i want to add a third 4TB drive. Is it possible to turn zfs mirror to raid z1 without loosing my data?
Update:
so i know i cant turn a mirror to a z1 but how hard is it to add drives to raid z1? for example from 3 to 4
Hi guys! I almost finished my home NAS and now choosing the best topology for the main data pool. For now I have 4 HDDs, 10 Tb each. For the moment raidz1 with a single vdev seems the best choice but considering the possibility of future storage expansion and the ability to expand the pool I also consider a 2 vdev raidz1 configuration. If I understand correctly, this gives more iops/write speed. So my questions on the matter are:
If now I build a raidz1 with 2 vdevs 2 disks wide (getting around 17.5 TiB of capacity) and somewhere in the future I buy 2 more drives of the same capacity, will I be able to expand each vdev to width of 3 getting about 36 TiB?
If the answer to the first question is “Yes, my dude”, will this work with adding only one drive to one of the vdevs in the pool so one of them is 3 disks wide and another one is 2? If not, is there another topology that allows something like that? Stripe of vdevs?
I used zfs for some time but only as a simple raidz1, so not much practical knowledge was accumulated. The host system is truenas, if this is important.
There are basically two ways of growing a ZFS pool.
Add more vdevs
This is what user1133275 is suggesting in their answer. It's done with zpool add (which has basically the same syntax as zpool create does for specifying storage), and it works well for what it does.
ZFS won't rebalance your stored data automatically, but it will start to write any new data to the new vdevs until the new vdev has about the same usage as the existing one(s).
Once you've added a vdev to a pool, you basically cannot remove it without recreating the pool from scratch.
All vdevs in a pool need to be above their respective redundancy thresholds for the pool to be importable. In other words, every vdev needs to be at least DEGRADED for the pool to function.
Replace disks with larger ones
This is what you're discussing in your question. It's the normal way of growing a ZFS pool when you have a pool layout that you are happy with.
To replace a device with a new one, the new device needs to be at least as large as the old one.
Operationally, you'd hook up the new disk along with the old, and then zpool replace the old disk with the new one. (This creates a temporary replacing device which becomes a parent to the old and new disk; when the resilver completes, the replacing device is removed from the device tree and it looks like the new device was there all along.) Once the resilver completes, the old disk can be removed from the system.
Once all disks in a vdev are replaced by larger ones, you can expand the pool by running zpool online -e or by having the autoexpand property set to on (though I wouldn't really recommend the latter; pool expansion should be a conscious decision).
So which way is better?
That basically depends on your pool. As mentioned, the downside to having multiple vdevs is that they all need to be functional, so by adding vdevs you are actually, in a sense, reducing your safety margin. The upside, though, is that it's much easier to grow the pool piecemeal. Replacing devices in-place is basically the opposite; you don't need to keep as many vdevs functioning, but it isn't as easy to grow a pool piecemeal.
For me, frankly, assuming for a second that you're using rotational hard disks (since this seems like bulk storage), 20 TB is still well within reason for a single vdev pool. My suggestion in your situation would be to get six drives of the 8 TB variety, and to set those up in a single raidz2 vdev. Doing so gives you a net storage capacity of around 32 TB, thus leaving you with an initial about 35% free, and the ability to lose any two drives before any of your data is at significant risk. You could also consider running eight 6 TB drives for a net storage capacity of around 36 TB and starting out at 45% free. (I'd consider 6-8 drives to be slightly on the large end for raidz1, but fine for raidz2.) Then plan to replace those drives either on a 4-5 year schedule (due to wear) or whenever the pool goes above about 80% full (because ZFS is much, much happier when it has good headroom). If your figures are accurate, you should be replacing those drives due to wear well before your pool starts getting full, while still allowing for a reasonable amount of unexpected growth in storage needs. When you replace the drives, you can decide whether you're happy with the pool size you've got based on then-current usage, or if you want to get larger drives and expand the pool.
In addition to the options in the excellent answer above, there will soon be another option.
The OpenZFS project (ZFS on Linux, ZFS on FreeBSD) is working on a feature to allow the addition of new physical devices to existing RAID-Z vdevs. This will allow, for instance, the expansion of a 6-drive RAID-Z2 vdev into a 7-drive RAID-Z2 vdev. This will happen while the filesystem is online, and will be repeatable once the expansion is complete (e.g., 7-drive vdev → 8-drive vdev).
As of September 2020, this feature is still in development: https://github.com/openzfs/zfs/pull/8853
As of October 2022, this feature is still in development: https://github.com/openzfs/zfs/pull/12225
As of July 2023, this feature is still in development: https://github.com/openzfs/zfs/pull/15022
As of November 2023, this feature was merged in main and is scheduled for release in OpenZFS 2.3: https://github.com/openzfs/zfs/pull/15022
Hello folks,
I'm very new to ZFS, so I'd like to apologize in advance if any of my questions have been answered already, but I couldn't find any post related to my questions.
I've built a zpool with a single raidz1 vdev containing 4x4TB disks. I'm thinking of what'll happen once I need to increase the size of this zpool or if I want to add more disks in the future for better reliability.
Given my setup, if I want to increase the size of my pool as well as having another disk for parity and be able to loose two disks on my pool, would adding another raidz1 vdev of 4x4TB be a good option? Would this give me two parity disks? can two disks on the same vdev fail in this scenario?
If anyone has better suggestions that would be very much appreciated. The end goal is once I need more space and can afford buying more disks, I want to be able to have two parity disks and more storage capacity.
Thank you!
Scenario...(I assume 3+ drives is going to be required for this scenario, not 2)...
I have 3 individual ZFS drives/partitions, which would be three vdev's.
Is it possible to pool the three drives/vdev's into a single pool without destroying the data?
I'm thinking of beginning my ZFS journey one step at a time, and wanted to start with individual ZFS drives/vdevs first. Then later, as I know more, plan more, buy more drives, then create the ZFS pool for full RAIDZ protection.
If not capable now, will the upcoming RAIDZ expansion have the capability of merging drives into a new pool, without data loss?
Or is ZFS storage planning set in stone? And I better know what I want, because there's no way to change the storage?
So I am currently building my Ugreen nasync dxp4800(4 possible bays) and while I got my ssds ready, I still need some HDDs. However my setup was already quite expensive and I want to start with 2 or 3 HDDs and add the rest later. How easy is it to add a drive later and add it to an existing pool and is this already possible in proxmox? Which raid level would you recommend?
You can't expand an existing raidz vdev, you have to blow it away and create it again with the new drive(s). See the other answer for better details.
Side note: Someone actually worked out that it's technically possibly to add drives to a raidz, but the functionality hasn't been implemented. The same is true of removing a disk.
See this thread from one of the FreeBSD lists: http://lists.freebsd.org/pipermail/freebsd-fs/2009-June/006327.html
To quote Dan Naumov,
To reiterate, you cant just add a single disk drive to a raidz1 or raidz2 pool. This is a known limitation (you can check with SUN ZFS docs). If you have an existing raidz and you MUST increase that particular pool's storage capabilities, you have 3 options:
1) Add a raidz of the same configuration to the pool (think 3 disk raidz + 3 disk raidz or 5 + 5, for example)
2) Replace each (and every) disk in your raidz pool one by one, letting it resilver after inserting each upgraded disk
3) Backup your data, destroy your pool and create a new raidz pool with a bigger amount of disks.