Managing ZFS Storage Pools
ZFS storage pools can be created, destroyed, and modified using the zpool command. Let's start by destroying an existing pool.
zpool destroy storagepool
[root@server ~]# zpool destroy storagepool
[root@server ~]# zpool status
no pools available
Now, let's create a simple mirrored pool. The mirror keyword allows for redundancy.
zpool create -f datapool mirror disk1 disk2 mirror disk3 disk4
[root@server ~]# zpool create -f datapool mirror sdc sdd mirror sde sdf
[root@server ~]# zpool status
pool: datapool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
datapool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
sde ONLINE 0 0 0
sdf ONLINE 0 0 0
errors: No known data errors
Similarly, a RAID-Z pool can be created for parity-based redundancy.
zpool create -f paritypool raidz sdc sdd sde sdf
[root@server ~]# zpool create -f paritypool raidz sdc sdd sde sdf
[root@server ~]# zpool status
pool: paritypool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
paritypool ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
sdf ONLINE 0 0 0
errors: No known data errors
Modifying ZFS Pools
Once a pool is created, devices can be added or removed, but only certain types of devices can be removed from specific pool configurations. Leet's create a basic pool and then add a device.
zpool create -f basepool sdc sdd
zpool add basepool sde
[root@server ~]# zpool create -f basepool sdc sdd
[root@server ~]# zpool add basepool sde
[root@server ~]# zpool status
pool: basepool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
basepool ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
errors: No known data errors
Attempting to remove a data device from a non-redundant pool will fail.
zpool remove basepool sde
[root@server ~]# zpool remove basepool sde
cannot remove sde: only inactive hot spares, cache, top-level, or log devices can be removed
However, a hot spare can be added and then removed.
zpool add basepool spare sdf
zpool remove basepool sdf
[root@server ~]# zpool add basepool spare sdf
[root@server ~]# zpool status
pool: basepool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
basepool ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
spares
sdf AVAIL
errors: No known data errors
[root@server ~]# zpool remove basepool sdf
[root@server ~]# zpool status
pool: basepool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
basepool ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
errors: No known data errors
For mirrored pools, devices can be attached or detached using the attach and detach commands.
zpool attach
zpool detach
If a device fails, it can be replaced using the replace command.
zpool replace
Let's simulate a device failure in a mirrored pool and then replace it.
zpool create -f testpool mirror sdd sde
dd if=/dev/zero of=/dev/sdd
zpool scrub testpool
zpool replace testpool sdd sdc
[root@server ~]# zpool create -f testpool mirror sdd sde
[root@server ~]# dd if=/dev/zero of=/dev/sdd
dd: writing to '/dev/sdd': No space left on device
2048001+0 records in
2048000+0 records out
1048576000 bytes (1.0 GB) copied, 22.4804 s, 46.6 MB/s
[root@server ~]# zpool scrub testpool
[root@server ~]# zpool status
pool: testpool
state: ONLINE
status: One or more devices could not be used because the label is missing or
invalid. Sufficient replicas exist for the pool to continue
functioning in a degraded state.
action: Replace the device using 'zpool replace'.
see: http://zfsonlinux.org/msg/ZFS-8000-4J
scan: scrub repaired 0 in 0h0m with 0 errors on Fri Mar 18 09:59:40 2016
config:
NAME STATE READ WRITE CKSUM
testpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
sdd UNAVAIL 0 0 0 corrupted data
sde ONLINE 0 0 0
errors: No known data errors
[root@server ~]# zpool replace testpool sdd sdc; zpool status
pool: testpool
state: ONLINE
scan: resilvered 83.5K in 0h0m with 0 errors on Fri Mar 18 10:05:17 2016
config:
NAME STATE READ WRITE CKSUM
testpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
replacing-0 UNAVAIL 0 0 0
sdd UNAVAIL 0 0 0 corrupted data
sdc ONLINE 0 0 0
sde ONLINE 0 0 0
errors: No known data errors
[root@server ~]# zpool status
pool: testpool
state: ONLINE
scan: resilvered 74.5K in 0h0m with 0 errors on Fri Mar 18 10:00:36 2016
config:
NAME STATE READ WRITE CKSUM
testpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
sdc ONLINE 0 0 0
sde ONLINE 0 0 0
errors: No known data errors
Migrating ZFS Pools
ZFS pools can be migrated between hosts using the export and import commands. The disks used by the pool must be available on both systems.
zpool export testpool
zpool import
zpool import testpool
[root@server ~]# zpool export testpool
[root@server ~]# zpool status
no pools available
[root@server2 ~]# zpool import
pool: testpool
id: 3823664125009563520
state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:
testpool ONLINE
sdc ONLINE
sdd ONLINE
sde ONLINE
[root@server2 ~]# zpool import testpool
[root@server2 ~]# zpool status
pool: testpool
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
testpool ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
errors: No known data errors
Monitoring Pool I/O
The iostat command provides I/O statistics for pool devices.
zpool iostat -v testpool
[root@server ~]# zpool iostat -v testpool
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
testpool 1.80M 2.86G 22 27 470K 417K
sdc 598K 975M 8 9 200K 139K
sdd 636K 975M 7 9 135K 139K
sde 610K 975M 6 9 135K 139K
---------- ----- ----- ----- ----- ----- -----
Managing ZFS Filesystems
ZFS filesystems are created and managed using the zfs command. Let's create some filesystems within a pool.
zfs create
zfs list
zfs destroy
[root@server ~]# zfs create datapool/academics
[root@server ~]# zfs create datapool/research
[root@server ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda 19G 1.4G 17G 8% /
devtmpfs 488M 0 488M 0% /dev
tmpfs 497M 0 497M 0% /dev/shm
tmpfs 497M 50M 447M 11% /run
tmpfs 497M 0 497M 0% /sys/fs/cgroup
datapool 2.8G 0 2.8G 0% /datapool
tmpfs 100M 0 100M 0% /run/user/0
datapool/academics 2.8G 0 2.8G 0% /datapool/academics
datapool/research 2.8G 0 2.8G 0% /datapool/research
[root@server ~]# zfs list
NAME USED AVAIL REFER MOUNTPOINT
datapool 100M 2.67G 19K /datapool
datapool/academics 31K 1024M 20.5K /datapool/academics
datapool/research 1.57M 98.4M 1.57M /datapool/research
Compression
ZFS supports transparent compression. Let's enable LZ4 compression on the academics filesystem.
zfs set compression=lz4 datapool/academics
[root@server ~]# zfs set compression=lz4 datapool/academics
Now, copy a file to the compressed filesystem and observe the size difference.
cp /var/log/secure /datapool/academics/
df -h /datapool/academics
zfs get compressratio datapool
[root@server ~]# cp /var/log/secure /datapool/academics/
[root@server academics]# df -h .
Filesystem Size Used Avail Use% Mounted on
datapool/academics 100M 1.7M 99M 2% /datapool/academics
[root@server ~]# zfs get compressratio datapool
NAME PROPERTY VALUE SOURCE
datapool compressratio 9.03x -
Quotas and Reservations
ZFS allows setting quotas and reservations to control disk space usage.
zfs set quota=
zfs set reservation=
[root@server ~]# zfs set quota=100M datapool/academics
[root@server ~]# zfs set reservation=100M datapool/academics
[root@server ~]# zfs list
NAME USED AVAIL REFER MOUNTPOINT
datapool 100M 2.67G 19K /datapool
datapool/academics 1.57M 98.4M 1.57M /datapool/academics
datapool/research 19K 2.67G 19K /datapool/research
[root@server ~]# zfs set quota=1G datapool/research
[root@server ~]# zfs list
NAME USED AVAIL REFER MOUNTPOINT
datapool 100M 2.67G 19K /datapool
datapool/academics 1.57M 98.4M 1.57M /datapool/academics
datapool/research 19K 1024M 19K /datapool/research
Creating and Managing Snapshots
Snapshots provide read-only, point-in-time copies of a filesystem. They consume no additional space untill changes are made.
zfs snapshot <filesystem@snapshot_name>
zfs list -t snapshot
[root@server ~]# cd /datapool/research/
[root@server research]# mkdir biology chemistry physics
[root@server research]# cat > experiment.log
Experiment data for March 2024
[root@server research]# ls -la
total 4
drwxr-xr-x 5 root root 6 Mar 19 10:34 .
drwxr-xr-x 4 root root 4 Mar 19 09:59 ..
drwxr-xr-x 2 root root 2 Mar 19 10:33 biology
drwxr-xr-x 2 root root 2 Mar 19 10:32 chemistry
drwxr-xr-x 2 root root 2 Mar 19 10:32 physics
-rw-r--r-- 1 root root 36 Mar 19 10:35 experiment.log
[root@server research]# zfs snapshot datapool/research@mar2024
[root@server research]# zfs list -t snapshot
NAME USED AVAIL REFER MOUNTPOINT
datapool/research@mar2024 0 - 20.5K -
To restore data from a snapshot, you can access the .zfs/snapshot directory.
rm -rf experiment.log
cp -a .zfs/snapshot/mar2024/experiment.log .
[root@server research]# rm -rf experiment.log
[root@server research]# ls
biology chemistry physics
[root@server research]# cd .zfs
[root@server .zfs]# cd snapshot/mar2024/
[root@server mar2024]# ls
biology chemistry physics experiment.log
[root@server mar2024]# cp -a experiment.log /datapool/research/
[root@server mar2024]# cd /datapool/research/
[root@server research]# ls
biology chemistry experiment.log physics