Copyright 2006, 2012, Oracle and/or its affiliates. All rights reserved.,.,,,,,,,,,,,,.,...,. U.S. GOVERNMENT END USERS: Oracle programs, including an

Transcription

1 Oracle Solaris 11.1 : ZFS : E

2 Copyright 2006, 2012, Oracle and/or its affiliates. All rights reserved.,.,,,,,,,,,,,,.,...,. U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government..., -,,. Oracle Corporation. Oracle Java Oracle Corporation /.. Intel Intel Xeon Intel Corporation. SPARC SPARC International, Inc.. AMD, Opteron, AMD, AMD Opteron Advanced Micro Devices. UNIX The Open Group. ( ) 3,. Oracle Corporation 3,. Oracle Corporation 3,, @25097

3 Oracle Solaris ZFS ( ) ZFS ZFS ZFS var EFI(GPT) ZFS ZFS ZFS (zfs.1m) aclmode ZFS ZFS ZFS ZFS (zfs diff)...21 ZFS ZFS ZFS ZFS ACL ZFS (zpool split)...25 ZFS iscsi ZFS ZFS Oracle Solaris ZFS? ZFS

4 ZFS ZFS ZFS Oracle Solaris ZFS ZFS ZFS ZFS NFSv4 Solaris ACL Oracle Solaris ZFS...35 ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS Oracle Solaris ZFS ZFS ZFS ZFS ZFS ZFS ZFS RAID-Z ZFS Oracle Solaris 11.1 : ZFS

5 ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS I/O ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ( ) ZFS ZFS ZFS ZFS BE

6 (SPARC x86/vtoc) (x86/efi(gpt)) ZFS (SPARC x86/vtoc) ZFS (x86/efi(gpt)) BE (SPARC x86/vtoc) BE (x86/efi(gpt)) ZFS ZFS ZFS ZFS ZFS SPARC ZFS x86 ZFS ZFS Oracle Solaris ZFS ZFS ( ) ZFS, ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS Oracle Solaris 11.1 : ZFS

7 ZFS ZFS ZFS ZFS ZFS / ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS, ZFS ZFS ZFS ZFS ZFS Oracle Solaris ZFS ZFS ZFS ZFS ZFS ZFS (zfs diff) ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS

8 ZFS ZFS ZFS ZFS ACL Oracle Solaris ZFS Solaris ACL ACL ACL ACL ZFS ACL Verbose ZFS ACL Verbose ZFS ACL Compact ZFS ACL ZFS Oracle Solaris ZFS ZFS ZFS ZFS ZFS (zfs allow) ZFS (zfs unallow) ZFS ( ) ZFS ( ) ZFS ( ) Oracle Solaris ZFS ZFS ZFS ZFS iscsi LUN Solaris ZFS ZFS ZFS Oracle Solaris 11.1 : ZFS

9 ZFS ZFS zoned ZFS ZFS Oracle Solaris ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS zpool status ZFS ZFS ZFS ZFS

10 ZFS ZFS ZFS ZFS ZFS Oracle Solaris ZFS ZFS ZFS ZFS A Oracle Solaris ZFS ZFS ZFS ZFS Oracle Solaris 11.1 : ZFS

11 Oracle Solaris 11.1 ZFS Oracle Solaris ZFS. SPARC x86. Oracle Solaris SPARC x86. hcl/index.html Oracle Solaris.. Oracle Solaris ZFS. Oracle Solaris OS( ) UNIX.. 1, Oracle Solaris ZFS ( ) 2, Oracle Solaris ZFS 3, Oracle Solaris ZFS 4, ZFS ZFS.. ZFS. ZFS. ZFS., ZFS, ZFS. 11

12 5, Oracle Solaris ZFS 6, Oracle Solaris ZFS 7, ACL Oracle Solaris ZFS 8, Oracle Solaris ZFS 9, Oracle Solaris ZFS 10, Oracle Solaris ZFS 11, 12, Oracle Solaris ZFS A, Oracle Solaris ZFS ZFS.,,. ZFS. (ACL) UNIX ZFS. ZFS ZFS. ZFS, Oracle Solaris ZFS,. ZFS... ZFS. ZFS,, ZFS Solaris OS. Oracle Solaris. Oracle Solaris 11.1, Oracle Solaris 11.1 Oracle Solaris 11.1 : Oracle Solaris 11.1 : Oracle Support Oracle My Oracle Support Oracle Solaris 11.1 : ZFS

13 . P 1 AaBbCc123 AaBbCc123 AaBbCc123 AaBbCc123, ;.. :.,,..login. ls -a. machine_name% you have mail. machine_name% su Password: rm filename :. Oracle Solaris OS UNIX.. P 2 Bash, Korn Bourne $ Bash, Korn Bourne # C C machine_name% machine_name# 13

14 14

15 1 Oracle Solaris ZFS ( ) ZFS Oracle Solaris ZFS ZFS 26 Oracle Solaris ZFS? 29 ZFS 30 ZFS 31 Oracle Solaris ZFS ZFS. 16 ZFS 16 ZFS 17 var 17 EFI(GPT) 18 ZFS 18 ZFS 19 ZFS (zfs.1m) 19 aclmode ZFS 21 ZFS 21 ZFS 21 ZFS (zfs diff) 22 ZFS 22 ZFS 15

16 ZFS 23 ZFS 24 ZFS ACL 25 ZFS (zpool split) 25 ZFS iscsi 25 ZFS 26 ZFS ZFS Oracle Solaris 11.1: zpool status. zpool status UNAVAIL (c0t5000c500335f907fd0). # zpool status -v pond pool: pond state: DEGRADED status: One or more devices are unavailable in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Determine if the device needs to be replaced, and clear the errors using zpool clear or fmadm repaired, or replace the device with zpool replace. scan: scrub repaired 0 in 0h0m with 0 errors on Wed Jun 20 15:38: config: device details: NAME STATE READ WRITE CKSUM pond DEGRADED mirror-0 DEGRADED c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 UNAVAIL mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE c0t5000c500335f907fd0 UNAVAIL cannot open status: ZFS detected errors on this device. The device was missing. see: for recovery errors: No known data errors ZFS Oracle Solaris 11.1: ZFS.. share.nfs share.smb. 16 Oracle Solaris 11.1 : ZFS

17 ZFS ZFS. var Oracle Solaris 11.1: Oracle Solaris 11.1 /var/share rpool/varshare. /var BE.. # ls /var/share audit cores crash mail /var /var/share. /var. Oracle Solaris 11 Oracle Solaris 11.1 /var /var/share. EFI(GPT) Oracle Solaris 11.1: x86 ZFS EFI(GPT).. # zpool status rpool pool: rpool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE c2t0d0 ONLINE errors: No known data errors SPARC. x86 EFI(GPT). zpool create -B EFI(GPT) zpool BE (SPARC x86/vtoc). 1 Oracle Solaris ZFS ( ) 17

18 ZFS EFI(GPT) zpool replace. # bootadm install-bootloader Oracle Solaris x86 2TiB. ZFS Oracle Solaris 11: zfs zpool zfs zpool help.. # zfs help The following commands are supported: allow clone create destroy diff get groupspace help hold holds inherit list mount promote receive release rename rollback send set share snapshot unallow unmount unshare upgrade userspace For more info, run: zfs help <command> # zfs help create usage: create [-p] [-o property=value]... <filesystem> create [-ps] [-b blocksize] [-o property=value]... -V <size> <volume> # zpool help The following commands are supported: add attach clear create destroy detach export get help history import iostat list offline online remove replace scrub set split status upgrade For more info, run: zpool help <command> # zpool help attach usage: attach [-f] <pool> <device> <new-device> zfs(1m) zpool(1m). ZFS Oracle Solaris 11: ZFS. zfs snapshot snap.. # zfs snap -r users/home@snap1 zfs diff -e Oracle Solaris 11.1 : ZFS

19 ZFS # zfs diff -e tank/cindy@yesterday tank/cindy@now + /tank/cindy/ + /tank/cindy/file.1 -o.. # zfs diff -e -o size -o name tank/cindy@yesterday tank/cindy@now + 7 /tank/cindy/ /tank/cindy/file.1 ZFS 6, Oracle Solaris ZFS. ZFS (zfs.1m) Oracle Solaris 11: ZFS zfs.1m zfs.1m,,,. zfs_allow(1m) zfs_encrypt(1m) zfs_share(1m) aclmode Oracle Solaris 11: aclmode chmod ACL ACL( ). aclmode. discard aclmode discard ACL.. mask aclmode mask. UID. ACL. ACL ACL. passthrough aclmode passthrough ACL ACL Oracle Solaris ZFS ( ) 19

20 ZFS Oracle Solaris 11: Solaris zpool status -l /dev/chassis.,. fmadm add-alias.. # fmadm add-alias SUN-Storage-J QAJ001 SUN-Storage-J4400.rack22. % zpool status -l export pool: export state: ONLINE scan: resilvered 492G in 8h22m with 0 errors on Wed Aug 1 17:22: config: NAME STATE READ WRITE CKSUM export ONLINE mirror-0 ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 2/disk ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 3/disk ONLINE mirror-1 ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 4/disk ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 5/disk ONLINE mirror-2 ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 6/disk ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 7/disk ONLINE mirror-3 ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 8/disk ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 9/disk ONLINE mirror-4 ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 10/disk ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 11/disk ONLINE spares /dev/chassis/sun-storage-j4400.rack22/scsi_device 0/disk AVAIL /dev/chassis/sun-storage-j4400.rack22/scsi_device 1/disk AVAIL errors: No known data errors zpool iostat. diskinfo, format prtconf. diskinfo(1m). ZFS Oracle Solaris 11:. 20 Oracle Solaris 11.1 : ZFS

21 ZFS ZFS shadow. shadow. file:///path nfs://host:path 192 ZFS. ZFS Oracle Solaris 11: ZFS., tank/cindy encryption.. 8. # zfs create -o encryption=on tank/cindy Enter passphrase for tank/cindy : xxx Enter again: xxx ZFS ZFS. ZFS Oracle Solaris 11:. mountpoint. 211 ZFS. ZFS (zfs diff) Oracle Solaris 11: zfs diff ZFS.,. $ ls /tank/cindy filea $ zfs snapshot tank/cindy@ Oracle Solaris ZFS ( ) 21

22 ZFS $ ls /tank/cindy filea fileb $ zfs snapshot tank/cindy@0914,. $ zfs diff tank/cindy@0913 tank/cindy@0914 M /tank/cindy/ + /tank/cindy/fileb M. + fileb. 203 ZFS (zfs diff). ZFS Oracle Solaris 11: ZFS. zpool import -m RAID-Z(raidz1, raidz2 raidz3) I/O. 29 RAID-Z. RAID-Z. RAID-Z.. ZFS Oracle Solaris 11: sync ZFS.... sync Oracle Solaris 11.1 : ZFS

23 ZFS # zfs set sync=always tank/neil zil_disable sync Oracle Solaris ZFS Oracle Solaris 11: -T zpool list zpool status. zpool status... scan: resilver in progress since Thu Jun 7 14:41: G scanned out of 73.3G at 106M/s, 0h11m to go 3.80G resilvered, 5.22% done.. scan: scrub in progress since Thu Jun 7 14:59: G scanned out of 73.3G at 118M/s, 0h10m to go 0 repaired, 2.66% done.. resilvered 73.3G in 0h13m with 0 errors on Thu Jun 7 14:54: scan: scrub repaired 512B in 1h2m with 0 errors on Thu Jun 7 15:10: scan: scrub canceled on Thu Jun 7 15:19:20 MDT T d -T u. # zpool status -T d tank 3 2 Thu Jun 14 14:08:21 MDT 2012 pool: tank state: DEGRADED status: One or more devices is currently being resilvered. The pool will continue to function in a degraded state. action: Wait for the resilver to complete. Run zpool status -v to see device specific details. scan: resilver in progress since Thu Jun 14 14:08: Oracle Solaris ZFS ( ) 23

24 ZFS 2.96G scanned out of 4.19G at 189M/s, 0h0m to go 1.48G resilvered, 70.60% done config: NAME STATE READ WRITE CKSUM tank DEGRADED mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 DEGRADED c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 DEGRADED (resilvering) errors: No known data errors ZFS ACL Oracle Solaris 11: ACL. ACL deny ACE( )., 0644, deny ACE 0705, 0060 deny ACE. ACL( : 644) deny ACE.. # ls -v file.1 -rw-r--r-- 1 root root Jun 14 11:52 file.1 0:owner@:execute:deny 1:owner@:read_data/write_data/append_data/write_xattr/write_attributes /write_acl/write_owner:allow 2:group@:write_data/append_data/execute:deny 3:group@:read_data:allow 4:everyone@:write_data/append_data/write_xattr/execute/write_attributes /write_acl/write_owner:deny 5:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow ACL( : 644) deny ACE.. # ls -v file.1 -rw-r--r-- 1 root root Jun 22 14:30 file.1 0:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 2:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow ACL ACE.,. aclinherit restricted.,acl ACE. 24 Oracle Solaris 11.1 : ZFS

25 ZFS ACL chmod(2). ZFS aclmode.,acl user ACE. group ACE. 7, ACL Oracle Solaris ZFS. ZFS (zpool split) Oracle Solaris 11: zpool split.. 67 ZFS. ZFS iscsi Oracle Solaris 11: iscsi COMSTAR(Common Multiprotocol SCSI Target). ZFS iscsi LUN shareiscsi. ZFS iscsi LUN stmfadm. 259 ZFS iscsi LUN. ZFS Oracle Solaris 11: ZFS zpool-poolname. I/O I/O. CPU. ps prstat.. SDC(7). 1 Oracle Solaris ZFS ( ) 25

26 Oracle Solaris ZFS? ZFS Oracle Solaris 11: (dedup) ZFS. dedup.,.. # zfs set dedup=on tank/home.,. # zpool list tank NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT tank 556G 4.19G 552G 0% 1.00x ONLINE - zpool list. 154 dedup. dedup dedup. Oracle Solaris ZFS? Oracle Solaris ZFS. ZFS,,. ZFS ZFS Oracle Solaris 11.1 : ZFS

27 Oracle Solaris ZFS? ZFS. ZFS. (, )..... DIMM.. ZFS.,..,. fsck.., fsck..,.,..,... (O_DSYNC ). ZFS... ZFS 1 Oracle Solaris ZFS ( ) 27

28 Oracle Solaris ZFS?.. ZFS. ZFS. ZFS. ZFS. 128, 256. inode (256 ). ZFS..... ZFS.,, NFS ZFS.,,... ZFS,,..,,.. 28 Oracle Solaris 11.1 : ZFS

29 ZFS ZFS. ZFS Oracle Solaris fletcher4( ) SHA256. clone. 204 ZFS.,, ZFS. ZFS.. pool. path. snapshot. 5, Oracle Solaris ZFS. filesystem ZFS. 5, Oracle Solaris ZFS Oracle Solaris ZFS ( ) 29

30 ZFS RAID-Z snapshot volume 3, Oracle Solaris ZFS.. RAID-Z 48 RAID-Z..,.. ZFS ZFS., ZFS. ZFS 257 ZFS. ZFS ZFS.. (_) (-) (:) (.), (_), (-) (.).. c[0-9]. log. 30 Oracle Solaris 11.1 : ZFS

31 Oracle Solaris ZFS mirror, raidz, raidz1, raidz2, raidz3 spare. (%).. (%).. Oracle Solaris ZFS 31 ZFS 31 ZFS 33 ZFS NFSv4 Solaris ACL ZFS,... ZFS. ZFS. ( : /export/home).. 38 ZFS. ZFS ZFS., ZFS. 1 Oracle Solaris ZFS ( ) 31

32 Oracle Solaris ZFS ( : df) ZFS ZFS. ( : /home) NFS. ZFS,.,. ZFS. ZFS df -g. total files. ZFS...,.. fsync(3c) O_SYNC. UFS du. ZFS du.. " ". ZFS UFS. df zfs list df. df., ZFS df.. recordsize 1/2. recordsize 128KB 64KB. RFE Oracle Solaris 11.1 : ZFS

33 Oracle Solaris ZFS RAIDZ-2 2 (512 ).. recordsize 512, KB( 3 ). recordsize (128KB) ( lzjb). df. ZFS. ZFS. ZFS 6, Oracle Solaris ZFS......, ENOSPC EDQUOT. ZFS ZFS., /etc/vfstab. ZFS. /etc/vfstab ZFS. ZFS 165 ZFS. 26 ZFS ZFS Volume Manager. ZFS. ZFS 1 Oracle Solaris ZFS ( ) 33

34 Oracle Solaris ZFS.. NFSv4 Solaris ACL Solaris OS POSIX ACL ACL. POSIX ACL UFS. NFSv4 Solaris ACL ZFS. Solaris ACL. NFSv4 NT ACL.. ACL setfacl getfacl chmod ls.. ZFS ACL 7, ACL Oracle Solaris ZFS. 34 Oracle Solaris 11.1 : ZFS

35 2 2 Oracle Solaris ZFS Oracle Solaris ZFS. ZFS ZFS 36 ZFS 36 ZFS 37 ZFS 38 ZFS ZFS ( ) ZFS ZFS. ZFS ZFS,. ZFS ZFS,. Oracle Solaris 11.1 :. ZFS RBAC, ZFS ZFS. 8, Oracle Solaris ZFS. 35

36 ZFS ZFS ZFS. Oracle Solaris SPARC x86. 64MB. 128MB. ZFS.. ZFS ZFS.., ZFS. tank tank ZFS. /dev/dsk/c1t0d0 /dev/dsk/c2t0d0. # zpool create tank mirror c1t0d0 c2t0d0 ZFS 47 ZFS. ZFS tank /tank. # mkfile 100m /tank/foo # df -h /tank Filesystem size used avail capacity Mounted on tank 80G 100M 80G 1% /tank.. tank fs. # zfs create tank/fs ZFS tank/fs /tank/fs. 36 Oracle Solaris 11.1 : ZFS

37 ZFS # mkfile 100m /tank/fs/foo # df -h /tank/fs Filesystem size used avail capacity Mounted on tank/fs 80G 100M 80G 1% /tank/fs. ZFS 38 ZFS. ZFS ZFS ZFS.. 128MB. ZFS. 38 ZFS /dev/dsk/c1t0d0 /dev/dsk/c2t0d0. 43 ZFS.. ZFS. ZFS ( ) RAID-Z(RAID-5 ). 38 ZFS. ZFS 47 ZFS. 2 Oracle Solaris ZFS 37

38 ZFS ZFS ZFS. ZFS 35 ZFS.. zpool zfs.., 30 ZFS.., tank. # zpool create tank mirror c1t0d0 c2t0d0. 50 ZFS zpool list. # zpool list NAME SIZE ALLOC FREE CAP HEALTH ALTROOT tank 80G 137K 80G 0% ONLINE - 83 ZFS. ZFS.,. ZFS.. ZFS. 38 Oracle Solaris 11.1 : ZFS

39 ZFS ZFS. ZFS,..,. 39 ZFS ZFS jeff bill. 5, Oracle Solaris ZFS.. ZFS ZFS home...,,,. 39 ZFS /export/zfs/ user NFS. jeff 10GB. 137 ZFS. 1 2 ZFS ZFS. ZFS 35 ZFS... # zfs create tank/home 2 Oracle Solaris ZFS 39

40 ZFS 3.. # zfs set mountpoint=/export/zfs tank/home # zfs set share.nfs=on tank/home # zfs set compression=on tank/home # zfs get compression tank/home NAME PROPERTY VALUE SOURCE tank/home compression on local.. # zfs create -o mountpoint=/export/zfs -o share.nfs=on -o compression=on tank/home 137 ZFS. tank home. 4. home.. # zfs create tank/home/jeff # zfs create tank/home/bill /export/zfs/ user NFS. /etc/vfstab /etc/dfs/dfstab. 134 ZFS. 165 ZFS jeff 10GB.. # zfs set quota=10g tank/home/jeff. zfs list. # zfs list NAME USED AVAIL REFER MOUNTPOINT tank 92.0K 67.0G 9.5K /tank tank/home 24.0K 67.0G 8K /export/zfs tank/home/bill 8K 67.0G 8K /export/zfs/bill tank/home/jeff 8K 10.0G 8K /export/zfs/jeff jeff 10GB, bill (67GB). 40 Oracle Solaris 11.1 : ZFS

41 ZFS 157 ZFS. 31 ZFS. 2 Oracle Solaris ZFS 41

42 42

43 3 3 Oracle Solaris ZFS Oracle Solaris ZFS.. 43 ZFS 47 ZFS 50 ZFS 60 ZFS 80 ZFS 83 ZFS 95 ZFS 103 ZFS ZFS. 43 ZFS 45 ZFS 46 ZFS ZFS. 128MB. /dev/dsk. (c1t0d0) (c0t0d0s7)., 43

44 ZFS. ZFS EFI. format. Current partition table (original): Total disk sectors available: (reserved sectors) Part Tag Flag First Sector Size Last Sector 0 usr wm GB unassigned wm unassigned wm unassigned wm unassigned wm unassigned wm unassigned wm reserved wm MB Oracle Solaris 11.1 EFI(GPT) x86. Current partition table (original): Total disk sectors available: (reserved sectors) Part Tag Flag First Sector Size Last Sector 0 BIOS_boot wm MB usr wm GB unassigned wm unassigned wm unassigned wm unassigned wm unassigned wm reserved wm MB (BIOS boot) GPT ZFS. /dev/dsk/cntndn. /dev/dsk. ZFS. x86 Solaris fdisk. Solaris fdisk Oracle Solaris 11.1 : ZFS ( ). ZFS EFI. EFI Oracle Solaris 11.1 : EFI(GPT). 44 Oracle Solaris 11.1 : ZFS

45 ZFS Oracle Solaris 11.1 x86 EFI(GPT). 108 ZFS. ( : /dev/dsk/c1t0d0) /dev/dsk ( : c1t0d0).,. c1t0d0 /dev/dsk/c1t0d0 /dev/foo/disk ZFS Solaris VTOC(SMI). format SPARC 72GB 68GB 0. # format... Specify disk (enter its number): 4 selecting c1t1d0 partition> p Current partition table (original): Total disk cylinders available: (reserved cylinders) Part Tag Flag Cylinders Size Blocks 0 root wm GB (14087/0/0) unassigned wm 0 0 (0/0/0) 0 2 backup wm GB (14087/0/0) unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0 format x86 72GB 68GB # format... selecting c1t0d0 3 Oracle Solaris ZFS 45

46 ZFS partition> p Current partition table (original): Total disk cylinders available: (reserved cylinders) Part Tag Flag Cylinders Size Blocks 0 root wm GB (49778/0/0) unassigned wu 0 0 (0/0/0) 0 2 backup wm GB (49779/0/0) unassigned wu 0 0 (0/0/0) 0 4 unassigned wu 0 0 (0/0/0) 0 5 unassigned wu 0 0 (0/0/0) 0 6 unassigned wu 0 0 (0/0/0) 0 7 unassigned wu 0 0 (0/0/0) 0 8 boot wu MB (1/0/0) unassigned wu 0 0 (0/0/0) 0 x86 fdisk. fdisk /dev/dsk/cn[tn]dnpn,. ZFS cn[tn]dnpn. ZFS ZFS.. UFS ZFS UFS. ZFS ZFS. ZFS. 64MB. ZFS ZFS. ZFS., RAID LUN, ZFS. ZFS. RAID LUN ZFS ZFS.,. 46 Oracle Solaris 11.1 : ZFS

47 ZFS ZFS.. ZFS. 12, Oracle Solaris ZFS. ID ( ). ID ZFS. ID. ZFS ID. ZFS ZFS RAID-Z RAID-Z ZFS.... mirror c1t0d0 c2t0d0. mirror c1t0d0 c2t0d0 c3t0d0 mirror c4t0d0 c5t0d0 c6t0d Oracle Solaris ZFS 47

48 ZFS RAID-Z ZFS, RAID-Z. RAID-Z(raidz raidz1) RAID-5. RAID-Z(raidz2) RAID-6. RAIDZ-3(raidz3). RAID-5 ( : RAID-4, RAID-6, RDP EVEN-ODD) RAID-5. RAID-5 ( ). RAID-Z ZFS RAID. RAID ZFS. RAID-Z RAID-5. X N P RAID-Z (N-P)*X P. RAID-Z 2, RAID-Z 3., RAID-Z 3 3. RAID-Z. 3 RAID-Z. raidz c1t0d0 c2t0d0 c3t0d0 RAID-Z. raidz c1t0d0 c2t0d0 c3t0d0 c4t0d0 c5t0d0 c6t0d0 c7t0d0 raidz c8t0d0 c9t0d0 c10t0d0 c11t0d0c12t0d0 c13t0d0 c14t0d0 RAID-Z.,14 RAID-Z 7. RAID-Z. RAID-Z 52 RAID-Z. RAID-Z. 48 Oracle Solaris 11.1 : ZFS

49 ZFS RAID-Z 12, Oracle Solaris ZFS. ZFS Oracle Sun Storage 7000 ZFS DRAM, SSD HDD. ZFS. Sun Storage Unified Storage System Administration Guide. ZFS RAID-Z. ZFS. ZFS.. ZFS. RAID-Z.., RAID-Z 2 ZFS., RAID-Z. RAID-Z.. 3 Oracle Solaris ZFS 49

50 ZFS ZFS ZFS. 50 ZFS ZFS 59 ZFS.. ZFS.. zpool destroy.. ZFS zpool create.. 30 ZFS. c1t0d0 c1t1d0 tank. # zpool create tank c1t0d0 c1t1d0 /dev/dsk, ZFS.. mirror. mirror. 2. # zpool create tank mirror c1d0 c2d0 mirror c3d0 c4d0 mirror.,. 12, Oracle Solaris ZFS. ZFS. 50 Oracle Solaris 11.1 : ZFS

51 ZFS (vdev) ( ). 73. ( ) ZFS.,. ZFS. Oracle Solaris 11.1 x86 EFI(GPT), SPARC SMI(VTOC). Oracle Solaris 11.1 EFI(GPT). ZFS EFI(GPT). # zpool create -B rpool2 c1t0d0. zpool add, zpool attach. RAID-Z... ERROR: ZFS pool <pool-name> does not support boot environments # zpool add -f rpool log c0t6d0s0 cannot add to rpool : root pool can not have multiple vdevs or separate logs ZFS 4, ZFS. 3 Oracle Solaris ZFS 51

52 ZFS RAID-Z RAID-Z raidz raidz1 mirror. 5 RAID-Z. # zpool create tank raidz c1t0d0 c2t0d0 c3t0d0 c4t0d0 /dev/dsk/c5t0d0. /dev/dsk/c5t0d0 c5t0d0. raidz2 raidz3 RAID-Z. : # zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0 c4t0d0 c5t0d0 # zpool status -v tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE raidz2-0 ONLINE c1t0d0 ONLINE c2t0d0 ONLINE c3t0d0 ONLINE c4t0d0 ONLINE c5t0d0 ONLINE errors: No known data errors # zpool create tank raidz3 c0t0d0 c1t0d0 c2t0d0 c3t0d0 c4t0d0 c5t0d0 c6t0d0 c7t0d0 c8t0d0 # zpool status -v tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE raidz3-0 ONLINE c0t0d0 ONLINE c1t0d0 ONLINE c2t0d0 ONLINE c3t0d0 ONLINE c4t0d0 ONLINE c5t0d0 ONLINE c6t0d0 ONLINE c7t0d0 ONLINE c8t0d0 ONLINE errors: No known data errors ZFS RAID-Z. 52 Oracle Solaris 11.1 : ZFS

53 ZFS RAID-Z. 61. RAID-Z ( ). 73. RAID-Z. RAID-Z RAID-Z ( ) RAID-Z. RFE. RAID-Z 48 RAID-Z. ZFS POSIX ZIL(ZFS ).,. NFS fsync(). ZIL. NVRAM. ZFS. ZFS.,.. RAID-Z..,,,,,... (64MB). in-play. ( ). 3 Oracle Solaris ZFS 53

54 ZFS in-play 1/2. 16GB 8GB. ZFS.. # zpool create datap mirror c0t5000c500335f95e3d0 c0t5000c500335f907fd0 mirror c0t5000c500335bd117d0 c0t5000c500335dc60fd0 log mirror c0t5000c500335e106bd0 c0t5000c500335fc3e7d0 # zpool status datap pool: datap state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM datap ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE logs mirror-2 ONLINE c0t5000c500335e106bd0 ONLINE c0t5000c500335fc3e7d0 ONLINE errors: No known data errors ZFS... : # zpool create tank mirror c2t0d0 c2t1d0 c2t3d0 cache c2t5d0 c2t8d0 # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c2t0d0 ONLINE c2t1d0 ONLINE Oracle Solaris 11.1 : ZFS

55 ZFS c2t3d0 ONLINE cache c2t5d0 ONLINE c2t8d0 ONLINE errors: No known data errors. 1. zpool iostat. # zpool iostat -v pool 5. ZFS.. zpool iostat RAID-Z. I/O RAID-Z.. ZFS.. OS. ( : )..... ZFS (RAIDZ ).. ZFS,... 3 Oracle Solaris ZFS 55

56 ZFS. ZFS.... ( vdev ). RAID-Z.,.. 2. # zpool create tank mirror c1d0 c2d0 mirror c3d0 c4d0 4. # zpool create mypool raidz2 c1d0 c2d0 c3d0 c4d0 zpool add.. # zpool add mypool raidz2 c2d1 c3d1 c4d1 c5d1,.. ZFS. ZFS zpool status.. # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE mirror-1 ONLINE Oracle Solaris 11.1 : ZFS

57 ZFS c0t2d0 ONLINE c1t2d0 ONLINE mirror-2 ONLINE c0t3d0 ONLINE c1t3d0 ONLINE errors: No known data errors ZFS.,. ZFS ZFS.. # zpool create tank c1t0d0 c1t1d0 invalid vdev specification use -f to override the following errors: /dev/dsk/c1t0d0s0 is currently mounted on /. Please see umount(1m). /dev/dsk/c1t0d0s1 is currently mounted on swap. Please see swap(1m). /dev/dsk/c1t1d0s0 is part of active ZFS pool zeepool. Please see zpool(1m). -f,. -f. /etc/vfstab ZFS. umount. /etc/vfstab,. /etc/vfstab.. dumpadm. ZFS. zpool destroy ( ). zpool detach.. -f. 3 Oracle Solaris ZFS 57

58 ZFS ZFS. Solaris Volume Manager.... -f. # zpool create tank c1t0d0 invalid vdev specification use -f to override the following errors: /dev/dsk/c1t0d0s0 contains a ufs filesystem. # zpool create -f tank c1t0d0 -f.. zpool.. # zpool create tank c1t0d0 mirror c2t0d0 c3t0d0 invalid vdev specification use -f to override the following errors: mismatched replication level: both disk and mirror vdevs are present # zpool create tank mirror c1t0d0 c2t0d0 mirror c3t0d0 c4t0d0 c5t0d0 invalid vdev specification use -f to override the following errors: mismatched replication level: 2-way mirror and 3-way mirror vdevs are present -f,. RAID-Z.,. -f. Dry Run,. zpool create -n. dry run.. 58 Oracle Solaris 11.1 : ZFS

59 ZFS # zpool create -n tank mirror c1t0d0 c1t1d0 would create tank with the following layout: tank mirror c1t0d0 c1t1d0.. zpool create -n,. /pool-name.... zpool create -m. : # zpool create home c1t0d0 default mountpoint /home exists and is not empty use -m option to provide a different default # zpool create -m /export/zfs home c1t0d0 /export/zfs home home. 165 ZFS. ZFS zpool destroy.. # zpool destroy tank ZFS. zpool destroy 101 ZFS. 3 Oracle Solaris ZFS 59

60 ZFS. format analyze->purge. ZFS ZFS UNAVAIL UNAVAIL ( UNAVAIL ),, -f... # zpool destroy tank cannot destroy tank : pool is faulted use -f to force destruction anyway # zpool destroy -f tank 90 ZFS. 98 ZFS. ZFS 43 ZFS ZFS Oracle Solaris 11.1 : ZFS

61 ZFS zpool add. : # zpool add zeepool mirror c2t1d0 c2t2d0 zpool create., -f. dry run -n.. # zpool add -n zeepool mirror c3t1d0 c3t2d0 would update zeepool to the following configuration: zeepool mirror c1t0d0 c1t1d0 mirror c2t1d0 c2t2d0 mirror c3t1d0 c3t2d0 c3t1d0 c3t2d0 zeepool ZFS ZFS. # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE mirror-1 ONLINE c0t2d0 ONLINE Oracle Solaris ZFS 61

62 ZFS 3 1 ZFS ( ) c1t2d0 ONLINE errors: No known data errors # zpool add tank mirror c0t3d0 c1t3d0 # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE mirror-1 ONLINE c0t2d0 ONLINE c1t2d0 ONLINE mirror-2 ONLINE c0t3d0 ONLINE c1t3d0 ONLINE errors: No known data errors 3 2 RAID-Z RAID-Z. 3 RAID-Z 3 RAID-Z. # zpool status rzpool pool: rzpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM rzpool ONLINE raidz1-0 ONLINE c1t2d0 ONLINE c1t3d0 ONLINE c1t4d0 ONLINE errors: No known data errors # zpool add rzpool raidz c2t2d0 c2t3d0 c2t4d0 # zpool status rzpool pool: rzpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM rzpool ONLINE raidz1-0 ONLINE Oracle Solaris 11.1 : ZFS

63 ZFS 3 2 RAID-Z ( ) c1t0d0 ONLINE c1t2d0 ONLINE c1t3d0 ONLINE raidz1-1 ONLINE c2t2d0 ONLINE c2t3d0 ONLINE c2t4d0 ONLINE errors: No known data errors 3 3. # zpool status newpool pool: newpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM newpool ONLINE mirror-0 ONLINE c0t4d0 ONLINE c0t5d0 ONLINE errors: No known data errors # zpool add newpool log mirror c0t6d0 c0t7d0 # zpool status newpool pool: newpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM newpool ONLINE mirror-0 ONLINE c0t4d0 ONLINE c0t5d0 ONLINE logs mirror-1 ONLINE c0t6d0 ONLINE c0t7d0 ONLINE errors: No known data errors.. zpool remove. mirror Oracle Solaris ZFS 63

64 ZFS 3 3 ( ) # zpool remove newpool mirror-1 # zpool status newpool pool: newpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM newpool ONLINE mirror-0 ONLINE c0t4d0 ONLINE c0t5d0 ONLINE errors: No known data errors.. # zpool status pool pool: pool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM pool ONLINE raidz1-0 ONLINE c0t8d0 ONLINE c0t9d0 ONLINE logs c0t10d0 ONLINE errors: No known data errors # zpool remove pool c0t10d0 3 4 ZFS. zpool add. : # zpool add tank cache c2t5d0 c2t8d0 # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c2t0d0 ONLINE c2t1d0 ONLINE Oracle Solaris 11.1 : ZFS

65 ZFS 3 4 ( ) c2t3d0 ONLINE cache c2t5d0 ONLINE c2t8d0 ONLINE errors: No known data errors RAID-Z. zpool remove.. # zpool remove tank c2t5d0 c2t8d0 # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c2t0d0 ONLINE c2t1d0 ONLINE c2t3d0 ONLINE errors: No known data errors zpool remove,. zpool detach. RAID-Z. ZFS 54 ZFS. zpool add zpool attach. 112 (SPARC x86/vtoc). ZFS 115 ZFS (SPARC x86/vtoc). 3 Oracle Solaris ZFS 65

66 ZFS 3 5 zeepool, c2t1d0 c1t1d0. # zpool status zeepool pool: zeepool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE errors: No known data errors # zpool attach zeepool c1t1d0 c2t1d0 # zpool status zeepool pool: zeepool state: ONLINE scrub: resilver completed after 0h0m with 0 errors on Fri Jan 8 12:59: config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE c2t1d0 ONLINE K resilvered errors: No known data errors, ZFS ZFS zpool attach.. # zpool create tank c0t1d0 # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE c0t1d0 ONLINE errors: No known data errors # zpool attach tank c0t1d0 c1t1d0 # zpool status tank 66 Oracle Solaris 11.1 : ZFS

67 ZFS 3 6 ZFS ZFS ( ) pool: tank state: ONLINE scrub: resilver completed after 0h0m with 0 errors on Fri Jan 8 14:28: config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE K resilvered errors: No known data errors zpool detach.. # zpool detach zeepool c2t1d0. : # zpool detach newpool c1t2d0 cannot detach c1t2d0: only applicable to mirror and replacing vdevs ZFS zpool split ZFS.. zpool split ZFS. ZFS. zpool split... # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM 3 Oracle Solaris ZFS 67

68 ZFS tank ONLINE mirror-0 ONLINE c1t0d0 ONLINE c1t2d0 ONLINE errors: No known data errors # zpool split tank tank2 # zpool import tank2 # zpool status tank tank2 pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE c1t0d0 ONLINE errors: No known data errors pool: tank2 state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank2 ONLINE c1t2d0 ONLINE errors: No known data errors zpool split.. # zpool split tank tank2 c1t0d0. GUID. GUID. zpool split -R.. # zpool split -R /tank2 tank tank2 zpool split -R -R.. 68 Oracle Solaris 11.1 : ZFS

69 ZFS zpool split. 3 7 RAID-Z. zpool split.. 2~3.. zpool attach. ( ) zpool split ZFS mothership. mothership luna.. luna. luna mothership.. # zpool status mothership pool: mothership state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM mothership ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335f907fd0 ONLINE Oracle Solaris ZFS 69

70 ZFS 3 7 ZFS ( ) errors: No known data errors # zpool split mothership luna # zpool import luna # zpool status mothership luna pool: luna state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM luna ONLINE c0t5000c500335f907fd0 ONLINE errors: No known data errors pool: mothership state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM mothership ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335bd117d0 ONLINE errors: No known data errors ZFS. ZFS. ZFS. ZFS.. zpool offline. ( ).. # zpool offline tank c0t5000c500335f95e3d0 70 Oracle Solaris 11.1 : ZFS

71 ZFS. UNAVAIL., raidz1 2,. # zpool offline tank c0t5000c500335f95e3d0 cannot offline c0t5000c500335f95e3d0: no valid replicas OFFLINE.. zpool offline -t.. # zpool offline -t tank c1t0d0 bringing device c1t0d0 offline ONLINE... device is part of exported or potentially active ZFS pool. Please see zpool(1m) OFFLINE. 83 ZFS. 90 ZFS. zpool online.. # zpool online tank c0t5000c500335f95e3d0.. UNAVAIL. 3 Oracle Solaris ZFS 71

72 ZFS UNAVAIL. /var/adm/messages.. SUNW-MSG-ID: ZFS-8000-LR, TYPE: Fault, VER: 1, SEVERITY: Major EVENT-TIME: Wed Jun 20 11:35:26 MDT 2012 PLATFORM: ORCL,SPARC-T3-4, CSN: 1120BDRCCD, HOSTNAME: tardis SOURCE: zfs-diagnosis, REV: 1.0 EVENT-ID: fb6699c8-6bfb-eefa-88bb e3b7 DESC: ZFS device in pool pond failed to open. AUTO-RESPONSE: An attempt will be made to activate a hot spare if available. IMPACT: Fault tolerance of the pool may be compromised. REC-ACTION: Use fmadm faulty to provide a more detailed view of this event. Run zpool status -lx for more information. Please refer to the associated reference document at for the latest service procedures and policies regarding this diagnosis zpool online -e LUN. LUN autoexpand.lun LUN zpool online -e LUN.. # zpool online -e tank c0t5000c500335f95e3d0 zpool status zpool clear... # zpool clear tank.. # zpool clear tank c0t5000c500335f95e3d0 zpool Oracle Solaris 11.1 : ZFS

73 ZFS zpool replace.. ZFS., (c1t1d0). # zpool replace tank c1t1d0.. # zpool replace tank c1t1d0 c1t2d0 ZFS 115 ZFS (SPARC x86/vtoc).. 1. zpool offline format. ID. WWN ID. 5. ZFS.. # zpool replace tank c1t1d0 ID ID. # zpool replace tank c0t5000c500335fc3e7d0 c0t5000c500335ba8c3d0 6. zpool online. 7. FMA. fmadm faulty Affects: zfs://pool=name/vdev=guid fmadm repaired. # fmadm faulty # fmadm repaired zfs://pool=name/vdev=guid SATA. zpool replace. 3 Oracle Solaris ZFS 73

74 ZFS SATA ZFS. autoreplace on. zpool replace.. REMOVED.,..,.., USB. cfgadm -c configure SATA.. RAID-Z.. autoexpand LUN. autoexpand. LUN LUN. 16GB 72GB.. autoexpand. # zpool create pool mirror c1t16d0 c1t17d0 # zpool status pool: pool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM pool ONLINE mirror ONLINE c1t16d0 ONLINE c1t17d0 ONLINE zpool list pool NAME SIZE ALLOC FREE CAP HEALTH ALTROOT pool 16.8G 76.5K 16.7G 0% ONLINE - # zpool replace pool c1t16d0 c1t1d0 # zpool replace pool c1t17d0 c1t2d0 74 Oracle Solaris 11.1 : ZFS

75 ZFS # zpool list pool NAME SIZE ALLOC FREE CAP HEALTH ALTROOT pool 16.8G 88.5K 16.7G 0% ONLINE - # zpool set autoexpand=on pool # zpool list pool NAME SIZE ALLOC FREE CAP HEALTH ALTROOT pool 68.2G 117K 68.2G 0% ONLINE -. zpool scrub.. zpool detach RAID-Z ,.. zpool create zpool add. # zpool create zeepool mirror c0t5000c500335f95e3d0 c0t5000c500335f907fd0 mirror c0t5000c500335bd117d0 c0t5000c500335dc60fd0 spare c0t5000c500335e106bd0 c0t5000c500335fc3e7d0 # zpool status zeepool pool: zeepool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE spares 3 Oracle Solaris ZFS 75

76 ZFS c0t5000c500335e106bd0 c0t5000c500335fc3e7d0 AVAIL AVAIL errors: No known data errors. # zpool add zeepool spare c0t5000c500335e106bd0 c0t5000c500335fc3e7d0 # zpool status zeepool pool: zeepool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE spares c0t5000c500335e106bd0 AVAIL c0t5000c500335fc3e7d0 AVAIL errors: No known data errors zpool remove.. # zpool remove zeepool c0t5000c500335fc3e7d0 # zpool status zeepool pool: zeepool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE spares c0t5000c500335e106bd0 AVAIL errors: No known data errors. ZFS. 76 Oracle Solaris 11.1 : ZFS

77 ZFS zpool remove,... zpool replace. cannot replace disk3 with disk4: device is too small. zpool replace. FMA.. FMA.. ZFS. zpool detach. autoreplace on. UNAVAIL.. # zpool status -x pool: zeepool state: DEGRADED status: One or more devices are unavailable in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Determine if the device needs to be replaced, and clear the errors using zpool clear or fmadm repaired, or replace the device with zpool replace. Run zpool status -v to see device specific details. scan: resilvered 3.15G in 0h0m with 0 errors on Thu Jun 21 16:46: config: NAME STATE READ WRITE CKSUM zeepool DEGRADED mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE Oracle Solaris ZFS 77

78 ZFS mirror-1 DEGRADED c0t5000c500335bd117d0 ONLINE spare-1 DEGRADED c0t5000c500335dc60fd0 UNAVAIL c0t5000c500335e106bd0 ONLINE spares c0t5000c500335e106bd0 INUSE errors: No known data errors (c0t5000c500335dc60fd0) zpool replace ZFS. # zpool replace zeepool c0t5000c500335dc60fd0 # zpool status zeepool pool: zeepool state: ONLINE scan: resilvered 3.15G in 0h0m with 0 errors on Thu Jun 21 16:53: config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE spares c0t5000c500335e106bd0 AVAIL zpool detach.. # zpool detach zeepool c0t5000c500335e106bd # zpool status zeepool pool: zeepool 78 Oracle Solaris 11.1 : ZFS

79 ZFS 3 9 ( ) state: DEGRADED status: One or more devices are unavailable in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Determine if the device needs to be replaced, and clear the errors using zpool clear or fmadm repaired, or replace the device with zpool replace. Run zpool status -v to see device specific details. scan: scrub in progress since Thu Jun 21 17:01: G scanned out of 6.29G at 220M/s, 0h0m to go 0 repaired, 17.05% done config: NAME STATE READ WRITE CKSUM zeepool DEGRADED mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 DEGRADED c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 UNAVAIL spares c0t5000c500335e106bd0 AVAIL errors: No known data errors # zpool detach zeepool c0t5000c500335dc60fd0 # zpool status zeepool pool: zeepool state: ONLINE scan: resilvered 3.15G in 0h0m with 0 errors on Thu Jun 21 17:02: config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335e106bd0 ONLINE errors: No known data errors (Original failed disk c0t5000c500335dc60fd0 is physically replaced) # zpool add zeepool spare c0t5000c500335dc60fd0 # zpool status zeepool pool: zeepool state: ONLINE scan: resilvered 3.15G in 0h0m with 0 errors on Thu Jun 21 17:02: config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE Oracle Solaris ZFS 79

80 ZFS 3 9 ( ) mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335e106bd0 ONLINE spares c0t5000c500335dc60fd0 AVAIL errors: No known data errors FMA. # fmadm faulty # fmadm repaired zfs://pool=name/vdev=guid ZFS zpool get.. # zpool get all zeepool NAME PROPERTY VALUE SOURCE zeepool allocated 6.29G - zeepool altroot - default zeepool autoexpand off default zeepool autoreplace off default zeepool bootfs - default zeepool cachefile - default zeepool capacity 1% - zeepool dedupditto 0 default zeepool dedupratio 1.00x - zeepool delegation on default zeepool failmode wait default zeepool free 550G - zeepool guid zeepool health ONLINE - zeepool listshares off default zeepool listsnapshots off default zeepool readonly off - zeepool size 556G - zeepool version 34 default zpool set.. # zpool set autoreplace=on zeepool # zpool get autoreplace zeepool NAME PROPERTY VALUE SOURCE zeepool autoreplace on local 80 Oracle Solaris 11.1 : ZFS

81 ZFS. # zpool set autoreplace=on tank cannot set property for tank : out of space 12, Oracle Solaris ZFS. 3 1 ZFS allocated. altroot off... autoreplace off. off zpool replace. on. replace. bootfs.. cachefile.... zpool import - c. ZFS. capacity. cap. dedupditto,. dedupratio ( ). 3 Oracle Solaris ZFS 81

82 ZFS 3 1 ZFS ( ) delegation on. 8, Oracle Solaris ZFS. failmode wait... wait zpool clear I/O. I/O. wait. continue I/O EIO.. zpool clear. panic. free. guid. health ONLINE, DEGRADED, SUSPENDED, REMOVED UNAVAIL. listshares Off zfs list. off. listsnapshots Off zfs list. zfslist-tsnapshot. readonly Off.. /. 82 Oracle Solaris 11.1 : ZFS

83 ZFS 3 1 ZFS ( ) size. version. zpool upgrade. 1 zpool upgrade -v. ZFS zpool list., I/O.. 83 ZFS 87 ZFS I/O 90 ZFS ZFS zpool list. zpool list. # zpool list NAME SIZE ALLOC FREE CAP HEALTH ALTROOT tank 80.0G 22.3G 47.7G 28% ONLINE - dozer 1.2T 384G 816G 32% ONLINE -. NAME. SIZE. ALLOC.. 31 ZFS. 3 Oracle Solaris ZFS 83

84 ZFS FREE CAP (CAPACITY) HEALTH ZFS. ALTROOT ( ). 265 ZFS.. : # zpool list tank NAME SIZE ALLOC FREE CAP HEALTH ALTROOT tank 80.0G 22.3G 47.7G 28% ONLINE - zpool list. -T.. # zpool list -T d32 Tue Nov 2 10:36:11 MDT 2010 NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT pool 33.8G 83.5K 33.7G 0% 1.00x ONLINE - rpool 33.8G 12.2G 21.5G 36% 1.00x ONLINE - Tue Nov 2 10:36:14 MDT 2010 pool 33.8G 83.5K 33.7G 0% 1.00x ONLINE - rpool 33.8G 12.2G 21.5G 36% 1.00x ONLINE - zpool status -l.. fmadm add-alias.. # fmadm add-alias SUN-Storage-J QCQ015 Lab10Rack5... # zpool status -l tank pool: tank state: ONLINE scan: scrub repaired 0 in 0h0m with 0 errors on Fri Aug 3 16:00: config: NAME STATE READ WRITE CKSUM tank ONLINE Oracle Solaris 11.1 : ZFS

85 ZFS mirror-0 ONLINE /dev/chassis/lab10rack5.../disk_02/disk ONLINE /dev/chassis/lab10rack5.../disk_20/disk ONLINE mirror-1 ONLINE /dev/chassis/lab10rack5.../disk_22/disk ONLINE /dev/chassis/lab10rack5.../disk_14/disk ONLINE mirror-2 ONLINE /dev/chassis/lab10rack5.../disk_10/disk ONLINE /dev/chassis/lab10rack5.../disk_16/disk ONLINE mirror-3 ONLINE /dev/chassis/lab10rack5.../disk_01/disk ONLINE /dev/chassis/lab10rack5.../disk_21/disk ONLINE mirror-4 ONLINE /dev/chassis/lab10rack5.../disk_23/disk ONLINE /dev/chassis/lab10rack5.../disk_15/disk ONLINE mirror-5 ONLINE /dev/chassis/lab10rack5.../disk_09/disk ONLINE /dev/chassis/lab10rack5.../disk_04/disk ONLINE mirror-6 ONLINE /dev/chassis/lab10rack5.../disk_08/disk ONLINE /dev/chassis/lab10rack5.../disk_05/disk ONLINE mirror-7 ONLINE /dev/chassis/lab10rack5.../disk_07/disk ONLINE /dev/chassis/lab10rack5.../disk_11/disk ONLINE mirror-8 ONLINE /dev/chassis/lab10rack5.../disk_06/disk ONLINE /dev/chassis/lab10rack5.../disk_19/disk ONLINE mirror-9 ONLINE /dev/chassis/lab10rack5.../disk_00/disk ONLINE /dev/chassis/lab10rack5.../disk_13/disk ONLINE mirror-10 ONLINE /dev/chassis/lab10rack5.../disk_03/disk ONLINE /dev/chassis/lab10rack5.../disk_18/disk ONLINE spares /dev/chassis/lab10rack5.../disk_17/disk AVAIL /dev/chassis/lab10rack5.../disk_12/disk AVAIL errors: No known data errors -o..,. # zpool list -o name,size NAME SIZE tank 80.0G dozer 1.2T Oracle Solaris ZFS 85

86 ZFS ZFS zpool list. -H.,. # zpool list -Ho name tank dozer. # zpool list -H -o name,size tank 80.0G dozer 1.2T ZFS ZFS zfs zpool. zpool history.,. # zpool history History for rpool : :02:55 zpool create -f rpool c3t0d0s :02:56 zfs create -p -o mountpoint=/export rpool/export :02:58 zfs set mountpoint=/export rpool/export :02:58 zfs create -p rpool/export/home :03:03 zfs create -p -V 2048m rpool/swap :03:08 zfs set primarycache=metadata rpool/swap :03:09 zfs create -p -V 4094m rpool/dump :26:47 zpool set bootfs=rpool/root/s11u1 rpool :31:15 zfs set primarycache=metadata rpool/swap :31:46 zfs create -o canmount=noauto -o mountpoint=/var/share rpool/varshare :22:33 zfs set primarycache=metadata rpool/swap :42:48 zfs set primarycache=metadata rpool/swap :17:24 zfs snapshot -r rpool/root@yesterday :17:54 zfs snapshot -r rpool/root@now ZFS....,.. 128KB. 32MB. 86 Oracle Solaris 11.1 : ZFS

87 ZFS 1%...,.. # zpool history tank :35:32 zpool create -f tank mirror c3t1d0 c3t2d0 spare c3t3d :04:10 zfs create tank/test :05:01 zfs snapshot -r tank/test@snap1 -l,.. # zpool history -l tank History for tank : :35:32 zpool create -f tank mirror c3t1d0 c3t2d0 spare c3t3d0 [user root on tardis:global] :04:10 zfs create tank/test [user root on tardis:global] :05:01 zfs snapshot -r tank/test@snap1 [user root on tardis:global] -i.. # zpool history -i tank History for tank : :35:32 zpool create -f tank mirror c3t1d0 c3t2d0 spare c3t3d :35:32 [internal pool create txg:5] pool spa 33; zfs spa 33; zpl 5; uts tardis sun4v :04:10 zfs create tank/test :04:10 [internal property set txg:66094] $share2=2 dataset = :04:31 [internal snapshot txg:66095] dataset = :05:01 zfs snapshot -r tank/test@snap :08:00 [internal user hold txg:66102] <.send > temp = 1... ZFS I/O I/O zpool iostat. iostat I/O.. alloc capacity.. 31 ZFS. 3 Oracle Solaris ZFS 87

88 ZFS free capacity read operations write operations read bandwidth write bandwidth. used. I/O. I/O. ( ).. I/O zpool iostat. : # zpool iostat capacity operations bandwidth pool alloc free read write read write rpool 6.05G 61.9G tank 31.3G 36.7G K 86.1K # zpool iostat tank 2 capacity operations bandwidth pool alloc free read write read write tank 18.5G 49.5G M tank 18.5G 49.5G M tank 18.5G 49.5G M tank 18.8G 49.2G M Ctrl-C 2 tank. count., zpool iostat Oracle Solaris 11.1 : ZFS

89 ZFS I/O I/O zpool iostat I/O. ZFS I/O. I/O zpool iostat -v. : # zpool iostat -v capacity operations bandwidth pool alloc free read write read write rpool 6.05G 61.9G mirror 6.05G 61.9G c1t0d0s c1t1d0s tank 36.5G 31.5G K 146K mirror 36.5G 31.5G M 4.01M c1t2d K 386K c1t3d K 386K I/O.,. RAID-Z.,., RAID-Z. I/O... ( )... # zpool iostat -lv capacity operations bandwidth pool alloc free read write read write export 2.39T 2.14T K 300K mirror 490G 438G K 60.3K /dev/chassis/lab10rack15/scsi_device 2/disk K 60.3K /dev/chassis/lab10rack15/scsi_device 3/disk K 60.3K mirror 490G 438G K 59.9K /dev/chassis/lab10rack15/scsi_device 4/disk K 59.9K /dev/chassis/lab10rack15/scsi_device 5/disk K 59.9K mirror 490G 438G K 60.2K 3 Oracle Solaris ZFS 89

90 ZFS /dev/chassis/lab10rack15/scsi_device 6/disk K 60.2K /dev/chassis/lab10rack15/scsi_device 7/disk K 60.2K mirror 490G 438G K 60.1K /dev/chassis/lab10rack15/scsi_device 8/disk K 60.1K /dev/chassis/lab10rack15/scsi_device 9/disk K 60.1K... ZFS ZFS.. zpool status. fmd,, /var/adm/messages... 10, Oracle Solaris ZFS. 4. DEGRADED. ONLINE. SUSPENDED. SUSPENDED. UNAVAIL.. DEGRADED. RAID-Z.. OFFLINE. ONLINE.. 90 Oracle Solaris 11.1 : ZFS

91 ZFS REMOVED UNAVAIL... UNAVAIL DEGRADED. UNAVAIL.. ONLINE ONLINE. DEGRADED UNAVAIL DEGRADED. UNAVAIL OFFLINE UNAVAIL SUSPENDED. UNAVAIL SUSPENDED.. DEGRADED,. zpool status... scan: resilver in progress since Wed Jun 20 14:19: G scanned out of 71.8G at 36.4M/s, 0h30m to go 7.43G resilvered, 10.35% done.. scan: scrub in progress since Wed Jun 20 14:56: M scanned out of 71.8G at 48.1M/s, 0h25m to go 0 repaired, 0.72% done.. scan: resilvered 71.8G in 0h14m with 0 errors on Wed Jun 20 14:33: scan: scrub repaired 0 in 0h11m with 0 errors on Wed Jun 20 15:08: scan: scrub canceled on Wed Jun 20 16:04: zpool status. # zpool status -x all pools are healthy. ONLINE. 3 Oracle Solaris ZFS 91

92 ZFS -v.. # zpool status -v pond pool: pond state: DEGRADED status: One or more devices are unavailable in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Determine if the device needs to be replaced, and clear the errors using zpool clear or fmadm repaired, or replace the device with zpool replace. scan: scrub repaired 0 in 0h0m with 0 errors on Wed Jun 20 15:38: config: device details: NAME STATE READ WRITE CKSUM pond DEGRADED mirror-0 DEGRADED c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 UNAVAIL mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE c0t5000c500335f907fd0 UNAVAIL cannot open status: ZFS detected errors on this device. The device was missing. see: for recovery errors: No known data errors.... UNAVAIL. zpool online.. # zpool online pond c0t5000c500335f907fd0 warning: device c0t5000c500335dc60fd0 onlined, but remains in degraded state # zpool status -x all pools are healthy. 92 Oracle Solaris 11.1 : ZFS

93 ZFS autoreplace on... # zpool status -x pool: pond state: DEGRADED status: One or more devices has been taken offline by the administrator. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Online the device using zpool online or replace the device with zpool replace. config: NAME STATE READ WRITE CKSUM pond DEGRADED mirror-0 DEGRADED c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 OFFLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE errors: No known data errors READ WRITE I/O, CKSUM.,. 0. errors:.. UNAVAIL 10, Oracle Solaris ZFS. ZFS zpool status. -T.. # zpool status -T d32 Wed Jun 20 16:10:09 MDT 2012 pool: pond state: ONLINE scan: resilvered 9.50K in 0h0m with 0 errors on Wed Jun 20 16:07: config: NAME STATE READ WRITE CKSUM 3 Oracle Solaris ZFS 93

94 ZFS pond ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE errors: No known data errors pool: rpool state: ONLINE scan: scrub repaired 0 in 0h11m with 0 errors on Wed Jun 20 15:08: config: NAME STATE READ WRITE CKSUM rpool ONLINE mirror-0 ONLINE c0t5000c500335ba8c3d0s0 ONLINE c0t5000c500335fc3e7d0s0 ONLINE errors: No known data errors Wed Jun 20 16:10:12 MDT 2012 pool: pond state: ONLINE scan: resilvered 9.50K in 0h0m with 0 errors on Wed Jun 20 16:07: config: NAME STATE READ WRITE CKSUM pond ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE errors: No known data errors pool: rpool state: ONLINE scan: scrub repaired 0 in 0h11m with 0 errors on Wed Jun 20 15:08: config: NAME STATE READ WRITE CKSUM rpool ONLINE mirror-0 ONLINE c0t5000c500335ba8c3d0s0 ONLINE c0t5000c500335fc3e7d0s0 ONLINE errors: No known data errors 94 Oracle Solaris 11.1 : ZFS

95 ZFS ZFS.. SAN. ZFS. 206 ZFS. 95 ZFS 95 ZFS ZFS 98 ZFS 101 ZFS ZFS.,.,., UNAVAIL... ZFS zpool export.. # zpool export tank. -f.. 3 Oracle Solaris ZFS 95

96 ZFS # zpool export tank cannot unmount /export/home/eric : Device busy # zpool export -f tank tank... ZFS -f. ZFS. ZFS 257 ZFS. ( ). ZFS,.. ZFS. zpool import.. # zpool import pool: tank id: state: ONLINE action: The pool can be imported using its name or numeric identifier. config: tank mirror-0 c1t0d0 c1t1d0 ONLINE ONLINE ONLINE ONLINE tank... zpool status zpool import Oracle Solaris 11.1 : ZFS

97 ZFS DEGRADED.. # zpool import pool: tank id: state: DEGRADED status: One or more devices are unavailable. action: The pool can be imported despite missing or damaged devices. fault tolerance of the pool may be compromised if imported. config: The tank DEGRADED mirror-0 DEGRADED c0t5000c500335e106bd0 ONLINE c0t5000c500335fc3e7d0 UNAVAIL cannot open device details: c0t5000c500335fc3e7d0 UNAVAIL cannot open status: ZFS detected errors on this device. The device was missing... 2 RAID-Z.. ZFS ZFS.. # zpool import pool: mothership id: state: UNAVAIL status: One or more devices are unavailable. action: The pool cannot be imported due to unavailable devices or data. config: device details: mothership UNAVAIL insufficient replicas raidz1-0 UNAVAIL insufficient replicas c8t0d0 UNAVAIL cannot open c8t1d0 UNAVAIL cannot open c8t2d0 ONLINE c8t3d0 ONLINE c8t0d0 UNAVAIL cannot open status: ZFS detected errors on this device. The device was missing. 3 Oracle Solaris ZFS 97

98 ZFS c8t1d0 UNAVAIL cannot open status: ZFS detected errors on this device. The device was missing. ZFS zpool import /dev/dsk. -d.. # zpool create dozer mirror /file/a /file/b # zpool export dozer # zpool import -d /file pool: dozer id: state: ONLINE action: The pool can be imported using its name or numeric identifier. config: dozer ONLINE mirror-0 ONLINE /file/a ONLINE /file/b ONLINE # zpool import -d /file dozer -d. ZFS zpool import.. # zpool import tank.. # zpool import pool: dozer id: state: ONLINE action: The pool can be imported using its name or numeric identifier. config: dozer c1t9d0 ONLINE ONLINE pool: dozer id: Oracle Solaris 11.1 : ZFS

99 ZFS state: ONLINE action: The pool can be imported using its name or numeric identifier. config: dozer ONLINE c1t8d0 ONLINE # zpool import dozer cannot import dozer : more than one matching pool import by numeric ID instead # zpool import # zpool import dozer zeepool zeepool dozer.. ZFS -f.. # zpool import dozer cannot import dozer : pool may be in use on another system use -f to import anyway # zpool import -f dozer. ZFS,. -R. 265 ZFS.. zpool import -m.. # zpool import dozer pool: dozer id: state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. see: 3 Oracle Solaris ZFS 99

100 ZFS config: device details: dozer UNAVAIL missing device mirror-0 ONLINE c8t0d0 ONLINE c8t1d0 ONLINE missing-1 UNAVAIL corrupted data status: ZFS detected errors on this device. The device has bad label or disk contents. Additional devices are known to be part of this pool, though their exact configuration cannot be determined... # zpool import -m dozer # zpool status dozer pool: dozer state: DEGRADED status: One or more devices are unavailable in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Determine if the device needs to be replaced, and clear the errors using zpool clear or fmadm repaired, or replace the device with zpool replace. Run zpool status -v to see device specific details. scan: none requested config: NAME STATE READ WRITE CKSUM dozer DEGRADED mirror-0 ONLINE c8t0d0 ONLINE c8t1d0 ONLINE logs UNAVAIL errors: No known data errors zpool clear... zpool clear Oracle Solaris 11.1 : ZFS

101 ZFS # zpool import -o readonly=on tank # zpool scrub tank cannot scrub tank: pool is read-only # zpool export tank # zpool import tank # zpool scrub tank /dev/dsk/c2t3d0 dpool. # zpool import -d /dev/dsk/c2t3d0s0 dpool # zpool status dpool pool: dpool state: ONLINE scan: resilvered 952K in 0h0m with 0 errors on Fri Jun 29 16:22: config: NAME STATE READ WRITE CKSUM dpool ONLINE mirror-0 ONLINE c2t3d0 ONLINE c2t1d0 ONLINE ZFS zpool import -D.. # zpool destroy tank # zpool import -D pool: tank id: state: ONLINE (DESTROYED) 3 Oracle Solaris ZFS 101

102 ZFS action: The pool can be imported using its name or numeric identifier. config: tank mirror-0 c1t0d0 c1t1d0 ONLINE ONLINE ONLINE ONLINE zpool import tank. state: ONLINE (DESTROYED) zpool import - D.. # zpool import -D tank # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c1t0d0 ONLINE c1t1d0 ONLINE errors: No known data errors -f.. : # zpool destroy dozer # zpool import -D pool: dozer id: state: DEGRADED (DESTROYED) status: One or more devices are unavailable. action: The pool can be imported despite missing or damaged devices. fault tolerance of the pool may be compromised if imported. config: The device details: dozer DEGRADED raidz2-0 DEGRADED c8t0d0 ONLINE c8t1d0 ONLINE c8t2d0 ONLINE c8t3d0 UNAVAIL cannot open c8t4d0 ONLINE 102 Oracle Solaris 11.1 : ZFS

103 ZFS c8t3d0 UNAVAIL cannot open status: ZFS detected errors on this device. The device was missing. # zpool import -Df dozer # zpool status -x pool: dozer state: DEGRADED status: One or more devices are unavailable in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Determine if the device needs to be replaced, and clear the errors using zpool clear or fmadm repaired, or replace the device with zpool replace. Run zpool status -v to see device specific details. scan: none requested config: NAME STATE READ WRITE CKSUM dozer DEGRADED raidz2-0 DEGRADED c8t0d0 ONLINE c8t1d0 ONLINE c8t2d0 ONLINE UNAVAIL c8t4d0 ONLINE errors: No known data errors # zpool online dozer c8t4d0 # zpool status -x all pools are healthy ZFS Solaris ZFS zpool upgrade. zpool status. : # zpool status pool: tank state: ONLINE status: The pool is formatted using an older on-disk format. The pool can still be used, but some features are unavailable. action: Upgrade the pool using zpool upgrade. Once this is done, the pool will no longer be accessible on older software versions. scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c1t0d0 ONLINE c1t1d0 ONLINE errors: No known data errors 3 Oracle Solaris ZFS 103

104 ZFS. # zpool upgrade -v This system is currently running ZFS pool version 33. The following versions are supported: VER DESCRIPTION Initial ZFS version 2 Ditto blocks (replicated metadata) 3 Hot spares and double parity RAID-Z 4 zpool history 5 Compression using the gzip algorithm 6 bootfs pool property 7 Separate intent log devices 8 Delegated administration 9 refquota and refreservation properties 10 Cache devices 11 Improved scrub performance 12 Snapshot properties 13 snapused property 14 passthrough-x aclinherit 15 user/group space accounting 16 stmf property support 17 Triple-parity RAID-Z 18 Snapshot user holds 19 Log device removal 20 Compression using zle (zero-length encoding) 21 Deduplication 22 Received properties 23 Slim ZIL 24 System attributes 25 Improved scrub stats 26 Improved snapshot deletion performance 27 Improved snapshot creation performance 28 Multiple vdev replacements 29 RAID-Z/mirror hybrid allocator 30 Encryption 31 Improved zfs list performance 32 One MB blocksize 33 Improved share support 34 Sharing with inheritance For more information on a particular version, including supported releases, see the ZFS Administration Guide. zpool upgrade.. # zpool upgrade -a 104 Oracle Solaris 11.1 : ZFS

105 ZFS ZFS ZFS. 3 Oracle Solaris ZFS 105

106 106

107 4 4 ZFS, ZFS Oracle Solaris ZFS ZFS ( ) 108 ZFS 109 ZFS 121 ZFS 124 ZFS 11,. Oracle Solaris ZFS ( ) ZFS Oracle Solaris 11. Oracle Solaris. Oracle Solaris 11 ZFS. Live CD(x86 ) ZFS. fdisk. (SPARC x86) ZFS. fdisk. (AI)(SPARC x86) ZFS.AI ZFS. 107

108 ZFS ( ) ZFS ZFS. ZFS 121 ZFS (SPARC x86/vtoc). ZFS. ZFS., 68GB ZFS (rpool/root/solaris) 67GB 1GB. 180 ZFS. ZFS ZFS. ZFS. ZFS, BE( ). ZFS. Oracle Solaris GB.. Solaris...,. 122 ZFS. BE( ) ZFS BE 4-6GB. ZFS BE ZFS BE. BE BE. ZFS BE. Oracle Solaris OS /var OS. Solaris OS. 108 Oracle Solaris 11.1 : ZFS

109 ZFS ZFS ZFS. Oracle Solaris 11.1 x86 EFI(GPT) SMI(VTOC) SPARC SMI(VTOC). x86 EFI(GPT). EFI(GPT) x86 43 ZFS. SPARC x86 SMI(VTOC). x86 EFI(GPT). beadm. ERROR: ZFS pool name does not support boot environments ZFS 51 ZFS. x86 Solaris fdisk. Solaris fdisk x86. Solaris fdisk Oracle Solaris 11.1 : fdisk.. gzip... ZFS ZFS. ZFS Oracle Solaris 11 Live CD ZFS. Oracle Solaris 11 AI( ) ZFS AI. 4 ZFS 109

110 ZFS AI ZFS. c1t0d0. MPxIO ID. ZFS... # zpool status rpool pool: rpool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE mirror-0 ONLINE c8t0d0 ONLINE c8t1d0 ONLINE # zfs list NAME USED AVAIL REFER MOUNTPOINT rpool 11.8G 55.1G 4.58M /rpool rpool/root 3.57G 55.1G 31K legacy rpool/root/solaris 3.57G 55.1G 3.40G / rpool/root/solaris/var 165M 55.1G 163M /var rpool/varshare 42.5K 55.1G 42.5K /var/share rpool/dump 6.19G 55.3G 6.00G - rpool/export 63K 55.1G 32K /export rpool/export/home 31K 55.1G 31K /export/home rpool/swap 2.06G 55.2G 2.00G - ZFS BE.. # beadm list BE Active Mountpoint Space Policy Created solaris NR / 3.75G static :10 Active BE (N ), (R ) (NR ). ZFS ZFS BE( ) solaris. beadm list BE.. # beadm list BE Active Mountpoint Space Policy Created solaris NR / 3.82G static :44 NR BE BE. 110 Oracle Solaris 11.1 : ZFS

111 ZFS pkg update ZFS. pkg update ZFS BE BE BE. 1 ZFS BE. # pkg update DOWNLOAD PKGS FILES XFER (MB) Completed 707/ / / BE solaris-1. BE. # beadm create solaris-1 # beadm activate solaris BE. BE. # init 6... # beadm list BE Active Mountpoint Space Policy Created solaris M static :25 solaris-1 NR / 3.82G static :45 BE BE. # beadm activate solaris # init BE BE.. BE. # beadm mount solaris-1 /mnt BE. # ls /mnt bin export media pkg rpool tmp boot home mine platform sbin usr 4 ZFS 111

112 ZFS dev import mnt proc scde var devices java net project shared doe kernel nfs4 re src etc lib opt root system 4 BE. # beadm umount solaris-1 (SPARC x86/vtoc). 115 ZFS (SPARC x86/vtoc). 1. # zpool status rpool pool: rpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE c2t0d0s0 ONLINE errors: No known data errors 2 3. SPARC: SMI(VTOC), 0. 0 Oracle Solaris 11.1 : ZFS. x86: fdisk, SMI, 0. 0 Oracle Solaris 11.1 : ZFS.. # zpool attach rpool c2t0d0s0 c2t1d0s0 Make sure to wait until resilver is done before rebooting Oracle Solaris 11.1 : ZFS

113 ZFS 4. # zpool status rpool # zpool status rpool pool: rpool state: DEGRADED status: One or more devices is currently being resilvered. The pool will continue to function in a degraded state. action: Wait for the resilver to complete. Run zpool status -v to see device specific details. scan: resilver in progress since Fri Jul 20 13:39: M scanned out of 11.7G at 46.9M/s, 0h3m to go 938M resilvered, 7.86% done config: NAME STATE READ WRITE CKSUM rpool DEGRADED mirror-0 DEGRADED c2t0d0s0 ONLINE c2t1d0s0 DEGRADED (resilvering).. resilvered 11.6G in 0h5m with 0 errors on Fri Jul 20 13:57: autoexpand. rpool : # zpool list rpool NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT rpool 29.8G 152K 29.7G 0% 1.00x ONLINE - # zpool set autoexpand=on rpool rpool : # zpool list rpool NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT rpool 279G 146K 279G 0% 1.00x ONLINE - 6. (x86/efi(gpt)) Oracle Solaris 11.1 x86 EFI(GPT).. 4 ZFS 113

114 ZFS 115 ZFS (SPARC x86/vtoc). 1. # zpool status rpool pool: rpool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE c2t0d0 ONLINE errors: No known data errors 2. # zpool attach rpool c2t0d0 c2t1d0 Make sure to wait until resilver is done before rebooting... # zpool attach rpool c2t0d0s0 c2t1d0 3. # zpool status rpool pool: rpool state: DEGRADED status: One or more devices is currently being resilvered. The pool will continue to function in a degraded state. action: Wait for the resilver to complete. Run zpool status -v to see device specific details. scan: resilver in progress since Fri Jul 20 13:52: M scanned out of 11.6G at 44.9M/s, 0h4m to go 776M resilvered, 6.82% done config: NAME STATE READ WRITE CKSUM rpool DEGRADED mirror-0 DEGRADED c8t0d0 ONLINE c8t1d0 DEGRADED (resilvering) errors: No known data errors.. resilvered 11.6G in 0h5m with 0 errors on Fri Jul 20 13:57: Oracle Solaris 11.1 : ZFS

115 ZFS 4 autoexpand. rpool : # zpool list rpool NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT rpool 29.8G 152K 29.7G 0% 1.00x ONLINE - # zpool set autoexpand=on rpool rpool : # zpool list rpool NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT rpool 279G 146K 279G 0% 1.00x ONLINE - 5. ZFS (SPARC x86/vtoc).. ( :CD ). zpool replace.. zpool replace, zpool attach.. SATA zpool replace.. # zpool offline rpool c1t0d0s0 # cfgadm -c unconfigure c1::dsk/c1t0d0 <Physically remove failed disk c1t0d0> <Physically insert replacement disk c1t0d0> # cfgadm -c configure c1::dsk/c1t0d0 <Confirm that the new disk has an SMI label and a slice 0> # zpool replace rpool c1t0d0s0 # zpool online rpool c1t0d0s0 # zpool status rpool 4 ZFS 115

116 ZFS <Let disk resilver before installing the boot blocks> # bootadm install-bootloader SPARC: ( ) SMI(VTOC), 0. Oracle Solaris 11.1 :. x86: fdisk, SMI, 0. 0 Oracle Solaris 11.1 : ZFS... # zpool attach rpool c2t0d0s0 c2t1d0s0 Make sure to wait until resilver is done before rebooting.... # zpool status rpool pool: rpool state: ONLINE scan: resilvered 11.7G in 0h5m with 0 errors on Fri Jul 20 13:45: config: NAME STATE READ WRITE CKSUM rpool ONLINE mirror-0 ONLINE c2t0d0s0 ONLINE c2t1d0s0 ONLINE errors: No known data errors 5., SPARC. ok boot /pci@1f,700000/scsi@2/disk@1,0 116 Oracle Solaris 11.1 : ZFS

117 ZFS. (c2t0d0s0). (c2t1d0s0). boot # zpool detach rpool c2t0d0s0. SPARC: eeprom PROM setenv. x86: BIOS. ZFS (x86/efi(gpt)) Oracle Solaris 11.1 x86 EFI(GPT)... ( :CD ). zpool replace.. zpool replace, zpool attach.. SATA zpool replace.. # zpool offline rpool c1t0d0 # cfgadm -c unconfigure c1::dsk/c1t0d0 <Physically remove failed disk c1t0d0> 4 ZFS 117

118 ZFS <Physically insert replacement disk c1t0d0> # cfgadm -c configure c1::dsk/c1t0d0 # zpool replace rpool c1t0d0 # zpool online rpool c1t0d0 # zpool status rpool <Let disk resilver before installing the boot blocks> x86# bootadm install-bootloader # zpool attach rpool c2t0d0 c2t1d0 Make sure to wait until resilver is done before rebooting.... # zpool status rpool pool: rpool state: ONLINE scan: resilvered 11.6G in 0h5m with 0 errors on Fri Jul 20 12:06: config: NAME STATE READ WRITE CKSUM rpool ONLINE mirror-0 ONLINE c2t0d0 ONLINE c2t1d0 ONLINE errors: No known data errors # zpool detach rpool c2t0d0. BIOS. 118 Oracle Solaris 11.1 : ZFS

119 ZFS BE (SPARC x86/vtoc) BE. BE BE. BE BE. BE SMI(VTOC).. # zpool create rpool2 c4t2d0s0 BE.. # beadm create -p rpool2 solaris2 bootfs.. # zpool set bootfs=rpool2/root/solaris2 rpool2 BE.. # beadm activate solaris2 BE. ok boot disk2 BE... # zfs create -V 4g rpool2/swap /etc/vfstab.. /dev/zvol/dsk/rpool2/swap - - swap - no -.. # zfs create -V 4g rpool2/dump.. # dumpadm -d /dev/zvol/dsk/rpool2/dump. SPARC - eeprom PROM setenv. 4 ZFS 119

120 ZFS x86 - BIOS. 11. # init 6 BE (x86/efi(gpt)) Oracle Solaris 11.1 x86 EFI(GPT). BE. BE BE. BE BE. BE. 1. # zpool create -B rpool2 c2t2d0.. # zpool create -B rpool2 mirror c2t2d0 c2t3d BE.. # beadm create -p rpool2 solaris2.. # bootadm install-bootloader -P rpool2 bootfs.. # zpool set bootfs=rpool2/root/solaris2 rpool2 BE.. # beadm activate solaris2 BE. SPARC - eeprom PROM setenv. x86 - BIOS. BE. 120 Oracle Solaris 11.1 : ZFS

121 ZFS # zfs create -V 4g rpool2/swap /etc/vfstab.. /dev/zvol/dsk/rpool2/swap - - swap - no -.. # zfs create -V 4g rpool2/dump.. # dumpadm -d /dev/zvol/dsk/rpool2/dump. # init 6 ZFS ZFS ZFS.. # swap -l swapfile dev swaplo blocks free /dev/zvol/dsk/rpool/swap 145, ZFS ZFS... # dumpadm Dump content: kernel pages Dump device: /dev/zvol/dsk/rpool/dump (dedicated) Savecore directory: /var/crash/ Savecore enabled: yes Save compressed: on dumpadm. zfs. 108 ZFS ZFS. ZFS. ZFS. 4 ZFS 121

122 ZFS ZFS. Solaris swap dumpadm. Oracle Solaris 11.1 : 16, ( ) Oracle Solaris , ( ). ZFS,. volsize.. # zfs set volsize=2g rpool/dump # zfs get volsize rpool/dump NAME PROPERTY VALUE SOURCE rpool/dump volsize 2G -.. # swap -d /dev/zvol/dsk/rpool/swap # zfs set volsize=2g rpool/swap # swap -a /dev/zvol/dsk/rpool/swap # init 6 Oracle Solaris 11.1 : Oracle Solaris ZFS... # zfs create -V 2G rpool/swap2.. # swap -a /dev/zvol/dsk/rpool/swap2 # swap -l swapfile dev swaplo blocks free /dev/zvol/dsk/rpool/swap 256, /dev/zvol/dsk/rpool/swap2 256, /etc/vfstab.. /dev/zvol/dsk/rpool/swap2 - - swap - no Oracle Solaris 11.1 : ZFS

123 ZFS ZFS. savecore. ZFS ZFS.., 40GB. # zfs set volsize=40g rpool/dump.. # dumpadm -d /dev/zvol/dsk/rpool/dump Dump content: kernel pages Dump device: /dev/zvol/dsk/rpool/dump (dedicated) Savecore directory: /var/crash/ Savecore enabled: yes Save compressed: on 128GB.. # dumpadm -d /dev/zvol/dsk/rpool/dump dumpadm: dump device /dev/zvol/dsk/rpool/dump is too small to hold a system dump dump size bytes, device size bytes Oracle Solaris 11.1 :... # dumpadm -d /dev/zvol/dsk/datapool/dump dump is not supported on device /dev/zvol/dsk/datapool/dump : datapool has multiple top level vdevs. 4 ZFS 123

124 ZFS ZFS SPARC x86. ZFS. ZFS UFS. ZFS. ZFS. ZFS. bootfs. SPARC boot -Z x86 BIOS. ZFS ZFS. 112 (SPARC x86/vtoc). ZFS. ZFS. PROM BIOS., (c1t0d0s0 c1t1d0s0). # zpool status pool: rpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE mirror-0 ONLINE c1t0d0s0 ONLINE c1t1d0s0 ONLINE SPARC ok. ok boot /pci@7c0/pci@0/pci@1/pci@0,2/lsilogic,sas@2/disk@1.. SPARC# prtconf -vp grep bootpath bootpath: /pci@7c0/pci@0/pci@1/pci@0,2/lsilogic,sas@2/disk@1,0:a 124 Oracle Solaris 11.1 : ZFS

125 ZFS x86. x86# prtconf -v sed -n /bootpath/,/value/p name= bootpath type=string items=1 value= x86 BIOS ZFS. SPARC x86: zpool replace bootadm. ZFS zpool attach. bootadm. # bootadm install-bootloader -P( ). # bootadm install-bootloader -P rpool2 GRUB installgrub. x86# installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c0t1d0s0 ZFS. PROM BIOS., (c1t0d0s0 c1t1d0s0). # zpool status pool: rpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE mirror-0 ONLINE c1t0d0s0 ONLINE c1t1d0s0 ONLINE SPARC ok. ok boot /pci@7c0/pci@0/pci@1/pci@0,2/lsilogic,sas@2/disk@1.. SPARC# prtconf -vp grep bootpath bootpath: /pci@7c0/pci@0/pci@1/pci@0,2/lsilogic,sas@2/disk@1,0:a x86. x86# prtconf -v sed -n /bootpath/,/value/p name= bootpath type=string items=1 value= /pci@0,0/pci8086,25f8@4/pci108e,286@0/disk@0,0:a x86 BIOS ZFS. 4 ZFS 125

126 ZFS SPARC ZFS ZFS BE SPARC beadm activate BE. beadm ZFS bootfs.. /pool-name/boot/menu.lst bootfs, menu.lst bootfs. menu.lst. ZFS menu.lst. title Oracle Solaris 11.1 SPARC bootfs rpool/root/solaris BE menu.lst. title Oracle Solaris 11.1 SPARC bootfs rpool/root/solaris title solaris bootfs rpool/root/solaris2 SPARC BE. ZFS BE boot -L ZFS.... -Z file system ZFS. BE. boot -L -Z BE BE. 4 1 ZFS ZFS ZFS BE beadm activate BE., ZFS BE beadm. # beadm list BE Active Mountpoint Space Policy Created solaris NR / 3.80G static :25 solaris M static : Oracle Solaris 11.1 : ZFS

127 ZFS 4 1 ZFS ( ) SPARC ZFS BE boot -L.. ok boot -L Boot device: /pci@7c0/pci@0/pci@1/pci@0,2/lsilogic,sas@2/disk@0,0:a File and args: -L 1 Oracle Solaris 11.1 SPARC 2 solaris Select environment to boot: [1-2]:1 To boot the selected entry, invoke: boot [<root-device>] -Z rpool/root/solaris-2 Program terminated ok boot -Z rpool/root/solaris-2 BE. boot -Z BE BE. x86 ZFS Oracle Solaris 11 x86 GRUB, ZFS beadm activate /pool-name /boot/grub/menu.lst. title solaris bootfs rpool/root/solaris kernel$ /platform/i86pc/kernel/amd64/unix -B $ZFS-BOOTFS module$ /platform/i86pc/amd64/boot_archive title solaris-1 bootfs rpool/root/solaris-1 kernel$ /platform/i86pc/kernel/amd64/unix -B $ZFS-BOOTFS module$ /platform/i86pc/amd64/boot_archive GRUB ZFS GRUB menu.lst. ZFS BE x86 GRUB BE. ZFS. -B $ZFS-BOOTFS Oracle Solaris 11.1 x86 GRUB2. menu.lst /rpool/boot/grub/grub.cfg. GRUB2 custom.cfg bootadm,. custom.cfg bootadm update-archive. 4 ZFS 127

128 ZFS GRUB Oracle Solaris x86: ZFS GRUB2 ZFS. # bootadm list-menu the location of the boot loader configuration files is: /rpool/boot/grub default 0 console text timeout 30 0 Oracle Solaris 11.1 GRUB ZFS boot -B $ZFS-BOOTFS.. title solaris bootfs rpool/root/solaris kernel$ /platform/i86pc/kernel/amd64/unix -B $ZFS-BOOTFS module$ /platform/i86pc/amd64/boot_archive title solaris-1 bootfs rpool/root/solaris-1 kernel$ /platform/i86pc/kernel/amd64/unix -B $ZFS-BOOTFS module$ /platform/i86pc/amd64/boot_archive 4 3 x86: ZFS x86. BIOS.. beadm activate BE init 6. reboot reboot -f.. # reboot -f ZFS root. menu.lst. 115 ZFS 128 Oracle Solaris 11.1 : ZFS

129 ZFS (SPARC x86/vtoc). ( ) 11,. 1. x86: - GNOME. SPARC: 3 Shell. x86: GRUB Text Installer command line 3 Shell. SPARC:. ok boot net:dhcp x86: PXE. GRUB Text Installer and command line. 3 Shell. 3 Shell. 1 Install Oracle Solaris 2 Install Additional Drivers 3 Shell 4 Terminal type (currently xterm) 5 Reboot Please enter a number [1]: 3 To return to the main menu, exit the shell # 2. /etc/passwd. x86 -s., SPARC. /etc/passwd. # init 0 ok boot -s Boot device: /pci@7c0/pci@0/pci@1/pci@0,2/lsilogic,sas@2/disk@0,0:a... SunOS Release 5.11 Version bit Copyright (c) 1983, 2012, Oracle and/or its affiliates. All rights reserved. Booting to milestone "milestone/single-user:default". Hostname: tardis.central Requesting System Maintenance Mode SINGLE USER MODE Enter user name for system maintenance (control-d to bypass): root 4 ZFS 129

130 ZFS Enter root password (control-d to bypass): xxxx single-user privilege assigned to root on /dev/console. Entering System Maintenance Mode Aug 3 15:46:21 su: su root succeeded for root on /dev/console Oracle Corporation SunOS October 2012 su: No shell /usr/bin/mybash. Trying fallback shell /sbin/sh. [email protected]:~# TERM =vt100; export TERM [email protected]:~# vi /etc/passwd [email protected]:~# <Press control-d> logout svc.startd: Returning to milestone all. x GRUB. bootadm list-menu GRUB2. set-menu. bootadm(1m). x86# zpool import -f rpool x86# bootadm list-menu x86# bootadm set-menu default=1 x86# exit 1 Install Oracle Solaris 2 Install Additional Drivers 3 Shell 4 Terminal type (currently sun-color) 5 Reboot Please enter a number [1]: (rpool) BE. SPARC x86. # zpool import -f rpool # beadm list be_find_current_be: failed to find current BE name be_find_current_be: failed to find current BE name BE Active Mountpoint Space Policy Created solaris M static :25 solaris-2 R G static :44 # mkdir /a # beadm mount solaris-2 /a # TERM=vt100 # export TERM # cd /a/etc # vi shadow <Carefully remove the unknown password> 130 Oracle Solaris 11.1 : ZFS

131 ZFS # cd / # beadm umount solaris-2 # halt. 3.. x86 -s. SPARC.. ok boot -s Boot device: /pci@7c0/pci@0/pci@1/pci@0,2/lsilogic,sas@2/disk@0,0:a... SunOS Release 5.11 Version bit Copyright (c) 1983, 2012, Oracle and/or its affiliates. All rights reserved Booting to milestone "milestone/single-user:default". Enter user name for system maintenance (control-d to bypass): root Enter root password (control-d to bypass): <Press return> single-user privilege assigned to root on /dev/console. Entering System Maintenance Mode Jul 20 14:09:59 su: su root succeeded for root on /dev/console Oracle Corporation SunOS October 2012 [email protected]:~# passwd -r files root New Password: xxxxxx Re-enter new Password: xxxxxx passwd: password successfully changed for root [email protected]:~# <Press control-d> logout svc.startd: Returning to milestone all. 4 ZFS 131

132 132

133 5 5 Oracle Solaris ZFS Oracle Solaris ZFS., ZFS ( ) 134 ZFS, 137 ZFS 157 ZFS 160 ZFS 165 ZFS 170 ZFS 179 ZFS 185 ZFS 192 ZFS 195 ZFS ZFS ( ) ZFS.. ZFS. ZFS zfs. zfs..,. 6, Oracle Solaris ZFS. ZFS 257 ZFS. 133

134 ZFS,,,. ZFS, zfs create zfs destroy ZFS. zfs rename ZFS. 134 ZFS 135 ZFS 136 ZFS ZFS ZFS zfs create. create.. pool-name/[filesystem-name/]filesystem-name.. 30 ZFS. ZFS. ZFS 185 ZFS. jeff tank/home. # zfs create tank/home/jeff ZFS,. create /dataset. jeff /tank/home/jeff. 165 ZFS. zfs create zfs(1m).. /export/zfs tank/home. 134 Oracle Solaris 11.1 : ZFS

135 ZFS, # zfs create -o mountpoint=/export/zfs tank/home 137 ZFS. ZFS ZFS zfs destroy tank/home/mark. # zfs destroy tank/home/mark destroy.. zfs destroy. -f.,. # zfs destroy tank/home/matt cannot unmount tank/home/matt : Device busy # zfs destroy -f tank/home/matt zfs destroy. -r.. # zfs destroy tank/ws cannot destroy tank/ws : filesystem has children use -r to destroy the following datasets: tank/ws/jeff tank/ws/bill tank/ws/mark # zfs destroy -r tank/ws. -R.. # zfs destroy -r tank/home/eric cannot destroy tank/home/eric : filesystem has dependent clones use -R to destroy the following datasets: tank//home/eric-clone # zfs destroy -R tank/home/eric 5 Oracle Solaris ZFS 135

136 ZFS, zfs destroy -f, -r -R. 6, Oracle Solaris ZFS. ZFS zfs rename. rename.. ZFS. ZFS. rename eric eric_old. # zfs rename tank/home/eric tank/home/eric_old zfs rename. # zfs rename tank/home/mark tank/ws/mark mark tank/home tank/ws.,. rename. 179 ZFS. rename /. rename ZFS. 136 Oracle Solaris 11.1 : ZFS

137 ZFS ZFS,, ZFS 150 ZFS 156 ZFS. ZFS.. ZFS. 156 ZFS..... local 160 ZFS zfs set. inherited from dataset-name default.. local. ZFS ZFS. 5 1 ZFS aclinherit secure ACL. discard, noallow, secure passthrough. 223 ACL. 5 Oracle Solaris ZFS 137

138 ZFS 5 1 ZFS ( ) aclmode groupmask chmod ACL. discard, groupmask passthrough. 223 ACL. atime on.. available.,,. avail. 31 ZFS. canmount on zfs mount.. off.. noauto., zfs mount-a zfs unmount-a. 152 canmount. 138 Oracle Solaris 11.1 : ZFS

139 ZFS 5 1 ZFS ( ) casesensitivity mixed casesensitive, caseinsensitive (mixed). UNIX POSIX. mixed. Oracle Solaris SMB. mixed 152 casesensitivity. casesensitivity.. on. ( fletcher4) on. on, off, fletcher2, fletcher4, sha256 sha256+mac. off. off. compression off. on, off, lzjb, gzip gzip- N. lzjb, gzip gzip- N on... compress. compressratio ( ). zfs set compression=on dataset.. compression. 5 Oracle Solaris ZFS 139

140 ZFS 5 1 ZFS ( ) copies 1. 1, used.. creation. dedup off ZFS. on, off, verify sha256[,verify]. sha dedup. devices on. encryption off.. exec on. off, PROT_EXEC mmap(2). keychangedate none zfs key -c.. keysource none. raw, hex, passphrase,prompt file. zfskey-l,. keysource passphrase,prompt. 140 Oracle Solaris 11.1 : ZFS

141 ZFS 5 1 ZFS ( ) keystatus none. available unavailable. none. logbias latency ZFS. logbias latency ZFS ( ). logbias throughput ZFS. ZFS. latency. mlslabel mlslabel multilevel. mlslabel. Trusted Extensions.. none. Trusted Extensions. mounted,.. yes no. mountpoint. mountpoint. legacy. legacy none ZFS. 5 Oracle Solaris ZFS 141

142 ZFS 5 1 ZFS ( ) multilevel off Trusted Extensions. off.., setlabel, setflabel., Oracle Solaris Solaris. mlslabel. mlslabel. mlslabel. mlslabel. mlslabel. ADMIN_HIGH mlslabel. mlslabel, none,. primarycache all (ARC). all, none metadata. all. none. metadata. I/O Oracle Solaris 11.1 : ZFS

143 ZFS 5 1 ZFS ( ) nbmand off nbmand( ). SMB.. normalization.,. none utf8only utf8only on. normalization none.. origin. -r -f. none. quota ( none) none... volsize. 180 ZFS. rekeydate zfs key -K zfs clone -K. rekey creation. 5 Oracle Solaris ZFS 143

144 ZFS 5 1 ZFS ( ) readonly off. on. rdonly. recordsize 128K. recsize. 155 recordsize. referenced,.. refer. refquota ( none) none... refreservation ( none) none. refreservation. refreservation. refreservation. refreserv. 144 Oracle Solaris 11.1 : ZFS

145 ZFS 5 1 ZFS ( ) reservation ( none) none... reserv. 183 ZFS. rstchown On. chown. rstchown off chown PRIV_FILE_CHOWN_SELF. secondarycache all (L2ARC). all, none metadata. all. none. metadata. setuid on setuid. shadow None ZFS ( URI ). URI.. 5 Oracle Solaris ZFS 145

146 ZFS 5 1 ZFS ( ) share.nfs off ZFS NFS. zfs share zfs unshare NFS. zfs share NFS NFS. NFS 170 ZFS. ZFS 170 ZFS. share.smb off ZFS SMB. zfs share zfs unshare SMB. zfs share SMB SMB. SMB 170 ZFS. snapdir hidden.zfs. 197 ZFS. 146 Oracle Solaris 11.1 : ZFS

147 ZFS 5 1 ZFS ( ) sync standard.. standard fsync, O_DSYNC, O_SYNC. always.. disabled... disabled ZFS NFS. /var,.. type filesystem( ), volume snapshot. used. 150 used. usedbychildren off ( ). usedchild. 5 Oracle Solaris ZFS 147

148 ZFS 5 1 ZFS ( ) usedbydataset off refreservation ( ). usedds. usedbyrefreservation off refreservation (refreservation ). usedrefreserv. usedbysnapshots off.. used. usedsnap. version.. zfs upgrade. utf8only Off UTF-8. off normalization none. utf8only off.. volsize. 156 volsize. volblocksize 8KB.. 8KB KB 2. volblock. 148 Oracle Solaris 11.1 : ZFS

149 ZFS 5 1 ZFS ( ) vscan Off.. off. zoned., ZFS.. ZFS 260 Solaris ZFS. xattr on (on) (off). ZFS available compressratio creation keystatus mounted origin referenced rekeydate type used 150 used. usedbychildren 5 Oracle Solaris ZFS 149

150 ZFS usedbydataset usedbyrefreservation usedbysnapshots used, referenced available 31 ZFS. used used....,... ( ). 33.,,.. fsync(3c) O_SYNC. usedbychildren, usedbydataset, usedbyrefreservation usedbysnapshots zfs list -o space. used ZFS. 160 ZFS zfs set 134 ZFS zfs create.,. 179 ZFS. 150 Oracle Solaris 11.1 : ZFS

151 ZFS. 5 1.,, aclinherit 223 ACL. atime canmount casesensitivity compression copies devices dedup encryption exec keysource logbias mlslabel mountpoint nbmand normalization primarycache quota readonly recordsize 155 recordsize. refquota refreservation reservation rstchown secondarycache share.smb share.nfs 5 Oracle Solaris ZFS 151

152 ZFS setuid snapdir version vscan utf8only volsize 156 volsize. volblocksize zoned xattr canmount canmount off zfs mount zfs mount -a. mountpoint off mountpoint none., off,. container. userpool canmount off. /export/home.. # zpool create userpool mirror c0t5d0 c1t6d0 # zfs set canmount=off userpool # zfs set mountpoint=/export/home userpool # zfs set compression=on userpool # zfs create userpool/user1 # zfs create userpool/user2 # zfs mount userpool/user1 /export/home/user1 userpool/user2 /export/home/user2 canmount noauto. casesensitivity casesensitive, caseinsensitive (mixed). 152 Oracle Solaris 11.1 : ZFS

153 ZFS.., foo, Foo FOO. foo ( : foo, FOO, FoO, foo ) 3. foo. foo, FOO, foo, Foo. utf8only, normalization casesensitivity ZFS. 246 ZFS. copies ZFS. (ditto blocks). zfs set copies.. # zfs set copies=2 users/home # zfs get copies users/home NAME PROPERTY VALUE SOURCE users/home copies 2 local 1, RAID-Z. ZFS. ZFS ( (bit rot) )... (ditto block). (ditto block). (ditto block). 5 Oracle Solaris ZFS 153

154 ZFS dedup dedup. dedup.,. dedup # zdb -S tank Simulated DDT histogram: bucket allocated referenced refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE M 239G 188G 194G 2.27M 239G 188G 194G 2 327K 34.3G 27.8G 28.1G 698K 73.3G 59.2G 59.9G K 2.91G 2.10G 2.11G 152K 14.9G 10.6G 10.6G K 691M 529M 529M 74.5K 6.25G 4.79G 4.80G M 25.8M 25.9M 13.1K 822M 492M 494M M 7.02M 7.03M 7.66K 480M 269M 270M M 626K 626K 3.86K 103M 51.2M 51.2M K 250K 251K 3.71K 150M 40.3M 40.3M K 48K 53.7K 2.27K 88.6M 17.3M 19.5M K 7.50K 7.75K 2.74K 102M 5.62M 5.79M 2K 1 2K 2K 2K 3.23K 6.47M 6.47M 6.47M 8K 1 128K 5K 5K 13.9K 1.74G 69.5M 69.5M Total 2.63M 277G 218G 225G 3.22M 337G 263G 270G dedup = 1.20, compress = 1.28, copies = 1.03, dedup * compress / copies = in-core DDT size = 2.63M x 320 = M 3...,. dedup dedup checksum. verify sha256,verify 154 Oracle Solaris 11.1 : ZFS

155 ZFS. verify ZFS... # zfs set dedup=on tank/home zfs get dedup..,.. # zpool list tank NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT rpool 136G 55.2G 80.8G 40% 2.30x ONLINE - DEDUP. dedup dedup DEDUP 1.00x. zpool get dedupratio.. # zpool get dedupratio export NAME PROPERTY VALUE SOURCE rpool dedupratio 3.00x -. encryption encryption ZFS. 185 ZFS. recordsize recordsize.. ZFS.. recordsize., KB 2. recordsize.. 5 Oracle Solaris ZFS 155

156 ZFS recsize. share.smb Oracle Solaris SMB ZFS. off on. off.share.smb 170 ZFS. volsize volsize.. volsize...,.. -s zfs create -V (sparse). (sparse). volsize. 257 ZFS. ZFS ZFS. ZFS.. (':')., ':', '+','.', '_' ZFS. module:property 156 Oracle Solaris 11.1 : ZFS

157 ZFS module DNS. com.oracle. Oracle Corporation # zfs set dept:users=finance userpool/user1 # zfs set dept:users=general userpool/user2 # zfs set dept:users=itops userpool/user3 zfs list, zfs get, zfs set.. zfs get -r dept:users userpool NAME PROPERTY VALUE SOURCE userpool dept:users all local userpool/user1 dept:users finance local userpool/user2 dept:users general local userpool/user3 dept:users itops local zfs inherit.. # zfs inherit -r dept:users userpool. ZFS zfs list.. ZFS zfs list. used, available, referenced mountpoint. 137 ZFS. 5 Oracle Solaris ZFS 157

158 ZFS. # zfs list users users/home 2.00G 2.00G 64.9G 64.9G 32K 35K /users /users/home users/home/cindy 548K 64.9G 548K /users/home/cindy users/home/mark 1.00G 64.9G 1.00G /users/home/mark users/home/neil 1.00G 64.9G 1.00G /users/home/neil. -r.. # zfs list -t all -r users/home/mark NAME USED AVAIL REFER MOUNTPOINT users/home/mark 1.00G 64.9G 1.00G /users/home/mark users/home/mark@yesterday G - users/home/mark@today G - zfs list.. # zfs list /user/home/mark NAME USED AVAIL REFER MOUNTPOINT users/home/mark 1.00G 64.9G 1.00G /users/home/mark tank/home/gina. # zfs list -r users/home/gina NAME USED AVAIL REFER MOUNTPOINT users/home/gina 2.00G 62.9G 32K /users/home/gina users/home/gina/projects 2.00G 62.9G 33K /users/home/gina/projects users/home/gina/projects/fs1 1.00G 62.9G 1.00G /users/home/gina/projects/fs1 users/home/gina/projects/fs2 1.00G 62.9G 1.00G /users/home/gina/projects/fs2 zfs list zfs(1m). ZFS zfs list -o, -t -H. -o ZFS. -o name. zfs list share.nfs mountpoint. 158 Oracle Solaris 11.1 : ZFS

159 ZFS # zfs list -r -o name,share.nfs,mountpoint users/home NAME NFS MOUNTPOINT users/home on /users/home users/home/cindy on /users/home/cindy users/home/gina on /users/home/gina users/home/gina/projects on /users/home/gina/projects users/home/gina/projects/fs1 on /users/home/gina/projects/fs1 users/home/gina/projects/fs2 on /users/home/gina/projects/fs2 users/home/mark on /users/home/mark users/home/neil on /users/home/neil -t ZFS filesystem volume share snapshot -t. -t -o used. # zfs list -r -t filesystem -o name,used users/home NAME USED users/home 4.00G users/home/cindy 548K users/home/gina 2.00G users/home/gina/projects 2.00G users/home/gina/projects/fs1 1.00G users/home/gina/projects/fs2 1.00G users/home/mark 1.00G users/home/neil 1.00G -H zfs list. -H.. -H zfs list. # zfs list -r -H -o name users/home users/home users/home/cindy users/home/gina users/home/gina/projects users/home/gina/projects/fs1 users/home/gina/projects/fs2 5 Oracle Solaris ZFS 159

160 ZFS users/home/mark users/home/neil ZFS zfs set, inherit get. 160 ZFS 161 ZFS 162 ZFS ZFS zfs set. zfs create. 150 ZFS. zfs set property=value /. zfs set. tank/home atime off. # zfs set atime=off tank/home.. # zfs create -o atime=off tank/home BKMGTPEZ ( ). b B. zfs set quota users/home/mark 20GB. # zfs set quota=20g users/home/mark # zfs set quota=20g users/home/mark # zfs set quota=20gb users/home/mark # zfs set quota=20gb users/home/mark. # zfs set quota=20gb users/home/mark cannot set property for /users/home/mark : out of space 160 Oracle Solaris 11.1 : ZFS

161 ZFS mountpoint.. zfs set zfs(1m). ZFS.. zfs inherit. zfs set tank/home/jeff. zfs inherit compression off. home tank compression. ( home). # zfs set compression=on tank/home/jeff # zfs get -r compression tank/home NAME PROPERTY VALUE SOURCE tank/home compression off default tank/home/eric compression off default tank/home/eric@today compression - - tank/home/jeff compression on local # zfs inherit compression tank/home/jeff # zfs get -r compression tank/home NAME PROPERTY VALUE SOURCE tank/home compression off default tank/home/eric compression off default tank/home/eric@today compression - - tank/home/jeff compression off default -r inherit. tank/home compression. # zfs inherit -r compression tank/home -r. zfs inherit zfs(1m). 5 Oracle Solaris ZFS 161

162 ZFS ZFS zfs list. 157 ZFS. zfs get. zfs get.. # zfs get checksum tank/ws NAME PROPERTY VALUE SOURCE tank/ws checksum on default SOURCE SOURCE (zfs get ) default inherited from dataset-name local temporary.. dataset-name. zfs set. zfs mount - o (none). ZFS. all. all. # zfs get all tank/home NAME PROPERTY VALUE SOURCE tank/home aclinherit restricted default tank/home aclmode discard default tank/home atime on default tank/home available 66.9G - tank/home canmount on default tank/home casesensitivity mixed - tank/home checksum on default 162 Oracle Solaris 11.1 : ZFS

163 ZFS tank/home compression off default tank/home compressratio 1.00x - tank/home copies 1 default tank/home creation Fri May 11 10: tank/home dedup off default tank/home devices on default tank/home encryption off - tank/home exec on default tank/home keysource none default tank/home keystatus none - tank/home logbias latency default tank/home mlslabel none - tank/home mounted yes - tank/home mountpoint /tank/home default tank/home multilevel off - tank/home nbmand off default tank/home normalization none - tank/home primarycache all default tank/home quota none default tank/home readonly off default tank/home recordsize 128K default tank/home referenced 43K - tank/home refquota none default tank/home refreservation none default tank/home rekeydate - default tank/home reservation none default tank/home rstchown on default tank/home secondarycache all default tank/home setuid on default tank/home shadow none - tank/home share.*... local tank/home snapdir hidden default tank/home sync standard default tank/home type filesystem - tank/home used 8.54M - tank/home usedbychildren 8.49M - tank/home usedbydataset 43K - tank/home usedbyrefreservation 0 - tank/home usedbysnapshots 0 - tank/home utf8only off - tank/home version 6 - tank/home vscan off default tank/home xattr on default tank/home zoned off default zfs get -s... local, default, inherited, temporary none. tank/ws. # zfs get -s local all tank/ws NAME PROPERTY VALUE SOURCE tank/ws compression on local 5 Oracle Solaris ZFS 163

164 ZFS -r. tank/home. # zfs get -r -s temporary all tank/home NAME PROPERTY VALUE SOURCE tank/home atime off temporary tank/home/jeff atime off temporary tank/home/mark quota 20G temporary zfs get.,.. # zfs get -s local all tank/home atime off local tank/home/jeff atime off local tank/home/mark quota 20G local zfs get zfs(1m). ZFS zfs get -H -o. -H.. -o. name 137 ZFS. ( ) name, value, property source. zfs get -H -o. # zfs get -H -o value compression tank/home on -p., 1MB # zfs get -H -o value -p used tank/home r. -H, -o -r export/home used. # zfs get -H -o name,value -r used export/home 164 Oracle Solaris 11.1 : ZFS

165 ZFS ZFS ZFS. 165 ZFS 167 ZFS ZFS ZFS ZFS.. zpool create -m. 50 ZFS. ZFS SMF( ) svc://system/filesystem/local ZFS. /path. path. zfs set mountpoint. ZFS. ZFS /etc/vfstab. mountpoint., pool/home mountpoint /export/stuff pool/home/user mountpoint /export/stuff/user. mountpoint none. canmount. canmount 152 canmount. zfs set mountpoint legacy. ZFS. mount umount /etc/vfstab Oracle Solaris ZFS 165

166 ZFS mountpoint legacy none ZFS. ZFS, mountpoint. mountpoint legacy ZFS. ZFS. # zfs create pool/filesystem # zfs get mountpoint pool/filesystem NAME PROPERTY VALUE SOURCE pool/filesystem mountpoint /pool/filesystem default # zfs get mounted pool/filesystem NAME PROPERTY VALUE SOURCE pool/filesystem mounted yes - mountpoint. # zfs set mountpoint=/mnt pool/filesystem # zfs get mountpoint pool/filesystem NAME PROPERTY VALUE SOURCE pool/filesystem mountpoint /mnt local # zfs get mounted pool/filesystem NAME PROPERTY VALUE SOURCE pool/filesystem mounted yes - mountpoint.. ZFS,,. mountpoint legacy ZFS. mount umount /etc/vfstab. ZFS, ZFS mount umount. ZFS. # zfs set mountpoint=legacy tank/home/eric # mount -F zfs tank/home/eschrock /mnt /etc/vfstab. /etc/vfstab. 166 Oracle Solaris 11.1 : ZFS

167 ZFS #device device mount FS fsck mount mount #to mount to fsck point type pass at boot options # tank/home/eric - /mnt zfs - yes - fsck ZFS device to fsck fsck pass -. ZFS 27. ZFS ZFS. zfs mount. zfs mount ZFS... # zfs mount grep tank/home zfs mount grep tank/home tank/home tank/home/jeff /tank/home /tank/home/jeff -a ZFS... # zfs mount -a ZFS.. # zfs mount tank/home/lori cannot mount tank/home/lori : filesystem already mounted. ZFS.. # zfs mount tank/home/bill cannot mount tank/home/bill : legacy mountpoint use mount(1m) to mount this filesystem # mount -F zfs tank/home/billm.. 5 Oracle Solaris ZFS 167

168 ZFS 5 4 ZFS atime devices exec nbmand readonly setuid xattr atime/noatime devices/nodevices exec/noexec nbmand/nonbmand ro/rw setuid/nosetuid xattr/noaxttr nosuid nodevices,nosetuid. NFSv4 NFS ZFS. NFS NFS., neo tank zee /tank/baz. zee# mount neo:/tank /mnt zee# ls /mnt baa bar neo# zfs create tank/baz zee% ls /mnt baa bar baz zee% ls /mnt/baz file1 file2 zfs mount -o. zfs get temporary.. tank/home/neil Oracle Solaris 11.1 : ZFS

169 ZFS # zfs mount -o ro users/home/neil remount. atime off. # zfs mount -o remount,noatime users/home/neil NAME PROPERTY VALUE SOURCE users/home/neil atime off temporary # zfs get atime users/home/perrin zfs mount zfs(1m). ZFS zfs unmount ZFS. unmount.. # zfs unmount users/home/mark. # zfs unmount /users/home/mark unmount. -f... # zfs unmount tank/home/eric cannot unmount /tank/home/eric : Device busy # zfs unmount -f tank/home/eric umount ZFS.. # umount /tank/home/bob zfs umount zfs(1m). 5 Oracle Solaris ZFS 169

170 ZFS ZFS Oracle Solaris 11.1 ZFS ZFS NFS. NFS. ZFS mountpoint legacy share. share.nfs sharenfs NFS. share.smb sharesmb SMB. sharenfs sharesmb share.nfs sharenfs. /etc/dfs/dfstab.. SMF ZFS UFS. sharenfs sharesmb off. sharemgr. share.. share -a share -ap. share -p., tank/home. # zfs set share.nfs=on tank/home.. # zfs set share.nfs.nosuid=on tank/home/usera share.nfs tank/home share.nfs.. # zfs create tank/home/usera # zfs create tank/home/userb 170 Oracle Solaris 11.1 : ZFS

171 ZFS ZFS Oracle Solaris 11.. zfs set share ZFS NFS SMB. # zfs create rpool/fs1 # zfs set share=name=fs1,path=/rpool/fs1,prot=nfs rpool/fs1 name=fs1,path=/rpool/fs1,prot=nfs sharenfs sharesmb on.. # zfs set sharenfs=on rpool/fs1 # grep fs1 /etc/dfs/sharetab /rpool/fs1 fs1 nfs sec=sys,rw zfs get share. # zfs get share rpool/fs1 NAME PROPERTY VALUE SOURCE rpool/fs1 share name=fs1,path=/rpool/fs1,prot=nfs local Oracle Solaris 10 share. ZFS. # share -F nfs /tank/zfsfs # grep zfsfs /etc/dfs/sharetab /tank/zfsfs tank_zfsfs nfs sec=sys,rw UFS. # share -F nfs /ufsfs # grep ufsfs /etc/dfs/sharetab /ufsfs - nfs rw /tank/zfsfs tank_zfsfs nfs rw ZFS zfs set NFS SMB ZFS. share.nfs share.smb. tank/sales. -. share.nfs tank/sales/log tank/sales/logs. 5 Oracle Solaris ZFS 171

172 ZFS # zfs create -o share.nfs=on tank/sales # zfs create -o share.nfs.ro=\* tank/sales/logs # zfs get -r share.nfs tank/sales NAME PROPERTY VALUE SOURCE tank/sales share.nfs on local tank/sales% share.nfs on inherited from tank/sales tank/sales/log share.nfs on inherited from tank/sales tank/sales/log% share.nfs on inherited from tank/sales. # zfs set share.nfs=on tank/home/data # zfs set share.nfs.sec.default.root=neo tank/home/data ZFS 34 ZFS. share. share share.. share share.desc, share.nfs.nosuid share.smb.guestok. share.nfs NFS. share.smb SMB. sharenfs share.nfs sharesmb share.smb sharenfs sharesmb. tank/home. # zfs set share.nfs=on tank/home share.nfs.. # zfs create tank/home/usera # zfs create tank/home/userb # grep tank/home /etc/dfs/sharetab /tank/home tank_home nfs sec=sys,rw /tank/home/usera tank_home_usera nfs sec=sys,rw /tank/home/userb tank_home_userb nfs sec=sys,rw ZFS sharenfs sharesmb..zfs/shares. sharenfs sharesmb sharenfs sharesmb. sharenfs NFS Oracle Solaris 11.1 : ZFS

173 ZFS # zpool get version tank NAME PROPERTY VALUE SOURCE tank version 33 default # zfs create -o sharenfs=on tank/home # zfs create tank/home/usera # grep tank/home /etc/dfs/sharetab /tank/home tank_home nfs sec=sys,rw /tank/home/usera tank_home_usera nfs sec=sys,r ZFS SMB.. # zfs share -o share.smb=on tank/workspace%myshare zfs share tank/workspace myshare SMB..zfs/shares SMB ACL. SMB.zfs/shares.. # ls -lv /tank/workspace/.zfs/shares -rwxrwxrwx+ 1 root root 0 May 15 10:31 myshare 0:everyone@:read_data/write_data/append_data/read_xattr/write_xattr /execute/delete_child/read_attributes/write_attributes/delete /read_acl/write_acl/write_owner/synchronize:allow. share.smb.guestok.. # zfs get -r share.smb.guestok tank/workspace NAME PROPERTY VALUE SOURCE tank/workspace share.smb.guestok on inherited from tank tank/workspace%myshare share.smb.guestok on inherited from tank NFS.. # zfs create -o share.nfs=on -o share.nfs.anon=99 -o share.auto=off tank/home # mkdir /tank/home/usera # mkdir /tank/home/userb # zfs share -o share.path=/tank/home/usera tank/home%usera # zfs share -o share.path=/tank/home/userb tank/home%userb # grep tank/home /etc/dfs/sharetab /tank/home/usera usera nfs anon=99,sec=sys,rw /tank/home/userb userb nfs anon=99,sec=sys,rw share.auto off. share.auto. 5 Oracle Solaris ZFS 173

174 ZFS NFS. NFS.. # zfs create -o mountpoint=/pub tank/public # zfs share -o share.nfs=on -o share.nfs.public=on tank/public%pubshare # grep pub /etc/dfs/sharetab /pub pubshare nfs public,sec=sys,rw NFS SMB share_nfs(1m) share_smb(1m). ZFS. (_)., data/home/john data_home_john. share.autoname... # zfs create -o share.smb=on -o share.autoname=john data/home/john # zfs create data/home/john/backups # grep john /etc/dfs/sharetab /data/home/john john smb /data/home/john/backups john_backups smb share zfs set share share.auto off... ZFS zfs get. share.nfs. # zfs get share.nfs tank/sales NAME PROPERTY VALUE SOURCE tank/sales share.nfs on local share.nfs. # zfs get -r share.nfs tank/sales NAME PROPERTY VALUE SOURCE tank/sales share.nfs on local tank/sales% share.nfs on inherited from tank/sales tank/sales/log share.nfs on inherited from tank/sales tank/sales/log% share.nfs on inherited from tank/sales 174 Oracle Solaris 11.1 : ZFS

175 ZFS zfs get all. NFS SMB. # zfs get share.nfs.all tank/sales NAME PROPERTY VALUE SOURCE tank/sales share.nfs.aclok off default tank/sales share.nfs.anon default tank/sales share.nfs.charset.*... default tank/sales share.nfs.cksum default tank/sales share.nfs.index default tank/sales share.nfs.log default tank/sales share.nfs.noaclfab off default tank/sales share.nfs.nosub off default tank/sales share.nfs.nosuid off default tank/sales share.nfs.public - - tank/sales share.nfs.sec default tank/sales share.nfs.sec.*... default.. # zfs get -e -s local,received,inherited share.all tank/home NAME PROPERTY VALUE SOURCE tank/home share.auto off local tank/home share.nfs on local tank/home share.nfs.anon 99 local tank/home share.protocols nfs local tank/home share.smb.guestok on inherited from tank ZFS., off. # zfs create -o share.nfs.ro=\* tank/data # zfs get share.nfs.ro tank/data NAME PROPERTY VALUE SOURCE tank/data share.nfs.sec.sys.ro on local # zfs set share.nfs.ro=none tank/data # zfs get share.nfs.ro tank/data NAME PROPERTY VALUE SOURCE tank/data share.nfs.sec.sys.ro off local SMB NFS.. # zfs set share.smb=on tank/multifs # zfs set share.nfs=on tank/multifs # grep multifs /etc/dfs/sharetab /tank/multifs tank_multifs nfs sec=sys,rw /tank/multifs tank_multifs smb - 5 Oracle Solaris ZFS 175

176 ZFS SMB. # zfs set share.smb=off tank/multifs # grep multifs /etc/dfs/sharetab /tank/multifs tank_multifs nfs sec=sys,rw.. # zfs share -o share.smb=on tank/home/abc%abcshare # grep abc /etc/dfs/sharetab /tank/home/abc abcshare smb - # zfs rename tank/home/abc%abcshare tank/home/abc%a1share # grep abc /etc/dfs/sharetab /tank/home/abc a1share smb - ZFS zfs unshare.. # zfs unshare tank/home/abc%a1share # grep abc /etc/dfs/sharetab # # zfs share tank/home/abc%a1share # grep abc /etc/dfs/sharetab /tank/home/abc a1share smb - zfs unshare. zfs share share.nfs share.smb. zfs unshare, zfs share share.nfs share.smb. ZFS share.nfs share.smb off.. # zfs set share.nfs=off tank/multifs # grep multifs /etc/dfs/sharetab # zfs destroy.. # zfs destroy tank/home/abc%a1share 176 Oracle Solaris 11.1 : ZFS

177 ZFS ZFS Oracle Solaris 11 Oracle Solaris NFS. ZFS.. share., /export/home/data /export/home/data1 zfszone. zfszone# share -F nfs /export/home/data zfszone# cat /etc/dfs/sharetab zfszone# zfs set share.nfs=on tank/zones/export/home/data1 zfszone# cat /etc/dfs/sharetab ZFS /. - Oracle Solaris 11 sharenfs sharesmb... Oracle Solaris : # zfs get share.nfs filesystem # zfs get share.smb filesystem BE sharenfs sharesmb. Oracle Solaris 11 Express - Oracle Solaris sharenfs sharesmb off on.. 5 Oracle Solaris ZFS 177

178 ZFS /etc/dfs/dfstab., ZFS. sharenfs sharesmb on. sharemgr. share. share -a share -ap. share -p. Oracle Solaris 11 Express BE ZFS. ZFS. BE ZFS pkg update. BE sharemgr show -vp.. # zfs get sharenfs filesystem # zfs get sharesmb filesystem BE sharenfs sharesmb sharemgr. unshare -a unshareall, SMF.. ZFS..zfs/shares zfs clone zfs send..zfs/shares zfs clone zfs send.. zfs send zfs receive 211 ZFS. ZFS.. share.auto off.,. 178 Oracle Solaris 11.1 : ZFS

179 ZFS.. ( ).. INVALID.. # zfs share FS%share.. SHARED.. UNSHARED. zfs share. UNVALIDATED..,. ZFS. zfs share.. share -A. ( ). ZFS quota. reservation.., tank/home tank/home. tank/home tank/home. used. refquota refreservation. 5 Oracle Solaris ZFS 179

180 ZFS Solaris., 4, 15.. quota reservation. refquota refreservation. refquota refreservation quota reservation. quota refquota. refquota quota. refquota quota ZFS 183 ZFS. ZFS ZFS zfs set zfs get. tank/home/jeff 10GB. # zfs set quota=10g tank/home/jeff # zfs get quota tank/home/jeff NAME PROPERTY VALUE SOURCE tank/home/jeff quota 10G local zfs list df.. # zfs list -r tank/home NAME USED AVAIL REFER MOUNTPOINT tank/home 1.45M 66.9G 36K /tank/home tank/home/eric 547K 66.9G 547K /tank/home/eric tank/home/jeff 322K 10.0G 291K /tank/home/jeff tank/home/jeff/ws 31K 10.0G 31K /tank/home/jeff/ws tank/home/lori 547K 66.9G 547K /tank/home/lori tank/home/mark 31K 66.9G 31K /tank/home/mark # df -h /tank/home/jeff Filesystem Size Used Avail Use% Mounted on tank/home/jeff 10G 306K 10G 1% /tank/home/jeff 180 Oracle Solaris 11.1 : ZFS

181 ZFS tank/home 66.9GB tank/home/jeff tank/home/jeff tank/home/jeff/ws 10GB... # zfs set quota=10k tank/home/jeff cannot set property for tank/home/jeff : size is less than current used or reserved space refquota.., studenta 10GB. # zfs set refquota=10g students/studenta # zfs list -t all -r students NAME USED AVAIL REFER MOUNTPOINT students 150M 66.8G 32K /students students/studenta 150M 9.85G 150M /students/studenta students/studenta@yesterday 0-150M - # zfs snapshot students/studenta@today # zfs list -t all -r students students 150M 66.8G 32K /students students/studenta 150M 9.90G 100M /students/studenta students/studenta@yesterday 50.0M - 150M - students/studenta@today 0-100M -.. # zfs set quota=20g students/studenta # zfs list -t all -r students NAME USED AVAIL REFER MOUNTPOINT students 150M 66.8G 32K /students students/studenta 150M 9.90G 100M /students/studenta students/studenta@yesterday 50.0M - 150M - students/studenta@today 0-100M - studenta refquota(10gb) studenta. (10GB 20GB) zfs list. zfs get.. # zfs get refquota,quota students/studenta NAME PROPERTY VALUE SOURCE students/studenta refquota 10G local students/studenta quota 20G local 5 Oracle Solaris ZFS 181

182 ZFS ZFS zfs userquota zfs groupquota.. # zfs create students/compsci # zfs set userquota@student1=10g students/compsci # zfs create students/labstaff # zfs set groupquota@labstaff=20gb students/labstaff. # zfs get userquota@student1 students/compsci NAME PROPERTY VALUE SOURCE students/compsci userquota@student1 10G local # zfs get groupquota@labstaff students/labstaff NAME PROPERTY VALUE SOURCE students/labstaff groupquota@labstaff 20G local. # zfs userspace students/compsci TYPE NAME USED QUOTA POSIX User root 350M none POSIX User student1 426M 10G # zfs groupspace students/labstaff TYPE NAME USED QUOTA POSIX Group labstaff 250M 20G POSIX Group root 350M none. # zfs get userused@student1 students/compsci NAME PROPERTY VALUE SOURCE students/compsci userused@student1 550M local # zfs get groupused@labstaff students/labstaff NAME PROPERTY VALUE SOURCE students/labstaff groupused@labstaff 250 local zfs get all dataset.. # zfs set userquota@student1=none students/compsci # zfs set groupquota@labstaff=none students/labstaff ZFS Oracle Solaris 11.1 : ZFS

183 ZFS. -R zfs send.. userused groupused. userquota groupquota ZFS, 4, 15.. EDQUOT. ZFS NFS quota. quota.. # zfs set userquota@student1=10m students/compsci # zfs userspace students/compsci TYPE NAME USED QUOTA POSIX User root 350M none POSIX User student1 550M 10M # quota student1 Block limit reached on /students/compsci quota -v.. # zfs set userquota@student1=10gb students/compsci # zfs userspace students/compsci TYPE NAME USED QUOTA POSIX User root 350M none POSIX User student1 550M 10G # quota student1 # quota -v student1 Disk quotas for student1 (uid 102): Filesystem usage quota limit timeleft files quota limit timeleft /students/compsci ZFS ZFS... ZFS zfs set zfs get.. 5 Oracle Solaris ZFS 183

184 ZFS # zfs set reservation=5g tank/home/bill # zfs get reservation tank/home/bill NAME PROPERTY VALUE SOURCE tank/home/bill reservation 5G local zfs list.. # zfs list -r tank/home NAME USED AVAIL REFER MOUNTPOINT tank/home 5.00G 61.9G 37K /tank/home tank/home/bill 31K 66.9G 31K /tank/home/bill tank/home/jeff 337K 10.0G 306K /tank/home/jeff tank/home/lori 547K 61.9G 547K /tank/home/lori tank/home/mark 31K 61.9G 31K /tank/home/mark tank/home 5GB tank/home 5GB. used tank/home/bill.,. # zfs set quota=5g pool/filesystem # zfs set reservation=10g pool/filesystem/user1 cannot set reservation for pool/filesystem/user1 : size is greater than available space..., zfs set... # zfs set reservation=10g tank/home/bill # zfs set reservation=5g tank/home/bill # zfs get reservation tank/home/bill NAME PROPERTY VALUE SOURCE tank/home/bill reservation 5G local refreservation. used.. # zfs set refreservation=10g profs/prof1 # zfs list NAME USED AVAIL REFER MOUNTPOINT profs 10.0G 23.2G 19K /profs profs/prof1 10G 33.2G 18K /profs/prof Oracle Solaris 11.1 : ZFS

185 ZFS # zfs set reservation=20g profs/prof1 # zfs list NAME USED AVAIL REFER MOUNTPOINT profs 20.0G 13.2G 19K /profs profs/prof1 10G 33.2G 18K /profs/prof1 used. (10GB 20GB) zfs list. zfs get.. # zfs get reservation,refreserv profs/prof1 NAME PROPERTY VALUE SOURCE profs/prof1 reservation 20G local profs/prof1 refreservation 10G local refreservation. ZFS. ZFS. ZFS ZFS. ZFS rekey... ZFS, ZFS. CCM GCM 128, AES(Advanced Encryption Standard). ZFS Oracle Solaris Cryptographic Framework. /var OS ZFS. ZFS., tank/home/darren Oracle Solaris ZFS 185

186 ZFS # zfs create -o encryption=on tank/home/darren Enter passphrase for tank/home/darren : xxxxxxx Enter again: xxxxxxxx.. # zfs get encryption tank/home/darren NAME PROPERTY VALUE SOURCE tank/home/darren encryption on local on aes-128-ccm.. zfs. raw hex. keysource. keysource=format,location. raw. hex 16. passphrase.. prompt. file:///filename. pkcs11 PKCS#11 URI. URL GET keysource. keysource libcurl OpenSSL. /etc/openssl/certs.pem /etc/certs/ca. # svcadm refresh ca-certificates keysource passphrase. keysource Oracle Solaris 11.1 : ZFS

187 ZFS keysource passphrase PKCS#5 PBKD2 salt.,... # zfs snapshot tank/home/darren@now # zfs clone tank/home/darren@now tank/home/darren-new Enter passphrase for tank/home/darren-new : xxxxxxx Enter again: xxxxxxxx # zfs set encryption=off tank/home/darren-new cannot set property for tank/home/darren-new : encryption is readonly ZFS.. /. cp -r find cpio tar rsync ZFS zfs key -c. (zfs key -l) (zfs mount filesystem). : # zfs key -c tank/home/darren Enter new passphrase for tank/home/darren : xxxxxxxx Enter again: xxxxxxxx keysource. # zfs key -c -o keysource=raw,file:///media/stick/key tank/home/darren zfs key -K. NIST Oracle Solaris ZFS 187

188 ZFS # zfs key -K tank/home/darren. keychange.. aes-128-ccm, aes-192-ccm, aes-256-ccm aes-128-gcm, aes-192-gcm, aes-256-gcm ZFS keysource.. # zfs get keysource tank/home/darren NAME PROPERTY VALUE SOURCE tank/home/darren keysource passphrase,prompt local ZFS rekeydate zfs key -K.. # zfs get rekeydate tank/home/darren NAME PROPERTY VALUE SOURCE tank/home/darren rekeydate Wed Jul 25 16: local creation rekeydate zfs key -K rekey. ZFS ZFS.. Oracle Key Manager http https URI GET. Oracle Key Manager PKCS#11 Oracle Solaris. ZFS. technetwork/articles/servers-storage-admin/manage-zfs-encryption html Oracle Key Manager. ZFS. 188 Oracle Solaris 11.1 : ZFS

189 ZFS zfs key -l zfs key -u key. mount. zfs key -c zfs key -K keychange. ( )..,... ZFS ZFS.., passphrase,prompt. passphrase,prompt zfs mount zfs key -l.. # zfs mount -a Enter passphrase for tank/home/darren : xxxxxxxx Enter passphrase for tank/home/ws : xxxxxxxx Enter passphrase for tank/home/mark : xxxxxxxx keysource. ZFS Solaris 11 Solaris # zfs mount -a Enter passphrase for pond/amy : xxxxxxxx Enter passphrase for pond/rory : xxxxxxxx # zfs mount grep pond pond /pond pond/amy /pond/amy pond/rory /pond/rory. 5 Oracle Solaris ZFS 189

190 ZFS # zfs upgrade -a ZFS. # zfs upgrade -a cannot set property for pond/amy : key not present zpool status. # zpool status -v pond... pond/amy:<0x1> pond/rory:<0x1>.. # zpool scrub pond # zpool clear pond 195 ZFS. ZFS, ZFS,..... dedup zfs key -K zfs clone -K. ZFS 5 1 ZFS aes-256-ccm pktool /cindykey.file. # pktool genkey keystore=file outkey=/cindykey.file keytype=aes keylen= Oracle Solaris 11.1 : ZFS

191 ZFS 5 1 ZFS ( ) tank/home/cindy /cindykey.file. # zfs create -o encryption=aes-256-ccm -o keysource=raw,file:///cindykey.file tank/home/cindy 5 2 ZFS ZFS. users users/home. aes-128-ccm. aes-256-ccm users/home/mark. # zpool create -O encryption=on users mirror c0t1d0 c1t1d0 mirror c2t1d0 c3t1d0 Enter passphrase for users : xxxxxxxx Enter again: xxxxxxxx # zfs create users/home # zfs get encryption users/home NAME PROPERTY VALUE SOURCE users/home encryption on inherited from users # zfs create -o encryption=aes-256-ccm users/home/mark # zfs get encryption users/home/mark NAME PROPERTY VALUE SOURCE users/home/mark encryption aes-256-ccm local 5 3 ZFS keysource keysource keysource=passphrase,prompt. keysource... # zfs create -o encryption=on tank/ws Enter passphrase for tank/ws : xxxxxxxx Enter again: xxxxxxxx # zfs create tank/ws/fs1 # zfs snapshot tank/ws/fs1@snap1 # zfs clone tank/ws/fs1@snap1 tank/ws/fs1clone zfs clone -K... # zfs create -o encryption=on tank/ws Enter passphrase for tank/ws : xxxxxxxx 5 Oracle Solaris ZFS 191

192 ZFS 5 3 ZFS ( ) Enter again: xxxxxxxx # zfs snapshot tank/ws@1 # zfs clone tank/ws@1 tank/ws1clone Enter passphrase for tank/ws1clone : xxxxxxxx Enter again: xxxxxxxx 5 4 ZFS tank/home/darren@snap1 /tank/home/darren. encryption bpool/snaps. tank/home/darren@snap1. # zfs get encryption tank/home/darren NAME PROPERTY VALUE SOURCE tank/home/darren encryption on local # zfs snapshot tank/home/darren@snap1 # zfs get encryption bpool/snaps NAME PROPERTY VALUE SOURCE bpool/snaps encryption on inherited from bpool # zfs send tank/home/darren@snap1 zfs receive bpool/snaps/darren1012 # zfs get encryption bpool/snaps/darren1012 NAME PROPERTY VALUE SOURCE bpool/snaps/darren1012 encryption on inherited from bpool. ZFS. ZFS ZFS UFS ZFS. ZFS. ( ) ZFS shadow.. shadow URI. shadow=file:///path. 192 Oracle Solaris 11.1 : ZFS

193 ZFS shadow=nfs://host:path NFS...,.... NFS UID, GID ACL,. NFS. NFS.. shadowstat. BYTES XFRD. BYTES LEFT. ZFS.. BYTES XFRD ELAPSED TIME. 1 ZFS NFS. NFS UID, GUID ACL. 5 Oracle Solaris ZFS 193

194 ZFS 2 shadowd. # pkg install shadow-migration # svcadm enable shadowd shadowd shadow none. 3. ZFS.. # zfs set readonly=on tank/home/data.. # share -F nfs -o ro /export/home/ufsdata # share - /export/home/ufsdata ro "" 4 shadow ZFS., ZFS rpool/old ZFS users/home/shadow users/home/shadow shadow rpool/old. # zfs create -o shadow=file:///rpool/old users/home/shadow, /export/home/ufsdata ZFS shadow. # zfs create -o shadow=nfs://v120-brm-02/export/home/ufsdata users/home/shadow2 5.. # shadowstat EST BYTES BYTES ELAPSED DATASET XFRD LEFT ERRORS TIME users/home/shadow 45.5M 2.75M - 00:02:31 users/home/shadow 55.8M :02:41 users/home/shadow 69.7M :02:51 No migrations in progress shadow none. # zfs get -r shadow users/home/shadow* NAME PROPERTY VALUE SOURCE users/home/shadow shadow none - users/home/shadow2 shadow none Oracle Solaris 11.1 : ZFS

195 ZFS ZFS ZFS.. shadow... # zfs set mountpoint=/users/home/data users/home/shadow3 cannot unmount /users/home/shadow3 : Device busy ZFS Solaris ZFS zfs upgrade.. 5. # zfs upgrade This system is currently running ZFS filesystem version 5. All filesystems are formatted with the current version.. # zfs upgrade -v The following filesystem versions are supported: VER DESCRIPTION Initial ZFS filesystem version 2 Enhanced directory entries 3 Case insensitive and File system unique identifier (FUID) 4 userquota, groupquota properties 5 System attributes For more information on a particular version, including supported releases, see the ZFS Administration Guide. 189 ZFS. 5 Oracle Solaris ZFS 195

196 196

197 6 6 Oracle Solaris ZFS Oracle Solaris ZFS ZFS 198 ZFS 201 ZFS 202 ZFS 204 ZFS 205 ZFS 205 ZFS 205 ZFS ZFS 206 ZFS ZFS... ZFS

198 ZFS.,..,,,. ZFS 206 ZFS. 198 ZFS 201 ZFS 202 ZFS ZFS zfs snapshot zfs snap, ZFS. friday tank/home/cindy. # zfs snapshot tank/home/cindy@friday -r.. # zfs snapshot -r tank/home@snap1 # zfs list -t snapshot -r tank/home NAME USED AVAIL REFER MOUNTPOINT tank/home@snap G - tank/home/cindy@snap M - tank/home/lori@snap G - tank/home/mark@snap G - tank/home/tim@snap M -... # zfs set compression=on tank/home/cindy@friday cannot set property for tank/home/cindy@friday : this property can not be modified for snapshots zfs destroy.. # zfs destroy tank/home/cindy@friday 198 Oracle Solaris 11.1 : ZFS

199 ZFS.. # zfs destroy tank/home/cindy cannot destroy tank/home/cindy : filesystem has children use -r to destroy the following datasets: tank/home/cindy@tuesday tank/home/cindy@wednesday tank/home/cindy@thursday. destroy 135 ZFS. ZFS zfs receive.. zfs destroy -d. user-reference, Oracle Solaris zfs destroy. Oracle Solaris user-reference 0.., tank/home/cindy/snap@1 keep. # zfs hold keep tank/home/cindy@snap1 -r.. # zfs snapshot -r tank/home@now # zfs hold -r keep tank/home@now keep.. zfs destroy.. # zfs destroy tank/home/cindy@snap1 cannot destroy tank/home/cindy@snap1 : dataset is busy 6 Oracle Solaris ZFS 199

200 ZFS -d.. # zfs destroy -d tank/home/cindy@snap1 zfs holds.. # zfs holds tank/home@now NAME TAG TIMESTAMP tank/home@now keep Fri Aug 3 15:15: # zfs holds -r tank/home@now NAME TAG TIMESTAMP tank/home/cindy@now keep Fri Aug 3 15:15: tank/home/lori@now keep Fri Aug 3 15:15: tank/home/mark@now keep Fri Aug 3 15:15: tank/home/tim@now keep Fri Aug 3 15:15: tank/home@now keep Fri Aug 3 15:15: zfs release.. # zfs release -r keep tank/home@now zfs destroy.. # zfs destroy -r tank/home@now. zfs destroy -d defer_destroy on. off. userrefs. user-reference. ZFS,.. # zfs rename tank/home/cindy@snap1 tank/home/cindy@today. # zfs rename tank/home/cindy@snap1 today. 200 Oracle Solaris 11.1 : ZFS

201 ZFS # zfs rename tank/home/cindy@today pool/home/cindy@saturday cannot rename to pool/home/cindy@today : snapshots must be part of same dataset zfs rename - r.. # zfs list -t snapshot -r users/home NAME USED AVAIL REFER MOUNTPOINT users/home@now 23.5K K - users/home@yesterday 0-38K - users/home/lori@yesterday G - users/home/mark@yesterday G - users/home/neil@yesterday G - # zfs rename -r # zfs list -t snapshot -r users/home NAME USED AVAIL REFER MOUNTPOINT users/home@now 23.5K K - users/home@2daysago 0-38K - users/home/lori@2daysago G - users/home/mark@2daysago G - users/home/neil@2daysago G - ZFS zfs list. zfs list -t snapshot. listsnapshots.. # zpool get listsnapshots tank NAME PROPERTY VALUE SOURCE tank listsnapshots off default # zpool set listsnapshots=on tank # zpool get listsnapshots tank NAME PROPERTY VALUE SOURCE tank listsnapshots on local.zfs/snapshot., tank/home/cindy /home/cindy tank/home/cindy@thursday /home/cindy/.zfs/snapshot/thursday. # ls /tank/home/cindy/.zfs/snapshot thursday tuesday wednesday. # zfs list -t snapshot -r tank/home NAME USED AVAIL REFER MOUNTPOINT tank/home/cindy@tuesday 45K G - tank/home/cindy@wednesday 45K G - tank/home/cindy@thursday G - 6 Oracle Solaris ZFS 201

202 ZFS. # zfs list -r -t snapshot -o name,creation tank/home NAME CREATION tank/home/cindy@tuesday Fri Aug 3 15: tank/home/cindy@wednesday Fri Aug 3 15: tank/home/cindy@thursday Fri Aug 3 15: tank/home/lori@today Fri Aug 3 15: tank/home/mark@today Fri Aug 3 15: ZFS.., used. (used ). referenced. used.,.. $ zfs list -o space -r rpool NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV USEDCHILD rpool 124G 9.57G 0 302K G rpool/root 124G 3.38G 0 31K G rpool/root/solaris 124G 20.5K K rpool/root/solaris/var 124G 20.5K K 0 0 rpool/root/solaris-1 124G 3.38G 66.3M 3.14G 0 184M rpool/root/solaris-1/var 124G 184M 49.9M 134M 0 0 rpool/varshare 124G 39.5K K 0 0 rpool/dump 124G 4.12G G 129M 0 rpool/export 124G 63K 0 32K 0 31K rpool/export/home 124G 31K 0 31K 0 0 rpool/swap 124G 2.06G G 64.7M ZFS zfs rollback.... -r. -R. 202 Oracle Solaris 11.1 : ZFS

203 ZFS.. -f. tank/home/cindy tuesday. # zfs rollback tank/home/cindy@tuesday cannot rollback to tank/home/cindy@tuesday : more recent snapshots exist use -r to force deletion of the following snapshots: tank/home/cindy@wednesday tank/home/cindy@thursday # zfs rollback -r tank/home/cindy@tuesday tuesday wednesday thursday. # zfs list -r -t snapshot -o name,creation tank/home/cindy NAME CREATION tank/home/cindy@tuesday Fri Aug 3 15: ZFS (zfs diff) ZFS zfs diff.,. $ ls /tank/home/tim filea $ zfs snapshot tank/home/tim@snap1 $ ls /tank/home/tim filea fileb $ zfs snapshot tank/home/tim@snap2,. $ zfs diff tank/home/tim@snap1 tank/home/tim@snap2 M /tank/home/tim/ + /tank/home/tim/fileb M. + fileb. M. $ mv /tank/cindy/fileb /tank/cindy/filec $ zfs snapshot tank/cindy@snap2 $ zfs diff tank/cindy@snap1 tank/cindy@snap2 M /tank/cindy/ R /tank/cindy/fileb -> /tank/cindy/filec 6 Oracle Solaris ZFS 203

204 ZFS zfs diff..... M + R zfs(1m). ZFS,.,..... origin, zfs destroy.. zfs get zfs set. ZFS 160 ZFS. used 0.. used. 205 ZFS 205 ZFS 205 ZFS ZFS 204 Oracle Solaris 11.1 : ZFS

205 ZFS ZFS zfs clone. ZFS. ( : ).. tank/ws/gate@yesterday tank/home/matt/bug123. # zfs snapshot tank/ws/gate@yesterday # zfs clone tank/ws/gate@yesterday tank/home/matt/bug123 projects/newproject@today projects/teama/tempuser.. # zfs snapshot projects/newproject@today # zfs clone projects/newproject@today projects/teama/tempuser # zfs set share.nfs=on projects/teama/tempuser # zfs set quota=5g projects/teama/tempuser ZFS ZFS zfs destroy.. # zfs destroy tank/home/matt/bug123. ZFS ZFS zfs promote ZFS ZFS. tank/test/producta tank/test/productabeta tank/test/producta. # zfs create tank/test # zfs create tank/test/producta # zfs snapshot tank/test/producta@today 6 Oracle Solaris ZFS 205

206 ZFS # zfs clone tank/test/producta@today tank/test/productabeta # zfs list -r tank/test NAME USED AVAIL REFER MOUNTPOINT tank/test 104M 66.2G 23K /tank/test tank/test/producta 104M 66.2G 104M /tank/test/producta tank/test/producta@today 0-104M - tank/test/productabeta G 104M /tank/test/productabeta # zfs promote tank/test/productabeta # zfs list -r tank/test NAME USED AVAIL REFER MOUNTPOINT tank/test 104M 66.2G 24K /tank/test tank/test/producta G 104M /tank/test/producta tank/test/productabeta 104M 66.2G 104M /tank/test/productabeta tank/test/productabeta@today 0-104M - zfs list producta productabeta... # zfs rename tank/test/producta tank/test/productalegacy # zfs rename tank/test/productabeta tank/test/producta # zfs list -r tank/test.. # zfs destroy tank/test/productalegacy ZFS zfs send... zfs receive.. ZFS ZFS ZFS 209 ZFS 210 ZFS 211 ZFS 213 ZFS 216 ZFS ZFS Oracle Solaris 11.1 : ZFS

207 ZFS zfs snapshot zfs rollback ZFS. zfs send zfs receive ZFS... ZFS. zfs send zfs receive. WAN.. ZFS, ( : RAID-Z). tar, cpio, pax ZFS. tar cpio NFSv4 ACL, pax. ZFS zfs send zfs receive tar cpio ZFS. ZFS ACL. tar cpio. ZFS Oracle Solaris 11. ZFS zfs send ZFS. zfs receive ZFS. zfs send.. 6 Oracle Solaris ZFS 207

208 ZFS zfs send NAME ORIGIN pool/a - pool/a/1 - pool/a/1@clone - pool/b - pool/b/1 pool/a/1@clone pool/b/1@clone2 - pool/b/2 pool/b/1@clone2 pool/b@pre-send - pool/b/1@pre-send - pool/b/2@pre-send - pool/b@send - pool/b/1@send - pool/b/2@send -. # zfs send -R pool/b@send.... TYPE SNAPSHOT INCREMENTAL FROM full pool/b@pre-send - incr pool/b@send pool/b@pre-send incr pool/b/1@clone2 pool/a/1@clone incr pool/b/1@pre-send pool/b/1@clone2 incr pool/b/1@send pool/b/1@send incr pool/b/2@pre-send pool/b/1@clone2 incr pool/b/2@send pool/b/2@pre-send pool/a/1@clone. pool/a/1@clone Oracle Solaris 11.1 : ZFS

209 ZFS. # zfs send -r pool/b@send.... TYPE SNAPSHOT INCREMENTAL FROM full pool/b@send - incr pool/b/1@clone2 pool/a/1@clone incr pool/b/1@send pool/b/1@clone2 incr pool/b/2@send pool/b/1@clone2 pool/a/1@clone. pool/a/1@clone # zfs send -rc pool/b@send.... TYPE SNAPSHOT INCREMENTAL FROM full pool/b@send - full pool/b/1@clone2 incr pool/b/1@send pool/b/1@clone2 incr pool/b/2@send pool/b/1@clone2 pool/b/1@clone2 pool/b/1. ZFS zfs send.. # zfs send tank/dana@snap1 zfs recv spool/ds01 zfs recv zfs receive. ssh zfs send.. sys1# zfs send tank/dana@snap1 ssh sys2 zfs recv newtank/dana. 6 Oracle Solaris ZFS 209

210 ZFS zfs send -i.. sys1# zfs send -i ssh sys2 zfs recv newtank/dana (snap1) (snap2). newtank/dana.. cannot receive incremental stream of into newtank/dana: most recent snapshot of does not match incremental source atime off. snap1. snap1 snap2.. sys1# zfs send -i snap1 ssh sys2 zfs recv newtank/dana. snapshot1. cannot send : not an earlier snapshot from the same fs gzip ZFS.. # zfs send pool/fs@snap gzip > backupfile.gz ZFS..... zfs rename. 210 Oracle Solaris 11.1 : ZFS

211 ZFS. # zfs send tank/gozer@0830 > /bkups/gozer # zfs receive tank/gozer2@today < /bkups/gozer # zfs rename tank/gozer tank/gozer.old # zfs rename tank/gozer2 tank/gozer... sys2# rm newtank/dana/file.1 tank/dana@snap3.. -F.. sys1# zfs send -i tank/dana@snap2 tank/dana@snap3 ssh sys2 zfs recv -F newtank/dana. -F,. sys1# zfs send -i tank/dana@snap4 tank/dana@snap5 ssh sys2 zfs recv newtank/dana cannot receive: destination has been modified since most recent snapshot -F.. zfs receive. cannot send pool/fs@name : not an earlier snapshot from the same fs ZFS ZFS,... 6 Oracle Solaris ZFS 211

212 ZFS ( ) zfs inherit -S. zfs inherit -S -S zfs inherit. zfs inherit -S zfs inherit. zfs get -o RECEIVED. zfs get -o all RECEIVED. -R zfs send -p. zfs receive -e. poola/bee/cee@1 poold/eee (cee@1). # zfs list -rt all poola NAME USED AVAIL REFER MOUNTPOINT poola 134K 134G 23K /poola poola/bee 44K 134G 23K /poola/bee poola/bee/cee 21K 134G 21K /poola/bee/cee poola/bee/cee@1 0-21K - # zfs send -R poola/bee/cee@1 zfs receive -e poold/eee # zfs list -rt all poold NAME USED AVAIL REFER MOUNTPOINT poold 134K 134G 23K /poold poold/eee 44K 134G 23K /poold/eee poold/eee/cee 21K 134G 21K /poold/eee/cee poold/eee/cee@1 0-21K - ( : mountpoint ). tank/data compression. tank/data (-p ) compression. # zfs get compression tank/data NAME PROPERTY VALUE SOURCE tank/data compression off default # zfs snapshot tank/data@snap1 # zfs send -p tank/data@snap1 zfs recv -o compression=on -d bpool # zfs get -o all compression bpool/data NAME PROPERTY VALUE RECEIVED SOURCE bpool/data compression on off local compression bpool. bpool/data compression on. restorepool. zfs send -b Oracle Solaris 11.1 : ZFS

213 ZFS # zfs send -b bpool/data@snap1 zfs recv -d restorepool # zfs get -o all compression restorepool/data NAME PROPERTY VALUE RECEIVED SOURCE restorepool/data compression off off received compression off, tank/data. zfs receive -x... # zfs send -R tank/home@snap1 zfs recv -x quota bpool/home # zfs get -r quota bpool/home NAME PROPERTY VALUE SOURCE bpool/home quota none local bpool/home@snap1 quota - - bpool/home/lori quota none default bpool/home/lori@snap1 quota - - bpool/home/mark quota none default bpool/home/mark@snap1 quota - - -x. # zfs send -R tank/home@snap1 zfs recv bpool/home # zfs get -r quota bpool/home NAME PROPERTY VALUE SOURCE bpool/home quota none received bpool/home@snap1 quota - - bpool/home/lori quota 10G received bpool/home/lori@snap1 quota - - bpool/home/mark quota 10G received bpool/home/mark@snap1 quota - - ZFS zfs send -I -R. 6 Oracle Solaris ZFS 213

214 ZFS ZFS. zfs send -I... zfs send -R.,,. zfs send -r -c zfs send -R origin. 207 ZFS zfs recv -F. zfs recv -F. (zfs send -R ) -i -I -I snapa snapd. -i snapd( ). zfs send ZFS zfs send -I.. # zfs send -I pool/fs@snapa pool/fs@snapd > /snaps/fs@all-i snapb, snapc snapd. # zfs destroy pool/fs@snapb # zfs destroy pool/fs@snapc # zfs destroy pool/fs@snapd. # zfs receive -d -F pool/fs < /snaps/fs@all-i # zfs list 214 Oracle Solaris 11.1 : ZFS

215 ZFS 6 1 ZFS ( ) NAME USED AVAIL REFER MOUNTPOINT pool 428K 16.5G 20K /pool pool/fs 71K 16.5G 21K /pool/fs pool/fs@snapa 16K K - pool/fs@snapb 17K - 20K - pool/fs@snapc 17K K - pool/fs@snapd 0-21K - zfs send -I.. # zfs create pool/fs # zfs snapshot pool/fs@snap1 # zfs clone pool/fs@snap1 pool/clone # zfs snapshot pool/clone@snapa # zfs send -I pool/fs@snap1 pool/clone@snapa > /snaps/fsclonesnap-i # zfs destroy pool/clone@snapa # zfs destroy pool/clone # zfs receive -F pool/clone < /snaps/fsclonesnap-i zfs send -R ZFS.,,...,. # zfs snapshot -r users@today # zfs list NAME USED AVAIL REFER MOUNTPOINT users 187K 33.2G 22K /users users@today 0-22K - users/user1 18K 33.2G 18K /users/user1 users/user1@today 0-18K - users/user2 18K 33.2G 18K /users/user2 users/user2@today 0-18K - users/user3 18K 33.2G 18K /users/user3 users/user3@today 0-18K - # zfs send -R users@today > /snaps/users-r # zfs destroy -r users # zfs receive -F -d users < /snaps/users-r # zfs list NAME USED AVAIL REFER MOUNTPOINT users 196K 33.2G 22K /users users@today 0-22K - users/user1 18K 33.2G 18K /users/user1 users/user1@today 0-18K - users/user2 18K 33.2G 18K /users/user2 users/user2@today 0-18K - users/user3 18K 33.2G 18K /users/user3 users/user3@today 0-18K - 6 Oracle Solaris ZFS 215

216 ZFS 6 1 ZFS ( ) zfs send -R users users2. # zfs create users2 mirror c0t1d0 c1t1d0 # zfs receive -F -d users2 < /snaps/users-r # zfs list NAME USED AVAIL REFER MOUNTPOINT users 224K 33.2G 22K /users users@today 0-22K - users/user1 33K 33.2G 18K /users/user1 users/user1@today 15K - 18K - users/user2 18K 33.2G 18K /users/user2 users/user2@today 0-18K - users/user3 18K 33.2G 18K /users/user3 users/user3@today 0-18K - users2 188K 16.5G 22K /users2 users2@today 0-22K - users2/user1 18K 16.5G 18K /users2/user1 users2/user1@today 0-18K - users2/user2 18K 16.5G 18K /users2/user2 users2/user2@today 0-18K - users2/user3 18K 16.5G 18K /users2/user3 users2/user3@today 0-18K - ZFS zfs send zfs recv.. # zfs send tank/cindy@today ssh newsys zfs recv sandbox/restfs@today tank/cindy@today sandbox/restfs. newsys restfs@today. ssh. 216 Oracle Solaris 11.1 : ZFS

217 7 7 ACL Oracle Solaris ZFS (ACL) UNIX ZFS Solaris ACL 224 ZFS ACL 226 Verbose ZFS ACL 236 Compact ZFS ACL 242 ZFS Solaris ACL Solaris POSIX ACL ACL. POSIX ACL UFS NFSv4 NFS. NFSv4 ACL UNIX UNIX NFSv4. ACL NFSv4,NT ACL. ACL. NFSv4 NT ACL setfacl getfacl, chmod ls ACL. 217

218 Solaris ACL ACL. POSIX ACL, ACL (ACE). POSIX ACL,. ACL ALLOW DENY ACE. ACE ACE. NFSv4 ACL POSIX ACL. cp, mv, tar, cpio, rcp ACL ACL UFS ZFS POSIX ACL NFSv4 ACL. NFSv4 ACL POSIX ACL. NFSv4 ACL POSIX ACL. # cp -p filea /var/tmp cp: failed to set acl entries on /var/tmp/filea Solaris ACL (tar -p cpio -P) UFS tar cpio, Solaris ACL. ACL. ufsrestore ZFS. POSIX ACL NFSv4 ACL. UFS NFSv4 ACL. chmod: ERROR: ACL type s are different ZFS POSIX ACL. # getfacl filea File system doesn t support aclent_t style ACL s. See acl(5) for more information on Solaris ACL support. ACL 207 ZFS. ACL ACL. ACL UNIX user, group owner. ACL,. 218 Oracle Solaris 11.1 : ZFS

219 Solaris ACL ACL chmod [options] A[index]{+ access-permissions/...[:inheritance-flags]: deny allow file chmod [options] /...[:inheritance-flags]:deny allow file... chmod [options] A[index]- file ACL chmod [options] A[index]{+ =}user group:name:access-permissions /...[:inheritance-flags]:deny allow file chmod [options] A-user group:name:access-permissions /...[:inheritance-flags]:deny allow file... chmod [options] A[index]- file ACL ACL-entry-type. ACL-entry-types 7 1. user or group:acl-entry-id=username or groupname ACL ACL-entry-type. ACL-entry-type ACL-entry-ID username groupname. ACL-entry-types 7 1. access-permissions/.../.acl 7 2. inheritance-flags ACL.ACL 7 4. deny allow. owner@, group@, everyone@ ACL-entry-ID. group@:write_data/append_data/execute:deny (ACL-entry-type) ACL ACL-entry-ID. 0:user:gozer:list_directory/read_data/execute:allow ACL. 2:group@:write_data/append_data/execute:deny 7 ACL Oracle Solaris ZFS 219

220 Solaris ACL 2 index-id, UID,, ACL ACL. chmod index-id ACL., ID 3 chmod A3. chmod A3=user:venkman:read_acl:allow filename ACL (owner/group/other ACL ). 7 1 ACL ACL owner@ group@ everyone@ user group.. ACL.. ACL-entry-ID username userid. UID username ACL.. ACL-entry-ID groupname groupid. GID groupname ACL. ACL. 7 2 ACL Compact add_file w. add_subdirectory p. append_data p. delete d. delete 7 3. delete_child D. delete_child 7 3. execute x. list_directory r. read_acl c ACL(ls). 220 Oracle Solaris 11.1 : ZFS

221 Solaris ACL 7 2 ACL ( ) Compact read_attributes a ( ACL). stat. ls(1) stat(2). read_data r. read_xattr R. synchronize s. write_xattr W... write_data w. write_attributes A. write_acl C ACL, chmod ACL. write_owner o. chown chgrp.,. PRIV_FILE_CHOWN. ACL delete delete_child. 7 3 ACL delete delete_child ACL ACL ACL delete_child ACL delete_child ACL write execute 7 ACL Oracle Solaris ZFS 221

222 Solaris ACL 7 3 ACL delete delete_child ( ) ACL write execute ZFS ACL ACL ACL. ACL. ACL full_set modify_set read_set write_set ACL write_acl write_owner read_data, read_attributes, read_xattr read_acl write_data, append_data, write_attributes write_xattr ACL. ACL ACL ACL. ACL. ACL. ACL ACL Compact file_inherit f ACL. dir_inherit d ACL. inherit_only i ACL,. file_inherit dir_inherit ( ). 222 Oracle Solaris 11.1 : ZFS

223 Solaris ACL 7 4 ACL ( ) Compact no_propagate n ( )ACL. file_inherit dir_inherit ( ). -. SMB. successful_access S. ACE. failed_access F. ACE. inherited I ACE., aclinherit / ACL.. ACL ZFS ACL chmod ACL ACL. aclinherit ACL.. discard, ACL. ACL. noallow, deny ACL. restricted,acl write_owner write_acl. passthrough passthrough ACE. ACE. passthrough-x passthrough, passthrough-x (x) ( ACE ). 7 ACL Oracle Solaris ZFS 223

224 ZFS ACL aclinherit restricted. aclmode ACL chmod ACL.. discard aclmode discard ACL.. mask aclmode mask. UID. ACL. ACL ACL. passthrough aclmode passthrough ACL ACL. aclmode discard. aclmode ZFS ACL ZFS ACL ACL. ZFS ACL ACL. ACL, UNIX owner/group/other. ZFS, ACL. ACL., / ACL, / /. ACL. ZFS ACL. ZFS ACL, ACL. " " ACL., ACL ACL. write_acl.., Oracle Solaris 11.1 : ZFS

225 ZFS ACL ACL ACL. ACL ACL Verbose ZFS ACL. umask ACL. $ ls -v file.1 -rw-r--r-- 1 root root Jun 23 15:06 file.1 0:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 2:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow (owner@, group@, everyone@) ACL. ACL. 0:owner@ (read_data/write_data/append_data/read_xattr).,,acl (write_xattr/read_attributes/write_attributes/ read_acl/write_acl)., (write_owner:allow). 1:group@ 2:everyone@ synchronize. (read_data/read_xattr/read_attributes/read_acl:allow). / (read_data/read_xattr/read_attributes/read_acl/ synchronize:allow). synchronize. umask ACL. $ ls -dv dir.1 drwxr-xr-x 2 root root 2 Jul 20 13:44 dir.1 0:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 2:everyone@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow ACL. 7 ACL Oracle Solaris ZFS 225

226 Verbose ZFS ACL (list_directory/read_data/add_file/write_data/add_subdirectory /append_data),,acl (/read_xattr/write_xattr/read_attributes/write_attributes/read_acl/ write_acl). (execute), (delete_child), (write_owner:allow). synchronize.., (list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow). / (list_directory/read_data/read_xattr/execute/read_ attributes/read_acl/synchronize:allow). synchronize. Verbose ZFS ACL chmod ZFS ACL. ACL chmod acl-specification ACL. acl-specification 218 ACL. ACL ACL % chmod A+acl-specification filename index-id ACL % chmod Aindex-ID+acl-specification filename index-id ACL. ACL % chmod A=acl-specification filename % chmod Aindex-ID=acl-specification filename ACL index-id ACL % chmod Aindex-ID- filename 226 Oracle Solaris 11.1 : ZFS

227 Verbose ZFS ACL ACL % chmod A-acl-specification filename ACE % chmod A- filename Verbose ACL ls-v.. # ls -v file.1 -rw-r--r-- 1 root root Jul 20 13:43 file.1 0:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 2:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow Compact ACL 236 Compact ZFS ACL. 7 1 ZFS ACL UNIX,, ACL ACL. ACL file.1. # ls -v file.1 -rw-r--r-- 1 root root Jul 20 13:43 file.1 0:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 2:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow group@ write_data. # chmod A1=group@:read_data/write_data:allow file.1 # ls -v file.1 -rw-rw-r-- 1 root root Jul 20 13:43 file.1 0:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:read_data/write_data:allow 2:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow file # chmod 644 file.1 # ls -v file.1 7 ACL Oracle Solaris ZFS 227

228 Verbose ZFS ACL 7 1 ZFS ACL ( ) -rw-r--r-- 1 root root Jul 20 13:43 file.1 0:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 2:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow 7 2 ZFS ACL ACL. test.dir gozer read_data/execute. # chmod A+user:gozer:read_data/execute:allow test.dir # ls -dv test.dir drwxr-xr-x+ 2 root root 2 Jul 20 14:23 test.dir 0:user:gozer:list_directory/read_data/execute:allow 1:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 2:group@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 3:everyone@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow gozer read_data/execute. # chmod A0- test.dir # ls -dv test.dir drwxr-xr-x 2 root root 2 Jul 20 14:23 test.dir 0:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 2:everyone@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 7 3 ZFS ACL ACL ACL /. ACL file.2. # ls -v file.2 -rw-r--r-- 1 root root 2693 Jul 20 14:26 file Oracle Solaris 11.1 : ZFS

229 Verbose ZFS ACL 7 3 ZFS ACL ( ) 0:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 2:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow ACL allow everyone@. # chmod A2- file.2 # ls -v file.2 -rw-r root root 2693 Jul 20 14:26 file.2 0:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow everyone@ ACL everyone@. ACL everyone@ read_data/write_data. # chmod A=everyone@:read_data/write_data:allow file.3 # ls -v file.3 -rw-rw-rw- 1 root root 2440 Jul 20 14:28 file.3 0:everyone@:read_data/write_data:allow chmod ACL owner/group/everyone@ read_data/write_data:allow ( / ). everyone@. owner@ group@ ACL 666. ACL gozer. # chmod A=user:gozer:read_data:allow file.3 # ls -v file root root 2440 Jul 20 14:28 file.3 0:user:gozer:read_data:allow owner@, group@, everyone@ ACL 000. ( ACL). # chmod 655 file.3 # ls -v file.3 -rw-r-xr-x 1 root root 2440 Jul 20 14:28 file.3 0:owner@:execute:deny 1:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner 7 ACL Oracle Solaris ZFS 229

230 Verbose ZFS ACL 7 3 ZFS ACL ( ) /synchronize:allow 2:group@:read_data/read_xattr/execute/read_attributes/read_acl /synchronize:allow 3:everyone@:read_data/read_xattr/execute/read_attributes/read_acl /synchronize:allow 7 4 ZFS ACL chmod ACL. 2 ACE test5.dir. # ls -dv test5.dir drwxr-xr-x+ 2 root root 2 Jul 20 14:32 test5.dir 0:user:lp:read_data:file_inherit:deny 1:user:gozer:read_data:file_inherit:deny 2:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 3:group@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 4:everyone@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow gozer lp ACL. ACL owner@, group@, everyone@. # chmod A- test5.dir # ls -dv test5.dir drwxr-xr-x 2 root root 2 Jul 20 14:32 test5.dir 0:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 2:everyone@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 7 5 ZFS ACL ACL ACL.ACL 222 ZFS ACL., read_set. # chmod A+user:otto:read_set:allow file.1 # ls -v file.1 -r--r--r--+ 1 root root Jul 20 13:43 file.1 0:user:otto:read_data/read_xattr/read_attributes/read_acl:allow 230 Oracle Solaris 11.1 : ZFS

231 Verbose ZFS ACL 7 5 ZFS ACL ( ) 1:owner@:read_data/read_xattr/write_xattr/read_attributes /write_attributes/read_acl/write_acl/write_owner/synchronize:allow 2:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 3:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow write_set read_set. # chmod A+user:otto:read_set/write_set:allow file.2 # ls -v file.2 -rw-r--r--+ 1 root root 2693 Jul 20 14:26 file.2 0:user:otto:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl:allow 1:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 2:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 3:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow Verbose ZFS ACL ACL. ACL. ACL ACL. ACL. aclinherit. aclinherit restricted. 222 ACL. 7 6 ACL ACL. read_data/write_data/execute ACE test.dir gozer. # chmod A+user:gozer:read_data/write_data/execute:allow test.dir # ls -dv test.dir drwxr-xr-x+ 2 root root 2 Jul 20 14:53 test.dir 0:user:gozer:list_directory/read_data/add_file/write_data/execute:allow 1:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 2:group@:list_directory/read_data/read_xattr/execute/read_attributes 7 ACL Oracle Solaris ZFS 231

232 Verbose ZFS ACL 7 6 ACL ( ) /read_acl/synchronize:allow 3:everyone@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow test.dir gozer ACE. gozer sub.dir, everyone@ sub.dir. # mkdir test.dir/sub.dir # ls -dv test.dir/sub.dir drwxr-xr-x 2 root root 2 Jul 20 14:54 test.dir/sub.dir 0:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 2:everyone@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 7 7 ACL file_inherit ACE. gozer test2.dir read_data/write_data. # chmod A+user:gozer:read_data/write_data:file_inherit:allow test2.dir # ls -dv test2.dir drwxr-xr-x+ 2 root root 2 Jul 20 14:55 test2.dir 0:user:gozer:read_data/write_data:file_inherit:allow 1:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 2:group@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 3:everyone@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow gozer test2.dir/file.2. ACL read_data:file_inherit:allow gozer. # touch test2.dir/file.2 # ls -v test2.dir/file.2 -rw-r--r--+ 1 root root 0 Jul 20 14:56 test2.dir/file.2 0:user:gozer:read_data:inherited:allow 232 Oracle Solaris 11.1 : ZFS

233 Verbose ZFS ACL 7 7 ACL ( ) 1:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 2:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 3:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow aclinherit restricted gozer file.2 write_data ( ). file_inherit dir_inherit inherit_only ACL. gozer, everyone@.. # mkdir test2.dir/subdir.2 # ls -dv test2.dir/subdir.2 drwxr-xr-x+ 2 root root 2 Jul 20 14:57 test2.dir/subdir.2 0:user:gozer:list_directory/read_data/add_file/write_data:file_inherit /inherit_only/inherited:allow 1:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 2:group@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 3:everyone@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow file_inherit dir_inherit ACL. gozer /,,. # chmod A+user:gozer:read_data/write_data/execute:file_inherit/dir_inherit:allow test3.dir # ls -dv test3.dir drwxr-xr-x+ 2 root root 2 Jul 20 15:00 test3.dir 0:user:gozer:list_directory/read_data/add_file/write_data/execute :file_inherit/dir_inherit:allow 1:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 2:group@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 3:everyone@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 7 ACL Oracle Solaris ZFS 233

234 Verbose ZFS ACL 7 7 ACL ( ) inherited ACE. # touch test3.dir/file.3 # ls -v test3.dir/file.3 -rw-r--r--+ 1 root root 0 Jul 20 15:01 test3.dir/file.3 0:user:gozer:read_data:inherited:allow 1:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 2:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 3:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow group@ everyone@ gozer. aclinherit restricted, write_data execute. gozer,,. # chmod A+user:gozer:read_data/write_data/execute:file_inherit/no_propagate:allow test4.dir # ls -dv test4.dir drwxr--r--+ 2 root root 2 Mar 1 12:11 test4.dir 0:user:gozer:list_directory/read_data/add_file/write_data/execute :file_inherit/no_propagate:allow 1:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 2:group@:list_directory/read_data/read_xattr/read_attributes/read_acl /synchronize:allow 3:everyone@:list_directory/read_data/read_xattr/read_attributes/read_acl /synchronize:allow, gozer read_data/write_data/execute. # touch test4.dir/file.4 # ls -v test4.dir/file.4 -rw-r--r--+ 1 root root 0 Jul 20 15:09 test4.dir/file.4 0:user:gozer:read_data:inherited:allow 1:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 2:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 3:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow 234 Oracle Solaris 11.1 : ZFS

235 Verbose ZFS ACL 7 8 ACL Pass Through ACL tank/cindy aclinherit passthrough gozer file.5 test4.dir ACL. # zfs set aclinherit=passthrough tank/cindy # touch test4.dir/file.5 # ls -v test4.dir/file.5 -rw-r--r--+ 1 root root 0 Jul 20 14:16 test4.dir/file.5 0:user:gozer:read_data/write_data/execute:inherited:allow 1:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 2:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 3:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow 7 9 ACL Discard ACL aclinherit discard ACL.. # zfs set aclinherit=discard tank/cindy # chmod A+user:gozer:read_data/write_data/execute:dir_inherit:allow test5.dir # ls -dv test5.dir drwxr-xr-x+ 2 root root 2 Jul 20 14:18 test5.dir 0:user:gozer:list_directory/read_data/add_file/write_data/execute :dir_inherit:allow 1:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 2:group@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 3:everyone@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow ACL.. # chmod 744 test5.dir # ls -dv test5.dir drwxr--r-- 2 root root 2 Jul 20 14:18 test5.dir 0:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:list_directory/read_data/read_xattr/read_attributes/read_acl /synchronize:allow 2:everyone@:list_directory/read_data/read_xattr/read_attributes/read_acl /synchronize:allow 7 ACL Oracle Solaris ZFS 235

236 Compact ZFS ACL 7 10 ACL Noallow ACL 2 ACL. ACL read_data, ACL read_data. chmod ACE. # zfs set aclinherit=noallow tank/cindy # chmod A+user:gozer:read_data:file_inherit:deny,user:lp:read_data:file_inherit:allow test6.dir # ls -dv test6.dir drwxr-xr-x+ 2 root root 2 Jul 20 14:22 test6.dir 0:user:gozer:read_data:file_inherit:deny 1:user:lp:read_data:file_inherit:allow 2:owner@:list_directory/read_data/add_file/write_data/add_subdirectory /append_data/read_xattr/write_xattr/execute/delete_child /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 3:group@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow 4:everyone@:list_directory/read_data/read_xattr/execute/read_attributes /read_acl/synchronize:allow, read_data ACL. # touch test6.dir/file.6 # ls -v test6.dir/file.6 -rw-r--r--+ 1 root root 0 Jul 20 14:23 test6.dir/file.6 0:user:gozer:read_data:inherited:deny 1:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 2:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 3:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow Compact ZFS ACL 14 Compact ZFS. Compact ls -V Compact ACL.. # ls -V file.1 -rw-r--r-- 1 root root Jul 20 14:27 file.1 owner@:rw-p--aarwccos: :allow group@:r-----a-r-c--s: :allow everyone@:r-----a-r-c--s: :allow 236 Oracle Solaris 11.1 : ZFS

237 Compact ZFS ACL Compact ACL. ( rw=read_data/write_data), (p= append_data).,,acl (a=read_attributes, W=write_xattr, R=read_xattr, A=write_attributes, c=read_acl, C=write_acl)., (o= write_owner). synchronize(s). (r= read_data) (a=read_attributes, R=read_xattr, c= read_acl). synchronize(s). / (r=read_data, a=append_data, R=read_xattr, c=read_acl, and s= synchronize). synchronize(s). Compact ACL Verbose ACL. chmod. (-).. Verbose ACL 226 Verbose ZFS ACL Compact ACL ACL file.1. # ls -V file.1 -rw-r--r-- 1 root root Jul 20 14:27 file.1 owner@:rw-p--aarwccos: :allow group@:r-----a-r-c--s: :allow everyone@:r-----a-r-c--s: :allow file.1 gozer read_data/execute. # chmod A+user:gozer:rx:allow file.1 # ls -V file.1 -rw-r--r--+ 1 root root Jul 20 14:27 file.1 user:gozer:r-x : :allow owner@:rw-p--aarwccos: :allow group@:r-----a-r-c--s: :allow 7 ACL Oracle Solaris ZFS 237

238 Compact ZFS ACL 7 11 Compact ACL ( ) everyone@:r-----a-r-c--s: :allow gozer Compact ACL,,. # chmod A+user:gozer:rwx:fd:allow dir.2 # ls -dv dir.2 drwxr-xr-x+ 2 root root 2 Jul 20 14:33 dir.2 user:gozer:rwx :fd-----:allow owner@:rwxp-daarwccos: :allow group@:r-x---a-r-c--s: :allow everyone@:r-x---a-r-c--s: :allow ls -V Compact chmod., gozer dir.2 dir.2 cindy (rwx :fd-----:allow) chmod.. # chmod A+user:cindy:rwx :fd-----:allow dir.2 # ls -dv dir.2 drwxr-xr-x+ 2 root root 2 Jul 20 14:33 dir.2 user:cindy:rwx :fd-----:allow user:gozer:rwx :fd-----:allow owner@:rwxp-daarwccos: :allow group@:r-x---a-r-c--s: :allow everyone@:r-x---a-r-c--s: :allow 7 12 ACL Pass Through ACL aclinherit passthrough ACL ACL. passthrough ACE. ACE. Compact ACL aclinherit passthrough. ACL test1.dir. owner@, group@, everyone@ group@, everyone@ ACL. # zfs set aclinherit=passthrough tank/cindy # pwd /tank/cindy # mkdir test1.dir 238 Oracle Solaris 11.1 : ZFS

239 Compact ZFS ACL 7 12 ACL Pass Through ACL ( ) # chmod A=owner@:rwxpcCosRrWaAdD:fd:allow,group@:rwxp:fd:allow, everyone@::fd:allow test1.dir # ls -Vd test1.dir drwxrwx root root 2 Jul 20 14:42 test1.dir owner@:rwxpddaarwccos:fd-----:allow group@:rwxp :fd-----:allow everyone@: :fd-----:allow ACL. # cd test1.dir # touch file.1 # ls -V file.1 -rwxrwx root root 0 Jul 20 14:44 file.1 owner@:rwxpddaarwccos:------i:allow group@:rwxp :------i:allow everyone@: :------i:allow ACE ACE. # mkdir subdir.1 # ls -dv subdir.1 drwxrwx root root 2 Jul 20 14:45 subdir.1 owner@:rwxpddaarwccos:fd----i:allow group@:rwxp :fd----i:allow everyone@: :fd----i:allow fd----i. ACE ACL. # cd /tank/cindy # mkdir test2.dir # cd test2.dir # touch file.2 # ls -V file.2 -rw-r--r-- 1 root root 0 Jul 20 14:48 file.2 owner@:rw-p--aarwccos: :allow group@:r-----a-r-c--s: :allow everyone@:r-----a-r-c--s: :allow 7 13 ACL Pass Through-X ACL aclinherit=passthrough-x owner@, group@, everyone@ (x) ( ACE ). 7 ACL Oracle Solaris ZFS 239

240 Compact ZFS ACL 7 13 ACL Pass Through-X ACL ( ) aclinherit passthrough-x. # zfs set aclinherit=passthrough-x tank/cindy ACL /tank/cindy/test1.dir owner@ ACL. # chmod A=owner@:rwxpcCosRrWaAdD:fd:allow,group@:rwxp:fd:allow, everyone@::fd:allow test1.dir # ls -Vd test1.dir drwxrwx root root 2 Jul 20 14:50 test1.dir owner@:rwxpddaarwccos:fd-----:allow group@:rwxp :fd-----:allow everyone@: :fd-----:allow (file1) # touch test1.dir/file1 # ls -V test1.dir/file1 -rw-rw root root 0 Jul 20 14:52 test1.dir/file1 owner@:rw-pddaarwccos:------i:allow group@:rw-p :------i:allow everyone@: :------i:allow, testdir cc t. # cc -o t t.c # ls -V t -rwxrwx root root 7396 Dec 3 15:19 t owner@:rwxpddaarwccos:------i:allow group@:rwxp :------i:allow everyone@: :------i:allow cc 0777, owner@, group@, everyone@ ZFS chmod ACL aclmode aclinherit ACL chmod ACL. aclmode mask, aclinherit restricted. ACL Compact. ACL. 240 Oracle Solaris 11.1 : ZFS

241 Compact ZFS ACL 7 14 ZFS chmod ACL ( ) # zfs set aclmode=mask pond/whoville # zfs set aclinherit=restricted pond/whoville # ls -lv file.1 -rwxrwx root root Aug 30 16:03 file.1 user:amy:r-----a-r-c---: :allow user:rory:r-----a-r-c---: :allow group:sysadmin:rw-p--aarwc---: :allow group:staff:rw-p--aarwc---: :allow owner@:rwxp--aarwccos: :allow group@:rwxp--aarwc--s: :allow everyone@:------a-r-c--s: :allow chown file.1, amy.. # chown amy:staff file.1 # su - amy $ ls -lv file.1 -rwxrwx amy staff Aug 30 16:03 file.1 user:amy:r-----a-r-c---: :allow user:rory:r-----a-r-c---: :allow group:sysadmin:rw-p--aarwc---: :allow group:staff:rw-p--aarwc---: :allow owner@:rwxp--aarwccos: :allow group@:rwxp--aarwc--s: :allow everyone@:------a-r-c--s: :allow chmod. sysadmin staff ACL. $ chmod 640 file.1 $ ls -lv file.1 -rw-r amy staff Aug 30 16:03 file.1 user:amy:r-----a-r-c---: :allow user:rory:r-----a-r-c---: :allow group:sysadmin:r-----a-r-c---: :allow group:staff:r-----a-r-c---: :allow owner@:rw-p--aarwccos: :allow group@:r-----a-r-c--s: :allow everyone@:------a-r-c--s: :allow chmod. sysadmin staff ACL. $ chmod 770 file.1 $ ls -lv file.1 -rwxrwx amy staff Aug 30 16:03 file.1 user:amy:r-----a-r-c---: :allow user:rory:r-----a-r-c---: :allow group:sysadmin:rw-p--aarwc---: :allow group:staff:rw-p--aarwc---: :allow 7 ACL Oracle Solaris ZFS 241

242 ZFS 7 14 ZFS chmod ACL ( ) owner@:rwxp--aarwccos: :allow group@:rwxp--aarwc--s: :allow everyone@:------a-r-c--s: :allow ZFS ZFS. ls(1) chmod(1) ZFS. # chmod S+ci file.1 # echo this >>file.1 -bash: file.1: Not owner # rm file.1 rm: cannot remove file.1 : Not owner ZFS. # ls -l/c file.1 -rw-r--r--+ 1 root root Jul 20 14:27 file.1 {A-----im----}. # chmod S-ci file.1 # ls -l/c file.1 -rw-r--r--+ 1 root root Jul 20 14:27 file.1 {A------m----} # rm file ZFS ZFS. # chmod S+cR file.2 # echo this >>file.2 -bash: file.2: Not owner 7 17 ZFS. 242 Oracle Solaris 11.1 : ZFS

243 ZFS 7 17 ZFS ( ) # ls -l/v file.3 -r--r--r-- 1 root root Jul 20 14:59 file.3 {archive,nohidden,noreadonly,nosystem,noappendonly,nonodump, noimmutable,av modified,noav_quarantined,nonounlink,nooffline,nosparse} # chmod S+cR file.3 # ls -l/v file.3 -r--r--r-- 1 root root Jul 20 14:59 file.3 {archive,nohidden,readonly,nosystem,noappendonly,nonodump,noimmutable, av_modified,noav_quarantined,nonounlink,nooffline,nosparse} Oracle Solaris SMB... # chmod S-a file.3 # ls -l/v file.3 -r--r--r-- 1 root root Jul 20 14:59 file.3 {noarchive,nohidden,noreadonly,nosystem,noappendonly,nonodump, noimmutable,noav_modified,noav_quarantined,nonounlink,nooffline,nosparse} 7 ACL Oracle Solaris ZFS 243

244 244

245 8 8 Oracle Solaris ZFS ZFS ZFS 246 ZFS 254 ZFS ( ) 250 ZFS ( ) 255 ZFS ( ) ZFS ZFS,.. create, destroy, mount, ZFS. ZFS RBAC. ZFS ZFS. ZFS

246 ZFS NFS., NFS.zfs/snapshot. ZFS.RBAC Oracle Solaris Oracle Solaris 11.1 : III,,. ZFS delegation.. # zpool get delegation users NAME PROPERTY VALUE SOURCE users delegation on default # zpool set delegation=off users # zpool get delegation users NAME PROPERTY VALUE SOURCE users delegation off local delegation. ZFS zfs allow ZFS.,.,.,.. ( ) allow clone create.... create mount. mount. 246 Oracle Solaris 11.1 : ZFS

247 ZFS ( ) destroy diff hold mount promote receive release rename rollback send share snapshot..... zfs receive mount. zfs diff. mount promote. mount create. create mount. NFS share share.nfs. SMB share share.smb.,. groupquota groupused key keychange userprop userquota userused 8 Oracle Solaris ZFS 247

248 ZFS ZFS. aclinherit aclmode atime canmount casesensitivity compression copies dedup devices encryption exec keysource logbias mountpoint nbmand normalization primarycache quota readonly recordsize refquota refreservation reservation rstchown secondarycache setuid shadow share.nfs share.smb snapdir sync utf8only version volblocksize volsize vscan xattr zoned. 137 ZFS. 248 Oracle Solaris 11.1 : ZFS

249 ZFS ZFS (zfs allow) zfs allow. zfs allow -[ldugecs] everyone user group[,...] filesystem volume zfs allow. zfs allow [-uge] user group everyone [,...] filesystem volume. -uge everyone,,. "everyone" -u -g. -g. -c. zfs allow. zfs allow [-s]... [,...] filesystem volume. ZFS.. -s. zfs allow.. ZFS 64. zfs allow. zfs allow [-ld] filesystem volume -d -l. -l -d.. ZFS (zfs unallow) zfs unallow., create, destroy, mount snapshot. 8 Oracle Solaris ZFS 249

250 ZFS ( ) ZFS ( ) # zfs allow cindy create,destroy,mount,snapshot tank/home/cindy # zfs allow tank/home/cindy ---- Permissions on tank/home/cindy Local+Descendent permissions: user cindy create,destroy,mount,snapshot. # zfs unallow cindy tank/home/cindy # zfs allow tank/home/cindy 8 1 create mount., mark tank create mount. # chmod A+user:mark:add_subdirectory:fd:allow /tank/home zfs allow create, destroy mount.. # zfs allow mark create,destroy,mount tank/home mark tank/home.. # su mark mark$ zfs create tank/home/mark mark$ ^D # su lp $ zfs create tank/home/lp cannot create tank/home/lp : permission denied 8 2 create destroy staff tank/home., staff. # zfs allow staff create,mount tank/home # zfs allow -c create,destroy tank/home # zfs allow tank/home ---- Permissions on tank/home Create time permissions: create,destroy 250 Oracle Solaris 11.1 : ZFS

251 ZFS ( ) 8 2 create destroy ( ) Local+Descendent permissions: group staff create,mount # su cindy cindy% zfs create tank/home/cindy/files cindy% exit # su mark mark% zfs create tank/home/mark/data mark% exit cindy% zfs destroy tank/home/mark/data cannot destroy tank/home/mark/data : permission denied 8 3., mark create, destroy mount. mark tank/home. snapshot. # zfs allow -l mark snapshot tank/home # zfs allow tank/home ---- Permissions on tank/home Create time permissions: create,destroy Local permissions: user mark snapshot Local+Descendent permissions: group staff create,mount # su mark mark$ zfs snapshot tank/home@snap1 mark$ zfs snapshot tank/home/mark@snap1 cannot create snapshot tank/home/mark@snap1 : permission denied mark zfs allow -d.. # zfs unallow -l mark snapshot tank/home # zfs allow -d mark snapshot tank/home # zfs allow tank/home ---- Permissions on tank/home Create time permissions: create,destroy Descendent permissions: user mark snapshot Local+Descendent permissions: group staff create,mount # su mark $ zfs snapshot tank/home@snap2 cannot create snapshot tank/home@snap2 : permission denied $ zfs snapshot tank/home/mark@snappy mark tank/home. 8 Oracle Solaris ZFS 251

252 ZFS ( ) 8 4., zfs allow staff. tank/home destroy snapshot. # zfs allow staff create,mount tank/home # zfs allow -c destroy,snapshot tank/home # zfs allow tank/home ---- Permissions on tank/home Create time permissions: create,destroy,snapshot Local+Descendent permissions: group staff create,mount mark staff tank/home. mark tank/home/mark2.. # su mark $ zfs create tank/home/mark2 $ zfs allow tank/home/mark Permissions on tank/home/mark Local permissions: user mark create,destroy,snapshot ---- Permissions on tank/home Create time permissions: create,destroy,snapshot Local+Descendent permissions: group staff create,mount mark tank/home/mark.. $ zfs snapshot tank/home/mark@snap1 cannot create snapshot tank/home/mark@snap1 : permission denied mark create. NFS. $ cd /tank/home/mark2 $ ls $ cd.zfs $ ls shares snapshot $ cd snapshot $ ls -l total 3 drwxr-xr-x 2 mark staff 2 Sep 27 15:55 snap1 $ pwd /tank/home/mark2/.zfs/snapshot $ mkdir snap2 $ zfs list 252 Oracle Solaris 11.1 : ZFS

253 ZFS ( ) 8 4 ( ) # zfs list -r tank/home NAME USED AVAIL REFER MOUNTPOINT tank/home/mark 63K 62.3G 32K /tank/home/mark tank/home/mark2 49K 62.3G 31K /tank/home/mark2 tank/home/mark2@snap1 18K - 31K - tank/home/mark2@snap2 0-31K - $ ls snap1 snap2 $ rmdir snap2 $ ls snap1 8 5 tank staff rename. cindy staff tank, lp tank. # zfs allow create,destroy,mount,snapshot,promote,clone,readonly tank # zfs allow tank ---- Permissions on tank Permission clone,create,destroy,mount,promote,readonly,snapshot # zfs allow tank # zfs allow tank ---- Permissions on tank Permission clone,create,destroy,mount,promote,readonly,snapshot Local+Descendent permissions: group # chmod A+group:staff:add_subdirectory:fd:allow tank # su cindy cindy% zfs create tank/data cindy% zfs allow tank ---- Permissions on tank Permission clone,create,destroy,mount,promote,readonly,snapshot Local+Descendent permissions: group cindy% ls -l /tank total 15 drwxr-xr-x 2 cindy staff 2 Jun 24 10:55 data cindy% exit # su lp $ zfs create tank/lp cannot create tank/lp : permission denied 8 Oracle Solaris ZFS 253

254 ZFS ( ) ZFS ( ). # zfs allow dataset cindy tank/cindy create, destroy, mount, snapshot. # zfs allow tank/cindy Local+Descendent permissions on (tank/cindy) user cindy create,destroy,mount,snapshot 8 7 pool/fred pool. pool/fred destroy, snapshot, mount, clone, promote, mountpoint. " mountpoint. joe create, destroy mount. share rename. barney pool destroy, mount). 254 Oracle Solaris 11.1 : ZFS

255 ZFS ( ) 8 7 ( ) $ zfs allow pool/fred ---- Permissions on pool/fred Permission create,mount Create time Local permissions: user user joe create,destroy,mount Local+Descendent permissions: user user group ---- Permissions on pool Permission create,destroy,mount Local permissions: group ZFS ( ) zfs unallow., cindy tank/cindy create, destroy, mount snapshot. # zfs allow cindy create,destroy,mount,snapshot tank/home/cindy # zfs allow tank/home/cindy ---- Permissions on tank/home/cindy Local+Descendent permissions: user cindy create,destroy,mount,snapshot zfs unallow cindy snapshot tank/home/cindy. # zfs unallow cindy snapshot tank/home/cindy # zfs allow tank/home/cindy ---- Permissions on tank/home/cindy Local+Descendent permissions: user cindy create,destroy,mount cindy% zfs create tank/home/cindy/data cindy% zfs snapshot tank/home/cindy@today cannot create snapshot tank/home/cindy@today : permission denied, mark tank/home/mark. 8 Oracle Solaris ZFS 255

256 ZFS ( ) # zfs allow tank/home/mark ---- Permissions on tank/home/mark Local+Descendent permissions: user mark create,destroy,mount zfs unallow mark tank/home/mark. # zfs unallow mark tank/home/mark zfs unallow tank. # zfs allow tank ---- Permissions on tank Permission clone,create,destroy,mount,promote,readonly,snapshot Create time permissions: create,destroy,mount Local+Descendent permissions: group staff create,mount # zfs unallow tank # zfs allow tank ---- Permissions on tank Create time permissions: create,destroy,mount Local+Descendent permissions: group staff create,mount 256 Oracle Solaris 11.1 : ZFS

257 9 9 Oracle Solaris ZFS ZFS, Solaris ZFS, ZFS ZFS ZFS 260 Solaris ZFS 265 ZFS ZFS ZFS. ZFS /dev/zvol/{dsk,rdsk}/pool. 5GB ZFS tank/vol. # zfs create -V 5gb tank/vol.,.., zfs get zfs get all ZFS.. # zfs get all tank/vol 257

258 ZFS zfs get volsize (?) I/O.. # zfs get -H volsize tank/vol tank/vol volsize? local I/O. 273 ZFS. Solaris ZFS.. ZFS 262 ZFS. ZFS ZFS UFS, ZFS ZFS.. # swap -l swapfile dev swaplo blocks free /dev/zvol/dsk/rpool/swap 253, ZFS UFS, ZFS ZFS... # dumpadm Dump content: kernel pages Dump device: /dev/zvol/dsk/rpool/dump (dedicated) Savecore directory: /var/crash/ Savecore enabled: yes Solaris swap dumpadm. ZFS. /etc/vfstab.. # zfs create -V 2G rpool/swap2 # swap -a /dev/zvol/dsk/rpool/swap2 # swap -l swapfile dev swaplo blocks free /dev/zvol/dsk/rpool/swap 256, /dev/zvol/dsk/rpool/swap2 256, ZFS. ZFS. 122 ZFS. 258 Oracle Solaris 11.1 : ZFS

259 ZFS ZFS iscsi LUN COMSTAR(Common Multiprotocol SCSI Target) SCSI Oracle Solaris. ZFS iscsi LUN( ). COMSTAR. # pkg install group/feature/storage-server iscsi ZFS SCSI LUN.. # zfs create -V 2g tank/volumes/v2 # sbdadm create-lu /dev/zvol/rdsk/tank/volumes/v2 Created the following LU: GUID DATA SIZE SOURCE f000144f1dafaa4c0faff /dev/zvol/rdsk/tank/volumes/v2 # sbdadm list-lu Found 1 LU(s) GUID DATA SIZE SOURCE f000144f1dafaa4c0faff /dev/zvol/rdsk/tank/volumes/v2 LUN. LUN GUID LUN. LUN. # stmfadm list-lu LU Name: F000144F1DAFAA4C0FAFF20001 # stmfadm add-view F000144F1DAFAA4C0FAFF20001 # stmfadm list-view -l F000144F1DAFAA4C0FAFF20001 View Entry: 0 Host group : All Target group : All LUN : 0 iscsi. iscsi Oracle Solaris 11.1 : 11, COMSTAR ( ). iscsi ZFS,, ZFS iscsi LUN ZFS.. # zfs rename tank/volumes/v2 tank/volumes/v1 cannot rename tank/volumes/v2 : dataset is busy # zpool export tank cannot export tank : pool is busy 9 Oracle Solaris ZFS 259

260 Solaris ZFS iscsi. NFS, iscsi. Solaris ZFS Oracle Solaris ZFS. 261 ZFS ZFS 263 ZFS 263 ZFS 264 zoned ZFS. ZFS. ZFS. ZFS. ZFS. ( ).,. ZFS.,,.. Oracle Solaris ZFS. ZFS mountpoint legacy. zonepath zonepath ZFS zoneadm clone ZFS. zoneadm clone zonepath ZFS zonepath. zfs clone. Oracle Solaris : Oracle Solaris Zones, Oracle Solaris 10 Zones II, Oracle Solaris Zones. 260 Oracle Solaris 11.1 : ZFS

261 Solaris ZFS ZFS ZFS. ZFS mountpoint legacy., tank/zone/zion mountpoint. # zfs set mountpoint=legacy tank/zone/zion zonecfg add fs ZFS. ZFS. # zonecfg -z zion zonecfg:zion> add fs zonecfg:zion:fs> set type=zfs zonecfg:zion:fs> set special=tank/zone/zion zonecfg:zion:fs> set dir=/opt/data zonecfg:zion:fs> end ZFS tank/zone/zion zion. /opt/data. mountpoint legacy.. atime, readonly, compression.. zonecfg zonecfg Oracle Solaris : Oracle Solaris Zones, Oracle Solaris 10 Zones II, Oracle Solaris Zones. ZFS zonecfg add dataset. ZFS. # zonecfg -z zion zonecfg:zion> add dataset zonecfg:zion:dataset> set name=tank/zone/zion zonecfg:zion:dataset> set alias=tank zonecfg:zion:dataset> end 9 Oracle Solaris ZFS 261

262 Solaris ZFS, ZFS tank/zone/zion zion. zion tank/zone/zion tank.... zion..,,. ZFS ZFS. dev(7fs). ZFS ZFS. ZFS.. # zfs create -V 2g tank/zone/zion/vol1. zonecfg add dataset ZFS.. # zonecfg -z zion zonecfg:zion> add dataset zonecfg:zion:dataset> set name=tank/volumes/vol1 zonecfg:zion:dataset> end. zonecfg add device ZFS.. # zonecfg -z zion zonecfg:zion> add device zonecfg:zion:device> set match=/dev/zvol/dsk/tank/volumes/vol2 zonecfg:zion:device> end. 262 Oracle Solaris 11.1 : ZFS

263 Solaris ZFS ZFS ZFS..,,. zonecfg add device zpool. ZFS.,.,. global# zfs list -Ho name tank tank/home tank/data tank/data/matrix tank/data/zion tank/data/zion/home tank/data/zion zion. tank - - tank/home - - tank/data - - tank/data/zion zoned, quota, reservation tank/data/zion/home zoned tank/zone/zion. zoned.. quota reservation,.. 9 Oracle Solaris ZFS 263

264 Solaris ZFS, share.nfs mountpoint. zoned.., setuid,., mountpoint.. ZFS zoned. zoned ZFS.. zoned. tank/zone/zion tank/zone/global. # zfs list -o name,zoned,mountpoint -r tank/zone NAME ZONED MOUNTPOINT tank/zone/global off /tank/zone/global tank/zone/zion on /tank/zone/zion # zfs mount tank/zone/global /tank/zone/global tank/zone/zion /export/zone/zion/root/tank/zone/zion mountpoint tank/zone/zion. mountpoint,. zoned.. mountpoint setuid., zoned. zoned off mountpoint setuid setuid. zfs set zfs inherit zoned. zoned 264 Oracle Solaris 11.1 : ZFS

265 ZFS.. zfs send zfs receive.... Oracle Solaris 11.1 : Oracle Solaris, Oracle Solaris 10 II, Oracle Solaris.. pkg update, beadm create zoneadm clone. sysa rpool/zones sys tank/zones. sysb. sysa# zfs snapshot -r rpool/zones@send-to-sysb sysa# zfs send -R rpool/zones@send-to-sysb ssh sysb zfs receive -d tank sysc sysd. ssh NFS. zfs send. sysc# zfs snapshot -r rpool/zones/zone1@send-to-nfs sysc# zfs send -rc rpool/zones/zone1@send-to-nfs > /net/nfssrv/export/scratch/zone1.zfs sysd# zfs create tank/zones sysd# zfs receive -d tank/zones < /net/nfssrv/export/scratch/zone1.zfs ZFS... ZFS.,. 9 Oracle Solaris ZFS 265

266 ZFS ZFS.. zpool create -R /. /mnt morpheus. # zpool create -R /mnt morpheus c0t0d0 # zfs list morpheus NAME USED AVAIL REFER MOUNTPOINT morpheus 32.5K 33.5G 8K /mnt morpheus /mnt. / /mnt., -R alternate root value. # zpool export morpheus # zpool import morpheus cannot mount / : directory is not empty # zpool export morpheus # zpool import -R /mnt morpheus # zfs list morpheus NAME USED AVAIL REFER MOUNTPOINT morpheus 32.5K 33.5G 8K /mnt... /mnt morpheus. morpheus. # zpool import -R /a pool # zpool list morpheus NAME SIZE ALLOC FREE CAP HEALTH ALTROOT pool 44.8G 78K 44.7G 0% ONLINE /a # zfs list pool NAME USED AVAIL REFER MOUNTPOINT pool 73.5K 44.1G 21K /a/pool 266 Oracle Solaris 11.1 : ZFS

267 10 Oracle Solaris ZFS ZFS ZFS ZFS 269 ZFS 270 ZFS 273 ZFS 278 ZFS ,. ZFS. 31 ZFS. ZFS zpool list zfs list df du.,. (rpool) 5.46GB 68.5GB. 267

268 ZFS # zpool list rpool NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT rpool 74G 5.46G 68.5G 7% 1.00x ONLINE - USED ALLOC USED.. # zfs list -r rpool NAME USED AVAIL REFER MOUNTPOINT rpool 5.41G 67.4G 74.5K /rpool rpool/root 3.37G 67.4G 31K legacy rpool/root/solaris 3.37G 67.4G 3.07G / rpool/root/solaris/var 302M 67.4G 214M /var rpool/dump 1.01G 67.5G 1000M - rpool/export 97.5K 67.4G 32K /rpool/export rpool/export/home 65.5K 67.4G 32K /rpool/export/home rpool/export/home/admin 33.5K 67.4G 33.5K /rpool/export/home/admin rpool/swap 1.03G 67.5G 1.00G - ZFS zpool list SIZE.. zfs list, ZFS ( ). 136GB zpool list SIZE FREE 136GB. zfs list AVAIL 134GB.. # zpool create tank c0t6d0 # zpool list tank NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT tank 136G 95.5K 136G 0% 1.00x ONLINE - # zfs list tank NAME USED AVAIL REFER MOUNTPOINT tank 72K 134G 21K /tank 136GB zpool list SIZE 136GB FREE 136GB.. zfs list AVAIL 134GB.. # zpool create tank mirror c0t6d0 c0t7d0 # zpool list tank NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT tank 136G 95.5K 136G 0% 1.00x ONLINE - # zfs list tank NAME USED AVAIL REFER MOUNTPOINT tank 72K 134G 21K /tank 268 Oracle Solaris 11.1 : ZFS

269 ZFS RAID-Z raidz2 136GB zpool list SIZE 408GB FREE 408GB., ( : ). zfs list AVAIL 133GB. RAID-Z zpool list zfs list zpool list. # zpool create tank raidz2 c0t6d0 c0t7d0 c0t8d0 # zpool list tank NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT tank 408G 286K 408G 0% 1.00x ONLINE - # zfs list tank NAME USED AVAIL REFER MOUNTPOINT tank 73.2K 133G 20.9K /tank ZFS ZFS..,. ZFS. 269 ZFS 270 ZFS 270 ZFS,. ZFS ZFS REMOVED.. RAID-Z. UNAVAIL.. RAID-Z(raidz1) 10 Oracle Solaris ZFS 269

270 ZFS ZFS " "..: I/O (cosmic ray) I/O,..,.. ZFS ( )......,.. ZFS fsck ZFS...,., 270 Oracle Solaris 11.1 : ZFS

271 ZFS. ( ) ZFS ZFS. fsck UFS. ZFS ZFS.. fsck,. fsck.. ZFS.. ZFS ZFS,. ZFS.., I/O.. zpool scrub.. # zpool scrub tank zpool status.. 10 Oracle Solaris ZFS 271

272 ZFS # zpool status -v tank pool: tank state: ONLINE scan: scrub in progress since Mon Jun 7 12:07: M scanned out of 222M at 9.55M/s, 0h0m to go 0 repaired, 90.44% done config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c1t0d0 ONLINE c1t1d0 ONLINE errors: No known data errors. -s.. # zpool scrub -s tank.. I/O.. I/O. zpool status 83 ZFS. ZFS Oracle Solaris 11.1 : ZFS

273 ZFS ZFS ZFS. 274 ZFS 274 zpool status 277 ZFS ZFS. ZFS zpool status. ZFS/FMA. ZFS zpool history. ZFS zpool status...,. ZFS., I/O,. fmd ZFS. fmd zpool status.. zpool history ZFS.. # zpool history tank History for tank : :06:50 zpool create tank mirror c0t1d0 c0t2d0 c0t3d :06:58 zfs create tank/eric :07:01 zfs set checksum=off tank/eric tank/eric.. /var/adm/messages fmd. zpool status -x Oracle Solaris ZFS 273

274 ZFS zpool status -x.. zpool status... ZFS zpool status -x... # zpool status -x all pools are healthy -x, ( ). zpool status 83 ZFS. zpool status zpool status. # zpool status pond pool: pond state: DEGRADED status: One or more devices are unavailable in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Determine if the device needs to be replaced, and clear the errors using zpool clear or fmadm repaired, or replace the device with zpool replace. Run zpool status -v to see device specific details. scan: scrub repaired 0 in 0h0m with 0 errors on Wed Jun 20 13:16: config: NAME STATE READ WRITE CKSUM pond DEGRADED mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 DEGRADED c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 UNAVAIL errors: No known data errors 274 Oracle Solaris 11.1 : ZFS

275 ZFS. zpool status.. pool. state status action see scrub errors ,.. zpool status config. ONLINE, FAULTED, DEGRADED SUSPENDED. ONLINE... READ I/O. WRITE I/O. CKSUM.. I/O,.., 10 Oracle Solaris ZFS 275

276 ZFS RAID-Z. ZFS. 282., zpool status. state. UNAVAIL zpool status scrub.,.. zpool status... scan: scrub in progress since Wed Jun 20 14:56: M scanned out of 71.8G at 48.1M/s, 0h25m to go 0 repaired, 0.72% done.. scan: scrub repaired 0 in 0h11m with 0 errors on Wed Jun 20 15:08: scan: scrub canceled on Wed Jun 20 16:04: ZFS. zpool status.. ZFS Oracle Solaris 11.1 : ZFS

277 ZFS. I/O... zpool status -v.. # zpool status -v tank pool: tank state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: scan: scrub repaired 0 in 0h0m with 2 errors on Fri Jun 29 16:58: config: NAME STATE READ WRITE CKSUM tank ONLINE c8t0d0 ONLINE c8t1d0 ONLINE errors: Permanent errors have been detected in the following files: /tank/file.1 fmd /var/adm/messages. fmdump ZFS ZFS syslog.. FAULTED ZFS.. ZFS.,. syslog fmdump. 10 Oracle Solaris ZFS 277

278 ZFS ZFS... ZFS ZFS.. ZFS,... ( ). 95 ZFS. zpool status UNAVAIL. ZFS.... UNAVAIL 284 ZFS. UNAVAIL. 124 ZFS 115 ZFS (SPARC x86/vtoc) 117 ZFS (x86/efi(gpt)) 11,, fmd. 278 Oracle Solaris 11.1 : ZFS

279 SUNW-MSG-ID: ZFS-8000-QJ, TYPE: Fault, VER: 1, SEVERITY: Minor EVENT-TIME: Wed Jun 20 13:09:55 MDT 2012 PLATFORM: ORCL,SPARC-T3-4, CSN: 1120BDRCCD, HOSTNAME: tardis SOURCE: zfs-diagnosis, REV: 1.0 EVENT-ID: e13312e0-be0a-439b-d7d3-cddaefe717b0 DESC: Outstanding dtls on ZFS device in pool pond. AUTO-RESPONSE: No automated response will occur. IMPACT: None at this time. REC-ACTION: Use fmadm faulty to provide a more detailed view of this event. Run zpool status -lx for more information. Please refer to the associated reference document at for the latest service procedures and policies regarding this diagnosis. zpool status -v.. # zpool status -v pool: pond state: DEGRADED status: One or more devices are unavailable in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Determine if the device needs to be replaced, and clear the errors using zpool clear or fmadm repaired, or replace the device with zpool replace. scan: scrub repaired 0 in 0h0m with 0 errors on Wed Jun 20 13:16: config: device details: NAME STATE READ WRITE CKSUM pond DEGRADED mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 DEGRADED c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 UNAVAIL c0t5000c500335dc60fd0 UNAVAIL cannot open status: ZFS detected errors on this device. The device was missing. see: for recovery c0t5000c500335dc60fd0.. zpool online.. fmadm faulty FMA.. # fmadm faulty Oracle Solaris ZFS 279

280 TIME EVENT-ID MSG-ID SEVERITY Jun 20 13:15: f c-c2d3-d940-93acbb881bd8 ZFS-8000-LR Major Problem Status : solved Diag Engine : zfs-diagnosis / 1.0 System Manufacturer : unknown Name : ORCL,SPARC-T3-4 Part_Number : unknown Serial_Number : 1120BDRCCD Host_ID : 84a02d Suspect 1 of 1 : Fault class : fault.fs.zfs.open_failed Certainty : 100% Affects : zfs://pool=86124fa573cad84e/vdev=25d36cd46e0a7f49/pool_name=pond/vdev_ name=id1,sd@n5000c500335dc60f/a Status : faulted and taken out of service FRU Name : "zfs://pool=86124fa573cad84e/vdev=25d36cd46e0a7f49/pool_name=pond/vdev_ name=id1,sd@n5000c500335dc60f/a" Status : faulty Description : ZFS device id1,sd@n5000c500335dc60f/a in pool pond failed to open. Response Impact Action : An attempt will be made to activate a hot spare if available. : Fault tolerance of the pool may be compromised. : Use fmadm faulty to provide a more detailed view of this event. Run zpool status -lx for more information. Please refer to the associated reference document at for the latest service procedures and policies regarding this diagnosis. fmadm faulty Affects: FMA. # fmadm repaired zfs://pool=86124fa573cad84e/vdev=25d36cd46e0a7f49/pool_name=pond/vdev_ name=id1,sd@n5000c500335dc60f/a fmadm: recorded repair to of zfs://pool=86124fa573cad84e/vdev=25d36cd46e0a7f49/pool_name=pond/vdev_ name=id1,sd@n5000c500335dc60f/a,.. # zpool status -x tank pool tank is healthy 280 Oracle Solaris 11.1 : ZFS

281 .. USB SUSPENDED. SUSPENDED.. # zpool status cybermen pool: cybermen state: SUSPENDED status: One or more devices are unavailable in response to IO failures. The pool is suspended. action: Make sure the affected devices are connected, then run zpool clear or fmadm repaired. Run zpool status -v to see device specific details. see: scan: none requested config: NAME STATE READ WRITE CKSUM cybermen UNAVAIL c8t3d0 UNAVAIL c8t1d0 UNAVAIL # zpool clear cybermen # fmadm repaired zfs://pool=name/vdev=guid ZFS ZFS. UNAVAIL SUSPENDED attach ZFS. zpool online ZFS.. # zpool online tank c0t1d0 10 Oracle Solaris ZFS 281

282 71.,.. (cosmic ray).,..,. ( : /dev/zero )... I/O.,.. I/O,..... zpool status.. # zpool status -v tank pool: tank state: ONLINE status: One or more devices has experienced an error resulting in data 282 Oracle Solaris 11.1 : ZFS

283 corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: config: NAME STATE READ WRITE CKSUM tank ONLINE c8t0d0 ONLINE c8t0d0 ONLINE errors: Permanent errors have been detected in the following files: /tank/file.1 I/O,. ( )... syslog. SCSI Fibre Channel. syslog..?.. ZFS, ,. RAID-Z zpool clear.. 10 Oracle Solaris ZFS 283

284 # zpool clear tank c1t1d0.. # zpool clear tank 72. ZFS ZFS UNAVAIL. 4 RAID-Z(raidz1) UNAVAIL,. UNAVAIL.. c1t1d0, c1t0d0. mirror c1t0d0 c1t1d0 DEGRADED ONLINE UNAVAIL c1t0d0,. UNAVAIL. ONLINE UNAVAIL. 284 Oracle Solaris 11.1 : ZFS

285 raidz1 c1t0d0 c2t0d0 c3t0d0 c4t0d0 UNAVAIL ONLINE UNAVAIL UNAVAIL ONLINE,. c1t0d0 c1t1d0 ONLINE ONLINE UNAVAIL UNAVAIL. UNAVAIL ZFS. ZFS zpool replace.. # zpool replace tank c1t1d0 c2t0d0 ( )..... # zpool replace tank c1t1d0. zpool replace Oracle Solaris ZFS 285

286 10 1 ZFS SATA SATA tank (c1t3d0). c1t3d0 (c1t3d0). SATA 73.. (c1t3d0). SATA. cfgadm SATA (c1t3d0).. (c1t3d0). UNAVAIL Ready to Remove LED ( ). SATA (c1t3d0). (c1t3d0). zpool replace (c1t3d0). autoreplace on, zpool replace... c2t4d0. # zpool detach tank c2t4d0 FMA. # fmadm faulty # fmadm repaired zfs://pool=name/vdev=guid ZFS. # zpool offline tank c1t3d0 # cfgadm grep c1t3d0 sata1/3::dsk/c1t3d0 disk connected configured ok # cfgadm -c unconfigure sata1/3 Unconfigure the device at: /devices/pci@0,0/pci1022,7458@2/pci11ab,11ab@1:3 This operation will suspend activity on the SATA device Continue (yes/no)? yes # cfgadm grep sata1/3 sata1/3 disk connected unconfigured ok 286 Oracle Solaris 11.1 : ZFS

287 10 1 ZFS SATA ( ) <Physically replace the failed disk c1t3d0> # cfgadm -c configure sata1/3 # cfgadm grep sata1/3 sata1/3::dsk/c1t3d0 disk connected configured ok # zpool online tank c1t3d0 # zpool replace tank c1t3d0 # zpool status tank pool: tank state: ONLINE scrub: resilver completed after 0h0m with 0 errors on Tue Feb 2 13:17: config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE mirror-1 ONLINE c0t2d0 ONLINE c1t2d0 ONLINE mirror-2 ONLINE c0t3d0 ONLINE c1t3d0 ONLINE errors: No known data errors zpool output replacing.. replacing DEGRADED c1t3d0s0/o FAULTED c1t3d0 ONLINE (c1t3d0) (c4t3d0) zpool replace.. # zpool replace tank c1t3d0 c4t3d0 # zpool status pool: tank state: DEGRADED scrub: resilver completed after 0h0m with 0 errors on Tue Feb 2 13:35: config: NAME STATE READ WRITE CKSUM tank DEGRADED mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE mirror-1 ONLINE c0t2d0 ONLINE Oracle Solaris ZFS 287

288 10 1 ZFS SATA ( ) c1t2d0 ONLINE mirror-2 DEGRADED c0t3d0 ONLINE replacing DEGRADED c1t3d0 OFFLINE c4t3d0 ONLINE errors: No known data errors zpool status. # zpool status tank pool: tank state: ONLINE scrub: resilver completed after 0h0m with 0 errors on Tue Feb 2 13:35: config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE mirror-1 ONLINE c0t2d0 ONLINE c1t2d0 ONLINE mirror-2 ONLINE c0t3d0 ONLINE c4t3d0 ONLINE ZFS zpool status. FMA(Fault Management Architecture). ZFS FMA. (pool) (c0t5d0).. zpool status -x FMA. CSP/main/ article?cmd=show&type=not&doctype=reference&alias=event:zfs-8000-k4... FMA. 288 Oracle Solaris 11.1 : ZFS

289 10 2 ( ). # zpool status -x pool: pool state: FAULTED status: One or more of the intent logs could not be read. Waiting for adminstrator intervention to fix the faulted pool. action: Either restore the affected device(s) and run zpool online, or ignore the intent log records by running zpool clear. scrub: none requested config: NAME STATE READ WRITE CKSUM pool FAULTED bad intent log mirror-0 ONLINE c0t1d0 ONLINE c0t4d0 ONLINE logs FAULTED bad intent log c0t5d0 UNAVAIL cannot open <Physically replace the failed log device> # zpool online pool c0t5d0 # zpool clear pool # fmadm faulty # fmadm repair zfs://pool=name/vdev=guid.. c0t5d0.. # zpool online pool c0t5d0. # zpool clear pool zpool clear.... zpool status. zpool status. 10 Oracle Solaris ZFS 289

290 .. scan: resilver in progress since Mon Jun 7 09:17: G scanned out of 16.2G at 18.5M/s, 0h2m to go 13.3G resilvered, 82.34% done.. resilvered 16.2G in 0h16m with 0 errors on Mon Jun 7 09:34: ZFS.. ZFS. ( ).. GB 500GB... zpool status.. # zpool status tank pool: tank state: ONLINE status: One or more devices is currently being resilvered. continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Mon Jun 7 10:49: M scanned out of 222M at 5.46M/s, 0h0m to go 54.5M resilvered, 24.64% done config: The pool will NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE replacing-0 ONLINE c1t0d0 ONLINE c2t0d0 ONLINE (resilvering) c1t1d0 ONLINE c1t0d0 c2t0d0. replacing Oracle Solaris 11.1 : ZFS

291 ONLINE DEGRADED., I/O I/O... # zpool status tank pool: tank state: ONLINE scrub: resilver completed after 0h1m with 0 errors on Tue Feb 2 13:54: config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c2t0d0 ONLINE M resilvered c1t1d0 ONLINE errors: No known data errors ONLINE, (c1t0d0) ZFS ZFS,.,,..,... - ZFS Oracle Solaris ZFS 291

292 . 270 ZFS. zpool status..: # zpool status tank pool: tank state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: config: NAME STATE READ WRITE CKSUM tank ONLINE c0t5000c500335e106bd0 ONLINE c0t5000c500335fc3e7d0 ONLINE errors: 2 data errors, use -v for a list..... zpool online... # zpool status -v morpheus pool: morpheus id: state: UNAVAIL status: The pool metadata is corrupted. action: The pool cannot be imported due to damaged devices or data. see: config: morpheus FAULTED corrupted data c1t10d0 ONLINE 292 Oracle Solaris 11.1 : ZFS

293 FAULTED zpool status -v.. # zpool status tank -v pool: tank state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: config: NAME STATE READ WRITE CKSUM tank ONLINE c0t5000c500335e106bd0 ONLINE c0t5000c500335fc3e7d0 ONLINE errors: Permanent errors have been detected in the following files: /tank/file.1 /tank/file.2.,.. /monkey/a.txt, (/).. monkey/ghost/e.txt dnode_t,.. monkey/dnode:<0x0> 10 Oracle Solaris ZFS 293

294 MOS(Metaobject Set) <metadata>... zpool status -v. ZFS.... zpool status - v. # find mount-point -inum $inode -print # find mount-point/.zfs/snapshot -inum $inode -print inode. inode. ZFS. zpool clear -F zpool import - F.. zpool status.. # zpool status pool: tpool state: UNAVAIL status: The pool metadata is corrupted and the pool cannot be opened. action: Recovery is possible, but will result in some data loss. Returning the pool to its state as of Fri Jun 29 17:22: should correct the problem. Approximately 5 seconds of data must be discarded, irreversibly. Recovery can be attempted by executing zpool clear -F tpool. A scrub of the pool is strongly recommended after recovery. see: scrub: none requested config: 294 Oracle Solaris 11.1 : ZFS

295 NAME STATE READ WRITE CKSUM tpool UNAVAIL corrupted data c1t1d0 ONLINE c1t3d0 ONLINE # zpool clear -F tpool. # zpool import tpool cannot import tpool : I/O error Recovery is possible, but will result in some data loss. Returning the pool to its state as of Fri Jun 29 17:22: should correct the problem. Approximately 5 seconds of data must be discarded, irreversibly. Recovery can be attempted by executing zpool import -F tpool. A scrub of the pool is strongly recommended after recovery.. # zpool import -F tpool Pool tpool returned to its state as of Fri Jun 29 17:22: Discarded approximately 5 seconds of transactions zpool.cache, zpool status. zpool.cache.... # zpool import -o readonly=on tpool 100. zpool import -m , zpool status. zpool destroy -f Oracle Solaris ZFS 295

296 ZFS.., -. ZFS. ZFS /etc/zfs/zpool.cache.. -m milestone=none none. /etc/zfs/zpool.cache. ZFS. svcadm milestone all.. zpool import... zpool.cache. fmdump -ev. fmdump. 296 Oracle Solaris 11.1 : ZFS

297 11 Oracle Solaris ZFS 298 ZFS 300 ZFS. OS ZFS.. Oracle Solaris 11.. ZFS. OS. USB.. Oracle Solaris

298 ZFS. ZFS Oracle Solaris 11. ZFS ( ). Oracle Solaris 11 SMI(VTOC). Oracle Solaris 11.1 x86 SMI(VTOC) EFI(GPT). EFI(GPT) 17 EFI(GPT).. ssh. ZFS ZFS.. sysa# zpool get all rpool. sysa# zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT rpool 74G 5.42G 68.6G 7% 1.00x ONLINE -. sysa# zfs list -r rpool NAME USED AVAIL REFER MOUNTPOINT rpool 13.8G 53.1G 73.5K /rpool rpool/root 3.54G 53.1G 31K legacy rpool/root/solaris 3.54G 53.1G 3.37G / rpool/root/solaris/var 165M 53.1G 163M /var rpool/varshare 37.5K 53.1G 37.5K /var/share rpool/dump 8.19G 53.4G 7.94G - rpool/export 63K 53.1G 32K /export rpool/export/home 31K 53.1G 31K /export/home rpool/swap 2.06G 53.2G 2.00G Oracle Solaris 11.1 : ZFS

299 ZFS ZFS.... NFS ssh.,. sysa sysb. rpool sysa# zfs snapshot -r [email protected]. sysa# zfs destroy rpool/[email protected] sysa# zfs destroy rpool/[email protected]... a.. /tank/snaps. sysb# zfs set share.nfs=on tank/snaps sysb# zfs set share.nfs.sec.default.root=sysa tank/snaps b... sysa# zfs send -Rv [email protected] gzip > /net/sysb/tank/snaps/ rpool.snap1.gz sending to [email protected] sending to rpool/[email protected] sending to rpool/[email protected] sending to rpool/export/[email protected]

300 sending to rpool/[email protected] sending to rpool/root/solaris@install sending to rpool/root/[email protected] sending to rpool/root/solaris/var@install sending to rpool/root/solaris/[email protected] OS ZFS (SPARC x86/vtoc)..,. Oracle Solaris 11. DVD USB (SPARC x86). Shell( ). (x86 ) GNOME. AI (SPARC x86) Shell( ). SPARC AI ( ) Shell( ). 300 Oracle Solaris 11.1 : ZFS

301 ok boot net:dhcp...welcome to the Oracle Solaris 11 installation menu 1 Install Oracle Solaris 2 Install Additional Drivers 3 Shell 4 Terminal type (currently xterm) 5 Reboot Please enter a number [1]: 3 3 SPARC x86(vtoc):. a. format. # format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c2t0d0 <FUJITSU-MAY2073RCSUN72G-0401 cyl alt 2 hd 24 sec 424> /pci@780/pci@0/pci@9/scsi@0/sd@0,0 1. c2t1d0 <FUJITSU-MAY2073RCSUN72G-0401 cyl alt 2 hd 24 sec 424> /pci@780/pci@0/pci@9/scsi@0/sd@1,0 2. c2t2d0 <SEAGATE-ST973402SSUN72G GB> /pci@780/pci@0/pci@9/scsi@0/sd@2,0 3. c2t3d0 <SEAGATE-ST973401LSUN72G GB> /pci@780/pci@0/pci@9/scsi@0/sd@3,0 Specify disk (enter its number): 0 b. SPARC x86(vtoc): SMI(VTOC) 0. SMI 0. selecting c2t0d0 [disk formatted] format> partition partition> print c. SPARC x86 (VTOC): SMI(VTOC)... SPARC: sysa# format -L vtoc -d c2t0d Oracle Solaris 11.1 : ZFS (EFI(GPT))

302 x86: sysa# fdisk -B /dev/rdsk/c2t0d0p0 sysa# format -L vtoc -d c2t0d Oracle Solaris 11.1 : ZFS (EFI(GPT)). 4. SPARC x86 (VTOC). sysa# zpool create rpool c2t0d0s0 EFI(GPT) x86. sysa# zpool create -B rpool c2t0d sysa# mount -F nfs sysb:/tank/snaps /mnt. sysa# gzcat /mnt/rpool.snap1.qz zfs receive -Fv rpool receiving full stream of [email protected] into [email protected] received 92.7KB stream in 1 seconds (92.7KB/sec) receiving full stream of rpool/[email protected] into rpool/[email protected] received 47.9KB stream in 1 seconds (47.9KB/sec)..... sysa# zfs create -V 4G rpool/swap sysa# zfs create -V 4G rpool/dump Oracle Solaris 11.1 :. BE. BE. sysa# beadm mount solaris /tmp/mnt. SPARC x86. sysa# bootadm install-bootloader -P rpool 302 Oracle Solaris 11.1 : ZFS

303 # devfsadm -Cn -r /tmp/mnt # touch /tmp/mnt/reconfigure BE. #beadm unmount solaris.. sysa# beadm list BE Active Mountpoint Space Policy Created solaris M static :25 solaris G static :44 # beadm activate solaris.. SPARC: eeprom PROM setenv. x86: BIOS

304 304

305 12 Oracle Solaris ZFS ZFS, ZFS. 10, Oracle Solaris ZFS. Solaris.. Oracle/Sun.. ARC ZFS.. ZFS. 305

306 # mdb -k > ::memstat Page Summary Pages MB %Tot Kernel % ZFS File Data % Anon % Exec and libs % Page cache % Free (cachelist) % Free (freelist) % Total Physical > $q ECC.. ZFS,.... ZFS ZFS. zpool split ZFS. RAID ZFS RAID JBOD. RAID ZFS ( ). ZFS ZFS. RAID RAID-Z, RAID-Z-2, RAID-Z-3, ZFS. ZFS /2-3/4. ZFS. 306 Oracle Solaris 11.1 : ZFS

307 .. ZFS ZFS.. # zpool create tank c4t1d0 c4t3d0 tank successfully created, but with no redundancy; failure of one device will cause loss of the pool. RAID-Z VDEV 3-9. RAID-Z... I/O. LUN LUN. metaslab ( : 128MB 2TB) LUN.. s*. p*. ZFS.. # zpool create rpool c0t1d0s0.. # zpool create rpool mirror c0t1d0s0 c0t2d0s0. RAID-Z. zpool add, zpool attach.. AI gzip Oracle Solaris ZFS 307

308 USB.. USB 1/2. ( ) d*. p*. ZFS. LUN. ZFS LUN ZFS I/O... # zpool create tank mirror c1d0 c2d0 mirror c3d0 c4d0 RAID-Z 1(raidz), 2(raidz2) 3(raidz3) 3. RAID-Z (128K ). 3 (2+1) 2 VDEV RAID-Z(raidz). # zpool create rzpool raidz1 c1t0d0 c2t0d0 c3t0d0 raidz1 c1t1d0 c2t1d0 c3t1d0 RAIDZ-2, RAID-Z. RAID-Z MTTDL(Mean Time To Data Loss).6 (4+2) RAID-Z(raidz2). # zpool create rzpool raidz2 c0t1d0 c1t1d0 c4t1d0 c5t1d0 c6t1d0 c7t1d0 raidz2 c0t2d0 c1t2d0 c4t2d0 c5t2d0 c6t2d0 c7t2d RAIDZ-3,.9 (6+3) RAID-Z(raidz3). # zpool create rzpool raidz3 c0t0d0 c1t0d0 c2t0d0 c3t0d0 c4t0d0 c5t0d0 c6t0d0 c7t0d0 c8t0d0 ZFS. 308 Oracle Solaris 11.1 : ZFS

309 SAN,, UNAVAIL.. SAN UNAVAIL. ZFS. Oracle Solaris ZFS. LUN. LUN ZFS I/O. LUN I/O. ( ) Oracle Solaris ZFS. Oracle Solaris ZFS LUN. Oracle Solaris ZFS. ZFS. ZFS LUN. LUN LUN. LUN LUN. (JBOD ), ZFS ZFS ( RAID-Z). Oracle Oracle. RAID. RAID-Z Oracle Solaris ZFS 309

310 VirtualBox ZFS Virtual Box.,. Virtual Box. VBoxManage setextradata <VM_NAME> "VBoxInternal/Devices/<type>/0/LUN#<n>/Config/IgnoreFlush" 0 <VM_NAME>. <type>, piix3ide( IDE ) ahci(sata ). <n>. 80%. / RAID-Z.... / - RAID-Z.. zpool status. zpool status fmdump fmdump -ev. ZFS 80%. ( : ) % %,,.. 80% ZFS. 310 Oracle Solaris 11.1 : ZFS

311 zpool status fmdump. zpool status fmdump. zpool scrub.,.,.. - zpool status. fmdump fmdump -ev. zpool status fmdump. zpool status fmdump. UNAVAIL OFFLINE. format. format ZFS. UNAVAIL OFFLINE. smtp-notify. smf(5) smtp-notify(1m).. /etc/aliases. From [email protected] Fri Jun 29 16:58: Date: Fri, 29 Jun :58: (MDT) From: No Access User <[email protected]> Message-Id: < [email protected]> Subject: Fault Management Event: tardis:zfs a To: [email protected] Content-Length: 771 SUNW-MSG-ID: ZFS A, TYPE: Fault, VER: 1, SEVERITY: Critical EVENT-TIME: Fri Jun 29 16:58:58 MDT 2012 PLATFORM: ORCL,SPARC-T3-4, CSN: 1120BDRCCD, HOSTNAME: tardis SOURCE: zfs-diagnosis, REV: 1.0 EVENT-ID: 76c2d1d dbbc-a3574b1ee807 DESC: A file or directory in pool pond could not be read due to corrupt data. AUTO-RESPONSE: No automated response will occur. IMPACT: The file or directory is unavailable. REC-ACTION: Use fmadm faulty to provide a more detailed view of this event. Run zpool status -xv and examine the list of damaged files to determine what 12 Oracle Solaris ZFS 311

312 has been affected. Please refer to the associated reference document at for the latest service procedures and policies regarding this diagnosis.. zpool list zfs list. ZFS, zfs list. zfs list -t.. ZFS.... ZFS. Oracle Oracle. ZFS recordsize Oracle db_block_size. 8KB recordsize primarycache. recordsize primarycache. recordsize primarycache metadata.. ZFS ZFS. 312 Oracle Solaris 11.1 : ZFS

313 zpool list zfs list. du df. 267 ZFS. zfs list -o space... # zfs list -t all /var. fsstat ZFS.. ZFS. # fsstat / new name name attr attr lookup rddir read read write write file remov chng get set ops ops ops bytes ops bytes K 3.23K 2.62M 20.8K 1.15M 1.75G 62.5K 348M /. /.. 12 Oracle Solaris ZFS 313

314 314

315 A Oracle Solaris ZFS ZFS, ZFS Solaris OS ZFS 315 ZFS 317 ZFS ZFS ZFS Solaris ZFS. zpool upgrade zfs upgrade Solaris.. zpool upgrade zfs upgrade 195 ZFS 103 ZFS. ZFS Oracle Solaris ZFS. Oracle Solaris 11 1 snv_36 ZFS 2 snv_38 Ditto ( ) 315

316 ZFS Oracle Solaris 11 3 snv_42 RAID-Z 4 snv_62 zpool history 5 snv_62 gzip 6 snv_62 bootfs 7 snv_68 8 snv_69 9 snv_77 refquota refreservation 10 snv_78 11 snv_94 12 snv_96 13 snv_98 snapused 14 snv_103 aclinherit passthrough-x 15 snv_ snv_116 stmf 17 snv_120 RAID-Z 18 snv_ snv_ snv_128 zle(0 ) 21 snv_ snv_ snv_135 Slim ZIL 24 snv_ snv_ snv_ snv_ snv_147 vdev 29 snv_148 RAID-Z/ 30 snv_ Oracle Solaris 11.1 : ZFS

317 ZFS Oracle Solaris snv_150 'zfs list' 32 snv_151 1MB 33 snv_ S11.1 ZFS Oracle Solaris ZFS.. Oracle Solaris 11 1 snv_36 ZFS 2 snv_69 3 snv_77 (FUID) 4 snv_114 userquota groupquota 5 snv_137 6 S11.1 A Oracle Solaris ZFS 317

318 318

319 A ACL ACL, 223 ACL, 222 ACL, 222 aclinherit, 223 POSIX ACL, 218 ZFS ACL, 225 ZFS ACL (Compact ) ( ), 237 ZFS ACL (Verbose ) ( ), 231 ZFS ACL (Compact ), 236 ZFS ACL (Verbose ), 226 ZFS ACL ( ) ( ), 227 ZFS, 224 ZFS ACL, 225 ZFS ACL (Verbose ) ( ), 230, 217, 220, 220, 218 ACL aclinherit, 137 ACL ( ) aclmode, 138 ACL, Solaris, ZFS, 34 aclinherit, 223 allocated,, 81 altroot,, 81 atime,, 138 autoreplace,, 81 available,, 138 B bootfs,, 81 C cachefile,, 81 canmount, 138, 152 capacity,, 81 casesensitivity,, 139 checksum,, 139 compression,, 139 compressratio,, 139 copies,, 140 creation,,

320 D dedup,, 140 dedupditto,, 81 dedupratio,, 81 delegation,, 246 delegation,, 82 devices,, 140 dry run ZFS (zpool create -n) ( ), 58 dumpadm,, 123 E EFI ZFS, 44, 44 exec,, 140 M mlslabel,, 141 mounted,, 141 mountpoint,, 141 N NFSv4 ACL ACL, 223 ACL, 222 ACL, 222 POSIX ACL, 218, 217, 218 O origin,, 143 F failmode,, 82 free,, 82 P POSIX ACL,, 218 primarycache,, 142 G guid,, 82 Q quota,, 143 H health,, 82 L listshares,, 82 listsnapshots,, 82 logbias,, 141 R RAID-Z,, 30 RAID-Z ( ), 52, 48,, 48,, 48, 48 RAID-Z,,( ), 62 read-only,, Oracle Solaris 11.1 : ZFS

321 recordsize, 144, 155 referenced,, 144 refquota,, 144 refreservation,, 144 reservation,, 145 V version,, 148 version,, 83 volblocksize,, 148 volsize, 148, 156 S savecore,, 123 secondarycache,, 145 setuid,, 145 shadow,, 145 share.nfs,, 146 share.smb, description, 146 share.smb,,, 156 size,, 83 snapdir,, 146 Solaris ACL ACL, 223 ACL, 222 ACL, 222 POSIX ACL, 218, 217, 218 sync,, 147 T type,, 147 U used, 147, 150 usedbychildren,, 147 usedbydataset,, 148 usedbyrefreservation,, 148 usedbysnapshots,, 148 X xattr,, 149 Z zfs allow, 249, 254 zfs create ( ), 39 ( ), 134, 134 zfs destroy, ( ), 135 zfs destroy -r, ( ), 135 zfs get, ( ), 162 zfs get -H -o, ( ), 164 zfs get -s, ( ), 163 zfs inherit, ( ), 161 zfs list ( ), 40 ( ), 157 zfs list -H, ( ), 159 zfs list -r, ( ), 158 zfs list -t, ( ), 159 zfs mount, ( ), 167 zfs promote, ( ), 205 zfs receive, ( ), 210 zfs rename, ( ), 136 zfs send, ( ), 209 zfs set atime,( ), 160 zfs set compression, ( ), 40 zfs set mountpoint ( ), 40 ( ),

322 zfs set mountpoint=legacy, ( ), 166 zfs set quota ( ), 40 zfs set quota,( ), 161 zfs set quota, 180 zfs set reservation, ( ), 183 zfs set share.nfs, ( ), 40 zfs unallow,, 249 zfs unmount, ( ), 169 zfs upgrade, 195 ZFS, ZFS, 32 ZFS,, 30 ZFS aclinherit, 137 aclmode, 138 atime, 138 available, 138 canmount, 138, 152 casesensitivity, 139 checksum, 139 compression, 139 compressratio, 139 copies, 140 creation, 140 dedup, 140 devices, 140 exec, 140 logbias, 141 mlslabel, 141 mounted, 141 mountpoint, 141 origin, 143 quota, 143 read-only, 144 recordsize, 144, 155 referenced, 144 refquota, 144 refreservation, 144 reservation, 145 secondarycache, 142, 145 ZFS ( ) setuid, 145 shadow, 145 share.nfs, 146 share.smb, 146 snapdir, 146 sync, 147 type, 147 used, 147, 150 usedbychildren, 147 usedbydataset, 148 usedbyrefreservation, 148 usedbysnapshots, 148 user, 156 version, 148 volblocksize, 148 volsize, 148, 156 xattr, 149 zoned, 149 zoned, 264,, 137, 137, 137, 150, 263, 149 ZFS ZFS Solaris OS, 315 ZFS,, 257 ZFS,, 245 ZFS dry run (zpool create -n) ( ), 58 RAID-Z, 30 RAID-Z,, 48 RAID-Z (zpool create) ( ), Oracle Solaris 11.1 : ZFS

323 ZFS ( ) vdev I/O ( ), 89 ZFS (zpool online) ( ), 281, 56, 30 ( ), 99 (zpool import -a) ( ), 96, 90 ( ), 91, 284, 43, 35, 59 ( ), 84 ( ), 96 (UNAVAIL), 269 ( ), 278 (zpool import -d) ( ), 98, 265, 271, 271 (zpool status -v) ( ), 292 ( ), 271, 271, 272, 49, 30 ZFS ( ) ( ), 290, 95 (zpool create) ( ), 50, 273, 275 (zpool status -x), 274, 29 (zpool create) ( ), 50,, 47 (zpool split) ( ), 67, 315, 296 (zpool destroy) ( ), 59 ( ), 102 ZFS, 278, 270, 270, 293, 277 (zpool status -v) ( ), 277, 103, 269 ( ),

324 ZFS ( ) (zpool replace) ( ), 73, 285 (zpool detach) ( ), 67 (zpool attach) ( ), 65, 282 (zpool clear) ( ), 283 (zpool offline) ( ), 71, 70 ( ), 72 (zpool add) ( ), 61 ( ), 86, 45, 46, 30 I/O ( ), 88, 295 ZFS (zpool online) ( ), 71 ZFS,, 95 ZFS atime ( ), 160 boot -L boot -Z ZFS BE (SPARC ), 126 quota ( ), 161 ZFS ACL, 225 ZFS ( ), 257 ZFS ( ) ZFS ACL (Verbose ) ( ), 231 ZFS ACL, 224 ZFS ACL (Compact ) ( ), 237, 236 ZFS ACL (Verbose ), 226 ZFS ACL ( ) ( ), 227 ZFS ACL, 225 ZFS ACL (Verbose ) ( ), 230, 28, 30, 35 ( ), 134 ( ), 157, 29, 159 (zfs receive) ( ), 210 (zfs send) ( ), 209 (zfs list) ( ), 162 (zfs inherit) ( ), 161, 165 ( ), 166, Oracle Solaris 11.1 : ZFS

325 ZFS ( ) ( ), 167, 165 (zfs set mountpoint) ( ), 166 ( ), 169, 193 ( ), 134, 315, 204, 29 ( ), 205, 205, 205, 30 ZFS ( ), 262 ZFS ( ), 261 ( ), 261 ( ), 135, 26, 133 ( ), 163, 203, 198, 199, 197, 201, 200, 30, 202, 121 ZFS, ( ), 121, 122 ( ), 164, 185, 195, 263 Solaris, 260 ( ), 183 ( ), 159 ( ), 136, 165, 206 ( ), 158 ( ), 135, 29, 28, 27, 29, 26 ( ), 159 ZFS (zfs set quota), 180 ZFS, share.smb, 156 ZFS, ZFS,

326 ZFS, 192, 195, 193 ZFS, 185, 185, 190, 187 ZFS allocated, 81 alroot, 81 autoreplace, 81 bootfs, 81 cachefile, 81 capacity, 81 dedupditto, 81 dedupratio, 81 delegation, 82 failmode, 82 free, 82 guid, 82 health, 82 listsharess, 82 listsnapshots, 82 size, 83 version, 83 ZFS ZFS, 32 ZFS, 33, 33, 34 Solaris ACL, 34, 31 ZFS, RAID-Z, 47 ZFS ( ), 156, 156 ZFS aclinherit, 137 aclmode, 138 atime, 138 canmount, 138, 152 casesensitivity, 139 ZFS ( ) checksum, 139 compression, 139 copies, 140 dedup, 140 devices, 140 exec, 140 mountpoint, 141 primarycache, 142 quota, 143 read-only, 144 recordsize, 144, 155 refquota, 144 refreservation, 144 reservation, 145 secondarycache, 145 setuid, 145 shadow, 145 share,nfs, 146 share.smb, 146 snapdir, 146 sync, 147 used, 150 version, 148 volblocksize, 148 volsize, 148, 156 xattr, 149 zoned, 149, 150 ZFS available, 138 compression, 139 creation, 140 mounted, 141 origin, 143 referenced, 144 type, 147 used, 147 usedbychildren, 147 usedbydataset, 148 usedbyrefreservation, Oracle Solaris 11.1 : ZFS

327 ZFS ( ) usedbysnapshots, 148, 149 ZIL(ZFS ),, 53 zoned, 149, 264 zpool add, ( ), 61 zpool attach, ( ), 65 zpool clear ( ), 72, 72 zpool create ( ), 36, 38 RAID-Z ( ), 52 ( ), 50 ( ), 50 zpool create -n, dry run( ), 58 zpool destroy, ( ), 59 zpool detach, ( ), 67 zpool export, ( ), 96 zpool import -a, ( ), 96 zpool import -D, ( ), 102 zpool import -d, ( ), 98 zpool import,( ), 99 zpool iostat, ( ), 88 zpool iostat -v, vdev( ), 89 zpool list ( ), 38, 84, 83 zpool list -Ho name, ( ), 86 zpool offline, ( ), 71 zpool online, ( ), 71 zpool replace, ( ), 73 zpool split, ( ), 67 zpool status -v, ( ), 92 zpool status -x, ( ), 91 zpool upgrade, 103 ZFS, 56, 30 ZFS ( ), 99 ZFS (zpool import -d) ( ), 98 ( ), 266,, 28 ( ), 57 ( ), 58,( ), 250, ZFS, 271, ZFS,

328 ( ), 278 (zpool replace) ( ), 73, 285, 290, ZFS, 43 ( ) ZFS ( ), 159 ZFS ( ), 158 ZFS, 38 ZFS ( ), 163 ZFS ( ), 164 ZFS ( ), 159,, 245, zfs allow, 249, zfs unallow, 249, ZFS, 35 ZFS ( ), 96,( ), 250, ZFS, 34 ZFS (zfs list) ( ), 162 ZFS ( ), 84, 83 ZFS ( ), 157 ZFS (zfs list) ( ), 40 ( ), 266 ( ), 266, 265 ( ), 271, 272, 271, 270 ( ), 271 (zpool status -v) ( ), 277, Oracle Solaris 11.1 : ZFS

329 ( ), 29,, 159,, 49 ZFS ( ), 169, 49, 49, ZFS, 45 ZFS ( ), 203,, 30,, 272 ZFS ( ), 167 ZFS, 165 ZFS, 59 ZFS, 134, 165, 165 ZFS ( ), 205 ZFS ( ), 257 ZFS ( ), 198 ZFS, 50 ZFS (zpool create) ( ), 36, 50 ZFS, 39 ( ), 134, 134 ZFS, 38 ZFS (zpool create) ( ), 36 RAID-Z (zpool create) ( ), 52 ( ), 266 ZFS ( ), 53 ZFS (zpool create) ( ), 50 (zpool split) ( ), 67 RAID-Z (zpool create) ( ), 52 RAID-Z (zpool create) ( ), 52 ZFS ( ), 54, ZFS,

330 FS (zpool online) ( ), 281 ZFS, 269 ZFS syslog, 277 ZFS, 195, 284 (UNAVAIL), 269 ( ), 278 (zpool status -v) ( ), 292, 273 (zpool status -x), 274, 296 ZFS, 278, 270, 293 (zpool status -v) ( ), 277 (zpool replace) ( ), 285, 290, 282 (zpool clear) ( ), 283, 275, 295,,( ), 63 (zpool split) ( ), 67,, 53, 296 ZFS ( ), 102 ZFS, 278, 293, 295 ZFS ACL(Verbose ) ( ), 230, 208, 204 ( ), 205 ( ), 205, 29,, 29 (zpool create), ( ), 50, 47, 47, 47, ZFS ( ), 53,, 30 boot -L boot -Z SPARC, Oracle Solaris 11.1 : ZFS

331 ( ), 124, bootadm, 125 bootadm ( ), 125 ZFS (zpool detach) ( ), 67 ( ), 57 ZFS ( ), 205 ZFS ( ), 199 ZFS, 50 ZFS (zpool destroy) ( ), 59 ZFS ( ), 135 ZFS ( ), 135 ZFS (zfs inherit), 161,, 192 compression ( ), 40 mountpoint, 40 quota ( ), 40 share.nfs ( ), 40 ZFS atime ( ), 160 ZFS (zfs set mountpoint) ( ), 166 ZFS ( ), 161 ZFS ( ), 183 ZFS (zfs set quota), 180 ZFS ACL, 224 ZFS ACL(Compact ) ( ), 237, 236 ZFS ACL(Verbose ) (, 226 ZFS ACL (Verbose ) ( ), 231 ( ), 166 ZFS (zfs receive) ( ), 210 ZFS ACL( ) ( ),

332 , 208, 202, 197 ( ), 203 ( ), 198 ( ), 199 ( ), 201 ( ), 200, 30, 121, 121, 122 ( ), 271, 271 ZFS ( ), 86, 208, 208 ZFS (zpool import -a) ( ), 96 (zpool status -v) ( ), 292, 37 FS (zpool online) ( ), 281 ZFS ( ), 201 ZFS, 103 ZFS, 195 ZFS (zpool attach) ( ), 65 ZFS, 260 zoned, 264 ZFS ( ), 262 ZFS ( ), 261 ( ), 261 ZFS, Oracle Solaris 11.1 : ZFS

333 , 269 (UNAVAIL), 269, 270, 270 RAID-Z, 30, 30, 29, 30, 29, 29, 30, 30, 29, 29, 30 ( ), 250 ( ), 261,, 245 ZFS ( ), 200 ZFS ( ), 136 ( ), 58 (zpool offline) ZFS ( ), 71 ZFS, 70 ZFS (zpool online) ( ), 71 ZFS ( ), 72 ZFS (zfs send) ( ), 209 savecore, 123,, 37 ZFS, 206, ZFS, 45, ( ), 64, ( ),

334 ,, 122,, 179 ZFS (zpool clear), 72 (zpool clear) ( ), 283,, 29,, 28,, 27, ZFS, 46,, 29,, 38, ZFS, 31 RAID-Z ( ), 62 ZFS (zpool add) ( ), 61 ( ), 63 ZFS ( ), 262 ZFS ( ), 261 ( ), 64,, 123 ZFS ( ), 54, 54,,( ), 64,,( ), 64 ZFS syslog, 277 ZFS I/O, 87 ZFS vdev I/O ( ), 89 ZFS ( ), 91 ZFS I/O ( ), 88 ( ), 254 ZFS ( ), 92, 90,, 30,, Oracle Solaris 11.1 : ZFS

335 , 36 ( ), 75 ( ), 75, 284,

336 336