Copyright 2006, 2012, Oracle and/or its affiliates. All rights reserved.,.,,,,,,,,,,,,.,...,. U.S. GOVERNMENT END USERS: Oracle programs, including an

Transcription

1 Oracle Solaris 11.1 : ZFS : E

2 Copyright 2006, 2012, Oracle and/or its affiliates. All rights reserved.,.,,,,,,,,,,,,.,...,. U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government..., -,,. Oracle Corporation. Oracle Java Oracle Corporation /.. Intel Intel Xeon Intel Corporation. SPARC SPARC International, Inc.. AMD, Opteron, AMD, AMD Opteron Advanced Micro Devices. UNIX The Open Group. ( ) 3,. Oracle Corporation 3,. Oracle Corporation 3,, @25097

3 Oracle Solaris ZFS ( ) ZFS ZFS ZFS var EFI(GPT) ZFS ZFS ZFS (zfs.1m) aclmode ZFS ZFS ZFS ZFS (zfs diff)...21 ZFS ZFS ZFS ZFS ACL ZFS (zpool split)...25 ZFS iscsi ZFS ZFS Oracle Solaris ZFS? ZFS

4 ZFS ZFS ZFS Oracle Solaris ZFS ZFS ZFS ZFS NFSv4 Solaris ACL Oracle Solaris ZFS...35 ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS Oracle Solaris ZFS ZFS ZFS ZFS ZFS ZFS ZFS RAID-Z ZFS Oracle Solaris 11.1 : ZFS

5 ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS I/O ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ( ) ZFS ZFS ZFS ZFS BE

6 (SPARC x86/vtoc) (x86/efi(gpt)) ZFS (SPARC x86/vtoc) ZFS (x86/efi(gpt)) BE (SPARC x86/vtoc) BE (x86/efi(gpt)) ZFS ZFS ZFS ZFS ZFS SPARC ZFS x86 ZFS ZFS Oracle Solaris ZFS ZFS ( ) ZFS, ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS Oracle Solaris 11.1 : ZFS

7 ZFS ZFS ZFS ZFS ZFS / ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS, ZFS ZFS ZFS ZFS ZFS Oracle Solaris ZFS ZFS ZFS ZFS ZFS ZFS (zfs diff) ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS

8 ZFS ZFS ZFS ZFS ACL Oracle Solaris ZFS Solaris ACL ACL ACL ACL ZFS ACL Verbose ZFS ACL Verbose ZFS ACL Compact ZFS ACL ZFS Oracle Solaris ZFS ZFS ZFS ZFS ZFS (zfs allow) ZFS (zfs unallow) ZFS ( ) ZFS ( ) ZFS ( ) Oracle Solaris ZFS ZFS ZFS ZFS iscsi LUN Solaris ZFS ZFS ZFS Oracle Solaris 11.1 : ZFS

9 ZFS ZFS zoned ZFS ZFS Oracle Solaris ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS ZFS zpool status ZFS ZFS ZFS ZFS

10 ZFS ZFS ZFS ZFS ZFS Oracle Solaris ZFS ZFS ZFS ZFS A Oracle Solaris ZFS ZFS ZFS ZFS Oracle Solaris 11.1 : ZFS

11 Oracle Solaris 11.1 ZFS Oracle Solaris ZFS. SPARC x86. Oracle Solaris SPARC x86. hcl/index.html Oracle Solaris.. Oracle Solaris ZFS. Oracle Solaris OS( ) UNIX.. 1, Oracle Solaris ZFS ( ) 2, Oracle Solaris ZFS 3, Oracle Solaris ZFS 4, ZFS ZFS.. ZFS. ZFS. ZFS., ZFS, ZFS. 11

12 5, Oracle Solaris ZFS 6, Oracle Solaris ZFS 7, ACL Oracle Solaris ZFS 8, Oracle Solaris ZFS 9, Oracle Solaris ZFS 10, Oracle Solaris ZFS 11, 12, Oracle Solaris ZFS A, Oracle Solaris ZFS ZFS.,,. ZFS. (ACL) UNIX ZFS. ZFS ZFS. ZFS, Oracle Solaris ZFS,. ZFS... ZFS. ZFS,, ZFS Solaris OS. Oracle Solaris. Oracle Solaris 11.1, Oracle Solaris 11.1 Oracle Solaris 11.1 : Oracle Solaris 11.1 : Oracle Support Oracle My Oracle Support Oracle Solaris 11.1 : ZFS

13 . P 1 AaBbCc123 AaBbCc123 AaBbCc123 AaBbCc123, ;.. :.,,..login. ls -a. machine_name% you have mail. machine_name% su Password: rm filename :. Oracle Solaris OS UNIX.. P 2 Bash, Korn Bourne $ Bash, Korn Bourne # C C machine_name% machine_name# 13

14 14

15 1 Oracle Solaris ZFS ( ) ZFS Oracle Solaris ZFS ZFS 26 Oracle Solaris ZFS? 29 ZFS 30 ZFS 31 Oracle Solaris ZFS ZFS. 16 ZFS 16 ZFS 17 var 17 EFI(GPT) 18 ZFS 18 ZFS 19 ZFS (zfs.1m) 19 aclmode ZFS 21 ZFS 21 ZFS 21 ZFS (zfs diff) 22 ZFS 22 ZFS 15

16 ZFS 23 ZFS 24 ZFS ACL 25 ZFS (zpool split) 25 ZFS iscsi 25 ZFS 26 ZFS ZFS Oracle Solaris 11.1: zpool status. zpool status UNAVAIL (c0t5000c500335f907fd0). # zpool status -v pond pool: pond state: DEGRADED status: One or more devices are unavailable in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Determine if the device needs to be replaced, and clear the errors using zpool clear or fmadm repaired, or replace the device with zpool replace. scan: scrub repaired 0 in 0h0m with 0 errors on Wed Jun 20 15:38: config: device details: NAME STATE READ WRITE CKSUM pond DEGRADED mirror-0 DEGRADED c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 UNAVAIL mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE c0t5000c500335f907fd0 UNAVAIL cannot open status: ZFS detected errors on this device. The device was missing. see: for recovery errors: No known data errors ZFS Oracle Solaris 11.1: ZFS.. share.nfs share.smb. 16 Oracle Solaris 11.1 : ZFS

17 ZFS ZFS. var Oracle Solaris 11.1: Oracle Solaris 11.1 /var/share rpool/varshare. /var BE.. # ls /var/share audit cores crash mail /var /var/share. /var. Oracle Solaris 11 Oracle Solaris 11.1 /var /var/share. EFI(GPT) Oracle Solaris 11.1: x86 ZFS EFI(GPT).. # zpool status rpool pool: rpool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE c2t0d0 ONLINE errors: No known data errors SPARC. x86 EFI(GPT). zpool create -B EFI(GPT) zpool BE (SPARC x86/vtoc). 1 Oracle Solaris ZFS ( ) 17

18 ZFS EFI(GPT) zpool replace. # bootadm install-bootloader Oracle Solaris x86 2TiB. ZFS Oracle Solaris 11: zfs zpool zfs zpool help.. # zfs help The following commands are supported: allow clone create destroy diff get groupspace help hold holds inherit list mount promote receive release rename rollback send set share snapshot unallow unmount unshare upgrade userspace For more info, run: zfs help <command> # zfs help create usage: create [-p] [-o property=value]... <filesystem> create [-ps] [-b blocksize] [-o property=value]... -V <size> <volume> # zpool help The following commands are supported: add attach clear create destroy detach export get help history import iostat list offline online remove replace scrub set split status upgrade For more info, run: zpool help <command> # zpool help attach usage: attach [-f] <pool> <device> <new-device> zfs(1m) zpool(1m). ZFS Oracle Solaris 11: ZFS. zfs snapshot snap.. # zfs snap -r users/home@snap1 zfs diff -e Oracle Solaris 11.1 : ZFS

19 ZFS # zfs diff -e tank/cindy@yesterday tank/cindy@now + /tank/cindy/ + /tank/cindy/file.1 -o.. # zfs diff -e -o size -o name tank/cindy@yesterday tank/cindy@now + 7 /tank/cindy/ /tank/cindy/file.1 ZFS 6, Oracle Solaris ZFS. ZFS (zfs.1m) Oracle Solaris 11: ZFS zfs.1m zfs.1m,,,. zfs_allow(1m) zfs_encrypt(1m) zfs_share(1m) aclmode Oracle Solaris 11: aclmode chmod ACL ACL( ). aclmode. discard aclmode discard ACL.. mask aclmode mask. UID. ACL. ACL ACL. passthrough aclmode passthrough ACL ACL Oracle Solaris ZFS ( ) 19

20 ZFS Oracle Solaris 11: Solaris zpool status -l /dev/chassis.,. fmadm add-alias.. # fmadm add-alias SUN-Storage-J QAJ001 SUN-Storage-J4400.rack22. % zpool status -l export pool: export state: ONLINE scan: resilvered 492G in 8h22m with 0 errors on Wed Aug 1 17:22: config: NAME STATE READ WRITE CKSUM export ONLINE mirror-0 ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 2/disk ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 3/disk ONLINE mirror-1 ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 4/disk ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 5/disk ONLINE mirror-2 ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 6/disk ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 7/disk ONLINE mirror-3 ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 8/disk ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 9/disk ONLINE mirror-4 ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 10/disk ONLINE /dev/chassis/sun-storage-j4400.rack22/scsi_device 11/disk ONLINE spares /dev/chassis/sun-storage-j4400.rack22/scsi_device 0/disk AVAIL /dev/chassis/sun-storage-j4400.rack22/scsi_device 1/disk AVAIL errors: No known data errors zpool iostat. diskinfo, format prtconf. diskinfo(1m). ZFS Oracle Solaris 11:. 20 Oracle Solaris 11.1 : ZFS

21 ZFS ZFS shadow. shadow. file:///path nfs://host:path 192 ZFS. ZFS Oracle Solaris 11: ZFS., tank/cindy encryption.. 8. # zfs create -o encryption=on tank/cindy Enter passphrase for tank/cindy : xxx Enter again: xxx ZFS ZFS. ZFS Oracle Solaris 11:. mountpoint. 211 ZFS. ZFS (zfs diff) Oracle Solaris 11: zfs diff ZFS.,. $ ls /tank/cindy filea $ zfs snapshot tank/cindy@ Oracle Solaris ZFS ( ) 21

22 ZFS $ ls /tank/cindy filea fileb $ zfs snapshot tank/cindy@0914,. $ zfs diff tank/cindy@0913 tank/cindy@0914 M /tank/cindy/ + /tank/cindy/fileb M. + fileb. 203 ZFS (zfs diff). ZFS Oracle Solaris 11: ZFS. zpool import -m RAID-Z(raidz1, raidz2 raidz3) I/O. 29 RAID-Z. RAID-Z. RAID-Z.. ZFS Oracle Solaris 11: sync ZFS.... sync Oracle Solaris 11.1 : ZFS

23 ZFS # zfs set sync=always tank/neil zil_disable sync Oracle Solaris ZFS Oracle Solaris 11: -T zpool list zpool status. zpool status... scan: resilver in progress since Thu Jun 7 14:41: G scanned out of 73.3G at 106M/s, 0h11m to go 3.80G resilvered, 5.22% done.. scan: scrub in progress since Thu Jun 7 14:59: G scanned out of 73.3G at 118M/s, 0h10m to go 0 repaired, 2.66% done.. resilvered 73.3G in 0h13m with 0 errors on Thu Jun 7 14:54: scan: scrub repaired 512B in 1h2m with 0 errors on Thu Jun 7 15:10: scan: scrub canceled on Thu Jun 7 15:19:20 MDT T d -T u. # zpool status -T d tank 3 2 Thu Jun 14 14:08:21 MDT 2012 pool: tank state: DEGRADED status: One or more devices is currently being resilvered. The pool will continue to function in a degraded state. action: Wait for the resilver to complete. Run zpool status -v to see device specific details. scan: resilver in progress since Thu Jun 14 14:08: Oracle Solaris ZFS ( ) 23

24 ZFS 2.96G scanned out of 4.19G at 189M/s, 0h0m to go 1.48G resilvered, 70.60% done config: NAME STATE READ WRITE CKSUM tank DEGRADED mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 DEGRADED c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 DEGRADED (resilvering) errors: No known data errors ZFS ACL Oracle Solaris 11: ACL. ACL deny ACE( )., 0644, deny ACE 0705, 0060 deny ACE. ACL( : 644) deny ACE.. # ls -v file.1 -rw-r--r-- 1 root root Jun 14 11:52 file.1 0:owner@:execute:deny 1:owner@:read_data/write_data/append_data/write_xattr/write_attributes /write_acl/write_owner:allow 2:group@:write_data/append_data/execute:deny 3:group@:read_data:allow 4:everyone@:write_data/append_data/write_xattr/execute/write_attributes /write_acl/write_owner:deny 5:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow ACL( : 644) deny ACE.. # ls -v file.1 -rw-r--r-- 1 root root Jun 22 14:30 file.1 0:owner@:read_data/write_data/append_data/read_xattr/write_xattr /read_attributes/write_attributes/read_acl/write_acl/write_owner /synchronize:allow 1:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow 2:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize :allow ACL ACE.,. aclinherit restricted.,acl ACE. 24 Oracle Solaris 11.1 : ZFS

25 ZFS ACL chmod(2). ZFS aclmode.,acl user ACE. group ACE. 7, ACL Oracle Solaris ZFS. ZFS (zpool split) Oracle Solaris 11: zpool split.. 67 ZFS. ZFS iscsi Oracle Solaris 11: iscsi COMSTAR(Common Multiprotocol SCSI Target). ZFS iscsi LUN shareiscsi. ZFS iscsi LUN stmfadm. 259 ZFS iscsi LUN. ZFS Oracle Solaris 11: ZFS zpool-poolname. I/O I/O. CPU. ps prstat.. SDC(7). 1 Oracle Solaris ZFS ( ) 25

26 Oracle Solaris ZFS? ZFS Oracle Solaris 11: (dedup) ZFS. dedup.,.. # zfs set dedup=on tank/home.,. # zpool list tank NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT tank 556G 4.19G 552G 0% 1.00x ONLINE - zpool list. 154 dedup. dedup dedup. Oracle Solaris ZFS? Oracle Solaris ZFS. ZFS,,. ZFS ZFS Oracle Solaris 11.1 : ZFS

27 Oracle Solaris ZFS? ZFS. ZFS. (, )..... DIMM.. ZFS.,..,. fsck.., fsck..,.,..,... (O_DSYNC ). ZFS... ZFS 1 Oracle Solaris ZFS ( ) 27

28 Oracle Solaris ZFS?.. ZFS. ZFS. ZFS. ZFS. 128, 256. inode (256 ). ZFS..... ZFS.,, NFS ZFS.,,... ZFS,,..,,.. 28 Oracle Solaris 11.1 : ZFS

29 ZFS ZFS. ZFS Oracle Solaris fletcher4( ) SHA256. clone. 204 ZFS.,, ZFS. ZFS.. pool. path. snapshot. 5, Oracle Solaris ZFS. filesystem ZFS. 5, Oracle Solaris ZFS Oracle Solaris ZFS ( ) 29

30 ZFS RAID-Z snapshot volume 3, Oracle Solaris ZFS.. RAID-Z 48 RAID-Z..,.. ZFS ZFS., ZFS. ZFS 257 ZFS. ZFS ZFS.. (_) (-) (:) (.), (_), (-) (.).. c[0-9]. log. 30 Oracle Solaris 11.1 : ZFS

31 Oracle Solaris ZFS mirror, raidz, raidz1, raidz2, raidz3 spare. (%).. (%).. Oracle Solaris ZFS 31 ZFS 31 ZFS 33 ZFS NFSv4 Solaris ACL ZFS,... ZFS. ZFS. ( : /export/home).. 38 ZFS. ZFS ZFS., ZFS. 1 Oracle Solaris ZFS ( ) 31

32 Oracle Solaris ZFS ( : df) ZFS ZFS. ( : /home) NFS. ZFS,.,. ZFS. ZFS df -g. total files. ZFS...,.. fsync(3c) O_SYNC. UFS du. ZFS du.. " ". ZFS UFS. df zfs list df. df., ZFS df.. recordsize 1/2. recordsize 128KB 64KB. RFE Oracle Solaris 11.1 : ZFS

33 Oracle Solaris ZFS RAIDZ-2 2 (512 ).. recordsize 512, KB( 3 ). recordsize (128KB) ( lzjb). df. ZFS. ZFS. ZFS 6, Oracle Solaris ZFS......, ENOSPC EDQUOT. ZFS ZFS., /etc/vfstab. ZFS. /etc/vfstab ZFS. ZFS 165 ZFS. 26 ZFS ZFS Volume Manager. ZFS. ZFS 1 Oracle Solaris ZFS ( ) 33

34 Oracle Solaris ZFS.. NFSv4 Solaris ACL Solaris OS POSIX ACL ACL. POSIX ACL UFS. NFSv4 Solaris ACL ZFS. Solaris ACL. NFSv4 NT ACL.. ACL setfacl getfacl chmod ls.. ZFS ACL 7, ACL Oracle Solaris ZFS. 34 Oracle Solaris 11.1 : ZFS

35 2 2 Oracle Solaris ZFS Oracle Solaris ZFS. ZFS ZFS 36 ZFS 36 ZFS 37 ZFS 38 ZFS ZFS ( ) ZFS ZFS. ZFS ZFS,. ZFS ZFS,. Oracle Solaris 11.1 :. ZFS RBAC, ZFS ZFS. 8, Oracle Solaris ZFS. 35

36 ZFS ZFS ZFS. Oracle Solaris SPARC x86. 64MB. 128MB. ZFS.. ZFS ZFS.., ZFS. tank tank ZFS. /dev/dsk/c1t0d0 /dev/dsk/c2t0d0. # zpool create tank mirror c1t0d0 c2t0d0 ZFS 47 ZFS. ZFS tank /tank. # mkfile 100m /tank/foo # df -h /tank Filesystem size used avail capacity Mounted on tank 80G 100M 80G 1% /tank.. tank fs. # zfs create tank/fs ZFS tank/fs /tank/fs. 36 Oracle Solaris 11.1 : ZFS

37 ZFS # mkfile 100m /tank/fs/foo # df -h /tank/fs Filesystem size used avail capacity Mounted on tank/fs 80G 100M 80G 1% /tank/fs. ZFS 38 ZFS. ZFS ZFS ZFS.. 128MB. ZFS. 38 ZFS /dev/dsk/c1t0d0 /dev/dsk/c2t0d0. 43 ZFS.. ZFS. ZFS ( ) RAID-Z(RAID-5 ). 38 ZFS. ZFS 47 ZFS. 2 Oracle Solaris ZFS 37

38 ZFS ZFS ZFS. ZFS 35 ZFS.. zpool zfs.., 30 ZFS.., tank. # zpool create tank mirror c1t0d0 c2t0d0. 50 ZFS zpool list. # zpool list NAME SIZE ALLOC FREE CAP HEALTH ALTROOT tank 80G 137K 80G 0% ONLINE - 83 ZFS. ZFS.,. ZFS.. ZFS. 38 Oracle Solaris 11.1 : ZFS

39 ZFS ZFS. ZFS,..,. 39 ZFS ZFS jeff bill. 5, Oracle Solaris ZFS.. ZFS ZFS home...,,,. 39 ZFS /export/zfs/ user NFS. jeff 10GB. 137 ZFS. 1 2 ZFS ZFS. ZFS 35 ZFS... # zfs create tank/home 2 Oracle Solaris ZFS 39

40 ZFS 3.. # zfs set mountpoint=/export/zfs tank/home # zfs set share.nfs=on tank/home # zfs set compression=on tank/home # zfs get compression tank/home NAME PROPERTY VALUE SOURCE tank/home compression on local.. # zfs create -o mountpoint=/export/zfs -o share.nfs=on -o compression=on tank/home 137 ZFS. tank home. 4. home.. # zfs create tank/home/jeff # zfs create tank/home/bill /export/zfs/ user NFS. /etc/vfstab /etc/dfs/dfstab. 134 ZFS. 165 ZFS jeff 10GB.. # zfs set quota=10g tank/home/jeff. zfs list. # zfs list NAME USED AVAIL REFER MOUNTPOINT tank 92.0K 67.0G 9.5K /tank tank/home 24.0K 67.0G 8K /export/zfs tank/home/bill 8K 67.0G 8K /export/zfs/bill tank/home/jeff 8K 10.0G 8K /export/zfs/jeff jeff 10GB, bill (67GB). 40 Oracle Solaris 11.1 : ZFS

41 ZFS 157 ZFS. 31 ZFS. 2 Oracle Solaris ZFS 41

42 42

43 3 3 Oracle Solaris ZFS Oracle Solaris ZFS.. 43 ZFS 47 ZFS 50 ZFS 60 ZFS 80 ZFS 83 ZFS 95 ZFS 103 ZFS ZFS. 43 ZFS 45 ZFS 46 ZFS ZFS. 128MB. /dev/dsk. (c1t0d0) (c0t0d0s7)., 43

44 ZFS. ZFS EFI. format. Current partition table (original): Total disk sectors available: (reserved sectors) Part Tag Flag First Sector Size Last Sector 0 usr wm GB unassigned wm unassigned wm unassigned wm unassigned wm unassigned wm unassigned wm reserved wm MB Oracle Solaris 11.1 EFI(GPT) x86. Current partition table (original): Total disk sectors available: (reserved sectors) Part Tag Flag First Sector Size Last Sector 0 BIOS_boot wm MB usr wm GB unassigned wm unassigned wm unassigned wm unassigned wm unassigned wm reserved wm MB (BIOS boot) GPT ZFS. /dev/dsk/cntndn. /dev/dsk. ZFS. x86 Solaris fdisk. Solaris fdisk Oracle Solaris 11.1 : ZFS ( ). ZFS EFI. EFI Oracle Solaris 11.1 : EFI(GPT). 44 Oracle Solaris 11.1 : ZFS

45 ZFS Oracle Solaris 11.1 x86 EFI(GPT). 108 ZFS. ( : /dev/dsk/c1t0d0) /dev/dsk ( : c1t0d0).,. c1t0d0 /dev/dsk/c1t0d0 /dev/foo/disk ZFS Solaris VTOC(SMI). format SPARC 72GB 68GB 0. # format... Specify disk (enter its number): 4 selecting c1t1d0 partition> p Current partition table (original): Total disk cylinders available: (reserved cylinders) Part Tag Flag Cylinders Size Blocks 0 root wm GB (14087/0/0) unassigned wm 0 0 (0/0/0) 0 2 backup wm GB (14087/0/0) unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0 format x86 72GB 68GB # format... selecting c1t0d0 3 Oracle Solaris ZFS 45

46 ZFS partition> p Current partition table (original): Total disk cylinders available: (reserved cylinders) Part Tag Flag Cylinders Size Blocks 0 root wm GB (49778/0/0) unassigned wu 0 0 (0/0/0) 0 2 backup wm GB (49779/0/0) unassigned wu 0 0 (0/0/0) 0 4 unassigned wu 0 0 (0/0/0) 0 5 unassigned wu 0 0 (0/0/0) 0 6 unassigned wu 0 0 (0/0/0) 0 7 unassigned wu 0 0 (0/0/0) 0 8 boot wu MB (1/0/0) unassigned wu 0 0 (0/0/0) 0 x86 fdisk. fdisk /dev/dsk/cn[tn]dnpn,. ZFS cn[tn]dnpn. ZFS ZFS.. UFS ZFS UFS. ZFS ZFS. ZFS. 64MB. ZFS ZFS. ZFS., RAID LUN, ZFS. ZFS. RAID LUN ZFS ZFS.,. 46 Oracle Solaris 11.1 : ZFS

47 ZFS ZFS.. ZFS. 12, Oracle Solaris ZFS. ID ( ). ID ZFS. ID. ZFS ID. ZFS ZFS RAID-Z RAID-Z ZFS.... mirror c1t0d0 c2t0d0. mirror c1t0d0 c2t0d0 c3t0d0 mirror c4t0d0 c5t0d0 c6t0d Oracle Solaris ZFS 47

48 ZFS RAID-Z ZFS, RAID-Z. RAID-Z(raidz raidz1) RAID-5. RAID-Z(raidz2) RAID-6. RAIDZ-3(raidz3). RAID-5 ( : RAID-4, RAID-6, RDP EVEN-ODD) RAID-5. RAID-5 ( ). RAID-Z ZFS RAID. RAID ZFS. RAID-Z RAID-5. X N P RAID-Z (N-P)*X P. RAID-Z 2, RAID-Z 3., RAID-Z 3 3. RAID-Z. 3 RAID-Z. raidz c1t0d0 c2t0d0 c3t0d0 RAID-Z. raidz c1t0d0 c2t0d0 c3t0d0 c4t0d0 c5t0d0 c6t0d0 c7t0d0 raidz c8t0d0 c9t0d0 c10t0d0 c11t0d0c12t0d0 c13t0d0 c14t0d0 RAID-Z.,14 RAID-Z 7. RAID-Z. RAID-Z 52 RAID-Z. RAID-Z. 48 Oracle Solaris 11.1 : ZFS

49 ZFS RAID-Z 12, Oracle Solaris ZFS. ZFS Oracle Sun Storage 7000 ZFS DRAM, SSD HDD. ZFS. Sun Storage Unified Storage System Administration Guide. ZFS RAID-Z. ZFS. ZFS.. ZFS. RAID-Z.., RAID-Z 2 ZFS., RAID-Z. RAID-Z.. 3 Oracle Solaris ZFS 49

50 ZFS ZFS ZFS. 50 ZFS ZFS 59 ZFS.. ZFS.. zpool destroy.. ZFS zpool create.. 30 ZFS. c1t0d0 c1t1d0 tank. # zpool create tank c1t0d0 c1t1d0 /dev/dsk, ZFS.. mirror. mirror. 2. # zpool create tank mirror c1d0 c2d0 mirror c3d0 c4d0 mirror.,. 12, Oracle Solaris ZFS. ZFS. 50 Oracle Solaris 11.1 : ZFS

51 ZFS (vdev) ( ). 73. ( ) ZFS.,. ZFS. Oracle Solaris 11.1 x86 EFI(GPT), SPARC SMI(VTOC). Oracle Solaris 11.1 EFI(GPT). ZFS EFI(GPT). # zpool create -B rpool2 c1t0d0. zpool add, zpool attach. RAID-Z... ERROR: ZFS pool <pool-name> does not support boot environments # zpool add -f rpool log c0t6d0s0 cannot add to rpool : root pool can not have multiple vdevs or separate logs ZFS 4, ZFS. 3 Oracle Solaris ZFS 51

52 ZFS RAID-Z RAID-Z raidz raidz1 mirror. 5 RAID-Z. # zpool create tank raidz c1t0d0 c2t0d0 c3t0d0 c4t0d0 /dev/dsk/c5t0d0. /dev/dsk/c5t0d0 c5t0d0. raidz2 raidz3 RAID-Z. : # zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0 c4t0d0 c5t0d0 # zpool status -v tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE raidz2-0 ONLINE c1t0d0 ONLINE c2t0d0 ONLINE c3t0d0 ONLINE c4t0d0 ONLINE c5t0d0 ONLINE errors: No known data errors # zpool create tank raidz3 c0t0d0 c1t0d0 c2t0d0 c3t0d0 c4t0d0 c5t0d0 c6t0d0 c7t0d0 c8t0d0 # zpool status -v tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE raidz3-0 ONLINE c0t0d0 ONLINE c1t0d0 ONLINE c2t0d0 ONLINE c3t0d0 ONLINE c4t0d0 ONLINE c5t0d0 ONLINE c6t0d0 ONLINE c7t0d0 ONLINE c8t0d0 ONLINE errors: No known data errors ZFS RAID-Z. 52 Oracle Solaris 11.1 : ZFS

53 ZFS RAID-Z. 61. RAID-Z ( ). 73. RAID-Z. RAID-Z RAID-Z ( ) RAID-Z. RFE. RAID-Z 48 RAID-Z. ZFS POSIX ZIL(ZFS ).,. NFS fsync(). ZIL. NVRAM. ZFS. ZFS.,.. RAID-Z..,,,,,... (64MB). in-play. ( ). 3 Oracle Solaris ZFS 53

54 ZFS in-play 1/2. 16GB 8GB. ZFS.. # zpool create datap mirror c0t5000c500335f95e3d0 c0t5000c500335f907fd0 mirror c0t5000c500335bd117d0 c0t5000c500335dc60fd0 log mirror c0t5000c500335e106bd0 c0t5000c500335fc3e7d0 # zpool status datap pool: datap state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM datap ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE logs mirror-2 ONLINE c0t5000c500335e106bd0 ONLINE c0t5000c500335fc3e7d0 ONLINE errors: No known data errors ZFS... : # zpool create tank mirror c2t0d0 c2t1d0 c2t3d0 cache c2t5d0 c2t8d0 # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c2t0d0 ONLINE c2t1d0 ONLINE Oracle Solaris 11.1 : ZFS

55 ZFS c2t3d0 ONLINE cache c2t5d0 ONLINE c2t8d0 ONLINE errors: No known data errors. 1. zpool iostat. # zpool iostat -v pool 5. ZFS.. zpool iostat RAID-Z. I/O RAID-Z.. ZFS.. OS. ( : )..... ZFS (RAIDZ ).. ZFS,... 3 Oracle Solaris ZFS 55

56 ZFS. ZFS.... ( vdev ). RAID-Z.,.. 2. # zpool create tank mirror c1d0 c2d0 mirror c3d0 c4d0 4. # zpool create mypool raidz2 c1d0 c2d0 c3d0 c4d0 zpool add.. # zpool add mypool raidz2 c2d1 c3d1 c4d1 c5d1,.. ZFS. ZFS zpool status.. # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE mirror-1 ONLINE Oracle Solaris 11.1 : ZFS

57 ZFS c0t2d0 ONLINE c1t2d0 ONLINE mirror-2 ONLINE c0t3d0 ONLINE c1t3d0 ONLINE errors: No known data errors ZFS.,. ZFS ZFS.. # zpool create tank c1t0d0 c1t1d0 invalid vdev specification use -f to override the following errors: /dev/dsk/c1t0d0s0 is currently mounted on /. Please see umount(1m). /dev/dsk/c1t0d0s1 is currently mounted on swap. Please see swap(1m). /dev/dsk/c1t1d0s0 is part of active ZFS pool zeepool. Please see zpool(1m). -f,. -f. /etc/vfstab ZFS. umount. /etc/vfstab,. /etc/vfstab.. dumpadm. ZFS. zpool destroy ( ). zpool detach.. -f. 3 Oracle Solaris ZFS 57

58 ZFS ZFS. Solaris Volume Manager.... -f. # zpool create tank c1t0d0 invalid vdev specification use -f to override the following errors: /dev/dsk/c1t0d0s0 contains a ufs filesystem. # zpool create -f tank c1t0d0 -f.. zpool.. # zpool create tank c1t0d0 mirror c2t0d0 c3t0d0 invalid vdev specification use -f to override the following errors: mismatched replication level: both disk and mirror vdevs are present # zpool create tank mirror c1t0d0 c2t0d0 mirror c3t0d0 c4t0d0 c5t0d0 invalid vdev specification use -f to override the following errors: mismatched replication level: 2-way mirror and 3-way mirror vdevs are present -f,. RAID-Z.,. -f. Dry Run,. zpool create -n. dry run.. 58 Oracle Solaris 11.1 : ZFS

59 ZFS # zpool create -n tank mirror c1t0d0 c1t1d0 would create tank with the following layout: tank mirror c1t0d0 c1t1d0.. zpool create -n,. /pool-name.... zpool create -m. : # zpool create home c1t0d0 default mountpoint /home exists and is not empty use -m option to provide a different default # zpool create -m /export/zfs home c1t0d0 /export/zfs home home. 165 ZFS. ZFS zpool destroy.. # zpool destroy tank ZFS. zpool destroy 101 ZFS. 3 Oracle Solaris ZFS 59

60 ZFS. format analyze->purge. ZFS ZFS UNAVAIL UNAVAIL ( UNAVAIL ),, -f... # zpool destroy tank cannot destroy tank : pool is faulted use -f to force destruction anyway # zpool destroy -f tank 90 ZFS. 98 ZFS. ZFS 43 ZFS ZFS Oracle Solaris 11.1 : ZFS

61 ZFS zpool add. : # zpool add zeepool mirror c2t1d0 c2t2d0 zpool create., -f. dry run -n.. # zpool add -n zeepool mirror c3t1d0 c3t2d0 would update zeepool to the following configuration: zeepool mirror c1t0d0 c1t1d0 mirror c2t1d0 c2t2d0 mirror c3t1d0 c3t2d0 c3t1d0 c3t2d0 zeepool ZFS ZFS. # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE mirror-1 ONLINE c0t2d0 ONLINE Oracle Solaris ZFS 61

62 ZFS 3 1 ZFS ( ) c1t2d0 ONLINE errors: No known data errors # zpool add tank mirror c0t3d0 c1t3d0 # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE mirror-1 ONLINE c0t2d0 ONLINE c1t2d0 ONLINE mirror-2 ONLINE c0t3d0 ONLINE c1t3d0 ONLINE errors: No known data errors 3 2 RAID-Z RAID-Z. 3 RAID-Z 3 RAID-Z. # zpool status rzpool pool: rzpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM rzpool ONLINE raidz1-0 ONLINE c1t2d0 ONLINE c1t3d0 ONLINE c1t4d0 ONLINE errors: No known data errors # zpool add rzpool raidz c2t2d0 c2t3d0 c2t4d0 # zpool status rzpool pool: rzpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM rzpool ONLINE raidz1-0 ONLINE Oracle Solaris 11.1 : ZFS

63 ZFS 3 2 RAID-Z ( ) c1t0d0 ONLINE c1t2d0 ONLINE c1t3d0 ONLINE raidz1-1 ONLINE c2t2d0 ONLINE c2t3d0 ONLINE c2t4d0 ONLINE errors: No known data errors 3 3. # zpool status newpool pool: newpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM newpool ONLINE mirror-0 ONLINE c0t4d0 ONLINE c0t5d0 ONLINE errors: No known data errors # zpool add newpool log mirror c0t6d0 c0t7d0 # zpool status newpool pool: newpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM newpool ONLINE mirror-0 ONLINE c0t4d0 ONLINE c0t5d0 ONLINE logs mirror-1 ONLINE c0t6d0 ONLINE c0t7d0 ONLINE errors: No known data errors.. zpool remove. mirror Oracle Solaris ZFS 63

64 ZFS 3 3 ( ) # zpool remove newpool mirror-1 # zpool status newpool pool: newpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM newpool ONLINE mirror-0 ONLINE c0t4d0 ONLINE c0t5d0 ONLINE errors: No known data errors.. # zpool status pool pool: pool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM pool ONLINE raidz1-0 ONLINE c0t8d0 ONLINE c0t9d0 ONLINE logs c0t10d0 ONLINE errors: No known data errors # zpool remove pool c0t10d0 3 4 ZFS. zpool add. : # zpool add tank cache c2t5d0 c2t8d0 # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c2t0d0 ONLINE c2t1d0 ONLINE Oracle Solaris 11.1 : ZFS

65 ZFS 3 4 ( ) c2t3d0 ONLINE cache c2t5d0 ONLINE c2t8d0 ONLINE errors: No known data errors RAID-Z. zpool remove.. # zpool remove tank c2t5d0 c2t8d0 # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c2t0d0 ONLINE c2t1d0 ONLINE c2t3d0 ONLINE errors: No known data errors zpool remove,. zpool detach. RAID-Z. ZFS 54 ZFS. zpool add zpool attach. 112 (SPARC x86/vtoc). ZFS 115 ZFS (SPARC x86/vtoc). 3 Oracle Solaris ZFS 65

66 ZFS 3 5 zeepool, c2t1d0 c1t1d0. # zpool status zeepool pool: zeepool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE errors: No known data errors # zpool attach zeepool c1t1d0 c2t1d0 # zpool status zeepool pool: zeepool state: ONLINE scrub: resilver completed after 0h0m with 0 errors on Fri Jan 8 12:59: config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE c2t1d0 ONLINE K resilvered errors: No known data errors, ZFS ZFS zpool attach.. # zpool create tank c0t1d0 # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE c0t1d0 ONLINE errors: No known data errors # zpool attach tank c0t1d0 c1t1d0 # zpool status tank 66 Oracle Solaris 11.1 : ZFS

67 ZFS 3 6 ZFS ZFS ( ) pool: tank state: ONLINE scrub: resilver completed after 0h0m with 0 errors on Fri Jan 8 14:28: config: NAME STATE READ WRITE CKSUM tank ONLINE mirror-0 ONLINE c0t1d0 ONLINE c1t1d0 ONLINE K resilvered errors: No known data errors zpool detach.. # zpool detach zeepool c2t1d0. : # zpool detach newpool c1t2d0 cannot detach c1t2d0: only applicable to mirror and replacing vdevs ZFS zpool split ZFS.. zpool split ZFS. ZFS. zpool split... # zpool status tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM 3 Oracle Solaris ZFS 67

68 ZFS tank ONLINE mirror-0 ONLINE c1t0d0 ONLINE c1t2d0 ONLINE errors: No known data errors # zpool split tank tank2 # zpool import tank2 # zpool status tank tank2 pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE c1t0d0 ONLINE errors: No known data errors pool: tank2 state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank2 ONLINE c1t2d0 ONLINE errors: No known data errors zpool split.. # zpool split tank tank2 c1t0d0. GUID. GUID. zpool split -R.. # zpool split -R /tank2 tank tank2 zpool split -R -R.. 68 Oracle Solaris 11.1 : ZFS

69 ZFS zpool split. 3 7 RAID-Z. zpool split.. 2~3.. zpool attach. ( ) zpool split ZFS mothership. mothership luna.. luna. luna mothership.. # zpool status mothership pool: mothership state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM mothership ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335f907fd0 ONLINE Oracle Solaris ZFS 69

70 ZFS 3 7 ZFS ( ) errors: No known data errors # zpool split mothership luna # zpool import luna # zpool status mothership luna pool: luna state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM luna ONLINE c0t5000c500335f907fd0 ONLINE errors: No known data errors pool: mothership state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM mothership ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335bd117d0 ONLINE errors: No known data errors ZFS. ZFS. ZFS. ZFS.. zpool offline. ( ).. # zpool offline tank c0t5000c500335f95e3d0 70 Oracle Solaris 11.1 : ZFS

71 ZFS. UNAVAIL., raidz1 2,. # zpool offline tank c0t5000c500335f95e3d0 cannot offline c0t5000c500335f95e3d0: no valid replicas OFFLINE.. zpool offline -t.. # zpool offline -t tank c1t0d0 bringing device c1t0d0 offline ONLINE... device is part of exported or potentially active ZFS pool. Please see zpool(1m) OFFLINE. 83 ZFS. 90 ZFS. zpool online.. # zpool online tank c0t5000c500335f95e3d0.. UNAVAIL. 3 Oracle Solaris ZFS 71

72 ZFS UNAVAIL. /var/adm/messages.. SUNW-MSG-ID: ZFS-8000-LR, TYPE: Fault, VER: 1, SEVERITY: Major EVENT-TIME: Wed Jun 20 11:35:26 MDT 2012 PLATFORM: ORCL,SPARC-T3-4, CSN: 1120BDRCCD, HOSTNAME: tardis SOURCE: zfs-diagnosis, REV: 1.0 EVENT-ID: fb6699c8-6bfb-eefa-88bb e3b7 DESC: ZFS device in pool pond failed to open. AUTO-RESPONSE: An attempt will be made to activate a hot spare if available. IMPACT: Fault tolerance of the pool may be compromised. REC-ACTION: Use fmadm faulty to provide a more detailed view of this event. Run zpool status -lx for more information. Please refer to the associated reference document at for the latest service procedures and policies regarding this diagnosis zpool online -e LUN. LUN autoexpand.lun LUN zpool online -e LUN.. # zpool online -e tank c0t5000c500335f95e3d0 zpool status zpool clear... # zpool clear tank.. # zpool clear tank c0t5000c500335f95e3d0 zpool Oracle Solaris 11.1 : ZFS

73 ZFS zpool replace.. ZFS., (c1t1d0). # zpool replace tank c1t1d0.. # zpool replace tank c1t1d0 c1t2d0 ZFS 115 ZFS (SPARC x86/vtoc).. 1. zpool offline format. ID. WWN ID. 5. ZFS.. # zpool replace tank c1t1d0 ID ID. # zpool replace tank c0t5000c500335fc3e7d0 c0t5000c500335ba8c3d0 6. zpool online. 7. FMA. fmadm faulty Affects: zfs://pool=name/vdev=guid fmadm repaired. # fmadm faulty # fmadm repaired zfs://pool=name/vdev=guid SATA. zpool replace. 3 Oracle Solaris ZFS 73

74 ZFS SATA ZFS. autoreplace on. zpool replace.. REMOVED.,..,.., USB. cfgadm -c configure SATA.. RAID-Z.. autoexpand LUN. autoexpand. LUN LUN. 16GB 72GB.. autoexpand. # zpool create pool mirror c1t16d0 c1t17d0 # zpool status pool: pool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM pool ONLINE mirror ONLINE c1t16d0 ONLINE c1t17d0 ONLINE zpool list pool NAME SIZE ALLOC FREE CAP HEALTH ALTROOT pool 16.8G 76.5K 16.7G 0% ONLINE - # zpool replace pool c1t16d0 c1t1d0 # zpool replace pool c1t17d0 c1t2d0 74 Oracle Solaris 11.1 : ZFS

75 ZFS # zpool list pool NAME SIZE ALLOC FREE CAP HEALTH ALTROOT pool 16.8G 88.5K 16.7G 0% ONLINE - # zpool set autoexpand=on pool # zpool list pool NAME SIZE ALLOC FREE CAP HEALTH ALTROOT pool 68.2G 117K 68.2G 0% ONLINE -. zpool scrub.. zpool detach RAID-Z ,.. zpool create zpool add. # zpool create zeepool mirror c0t5000c500335f95e3d0 c0t5000c500335f907fd0 mirror c0t5000c500335bd117d0 c0t5000c500335dc60fd0 spare c0t5000c500335e106bd0 c0t5000c500335fc3e7d0 # zpool status zeepool pool: zeepool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE spares 3 Oracle Solaris ZFS 75

76 ZFS c0t5000c500335e106bd0 c0t5000c500335fc3e7d0 AVAIL AVAIL errors: No known data errors. # zpool add zeepool spare c0t5000c500335e106bd0 c0t5000c500335fc3e7d0 # zpool status zeepool pool: zeepool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE spares c0t5000c500335e106bd0 AVAIL c0t5000c500335fc3e7d0 AVAIL errors: No known data errors zpool remove.. # zpool remove zeepool c0t5000c500335fc3e7d0 # zpool status zeepool pool: zeepool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE spares c0t5000c500335e106bd0 AVAIL errors: No known data errors. ZFS. 76 Oracle Solaris 11.1 : ZFS

77 ZFS zpool remove,... zpool replace. cannot replace disk3 with disk4: device is too small. zpool replace. FMA.. FMA.. ZFS. zpool detach. autoreplace on. UNAVAIL.. # zpool status -x pool: zeepool state: DEGRADED status: One or more devices are unavailable in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Determine if the device needs to be replaced, and clear the errors using zpool clear or fmadm repaired, or replace the device with zpool replace. Run zpool status -v to see device specific details. scan: resilvered 3.15G in 0h0m with 0 errors on Thu Jun 21 16:46: config: NAME STATE READ WRITE CKSUM zeepool DEGRADED mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE Oracle Solaris ZFS 77

78 ZFS mirror-1 DEGRADED c0t5000c500335bd117d0 ONLINE spare-1 DEGRADED c0t5000c500335dc60fd0 UNAVAIL c0t5000c500335e106bd0 ONLINE spares c0t5000c500335e106bd0 INUSE errors: No known data errors (c0t5000c500335dc60fd0) zpool replace ZFS. # zpool replace zeepool c0t5000c500335dc60fd0 # zpool status zeepool pool: zeepool state: ONLINE scan: resilvered 3.15G in 0h0m with 0 errors on Thu Jun 21 16:53: config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 ONLINE spares c0t5000c500335e106bd0 AVAIL zpool detach.. # zpool detach zeepool c0t5000c500335e106bd # zpool status zeepool pool: zeepool 78 Oracle Solaris 11.1 : ZFS

79 ZFS 3 9 ( ) state: DEGRADED status: One or more devices are unavailable in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Determine if the device needs to be replaced, and clear the errors using zpool clear or fmadm repaired, or replace the device with zpool replace. Run zpool status -v to see device specific details. scan: scrub in progress since Thu Jun 21 17:01: G scanned out of 6.29G at 220M/s, 0h0m to go 0 repaired, 17.05% done config: NAME STATE READ WRITE CKSUM zeepool DEGRADED mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 DEGRADED c0t5000c500335bd117d0 ONLINE c0t5000c500335dc60fd0 UNAVAIL spares c0t5000c500335e106bd0 AVAIL errors: No known data errors # zpool detach zeepool c0t5000c500335dc60fd0 # zpool status zeepool pool: zeepool state: ONLINE scan: resilvered 3.15G in 0h0m with 0 errors on Thu Jun 21 17:02: config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335e106bd0 ONLINE errors: No known data errors (Original failed disk c0t5000c500335dc60fd0 is physically replaced) # zpool add zeepool spare c0t5000c500335dc60fd0 # zpool status zeepool pool: zeepool state: ONLINE scan: resilvered 3.15G in 0h0m with 0 errors on Thu Jun 21 17:02: config: NAME STATE READ WRITE CKSUM zeepool ONLINE mirror-0 ONLINE c0t5000c500335f95e3d0 ONLINE c0t5000c500335f907fd0 ONLINE Oracle Solaris ZFS 79

80 ZFS 3 9 ( ) mirror-1 ONLINE c0t5000c500335bd117d0 ONLINE c0t5000c500335e106bd0 ONLINE spares c0t5000c500335dc60fd0 AVAIL errors: No known data errors FMA. # fmadm faulty # fmadm repaired zfs://pool=name/vdev=guid ZFS zpool get.. # zpool get all zeepool NAME PROPERTY VALUE SOURCE zeepool allocated 6.29G - zeepool altroot - default zeepool autoexpand off default zeepool autoreplace off default zeepool bootfs - default zeepool cachefile - default zeepool capacity 1% - zeepool dedupditto 0 default zeepool dedupratio 1.00x - zeepool delegation on default zeepool failmode wait default zeepool free 550G - zeepool guid zeepool health ONLINE - zeepool listshares off default zeepool listsnapshots off default zeepool readonly off - zeepool size 556G - zeepool version 34 default zpool set.. # zpool set autoreplace=on zeepool # zpool get autoreplace zeepool NAME PROPERTY VALUE SOURCE zeepool autoreplace on local 80 Oracle Solaris 11.1 : ZFS

81 ZFS. # zpool set autoreplace=on tank cannot set property for tank : out of space 12, Oracle Solaris ZFS. 3 1 ZFS allocated. altroot off... autoreplace off. off zpool replace. on. replace. bootfs.. cachefile.... zpool import - c. ZFS. capacity. cap. dedupditto,. dedupratio ( ). 3 Oracle Solaris ZFS 81

82 ZFS 3 1 ZFS ( ) delegation on. 8, Oracle Solaris ZFS. failmode wait... wait zpool clear I/O. I/O. wait. continue I/O EIO.. zpool clear. panic. free. guid. health ONLINE, DEGRADED, SUSPENDED, REMOVED UNAVAIL. listshares Off zfs list. off. listsnapshots Off zfs list. zfslist-tsnapshot. readonly Off.. /. 82 Oracle Solaris 11.1 : ZFS

83 ZFS 3 1 ZFS ( ) size. version. zpool upgrade. 1 zpool upgrade -v. ZFS zpool list., I/O.. 83 ZFS 87 ZFS I/O 90 ZFS ZFS zpool list. zpool list. # zpool list NAME SIZE ALLOC FREE CAP HEALTH ALTROOT tank 80.0G 22.3G 47.7G 28% ONLINE - dozer 1.2T 384G 816G 32% ONLINE -. NAME. SIZE. ALLOC.. 31 ZFS. 3 Oracle Solaris ZFS 83

84 ZFS FREE CAP (CAPACITY) HEALTH ZFS. ALTROOT ( ). 265 ZFS.. : # zpool list tank NAME SIZE ALLOC FREE CAP HEALTH ALTROOT tank 80.0G 22.3G 47.7G 28% ONLINE - zpool list. -T.. # zpool list -T d32 Tue Nov 2 10:36:11 MDT 2010 NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT pool 33.8G 83.5K 33.7G 0% 1.00x ONLINE - rpool 33.8G 12.2G 21.5G 36% 1.00x ONLINE - Tue Nov 2 10:36:14 MDT 2010 pool 33.8G 83.5K 33.7G 0% 1.00x ONLINE - rpool 33.8G 12.2G 21.5G 36% 1.00x ONLINE - zpool status -l.. fmadm add-alias.. # fmadm add-alias SUN-Storage-J QCQ015 Lab10Rack5... # zpool status -l tank pool: tank state: ONLINE scan: scrub repaired 0 in 0h0m with 0 errors on Fri Aug 3 16:00: config: NAME STATE READ WRITE CKSUM tank ONLINE Oracle Solaris 11.1 : ZFS

85 ZFS mirror-0 ONLINE /dev/chassis/lab10rack5.../disk_02/disk ONLINE /dev/chassis/lab10rack5.../disk_20/disk ONLINE mirror-1 ONLINE /dev/chassis/lab10rack5.../disk_22/disk ONLINE /dev/chassis/lab10rack5.../disk_14/disk ONLINE mirror-2 ONLINE /dev/chassis/lab10rack5.../disk_10/disk ONLINE /dev/chassis/lab10rack5.../disk_16/disk ONLINE mirror-3 ONLINE /dev/chassis/lab10rack5.../disk_01/disk ONLINE /dev/chassis/lab10rack5.../disk_21/disk ONLINE mirror-4 ONLINE /dev/chassis/lab10rack5.../disk_23/disk ONLINE /dev/chassis/lab10rack5.../disk_15/disk ONLINE mirror-5 ONLINE /dev/chassis/lab10rack5.../disk_09/disk ONLINE /dev/chassis/lab10rack5.../disk_04/disk ONLINE mirror-6 ONLINE /dev/chassis/lab10rack5.../disk_08/disk ONLINE /dev/chassis/lab10rack5.../disk_05/disk ONLINE mirror-7 ONLINE /dev/chassis/lab10rack5.../disk_07/disk ONLINE /dev/chassis/lab10rack5.../disk_11/disk ONLINE mirror-8 ONLINE /dev/chassis/lab10rack5.../disk_06/disk ONLINE /dev/chassis/lab10rack5.../disk_19/disk ONLINE mirror-9 ONLINE /dev/chassis/lab10rack5.../disk_00/disk ONLINE /dev/chassis/lab10rack5.../disk_13/disk ONLINE mirror-10 ONLINE /dev/chassis/lab10rack5.../disk_03/disk ONLINE /dev/chassis/lab10rack5.../disk_18/disk ONLINE spares /dev/chassis/lab10rack5.../disk_17/disk AVAIL /dev/chassis/lab10rack5.../disk_12/disk AVAIL errors: No known data errors -o..,. # zpool list -o name,size NAME SIZE tank 80.0G dozer 1.2T Oracle Solaris ZFS 85

86 ZFS ZFS zpool list. -H.,. # zpool list -Ho name tank dozer. # zpool list -H -o name,size tank 80.0G dozer 1.2T ZFS ZFS zfs zpool. zpool history.,. # zpool history History for rpool : :02:55 zpool create -f rpool c3t0d0s :02:56 zfs create -p -o mountpoint=/export rpool/export :02:58 zfs set mountpoint=/export rpool/export :02:58 zfs create -p rpool/export/home :03:03 zfs create -p -V 2048m rpool/swap :03:08 zfs set primarycache=metadata rpool/swap :03:09 zfs create -p -V 4094m rpool/dump :26:47 zpool set bootfs=rpool/root/s11u1 rpool :31:15 zfs set primarycache=metadata rpool/swap :31:46 zfs create -o canmount=noauto -o mountpoint=/var/share rpool/varshare :22:33 zfs set primarycache=metadata rpool/swap :42:48 zfs set primarycache=metadata rpool/swap :17:24 zfs snapshot -r rpool/root@yesterday :17:54 zfs snapshot -r rpool/root@now ZFS....,.. 128KB. 32MB. 86 Oracle Solaris 11.1 : ZFS

87 ZFS 1%...,.. # zpool history tank :35:32 zpool create -f tank mirror c3t1d0 c3t2d0 spare c3t3d :04:10 zfs create tank/test :05:01 zfs snapshot -r tank/test@snap1 -l,.. # zpool history -l tank History for tank : :35:32 zpool create -f tank mirror c3t1d0 c3t2d0 spare c3t3d0 [user root on tardis:global] :04:10 zfs create tank/test [user root on tardis:global] :05:01 zfs snapshot -r tank/test@snap1 [user root on tardis:global] -i.. # zpool history -i tank History for tank : :35:32 zpool create -f tank mirror c3t1d0 c3t2d0 spare c3t3d :35:32 [internal pool create txg:5] pool spa 33; zfs spa 33; zpl 5; uts tardis sun4v :04:10 zfs create tank/test :04:10 [internal property set txg:66094] $share2=2 dataset = :04:31 [internal snapshot txg:66095] dataset = :05:01 zfs snapshot -r tank/test@snap :08:00 [internal user hold txg:66102] <.send > temp = 1... ZFS I/O I/O zpool iostat. iostat I/O.. alloc capacity.. 31 ZFS. 3 Oracle Solaris ZFS 87

88 ZFS free capacity read operations write operations read bandwidth write bandwidth. used. I/O. I/O. ( ).. I/O zpool iostat. : # zpool iostat capacity operations bandwidth pool alloc free read write read write rpool 6.05G 61.9G tank 31.3G 36.7G K 86.1K # zpool iostat tank 2 capacity operations bandwidth pool alloc free read write read write tank 18.5G 49.5G M tank 18.5G 49.5G M tank 18.5G 49.5G M tank 18.8G 49.2G M Ctrl-C 2 tank. count., zpool iostat Oracle Solaris 11.1 : ZFS

89 ZFS I/O I/O zpool iostat I/O. ZFS I/O. I/O zpool iostat -v. : # zpool iostat -v capacity operations bandwidth pool alloc free read write read write rpool 6.05G 61.9G mirror 6.05G 61.9G c1t0d0s c1t1d0s tank 36.5G 31.5G K 146K mirror 36.5G 31.5G M 4.01M c1t2d K 386K c1t3d K 386K I/O.,. RAID-Z.,., RAID-Z. I/O... ( )... # zpool iostat -lv capacity operations bandwidth pool alloc free read write read write export 2.39T 2.14T K 300K mirror 490G 438G K 60.3K /dev/chassis/lab10rack15/scsi_device 2/disk K 60.3K /dev/chassis/lab10rack15/scsi_device 3/disk K 60.3K mirror 490G 438G K 59.9K /dev/chassis/lab10rack15/scsi_device 4/disk K 59.9K /dev/chassis/lab10rack15/scsi_device 5/disk K 59.9K mirror 490G 438G K 60.2K 3 Oracle Solaris ZFS 89