Copyright 2009, 2011, 2012, Oracle and/or its affiliates. All rights reserved.,.,,,,,,,,,,,,.,...,. U.S. GOVERNMENT END USERS: Oracle programs, includ

Sun ZFS Storage 7000 : E38995 01 2012 12 E38995-01

Copyright 2009, 2011, 2012, Oracle and/or its affiliates. All rights reserved.,.,,,,,,,,,,,,.,...,. U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government..., -,,. Oracle Corporation. Oracle Java Oracle Corporation /.. Intel Intel Xeon Intel Corporation. SPARC SPARC International, Inc.. AMD, Opteron, AMD, AMD Opteron Advanced Micro Devices. UNIX The Open Group. ( )3,. Oracle Corporation 3,. Oracle Corporation 3,,. 130322@25097

...19 1...21... 21... 21... 24 BUI( )... 25 CLI( )... 25... 26... 26... 26... 27... 27... 28... 28 BUI... 29... 32... 32 CLI... 36 CLI... 36... 37 CLI... 37... 38... 39... 39... 40... 40... 41 3

CLI... 41... 41... 42... 43... 44... 44... 44... 50... 50... 50 BUI... 65 CLI... 65 2 Status...69 Status... 69... 69 Dashboard... 70... 70 CLI... 75... 76 Settings... 76... 76 BUI... 76 CLI... 78... 78 NDMP... 79 BUI... 79 CLI... 80 3 Configuration...81 Configuration... 81... 81... 82... 82... 82 4 Sun ZFS Storage 7000 2012 12 E38995-01

... 82 BUI... 83 CLI... 83 Network... 88 Network... 88 BUI... 96 CLI... 99... 100 Storage... 105... 106... 112 SAN... 112 SAN... 112 BUI... 114 CLI... 115... 116 SAN... 116 FC... 117... 117 BUI... 120 CLI... 122 FCMPxIO... 125 FC... 125 iscsi... 131... 131 BUI... 134 CLI... 135 SRP... 136... 136 BUI... 143 CLI... 147... 147... 147... 148... 148... 149 5

BUI... 150 CLI... 150... 152 Preferences... 155... 155 BUI... 156 CLI... 156 SSH Public Keys... 157... 157... 157... 158 Threshold Alerts... 160 BUI... 161 CLI... 161... 161... 162... 162... 162... 163... 164... 165... 170... 171... 172 Infiniband... 174 ' '... 175... 177... 179... 181 JBOD... 181 BUI... 181... 183 4 Services... 185 Services... 185 6 Sun ZFS Storage 7000 2012 12 E38995-01

... 185 BUI... 188 CLI... 190 NFS... 193... 193... 194 Kerberos... 195... 195 Analytics... 196 CLI... 196... 197 iscsi... 197... 197... 198... 198... 198... 198 CLI... 198... 199 SMB... 199... 199... 199... 200 NFS/SMB... 201 DFS... 201... 202... 202... 203 MMC... 203 CLI... 208... 209 FTP... 212... 212... 212... 213... 213 7

HTTP... 214... 214... 214... 215... 215... 215 NDMP... 216... 216... 221... 222 SFTP... 222... 222... 223... 223... 224 Virus Scan... 224... 224... 224... 226... 226 NIS... 226... 226... 227... 227... 227 LDAP... 228... 228... 228... 230... 230 Active Directory... 231... 231... 231... 232 LDAP... 232 Windows Server 2008... 233 8 Sun ZFS Storage 7000 2012 12 E38995-01

BUI... 234 CLI... 234... 235 Identity Mapping... 236... 236 IDMU... 237... 237... 238... 240... 240... 241... 241... 242 DNS... 242... 242... 243 CLI... 243... 243 Active Directory DNS... 244 DNS... 244 DNS... 244 IPMP... 245... 245... 245... 245... 246 NTP... 246... 246... 246 BUI... 248... 248... 249 Remote Replication... 249... 249 Dynamic Routing... 250 RIP RIPng... 250 9

... 250 Phone Home... 250... 250... 251... 252... 252 SNMP... 252... 252... 253 MIB... 253 Sun FM MIB... 254 Sun AK MIB... 254... 255 SMTP... 255... 255... 256... 256 Service Tags... 257... 257... 257 System Identity... 257... 257... 258... 258 SSH... 258... 258... 258... 259... 259 Shadow Migration... 259... 259... 260... 260 Syslog... 260... 260... 261 10 Sun ZFS Storage 7000 2012 12 E38995-01

Syslog: RFC 3164... 261 Syslog: RFC 5424... 262... 262... 264 5 Shares... 267 Shares... 267... 267... 268... 268... 269 Shares... 269... 269... 270... 271 Shadow Migration... 271... 271... 274... 275... 279... 279... 280... 280... 281... 282... 283... 284... 288... 288 Shares... 290 BUI... 290 CLI... 295... 299... 299... 300 11

... 300... 306... 306... 306 NFS... 307 SMB... 311 SCSI... 311 HTTP... 312 FTP... 312 SFTP... 313... 313... 313... 313 ACL... 314 ACL... 316... 319... 319... 319 BUI... 319 CLI... 322... 325 BUI... 325 CLI... 327... 331... 331... 331... 331... 332... 332 LUN... 332... 333... 333 NFS... 333 SMB... 333 iscsi... 333 HTTP... 334 12 Sun ZFS Storage 7000 2012 12 E38995-01

FTP... 334... 334... 334 ACL... 334... 334... 334... 334 BUI... 335 CLI... 335... 336 Remote Replication... 336... 337... 341... 345 Remote Replication... 356... 363... 363 BUI... 363 CLI... 364... 365 6 Analytics... 367 Analytics... 367... 368... 368 Analytics... 368... 368... 369... 369... 370... 370... 371... 371... 371... 372 13

... 374 CPU... 375 CPU: Percent utilization... 375 ARC... 376 Cache: ARC accesses... 376 L2ARC IO... 379 Cache: L2ARC I/O bytes... 379 L2ARC... 379 Cache: L2ARC accesses... 379 NDMP... 381 Data Movement: NDMP bytes transferred to/from disk... 381 NDMP... 381 Data Movement: NDMP bytes transferred to/from tape... 381... 382 Data Movement: Shadow migration bytes... 382... 383 Data Movement: Shadow migration ops... 383... 383 Data Movement: Shadow migration requests... 383... 384 Disk: Disks... 384 IO... 385 Disk: I/O bytes... 385 IO... 387 Disk: I/O operations... 387... 388 Network: Device bytes... 388... 389 Network: Interface bytes... 389 SMB... 390 Protocol: SMB operations... 390... 392 Protocol: Fibre Channel bytes... 392... 393 Protocol: Fibre Channel operations... 393 FTP... 394 14 Sun ZFS Storage 7000 2012 12 E38995-01

Protocol: FTP bytes... 394 HTTPWebDAV... 395 Protocol: HTTP/WebDAV requests... 395 iscsi... 397 Protocol: iscsi bytes... 397 iscsi... 397 Protocol: iscsi operations... 397 NFSv2... 399 Protocol: NFSv2 operations... 399 NFSv3... 401 Protocol: NFSv3 operations... 401 NFSv4... 402 Protocol: NFSv4 operations... 402 SFTP... 404 Protocol: SFTP bytes... 404 SRP... 405 Protocol: SRP bytes... 405 SRP... 406 Protocol: SRP operations... 406 CPU CPU... 408 CPU: CPUs... 408 CPU... 409 CPU: Kernel spins... 409 ARC... 409 Cache: ARC adaptive parameter... 409 ARC... 410 Cache: ARC evicted bytes... 410 ARC... 410 Cache: ARC size... 410 ARC... 411 Cache: ARC target size... 411 DNLC... 412 Cache: DNLC accesses... 412 DNLC... 413 Cache: DNLC entries... 413 L2ARC... 413 15

Cache: L2ARC errors... 413 L2ARC... 414 Cache: L2ARC size... 414 NDMP... 415 Data Movement: NDMP file system operations... 415 NDMP... 415 Data Movement: NDMP jobs... 415... 416 Disk: Percent utilization... 416 ZFS DMU... 416 Disk: ZFS DMU operations... 416 ZFS IO... 417 Disk: ZFS logical I/O bytes... 417 ZFS IO... 418 Disk: ZFS logical I/O operations... 418... 418 Memory: Dynamic memory usage... 418... 419 Memory: Kernel memory... 419... 419 Memory: Kernel memory in use... 419... 420 Memory: Kernel memory lost to fragmentation... 420 IP... 421 Network: IP bytes... 421 IP... 421 Network: IP packets... 421 TCP... 422 Network: TCP bytes... 422 TCP... 422 Network: TCP packets... 422 NSCD... 423 System: NSCD backend requests... 423 NSCD... 424 System: NSCD operations... 424 Open Worksheets... 424 16 Sun ZFS Storage 7000 2012 12 E38995-01

... 425... 428... 429 CLI... 430... 430... 431 Saved Worksheets... 432... 432... 432 BUI... 432 CLI... 433... 434... 434 BUI... 434 CLI... 435 7... 439... 439... 439 Symantec... 440 Symantec 'DMP'/Storage Foundation FC... 440 Oracle Solaris Cluster... 441 Oracle Solaris Cluster Sun ZFS Storage Appliance 441 Microsoft... 441 Volume Shadow Copy Service Software Sun ZFS Storage Appliance Provider... 441 Oracle Grid Control... 442 Oracle Enterprise Manager Grid Control Sun ZFS Storage... 442 Oracle Storage Connect... 443 Sun ZFS Storage Appliance Oracle Virtual Machine... 443 VMware Site Recovery Manager... 444 VMware Site Recovery Manager Sun ZFS Storage 7000... 444 Exadata... 444 Oracle Exadata... 444... 445 Sun ZFS Storage Appliance... 445 17

Exadata... 448 Sun ZFS Storage Appliance Oracle Exadata... 448... 453 18 Sun ZFS Storage 7000 2012 12 E38995-01

Sun ZFS Storage 7000 NAS Oracle Sun ZFS Storage 7000. Help.. Sun ZFS Storage 7000.,,. Installation Guide, Analytics Guide and Service Manual (http://www.oracle.com/ technetwork/documentation/) URL. ( )3,. Oracle Corporation 3,. Oracle Corporation 3,,. 19

Oracle Oracle My Oracle Support. http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs( ). 20 Sun ZFS Storage 7000 2012 12 E38995-01

1 Sun ZFS Storage 7000. 7120 7320 7420 : 7110 7210 7310 7410 Disk Shelf : J4400/J4500 21

Sun ZFS Storage. SMB NFS HTTP HTTPS WebDAV iscsi FC SRP iser FTP SFTP Sun ZFS Storage /. Analytics - ZFS -,, DRAM Sun ZFS Storage. :.. Oracle. "Oracle ("SLA") ". RAID-Z(RAID-5 RAID-6), /, - (7310, 7320, 7410, 7420) 22 Sun ZFS Storage 7000 2012 12 E38995-01

iscsi LUN NDMP Sun ZFS Storage.. (CPU, DRAM, I/O,,, ) ZFS RAID-6() Disk Shelf RAID-6 - (7310, 7320, 7410, 7420) IP Disk Shelf I/O Phone-Home Lights Out BUI( ) BUI.BUI,,. 1 23

. NET-0 IP ( : https://ipaddress:215 https://hostname:215).. BUI.BUI Help. CLI( ) CLI BUI. CLI. CLI. Tab. Tab. Tab 'Tab'.. help. ( : help commands). help help topics.. dory:> help tab builtins commands general help properties script 24 Sun ZFS Storage 7000 2012 12 E38995-01

BUI( ) BUI.BUI,,. BUI. -BUI - - CLI( ) CLI BUI. CLI. CLI - - - - - 1 25

BUI. 1. 1 BUI 1. Firefox 3.x 5.x Internet Explorer 7, 8, 9 Safari 3.1 WebKit 525.13 2 BUI 2,.. 2. Firefox 2.x Solaris 10 Mozilla 1.7 Opera 9 Internet Explorer 6. 26 Sun ZFS Storage 7000 2012 12 E38995-01

Projects BUI.. Sun,,.,,,. BUI Configuration, Maintenance, Shares, Status Analytics.. 1 27

. Dashboard Maintenance: Logs. ID.. Configuration: Users.,.,ID. ID ID ID Properties, Rules Logs Service Project. 28 Sun ZFS Storage 7000 2012 12 E38995-01

Projects Add.... Shares Projects Shares. : ( ). Share Share. BUI BUI. BUI... Apply. 1 29

BUI Revert. ( )..... BUI Oracle oracle.com.. Let us know. Permissions Permissions RWX. User, Group, Other. Share Share Inherit from project. 30 Sun ZFS Storage 7000 2012 12 E38995-01

... 1 31

.....,. * * -- ( ) -- -- -- -- -- 32 Sun ZFS Storage 7000 2012 12 E38995-01

* * -- -- / -- -- -- ( ) -- -- () / -- / -- -- -- -- *.. InfiniBand InfiniBand (IB ) VLAN 1 33

VLAN Settings. 2 3 4 5 Analytics Analytics. 34 Sun ZFS Storage 7000 2012 12 E38995-01

1 1 1 1 ( ) 1 ID Windows Unix. * * Windows Unix Windows Unix Unix Windows Unix Windows *.. SAS SAS 1 35

CLI CLI CLI. SSH.. -.. -. -. -.. CLI CLI ssh. root. CLI,,. % ssh root@dory Password: Last login: Mon Oct 13 15:43:05 2009 from kiowa.sf.fishpo dory:> CLI. - Tab. Tab 'Tab'. - help. help ( : help commands). help help topics.. dory:> help tab builtins commands general help properties script 36 Sun ZFS Storage 7000 2012 12 E38995-01

CLI CLI.. BUI...., Configuration CLI configuration.. dory:> configuration dory:configuration>,. show., configuration. dory:configuration> show Children: net => Configure networking services => Configure services version => Display system version users => Configure administrative users roles => Configure administrative roles preferences => Configure user preferences alerts => Configure alerts storage => Configure Storage Network, Services, Users, Preferences Configuration.. dory:configuration> preferences dory:configuration preferences> ( )., configuration preferences. 1 37

dory:> configuration preferences dory:configuration preferences>. select,. list., users. dory:> configuration users dory:configuration users> list NAME USERNAME UID TYPE John Doe bmc 12345 Dir Super-User root 0 Loc bmc select bmc. dory:configuration users> select bmc dory:configuration users bmc> select destroy., maintenance logs system reboot. dory:maintenance logs system> select module=reboot dory:maintenance logs system entry-034> show Properties: timestamp = 2010-8-14 06:24:41 module = reboot priority = crit text = initiated by root on /dev/console syslogd: going down on signal 15 select., bmc. dory:> configuration users select bmc dory:configuration users bmc> done. dory:configuration> done dory:>. 38 Sun ZFS Storage 7000 2012 12 E38995-01

dory:> configuration users select bmc dory:configuration users bmc> done dory:> done. dory:> configuration dory:configuration> users dory:configuration users> select bmc dory:configuration users bmc> done dory:configuration users> done dory:configuration> done dory:> cd. UNIX cd "..". dory:> configuration users select bmc dory:configuration users bmc> cd.. dory:configuration users> UNIX "cd /". dory:> configuration dory:configuration> users dory:configuration users> select bmc dory:configuration users bmc> cd / dory:> UNIX "cd../..". dory:> configuration dory:configuration> users dory:configuration users> select bmc dory:configuration users bmc> cd../.. dory:configuration> (, select ). 15 bmc. 31. dory:> configtab dory:> configuration utab dory:> configuration users setab 1 39

dory:> configuration users select tab bmc root dory:> configuration users select btab dory:> configuration users select bmcenter dory:configuration users bmc>., configuration preferences get. dory:configuration preferences> get locale = C login_screen = status/dashboard session_timeout = 15 session_annotation = advanced_analytics = false., get. dory:> configuration preferences get locale = C login_screen = status/dashboard session_timeout = 15 session_annotation = advanced_analytics = false., configuration alerts threshold create threshold alert. dory:> configuration alerts thresholds create dory:configuration alerts threshold (uncommitted)> (uncommitted). commit.. dory:configuration alerts threshold (uncommitted)> cd / Leaving will abort creation of "threshold". Are you sure? (Y/N).,.. 40 Sun ZFS Storage 7000 2012 12 E38995-01

dory:configuration alerts threshold (uncommitted)> commit error: missing value for property "statname". dory:configuration alerts threshold (uncommitted)> set statname=cpu.utilization statname = cpu.utilization (uncommitted) dory:configuration alerts threshold (uncommitted)> commit error: missing value for property "limit" dory:configuration alerts threshold (uncommitted)> set limit=90 limit = 90 (uncommitted) dory:configuration alerts threshold (uncommitted)> commit dory:configuration alerts thresholds> list THRESHOLD LIMIT TYPE STATNAME threshold-000 90 normal cpu.utilization CLI /. "help properties".. dory:configuration preferences> help properties Properties that are valid in this context: locale login_screen session_timeout session_annotation advanced_analytics => Locality => Initial login screen => Session timeout => Current session annotation => Make available advanced analytics statistics get. get. dory:configuration preferences> get locale = C login_screen = status/dashboard session_timeout = 15 session_annotation = advanced_analytics = false 1 41

get., login_screen. dory:configuration preferences> get login_screen login_screen = status/dashboard get., iscsi. dory:> configuration services iscsi get tab <status> isns_server radius_secret target_chap_name isns_access radius_access radius_server target_chap_secret set.., login_screen "shares". dory:configuration preferences> set login_screen=shares login_screen = shares (uncommitted).. commit. dory:configuration preferences> get login_screen login_screen = shares (uncommitted) dory:configuration preferences> commit dory:configuration preferences> get login_screen login_screen = shares. : dory:configuration preferences> set login_screen=maintenance/hardware login_screen = maintenance/hardware (uncommitted) dory:configuration preferences> done You have uncommitted changes that will be discarded. Are you sure? (Y/N) 42 Sun ZFS Storage 7000 2012 12 E38995-01

, set. : dory:> configuration preferences set login_screen=analytics/worksheets login_screen = analytics/worksheets dory:>.., NTP servers NTP. dory:configuration services ntp> set servers=0.pool.ntp.org,1.pool.ntp.org servers = 0.pool.ntp.org,1.pool.ntp.org (uncommitted) dory:configuration services ntp> commit,,., sharenfs shares "kiowa" /. dory:> shares select default dory:shares default> set sharenfs="ro,rw=kiowa" sharenfs = ro,rw=kiowa (uncommitted) dory:shares default> commit.,.., space_available. dory:> shares select default dory:shares default> get space_available space_available = 1.15T dory:shares default> set space_available=100p error: cannot set immutable property "space_available". set., bmc fullname. dory:> configuration users select bmc set fullname="rembrandt Q. Einstein" error: cannot set immutable property "fullname" 1 43

., "myproj" "myfs" "newsnap". shares select myproj select myfs snapshots snapshot newsnap ssh. % ssh root@dory < myfile.txt "here file".. here file. % ssh root@dory << EOF shares select myproj select myfs snapshots snapshot newsnap EOF.., CLI.,. ECMAScript 3. ECMAScript C,. (if/else) (while, for ) Perl(split(), join() ) 44 Sun ZFS Storage 7000 2012 12 E38995-01

CLI script. dory:> script ("." to run)>, ".". dory:> script ("." to run)> for(i=10;i>0;i--) ("." to run)> printf("%d... ", i); ("." to run)> printf("blastoff!\n"); ("." to run)>. 10... 9... 8... 7... 6... 5... 4... 3... 2... 1... Blastoff! script. dory:> script print("it is now " + new Date()) It is now Tue Oct 14 2009 05:33:01 GMT+0000 (UTC),.. get list run props set. ( : Date )...... Run "run".. : 1 45

dory:> configuration version script dump(run( get boot_time )) boot_time = 2009-10-12 07:02:17\n dump. ECMAScript.,. dory:> configuration version script dump(run( get boot_time ).split(/\s+/)) [', boot_time, =, 2009-10-12, 07:02:17, ' ] Get run.. "get". boot_time ECMAScript Date., boot_time. script run( configuration version ); now = new Date(); uptime = (now.valueof() - get( boot_time ).valueof()) / 1000; printf( up %d day%s, %d hour%s, %d minute%s, %d second%s\n, d = uptime / 86400, d<1 d>=2? s :, h = (uptime / 3600) % 24, h<1 h>=2? s :, m = (uptime / 60) % 60, m<1 m>=2? s :, s = uptime % 60, s<1 s>=2? s : ); "uptime.aksh". % ssh root@dory < uptime.aksh Pseudo-terminal will not be allocated because stdin is not a terminal. Password: up 2 days, 10 hours, 47 minutes, 48 seconds SSH. SSH "-T". List. list.,. script run( shares ); projects = list(); 46 Sun ZFS Storage 7000 2012 12 E38995-01

for (i = 0; i < projects.length; i++) { run( select + projects[i]); shares = list(); } for (j = 0; j < shares.length; j++) { run( select + shares[j]); printf("%s/%s %1.64g %1.64g\n", projects[i], shares[j], get( space_data ), get( space_available )); run( cd.. ); } run( cd.. ); "space.aksh". % ssh root@koi < space.aksh Password: admin/accounts 18432 266617007104 admin/exports 18432 266617007104 admin/primary 18432 266617007104 admin/traffic 18432 266617007104 admin/workflow 18432 266617007104 aleventhal/hw_eng 18432 266617007104 bcantrill/analytx 1073964032 266617007104 bgregg/dashbd 18432 266617007104 bgregg/filesys01 26112 107374156288 bpijewski/access_ctrl 18432 266617007104... " "( ) get. script run( shares ); projects = list(); printf( %-40s %-10s %-10s\n, SHARE, USED, AVAILABLE ); for (i = 0; i < projects.length; i++) { run( select + projects[i]); shares = list(); for (j = 0; j < shares.length; j++) { run( select + shares[j]); share = projects[i] + / + shares[j]; used = run( get space_data ).split(/\s+/)[3]; avail = run( get space_available ).split(/\s+/)[3]; } } run( cd.. ); printf( %-40s %-10s %-10s\n, share, used, avail); run( cd.. ); 1 47

"prettyspace.aksh". % ssh root@koi < prettyspace.aksh Password: SHARE USED AVAILABLE admin/accounts 18K 248G admin/exports 18K 248G admin/primary 18K 248G admin/traffic 18K 248G admin/workflow 18K 248G aleventhal/hw_eng 18K 248G bcantrill/analytx 1.00G 248G bgregg/dashbd 18K 248G bgregg/filesys01 25.5K 100G bpijewski/access_ctrl 18K 248G... Children. children.,. configuration services script var svcs = children(); for(vari=0; i<svcs.length; ++i) { run(svcs[i]); if (props().length!== 0) printf("%-10s %s\n", svcs[i], get( <status> )); run("done"); }. "svcinfo.aksh". % ssh root@koi < space.aksh Password: cifs disabled dns online ftp disabled http disabled identity online idmap online ipmp online iscsi online ldap disabled ndmp online nfs online nis online ntp online scrk online 48 Sun ZFS Storage 7000 2012 12 E38995-01

sftp smtp snmp ssh tags vscan disabled online disabled online online disabled.. dump print printf. JSON... tostring. C printf(3c)... code - message -. CLI. : dory:> script run( not a cmd ) error: uncaught error exception (code EAKSH_BADCMD) in script: invalid command "not a cmd" (encountered while attempting to run command "not a cmd"). dory:> script try { run( not a cmd ) } catch (err) { dump(err); } { tostring: <function>, code: 10004, message: invalid command "not a cmd" (encountered while attempting to run command "not a cmd") }. #!/usr/bin/ksh -p ssh -T root@dory <<EOF 1 49

script try { run( shares select default select $1 ); } catch (err) { if (err.code == EAKSH_ENTITY_BADSELECT) { printf( error: "$1" is not a share in the + default project\n ); exit(1); } } throw (err); EOF printf( "default/$1": compression is %s\n, get( compression )); exit(0); "share.ksh". % ksh./share.ksh bogus error: "bogus" is not a share in the default project,.,, SSH....,. workflow ECMAScript. 3. 50 Sun ZFS Storage 7000 2012 12 E38995-01

name description execute. var workflow = { name: Hello world, description: Bids a greeting to the world, execute: function () { return ( hello world! ) } }; "Hello world". "hello world!"..,.,,,. run CPU. var workflow = { name: CPU utilization, description: Displays the current CPU utilization, execute: function () { run( analytics datasets select name=cpu.utilization ); cpu = run( csv 1 ).split( \n )[1].split(, ); return ( At + cpu[0] +, utilization is + cpu[1] + % ); } };.. workflow parameters. parameters. parameters. label type 1 51

type. Boolean ChooseOne EmailAddress File Host HostName HostPort Integer NetAddress NodeName NonNegativeInteger Number Password Permissions Port Size String StringList ( ) 0 ( ) POSIX., 2, ( ) ( ). var workflow = { name: New share, description: Creates a new share in a business unit, parameters: { name: { label: Name of new share, type: String }, unit: { label: Business unit, type: String 52 Sun ZFS Storage 7000 2012 12 E38995-01

}; } }, execute: function (params) { run( shares select + params.unit); run( filesystem + params.name); run( commit ); return ( Created new share " + params.name + " ); }... ChooseOne,. options optionlabels options ChooseOne. var workflow = { name: Create share, description: Creates a new share in a business unit, parameters: { name: { label: Name of new share, type: String }, unit: { label: Business unit, type: ChooseOne, options: [ development, finance, qa, sales ], optionlabels: [ Development, Finance, Quality Assurance, Sales/Administrative ], } }, execute: function (params) { run( shares select + params.unit); run( filesystem + params.name); run( commit ); return ( Created new share " + params.name + " ); } }; unit. 1 53

UI. parameters optional. optional true., UI. execute undefined....,.. var workflow = { name: Create share, description: Creates a new share in a business unit, parameters: { name: { label: Name of new share, type: String }, unit: { label: Business unit, type: ChooseOne, options: [ development, finance, qa, sales ], optionlabels: [ Development, Finance, Quality Assurance, Sales/Administrative ], } }, execute: function (params) { try { run( shares select + params.unit); } catch (err) { if (err.code!= EAKSH_ENTITY_BADSELECT) throw (err); /* * We haven t yet created a project that corresponds to * this business unit; create it now. */ 54 Sun ZFS Storage 7000 2012 12 E38995-01

} run( shares project + params.unit); run( commit ); run( shares select + params.unit); }; } run( filesystem + params.name); run( commit ); return ( Created new share " + params.name + " ); validate. validate.. var workflow = { name: Create share, description: Creates a new share in a business unit, parameters: { name: { label: Name of new share, type: String }, unit: { label: Business unit, type: ChooseOne, options: [ development, finance, qa, sales ], optionlabels: [ Development, Finance, Quality Assurance, Sales/Administrative ], } }, validate: function (params) { try { run( shares select + params.unit); run( select + params.name); } catch (err) { if (err.code == EAKSH_ENTITY_BADSELECT) return; } return ({ name: share already exists }); }, execute: function (params) { try { run( shares select + params.unit); } catch (err) { if (err.code!= EAKSH_ENTITY_BADSELECT) throw (err); /* * We haven t yet created a project that corresponds to * this business unit; create it now. 1 55

} */ run( shares project + params.unit); set( mountpoint, /export/ + params.unit); run( commit ); run( shares select + params.unit); }; } run( filesystem + params.name); run( commit ); return ( Created new share " + params.name + " ); audit. audit.. execute.. step execute execute execute.,. var steps =[{ step: Checking for associated project, execute: function (params) { try { run( shares select + params.unit); } catch (err) { if (err.code!= EAKSH_ENTITY_BADSELECT) throw (err); /* * We haven t yet created a project that corresponds to * this business unit; create it now. */ run( shares project + params.unit); set( mountpoint, /export/ + params.unit); run( commit ); 56 Sun ZFS Storage 7000 2012 12 E38995-01

run( shares select + params.unit); } } }, { step: Creating share, execute: function (params) { run( filesystem + params.name); run( commit ); } }, { step: Creating audit record, execute: function (params) { audit( created " + params.name + " in " + params.unit); } }]; var workflow = { name: Create share, description: Creates a new share in a business unit, parameters: { name: { label: Name of new share, type: String }, unit: { label: Business unit, type: ChooseOne, options: [ development, finance, qa, sales ], optionlabels: [ Development, Finance, Quality Assurance, Sales/Administrative ], } }, validate: function (params) { try { run( shares select + params.unit); run( select + params.name); } catch (err) { if (err.code == EAKSH_ENTITY_BADSELECT) return; } }; return ({ name: share already exists }); }, execute: function (params) { return (steps); }.,.. 1 57

required (,,, ). version (major.minor.micro). required.,, "year.month.day.build-branch"., "2009.04.10,12-0" 2009 4 10 12. "configuration version get version" CLI BUI "Version". required. var workflow = { name: Configure FC, description: Configures fibre channel target groups, required: 2009.12.25,1-0,.... version.,,..,. "2.1" "2.0" "2.0.1" "1.2" "0.1".. alert true. 58 Sun ZFS Storage 7000 2012 12 E38995-01

ID. setid true.. class. code. items. timestamp. items. url URL. action. impact. description. severity. audit. audit.,. var workflow = { name: Failover, description: Fail the node over to its clustered peer, alert: true, setid: true, execute: function (params) { /* * To failover, we first confirm that clustering is configured * and that we are in the clustered state. We then reboot, * which will force our peer to takeover. Note that we re * being very conservative by only rebooting if in the * AKCS_CLUSTERED state: there are other states in which it 1 59

}; } * may well be valid to failback (e.g., we are in AKCS_OWNER, * and our peer is AKCS_STRIPPED), but those states may also * indicate aberrent operation, and we therefore refuse to * failback. (Even in an active/passive clustered config, a * FAILBACK should always be performed to transition the * cluster peers from OWNER/STRIPPED to CLUSTERED/CLUSTERED.) */ var uuid = params.uuid; var clustered = AKCS_CLUSTERED ; audit( attempting failover in response to alert + uuid); try { run( configuration cluster ); } catch (err) { audit( could not get clustered state; aborting ); return; } if ((state = get( state ))!= clustered) { audit( state is + state + ; aborting ); return; } if ((state = get( peer_state ))!= clustered) { audit( peer state is + state + ; aborting ); return; } run( cd / ); run( confirm maintenance system reboot );. scheduled Workflow true. CLI schedule Workflow. CLI CLI. dory:> maintenance workflows dory:maintenance workflows> "select workflow-002 dory:maintenance workflow-002> schedules dory:maintenance workflow-002 schedules>create dory:maintenance workflow-002 schedule (uncommitted)> set frequency=day frequency = day (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> set hour=10 hour = 10 (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> set minute=05 60 Sun ZFS Storage 7000 2012 12 E38995-01

minute = 05 (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> commit dory:maintenance workflow-002 schedules> list NAME FREQUENCY DAY HH:MM schedule-001 day - 10:05 dory:maintenance workflow-002 schedules> create dory:maintenance workflow-002 schedule (uncommitted)> set frequency=week frequency = week (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> set day=monday day = Monday (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> set hour=13 hour = 13 (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> set minute=15 minute = 15 (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> commit dory:maintenance workflow-002 schedules> list NAME FREQUENCY DAY HH:MM schedule-001 day - 10:05 schedule-002 week Monday 13:15 dory:maintenance workflow-002 schedules>. NAME. frequency minute, halfhour, hour, day, week, month. day hour minute Monday, Tuesday, Wednesday, Thursday, Friday, Saturday Sunday. frequency week month. 00-23. frequency day, week month. 00-59. workflow. CLI.. offset. period. unit seconds month. 1 61

.. // Example of using Schedule definitions within a workflow var MyTextObject = { MyVersion: 1.0, MyName: Example 9, MyDescription: Example of use of Timer, Origin: Oracle }; var MySchedules = [ // half hr interval { offset: 0, period: 1800, units: "seconds" }, // offset 2 days, 4hr, 30min, week interval {offset: 2*24*60*60+4*60*60+30*60, period: 604800,units: "seconds" } ]; var workflow = { name: MyTextObject.MyName, description: MyTextObject.MyDescription, version: MyTextObject.MyVersion, alert: false, setid: true, schedules: MySchedules, scheduled: true, origin: MyTextObject.Origin, execute: function () { audit( workflow started for timer; ); } } }; MySchedules units offset period. seconds month. period offset., 4:30. schedules. 0. CLI. dory:> maintenance workflows dory:maintenance workflows>list WORKFLOW NAME OWNER SETID ORIGIN workflow-002 Example 9 root true Oracle dory:maintenance workflows> select workflow-002 dory:maintenance workflow-002> schedules dory:maintenance workflow-002 schedules> list NAME FREQUENCY DAY HH:MM schedule-000 halfhour - --:00 schedule-001 week Saturday 04:30 dory:maintenance workflow-002 schedules> :. 62 Sun ZFS Storage 7000 2012 12 E38995-01

if (get( use ) == params.type) { params.disks.push(name + / + get( label )); } } run( cd.. ); } run( cd../.. ); if (params.disks.length === 0) throw ( No + params.type + disks found ); run( cd / ); } }, { step: Creating worksheet, execute: function (params) { /* * In this step, we re ready to actually create the worksheet * itself: we have the disks of the specified type and * we know that we can create the worksheet. Note that we * create several datasets: first, I/O bytes broken down * by disk, with each disk of the specified type highlighted * as a drilldown. Then, we create a separate dataset for * each disk of the specified type. Finally, note that we * aren t saving the datasets -- we ll let the user do that * from the created worksheet if they so desire. (It would * be straightforward to add a boolean parameter to this * workflow that allows that last behavior to be optionally * changed.) */ var disks = [], i; run( analytics worksheets ); run( create " + params.worksheet + " ); run( select name=" + params.worksheet + " ); run( dataset ); run( set name=io.bytes[disk] ); for (i = 0; i < params.disks.length; i++) disks.push( " + params.disks[i] + " ); run( set drilldowns= + disks.join(, )); run( commit ); } }]; for (i = 0; i < params.disks.length; i++) { run( dataset ); run( set name="io.bytes[disk= + params.disks[i] + ]" ); run( commit ); } var workflow = { name: Disk drilldown, description: Creates a worksheet that drills down on system, + cache, or log devices, 64 Sun ZFS Storage 7000 2012 12 E38995-01

}; parameters: { type: { label: Create a new worksheet drilling down on, type: ChooseOne, options: [ cache, log, system ], optionlabels: [ Cache, Log, System ] }, overwrite: { label: Overwrite the worksheet if it exists, type: Boolean } }, execute: function (params) { return (steps); } BUI. CLI CLI maintenance workflows. download.. dory:maintenance workflows> download dory:maintenance workflows download (uncommitted)> get url = (unset) user = (unset) password = (unset) "url" URL.. URL HTTP("http://" ) FTP("ftp://" ). URL ( : "ftp://myusername:mypasswd@myserver/export/foo"), URL. dory:maintenance workflows download (uncommitted)> set url= ftp://foo/example1.akwf url = ftp://foo/example1.akwf dory:maintenance workflows download (uncommitted)> set user=bmc user = bmc dory:maintenance workflows download (uncommitted)> set password Enter password: password = ******** dory:maintenance workflows download (uncommitted)> commit Transferred 138 of 138 (100%)... done 1 65

maintenance workflows list. dory:maintenance workflows> list WORKFLOW NAME OWNER SETID ORIGIN workflow-000 Hello world root false <local> select. dory:maintenance workflows> select workflow-000 dory:maintenance workflow-000> get. dory:maintenance workflow-000> get name = Hello world description = Bids a greeting to the world owner = root origin = <local> setid = false alert = false scheduled = false execute.. dory:maintenance workflow-000> execute hello world!. dory:maintenance workflow-000> execute dory:maintenance workflow-000 execute (uncommitted)> get type = (unset) overwrite = (unset). dory:maintenance workflow-000 execute (uncommitted)> commit error: cannot execute workflow without setting property "type" commit. 66 Sun ZFS Storage 7000 2012 12 E38995-01

dory:maintenance workflow-000 execute (uncommitted)> set type=system type = system dory:maintenance workflow-000 execute (uncommitted)> set overwrite=true overwrite = true dory:maintenance workflow-000 execute (uncommitted)> commit CLI. : dory:maintenance workflow-000 execute (uncommitted)> commit Checking for existing worksheet... done Finding disks of specified type... done Creating worksheet... done 1 67

2 2 Status Status Sun Storage 7000 Status > Dashboard Status.. Status > Dashboard,,,,. Status > Settings Dashboard Dashboard. 69

Dashboard Status > NDMP NDMP NDMP. Dashboard Dashboard Status Dashboard BUI( ). Dashboard 100 BUI,. Dashboard. Usage Dashboard Usage.. 70 Sun ZFS Storage 7000 2012 12 E38995-01

Dashboard Storage Usage.. Shares Storage. Memory. Analytics Memory. Used Avail.. Shares. Compression. 1x. Dedup. 1x. (RAM) Cache. Unused.. Mgmt Other.. 2 Status 71

Dashboard (RAM) Kernel. analytics/component create+read. Dashboard. Services Dashboard.....,,, Logs Properties. Hardware Dashboard. 72 Sun ZFS Storage 7000 2012 12 E38995-01

Dashboard. Hardware Maintenance. Dashboard 8. Disk /.. Analytics 4 (,,, )... Status Settings. 7 day graph (7d). 24 hour graph (24h) 1. 60 minute graph (60m) 1 (24 1 ). 1 second graph.. (7d, 24h 60m). 2 Status 73

Dashboard,.. 100%. 60 60..., 1000 NFSv3 /. 24 7.. 60. 60. 7 7d.....BUI.. ( ). Dashboard / / Analytics. 4.. 74 Sun ZFS Storage 7000 2012 12 E38995-01

Dashboard CLI CLI status dashboard Status > Dashboard. walu:> status dashboard Storage: pool_0: Used 10.0G bytes Avail 6.52T bytes State online Compression 1x Memory: Cache Unused Mgmt Other Kernel 550M bytes 121G bytes 272M bytes 4.10G bytes 1.90G bytes Services: ad disabled smb disabled dns online ftp disabled http online identity online idmap online ipmp online iscsi online ldap disabled ndmp online nfs online nis online ntp online routing online scrk maintenance snmp online ssh online tags online vscan online Hardware: CPU online Cards online Disks faulted Fans online Memory online PSU online Activity: CPU 1 %util Sunny Disk 32 ops/sec Sunny iscsi 0 ops/sec Sunny NDMP 0 bytes/sec Sunny NFSv3 0 ops/sec Sunny NFSv4 0 ops/sec Sunny Network 13K bytes/sec Sunny SMB 0 ops/sec Sunny Recent Alerts: 2009-10-13 07:46: A cluster interconnect link has been restored. BUI. aalib. CLI BUI. status activity show. 2 Status 75

Settings caji:> status activity show Activity: CPU 10 %util Sunny Disk 478 ops/sec Partly Cloudy iscsi 0 ops/sec Sunny NDMP 0 bytes/sec Sunny NFSv3 681 ops/sec Partly Cloudy NFSv4 0 ops/sec Sunny Network 22.8M bytes/sec Partly Cloudy SMB 0 ops/sec Sunny caji:> Dashboard (24x7). ( ),.. Dashboard. Firefox. 1 2 3 about:config. "memory". browser.cache.memory.enable = false. Settings Status > Settings Status Dashboard. BUI 76 Sun ZFS Storage 7000 2012 12 E38995-01

Settings Layout Layout Dashboard ( ). <empty> -. SMB / SMB. CPU CPU. CPU. Disk /. HTTP / HTTP. iscsi / iscsi. FC /. Network NDMP / /. / NDMP. NFSv2 / NFSv2. NFSv3 / NFSv3. NFSv4 / NFSv4. FTP SFTP / FTP. / SFTP. Dashboard "<empty>". 2 Status 77

Settings Thresholds Thresholds Dashboard.. Dashboard (60 )., CPU 41% Cloudy. 40%( ). Custom.. CLI CLI.BUI CLI. ( ). BUI 1 2 Status > Settings > Layout. Dashboard. 78 Sun ZFS Storage 7000 2012 12 E38995-01

NDMP 3 Apply. 1 2 3 4 5 Status > Settings >Thresholds.. Custom.. Kilo/Mega/Giga Units. Apply. NDMP BUI NDMP NDMP.. NDMP. Type NDMP, Path NDMP /dev/rmt/0n Vendor STK Product SL500 NDMP. 2 Status 79

NDMP ID NDMP ID 49 Active Remote Client NDMP 192.168.1.219:4760 Authenticated, Data State,... Mover State Mover,... Current Operation NDMP,, Progress NDMP.. Active:. Idle:. Connected:. Halted:. Listen:. NDMP Mover NDMP.. Active:. Idle:. Paused:. Halted: /. Listen:. CLI CLI NDMP. 80 Sun ZFS Storage 7000 2012 12 E38995-01

3 3 Configuration Configuration Configure Network,. Initial - Network - Services - SAN - SAN(Storage Area Network) Cluster - Users - Preferences - Alerts - Storage - 81

Shares. Maintenance System Backup. 6. 1. 2. DNS 3. 4. (NIS, LDAP, Active Directory) 5. 6.. BUI....,.... Maintenance > System "INITIAL SETUP" CLI maintenance system setup. 82 Sun ZFS Storage 7000 2012 12 E38995-01

BUI BUI.,, Start. Commit. Commit. NIC.BUI Allow Admin 215 BUI ssh 22 CLI. NIC. NIC. CLI CLI. help. done. 3 Configuration 83

CLI : caji console login: root Password: Last login: Sun Oct 19 02:55:31 on console To setup your system, you will be taken through a series of steps; as the setup process advances to each step, the help message for that step will be displayed. Press any key to begin initial configuration... (DHCP ) done. (, ) help... aksh: starting configuration with "net"... Configure Networking. Configure the appliance network interfaces. The first network interface has been configured for you, using the settings you provided at the serial console. Subcommands that are valid in this context: datalinks devices interfaces help [topic] show abort done => Manage datalinks => Manage devices => Manage interfaces => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help" or "script". => Show information pertinent to the current context => Abort this task (potentially resulting in a misconfigured system) => Finish operating on "net" caji:maintenance system setup net> devices show Devices: DEVICE UP MAC SPEED nge0 true 0:14:4f:8d:59:aa 1000 Mbit/s nge1 false 0:14:4f:8d:59:ab 0 Mbit/s nge2 false 0:14:4f:8d:59:ac 0 Mbit/s nge3 false 0:14:4f:8d:59:ad 0 Mbit/s caji:maintenance system setup net> datalinks show Datalinks: 84 Sun ZFS Storage 7000 2012 12 E38995-01

DATALINK CLASS LINKS LABEL nge0 device nge0 Untitled Datalink caji:maintenance system setup net> interfaces show Interfaces: INTERFACE STATE CLASS LINKS ADDRS LABEL nge0 up ip nge0 192.168.2.80/22 Untitled Interface caji:maintenance system setup net> done DNS DNS. Configure DNS. Configure the Domain Name Service. Subcommands that are valid in this context: help [topic] show commit abort done get [prop] set [prop] => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help", "script" or "properties". => Show information pertinent to the current context => Commit current state, including any changes => Abort this task (potentially resulting in a misconfigured system) => Finish operating on "dns" => Get value for property [prop]. ("help properties" for valid properties.) If [prop] is not specified, returns values for all properties. => Set property [prop] to [value]. ("help properties" for valid properties.) For properties taking list values, [value] should be a comma-separated list of values. caji:maintenance system setup dns> show Properties: <status> = online domain = sun.com servers = 192.168.1.4 caji:maintenance system setup dns> set domain=sf.fishworks.com domain = sf.fishworks.com (uncommitted) caji:maintenance system setup dns> set servers=192.168.1.5 servers = 192.168.1.5 (uncommitted) caji:maintenance system setup dns> commit caji:maintenance system setup dns> done aksh: done with "dns", advancing configuration to "ntp"... NTP(Network Time Protocol). NTP. 3 Configuration 85

Configure Time. Configure the Network Time Protocol. Subcommands that are valid in this context: help [topic] show commit abort done enable disable get [prop] set [prop] => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help", "script" or "properties". => Show information pertinent to the current context => Commit current state, including any changes => Abort this task (potentially resulting in a misconfigured system) => Finish operating on "ntp" => Enable the ntp service => Disable the ntp service => Get value for property [prop]. ("help properties" for valid properties.) If [prop] is not specified, returns values for all properties. => Set property [prop] to [value]. ("help properties" for valid properties.) For properties taking list values, [value] should be a comma-separated list of values. caji:maintenance system setup ntp> set servers=0.pool.ntp.org servers = 0.pool.ntp.org (uncommitted) caji:maintenance system setup ntp> commit caji:maintenance system setup ntp> done aksh: done with "ntp", advancing configuration to "directory"... NIS, LDAP Active Directory. Configure Name Services. Configure directory services for users and groups. You can configure and enable each directory service independently, and you can configure more than one directory service. Subcommands that are valid in this context: nis ldap ad help [topic] show abort => Configure NIS => Configure LDAP => Configure Active Directory => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help" or "script". => Show information pertinent to the current context => Abort this task (potentially resulting in a misconfigured system) 86 Sun ZFS Storage 7000 2012 12 E38995-01

done => Finish operating on "directory" caji:maintenance system setup directory> nis caji:maintenance system setup directory nis> show Properties: <status> = online domain = sun.com broadcast = true ypservers = caji:maintenance system setup directory nis> set domain=fishworks domain = fishworks (uncommitted) caji:maintenance system setup directory nis> commit caji:maintenance system setup directory nis> done caji:maintenance system setup directory> done aksh: done with "directory", advancing configuration to "support"... LUN.. Configure Storage. Subcommands that are valid in this context: help [topic] show commit done config <pool> unconfig add import scrub <start stop> get [prop] set pool=[pool] => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help", "script" or "properties". => Show information pertinent to the current context => Commit current state, including any changes => Finish operating on "storage" => Configure the storage pool => Unconfigure the storage pool => Add additional storage to the storage pool => Search for existing or destroyed pools to import => Start or stop a scrub => Get value for property [prop]. ("help properties" for valid properties.) If [prop] is not specified, returns values for all properties. => Change current pool caji:maintenance system setup storage> show Properties: pool = pool-0 status = online profile = mirror log_profile = - cache_profile = - 3 Configuration 87

Network caji:maintenance system setup storage> done aksh: done with "storage", advancing configuration to "support"... (Phone Home). Remote Support. Register your appliance and configure remote monitoring. Subcommands that are valid in this context: tags scrk help [topic] show abort done => Configure service tags => Configure phone home => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help" or "script". => Show information pertinent to the current context => Abort this task (potentially resulting in a misconfigured system) => Finish operating on "support" caji:maintenance system setup support> done aksh: initial configuration complete! Network Network Networking Configuration, VLAN( LAN). IPv4 IPv6. 88 Sun ZFS Storage 7000 2012 12 E38995-01

Network 4. -. IPoIB(IP on InfiniBand). -. ( ) IB 1:1, VLAN. -IP. IP IPMP(IP ). -IP. IP.,. 2, LACP. IP 3. ( 2 3 ). IP ( ). nge0 datalink1 deimos(192.168.2.80/22). nge1, nge2, nge3 aggr1(lacp ) phobos(192.168.2.81/22) ("aggr1" ) (LACP ). ("phobos" ) IP. ( "nge1", "nge2" ).. 3 Configuration 89

Network InfiniBand,... VLAN - LAN. LACP -. ( ) ( ) LACP. IB - IB InfiniBand.. Name Speed Duplex VLAN VLAN ID. : "internal", "external", "adminnet". auto( ), 10( 10Mbit/ ), 100( 100Mbit/ ), 1000( 1Gbit/ ) 10000( 10Gbit/ ).. /.... CLI auto( ), half( ) full( ).. VLAN. VLAN ID 90 Sun ZFS Storage 7000 2012 12 E38995-01

Network MTU LACP Aggregation LACP Policy LACP Mode LACP Timer IB Partition Partition Key MTU 1500. MTU( 9000 ). MTU MTU MTU. LAN MTU. MTU MTU ( ). MTU VLAN. MTU Q3.2010 "". "" CLI...L2MAC L3IP L4.. LACP. LACP.. Cisco Etherchannel LACP. LACP LACP "". LACP. IB. ( ). (pkey). pkey GUID " ".HCA IPMP. IP.. IPv4 IPv6 IPMP - IP IP. 3 Configuration 91

Network Name Allow Administration Enable Interface IPv4 Configure with IPv4 Address/Mask IPv6 Configure with IPv6 Address/Mask IP MultiPathing Group. BUI CLI.. IP. IP / IP. IPMP IP. " " "DHCP". CIDR (192.168.1.1/24)IPv4. " " - - (IPv6 ) "IPv6 ". CIDR (1080::8:800:200C:417A/32)IPv6. IP. IPMP(IP ) IP IP ( : ) IP. IP IP. (LAN, IB VLAN)IP IPMP IPMP. IPMP IP <i>active</i> <i>standby</i>. Active: IPMP IP IP. Standby: IP. IP IPMP IP. IPMP., IPMP. 92 Sun ZFS Storage 7000 2012 12 E38995-01

Network... IP,.,IP ( ) 5. ICMP 224.0.01(IPv4) ff02::1(ipv6) 5. IPMP ICMP. IPMP IPv4 IPv6.IPv6 -.. 5 IP., 10 IP. IPMP IPMP.., 10 2 20..., IP.,. IPMP. IPMP. IPMP.. NIC. IPMP IPMP., IP IP NIC IPMP. 3 Configuration 93

Network,. IP. IP (, ). IP....,. Destination IP (CIDR ) 192.168.0.0/22 Gateway (IP )(" ", ) 192.168.2.80 Family IPv4, IPv6 Type dhcp, static, system Interface IP nge0 "Destination" 0.0.0.0/0 ''.BUI. Kind, 94 Sun ZFS Storage 7000 2012 12 E38995-01

Network "Interface" IP. IPMP IPMP IP, IP., IP IPMP "Interface".., "Type"... Static System DHCP Dynamic. IP. IP. "Gateway" IP. DHCP IP.DHCP DHCP. RIP RIPng ( ).. Inactive IP. Multihoming model IP IP. "loose", "adaptive" "strict".. IP IP 3 Configuration 95

Network., IP IP IP... Loose Adaptive Strict IP IP.1) IP IP IP.2)IP IP IP IP IP.. IP.1) IP IP IP.2)IP IP IP... IP IP.1) IP IP IP IP.2) IP IP IP IP... IP BUI., "Loose". "Adaptive" "Strict" IP. IP "Adaptive", "Strict". BUI BUI. 96 Sun ZFS Storage 7000 2012 12 E38995-01

Network. IP. CLI. Configuration -> Network. / / / / / / I/O (,?) InfiniBand I/O InfiniBand InfiniBand (, ) InfiniBand InfiniBand ( ) VLAN VLAN IB ( ) 3 Configuration 97

Network ( ) IP Configuration, Addresses Routing,. Configuration Configuration Devices, Datalinks Interfaces.,. Devices.. IP. Datalink Interface....... Network Datalink datalink1(nge0 ) Network Interface IPv4 DHCP( 1 ) Network Addresses 192.168.2.80/22 Host Names caji.sf.example.com 98 Sun ZFS Storage 7000 2012 12 E38995-01

Network IP.. CLI configuration net devices, datalinks, interfaces routing. show. caji:> configuration net caji:configuration net> devices show Devices: DEVICE UP SPEED MAC nge0 true 1000 Mbit/s 0:14:4f:9a:b9:0 nge1 true 1000 Mbit/s 0:14:4f:9a:b9:1 nge2 true 1000 Mbit/s 0:14:4f:9a:b8:fe nge3 true 1000 Mbit/s 0:14:4f:9a:b8:ff caji:configuration net> datalinks show Datalinks: DATALINK CLASS LINKS LABEL nge0 device nge0 datalink1 caji:configuration net> interfaces show Interfaces: INTERFACE STATE CLASS LINKS ADDRS LABEL nge0 up ip nge0 192.168.2.80/22 caji caji:configuration net> routing show Properties: multihoming = loose Routes: ROUTE DESTINATION GATEWAY INTERFACE TYPE route-000 0.0.0.0/0 192.168.1.1 nge0 dhcp route-001 192.168.0.0/22 192.168.2.142 nge0 system, help. device ip. caji:configuration net> datalinks caji:configuration net datalinks> device caji:configuration net datalinks device (uncommitted)> set links=nge1 3 Configuration 99

Network links = nge1 (uncommitted) caji:configuration net datalinks device (uncommitted)> set label=datalink2 label = datalink2 (uncommitted) caji:configuration net datalinks device (uncommitted)> set mtu=9000 mtu = 9000 (uncommitted) caji:configuration net datalinks device (uncommitted)> commit caji:configuration net datalinks> show Datalinks: DATALINK CLASS LINKS LABEL nge0 device nge0 datalink1 nge1 device nge1 datalink2 caji:configuration net datalinks> cd.. caji:configuration net> interfaces caji:configuration net interfaces> ip caji:configuration net interfaces ip (uncommitted)> set label="caji2" label = caji2 (uncommitted) caji:configuration net interfaces ip (uncommitted)> set links=nge1 links = nge1 (uncommitted) caji:configuration net interfaces ip (uncommitted)> set v4addrs=10.0.1.1/8 v4addrs = 10.0.1.1/8 (uncommitted) caji:configuration net interfaces ip (uncommitted)> commit caji:configuration net interfaces> show Interfaces: INTERFACE STATE CLASS LINKS ADDRS LABEL nge0 up ip nge0 192.168.2.80/22 caji nge1 up ip nge1 10.0.1.1/8 caji2 nge1 IP 10.0.1.2. caji:configuration net routing> create caji:configuration net route (uncommitted)> set family=ipv4 family = IPv4 (uncommitted) caji:configuration net route (uncommitted)> set destination=0.0.0.0 destination = 0.0.0.0 (uncommitted) caji:configuration net route (uncommitted)> set mask=0 mask = 0 (uncommitted) caji:configuration net route (uncommitted)> set interface=nge1 interface = nge1 (uncommitted) caji:configuration net route (uncommitted)> set gateway=10.0.1.2 gateway = 10.0.1.2 (uncommitted) caji:configuration net route (uncommitted)> commit BUI 1 Datalink. 100 Sun ZFS Storage 7000 2012 12 E38995-01

Network 2 3 4 MTU ( 9000 ). Devices. "APPLY". Datalinks. 5 Interface. 6 7 8. "APPLY". Interfaces.. "APPLY". 1 2 3 4.. "APPLY". "APPLY"., 1 ( ). 2 3 4 5 6 7 Datalink. MTU. "APPLY". Interfaces. "APPLY". "APPLY". 3 Configuration 101

Network LACP 1 Datalinks. 2 3 4. LACP Aggregation. Devices 2 "APPLY". 5 Interfaces. 6 7 Datalinks "APPLY". "APPLY". 1 IPMP IPMP "" IP. IP ( ). 2 Interface. 3 4 5 6 7 8. IP MultiPathing Group. UseIPv4Protocol / UseIPv6Protocol IPMP IP. Interfaces 1. "Active" "Standby". "APPLY". 1 IPMP IP 0.0.0.0/8 IPMP "" IP ( ). 102 Sun ZFS Storage 7000 2012 12 E38995-01

Network 2 Interface. 3 4 5 6 7 8 1 2 3 1 2 3. IP MultiPathing Group. UseIPv4Protocol / UseIPv6Protocol IPMP IP. Interfaces 1. "Active" "Standby". "APPLY". LACP Devices.. "APPLY". IPMP Interfaces. IPMP. "APPLY". InfiniBand 1 Datalink. 2 3 4 5. IB Partition. Partition Devices. "APPLY". Datalinks. 3 Configuration 103

Network 6 Interface. 7 8 9. "APPLY". Interfaces.. "APPLY". 1 2 3 4 Configuration->Network->Routing... "ADD".. 1 2 Configuration->Network->Routing.. CLI 1 2 3 4 configuration net routing. create. show set. commit. 104 Sun ZFS Storage 7000 2012 12 E38995-01

Storage 1 2 3 configuration net routing. show ( : route-002). destroy route name. 1 2 3 multihoming strict configuration net routing. set multihoming=strict. commit. Q3.2010 InfiniBand InfiniBand. Q3.2010 InfiniBand. Q3.2010 InfiniBand ( ). InfiniBand... Storage 3 Configuration 105

Storage., LUN. LUN Shares..,.. RAID-Z... Sun Storage 7000,. 106 Sun ZFS Storage 7000 2012 12 E38995-01

Storage, LUN. 80% 90%. 20%. Shares UI..BUI.. CLI config.... 1-- JBOD " " ( ). ZFSSA. 1--. 2--, SAS (HBA ) JBOD.. 3--, ZFS.,... 3 Configuration 107

Storage.. JBOD JBOD. JBOD..... 0.. SAS-1 JBOD(, J4400 J4500). JBOD.,JBOD '' ''. JBOD NSPF JBOD. SAS-2 JBOD.,. JBOD 8. 108 Sun ZFS Storage 7000 2012 12 E38995-01

Storage RAS... 3 RAID 3 ( : ).. 3 IOPS. 2 RAID.3. RAID, ( : ). RAID,. ( : ). 3 1 RAID., RAID.. RAID. RAID. 3 Configuration 109

Storage 3 RAID,... RAID. 3 RAID.. I/O. RAID.. 'NSPF'. NSPF ' ', JBOD RAID.. JBOD,. NSPF ( ) ( JBOD ). NSPF..,... ( )....... :. 3.. 110 Sun ZFS Storage 7000 2012 12 E38995-01

Storage....BUI 'IMPORT', CLI 'import'....,bui CLI 'name'..... RAID-Z NSPF config JBOD NSPF.JBOD JBOD RAID NSPF. LUN.,.. BUI.. 3 Configuration 111

SAN BUI Configuration->Storage. 1. 2 3 4 5 6. "Allocate and verify storage" JBOD. JBOD none, half all. JBOD JBOD. "COMMIT". "Configure Added Storage".,. "COMMIT". SAN SAN SAN SAN(Storage Area Network). SAN 3. 3.. 112 Sun ZFS Storage 7000 2012 12 E38995-01

SAN SAN.. LUN. LUN. LUN SCSI SCSI, SCSI. SCSI I/O.. 0..LUN. SCSI, SCSI I/O... LUN LUN. SCSI. FC iscsi iser SRP SAN(Storage Area Network) SCSI,,FC HBA. SCSI IP SCSI. RDMA iscsi,rdma (, InfiniBand) iscsi. iser IB iscsi. CLI BUI iser ( ) iscsi. SCSI RDMA RDMA (, InfiniBand) SCSI.. (FC, iscsi SRP). 3 Configuration 113

SAN LUN. LUN LUN.LUN LUN. LUN LUN.LUN LUN., LUN. BUI Configuration > SAN. Target Initiator. Fibre Channel Ports, iscsi Targets SRP Targets. 114 Sun ZFS Storage 7000 2012 12 E38995-01

SAN Fibre Channel FC. Shares > Shares > Protocols. Target Group Initiator Group LUN. CLI CLI configuration san. shares CLI LUN. 3 Configuration 115

SAN SAN(Storage Area Network).. SCSI SCSI iscsi iser FC SRP IQN SCSI SCSI I/O. SCSI. SCSI 0. SCSI SCSI, SCSI I/O. SCSI (SCSI ).. LUN. LUN. LUNSCSI SCSI, SCSI. SCSI IP SCSI. RDMA iscsi,rdma (, InfiniBand) iscsi. iser IB iscsi. CLI BUI iser ( ) iscsi. SAN(Storage Area Network) SCSI,,FC HBA. SCSI RDMA RDMA (, InfiniBand) SCSI. iscsi Qualified Name iscsi. iscsi IQN iqn.date.authority:uniqueid. iqn.1986-03.com.sun:02:c7824a5b-f3ea-6038-c79d-ca443337d92c IQN iscsi. 1986 3 iscsi. DNS ( "com.sun"). DNS Sun ID. 116 Sun ZFS Storage 7000 2012 12 E38995-01

FC CHAP RADIUS iscsi, IP TCP. iscsi,..,. iscsi. Challenge-Handshake,. CHAP..LUN.. LUN LUN. LUN.. FC FC( ) SCSI.FC FC LUN, FC ( ). FC SAN.FC FC... root. FC WWN(World Wide Name), FC LUN LUN 3 Configuration 117

FC.FC. Sun ZFS Storage Appliance FC SAN Implementing Fibre Channel SAN Boot with Oracle's Sun ZFS Storage Appliance (http://www.oracle.com/technetwork/articles/servers-storage-admin/ fbsanboot-365291.html (http://www.oracle.com/ technetwork/articles/servers-storage-admin/fbsanboot-365291.html)). LUN 2 ( ). ( ) LUN, ( ). ( ) ( ). I/O. ALUA( ) ALUA. WWN. WWN WWNFC..LUN LUN. FC SAN LUN LUN. ALUA. ALUA FC ALUA. FC analytics., LUN. 118 Sun ZFS Storage 7000 2012 12 E38995-01

FC,, SCSI FC. LUN. FC. SCSI.. - FC FC LUN ( : - ) 3 Configuration 119

FC. (2048) LUN. LUN LUN. - 7410 ( FC 2 LUN 100, FC 4 LUN 28 ), 2 2048 LUN 100 +28, LUN 32. Solaris ssd_max_throttle. FC.0. BUI FC FC BUI Configuration > SAN FC Target. root. Target.. Target Apply... 120 Sun ZFS Storage 7000 2012 12 E38995-01

FC FC. FC Discovered Ports. Discovered Ports WWN. FC Initiators.. WWN Alias. 3 Configuration 121

FC Initiators FC Initiator Groups. Apply. LUN. FC LUN LUN LUN. Create LUN. Apply LUN. CLI FC dory:configuration san targets fc> set targets="wwn.2101001b32a11639" targets = wwn.2101001b32a11639 (uncommitted) dory:configuration san targets fc> commit 122 Sun ZFS Storage 7000 2012 12 E38995-01

FC FC dory:configuration san targets fc> show Properties: targets = wwn.2100001b32811639,wwn.2101001b32a12239 Targets: 5: Port 1 target-001 initiator wwn.2101001b32a11639 PCIe 5: Port 2 0 Gbit/s target-002 initiator wwn.2100001b32812239 PCIe 2: Port 1 0 Gbit/s target-003 target wwn.2101001b32a12239 PCIe 2: Port 2 0 Gbit/s dory:configuration san targets fc> select target-000 dory:configuration san targets fc target-000> show Properties: wwn = wwn.2100001b32811639 port = PCIe 5: Port 1 mode = target NAME target-000 MODE target WWN wwn.2100001b32811639 PORT PCIe SPEED 4 Gbit/s speed = 4 Gbit/s discovered_ports = 6 link_failure_count = 0 loss_of_sync_count = 0 loss_of_signal_count = 0 protocol_error_count = 0 invalid_tx_word_count = 0 invalid_crc_count = 0 Ports: PORT WWN ALIAS MANUFACTURER port-000 wwn.2100001b3281a339 longjaw-1 QLogic Corporation port-001 wwn.2101001b32a1a339 longjaw-2 QLogic Corporation port-002 wwn.2100001b3281ac39 thicktail-1 QLogic Corporation port-003 wwn.2101001b32a1ac39 thicktail-2 QLogic Corporation port-004 wwn.2100001b3281e339 <none> QLogic Corporation port-005 wwn.2101001b32a1e339 <none> QLogic Corporation FC dory:configuration san initiators fc groups> create dory:configuration san initiators fc group (uncommitted)> set name=lefteye dory:configuration san initiators fc group (uncommitted)> set initiators=wwn.2101001b32a1ac39,wwn.2100001b3281ac39 dory:configuration san initiators fc group (uncommitted)> commit dory:configuration san initiators fc groups> list GROUP NAME group-001 lefteye +-> INITIATORS wwn.2101001b32a1ac39 wwn.2100001b3281ac39 FC LUN lefty LUN fera. dory:shares default> lun lefty dory:shares default/lefty (uncommitted)> set volsize=10 volsize = 10 (uncommitted) 3 Configuration 123

FC dory:shares default/lefty (uncommitted)> set initiatorgroup=fera initiatorgroup = default (uncommitted) dory:shares default/lefty (uncommitted)> commit CLI CLI. script /** This script creates both aliases for initiators and initiator * groups, as specified by the below data structure. In this * particular example, there are five initiator groups, each of * which is associated with a single host (thicktail, longjaw, etc.), * and each initiator group consists of two initiators, each of which * is associated with one of the two ports on the FC HBA. (Note that * there is nothing in the code that uses this data structure that * assumes the number of initiators per group.) */ groups = { thicktail: { thicktail-1 : wwn.2100001b3281ac39, thicktail-2 : wwn.2101001b32a1ac39 }, longjaw: { longjaw-1 : wwn.2100001b3281a339, longjaw-2 : wwn.2101001b32a1a339 }, tecopa: { tecopa-1 : wwn.2100001b3281e339, tecopa-2 : wwn.2101001b32a1e339 }, spinedace: { spinedace-1 : wwn.2100001b3281df39, spinedace-2 : wwn.2101001b32a1df39 }, fera: { fera-1 : wwn.2100001b32817939, fera-2 : wwn.2101001b32a17939 } }; for (group in groups) { initiators = []; for (initiator in groups[group]) { printf( Adding %s for %s..., groups[group][initiator], initiator); try { run( select alias= + initiator); printf( (already exists)\n ); run( cd.. ); } catch (err) { if (err.code!= EAKSH_ENTITY_BADSELECT) throw err; run( create ); set( alias, initiator); set( initiator, groups[group][initiator]); 124 Sun ZFS Storage 7000 2012 12 E38995-01

FCMPxIO } run( commit ); printf( done\n ); } run( select alias= + initiator); initiators.push(get( initiator )); run( cd.. ); } printf( Creating group for %s..., group); run( groups ); try { run( select name= + group); printf( (already exists)\n ); run( cd.. ); } catch (err) { if (err.code!= EAKSH_ENTITY_BADSELECT) throw err; run( create ); set( name, group); run( set initiators= + initiators); run( commit ); printf( done\n ); } run( cd.. ); FCMPxIO FC Sun ZFS Storage 7000 ALUA( ) FC. http://www.t10.org (http://www.t10.org) t10 SPC(SCSI Primary Commands). FC FC. Solaris MPxIO Solaris x86 SPARC. mpathadm show LU. luxadm display. stmsboot MPxIO /.. 1. MPxIO stmsboot -D fp -e, 2. MPxIO stmsboot -D fp -d. 3. mpathadm show LU. 3 Configuration 125

FCMPxIO Windows ALUA Windows 2008/R2 MPIO. R2. SP2. 1. FC HBA Windows HBA. 2. Windows Server 2008 MPIO. Windows mpclaim.exe -r -i -a "" SS7000.. MPIO /LUN. 3. Windows SS7000 LUN. Windows. GUI SS7000LUN. 4. SS7000 Microsoft DSM I/O. Multi-Path Disk Device DSM. Windows - Microsoft DSM Windows PathVerifyEnabled MPIO"N".N PathVerificationPeriod. Disabled Enabled PathVerificationPeriod MPIO ( ). PathVerifyEnabled = TRUE. 30 seconds 5 seconds RetryInterval (DSM IO RetryCount ). 1 second 5 seconds RetryCount PDORemovePeriod DSM IO. LUN ( ). 3 300 20 seconds 1500 seconds : Emulex HBA Windows Server 2008: Emulex HBA Windows Server 2008 HBA. / I/O Emulex HBA NodeTimeout 126 Sun ZFS Storage 7000 2012 12 E38995-01

FCMPxIO 0. http://www.emulex.com (http:// www.emulex.com) OCManager Utility. Linux. OEL 5.4(Oracle Enterprise Linux 5.4) OEL 5.5(Oracle Enterprise Linux 5.5) SUSE Linux 11 SP 1 1.. 2. multipathd. service multipathd stop Stopping multipathd daemon: [ OK ] 3. /etc/multipath.conf PRODUCT; CLI. run( cd / ); run( configuration version ); printf(get( product ).replace(/^sun /,"") + \n ); : spadefish:> script ("." to run)> run( cd / ); ("." to run)> run( configuration version ); ("." to run)> printf(get( product ).replace(/^sun /,"") + \n ); ("." to run)>. ZFS Storage 7420 OEL 5.4: device { vendor "SUN" } product "PRODUCT" getuid_callout "/sbin/scsi_id -g -u -s /block/%n" prio_callout "/sbin/mpath_prio_alua /dev/%n" hardware_handler "0" path_grouping_policy group_by_prio failback immediate no_path_retry queue rr_min_io 100 path_checker tur rr_weight uniform 3 Configuration 127

FCMPxIO OEL 5.5: defaults { find_multipaths user_friendly_names } yes yes devices { device { vendor "SUN" product "PRODUCT" getuid_callout "/sbin/scsi_id -g -u -s /block/%n" prio alua hardware_handler "1 alua" path_grouping_policy group_by_prio failback immediate no_path_retry 600 rr_min_io 100 path_checker tur rr_weight uniform features "0" } } SUSE Linux 11 SP 1: defaults { multipath_tool "/sbin/multipath -v0" udev_dir /dev polling_interval 100 default_selector "round-robin 0" default_path_grouping_policy group_by_prio default_getuid_callout "/lib/udev/scsi_id -g -u -d /dev/%n" default_prio_callout "/bin/true" prio "alua" default_features "0" rr_min_io 100 failback immediate user_friendly_names yes path_checker tur no_path_retry 1000 } device { vendor product } "SUN" "PRODUCT" 4. multipathd. #service multipathd start Starting multipathd daemon: [ OK ] 5. SCSI multipath I/O. Linux... 128 Sun ZFS Storage 7000 2012 12 E38995-01

FCMPxIO #multipath ll sdd: checker msg is "tur checker reports path is down" mpath1 (3600144f094f0bd0300004b31c88f0001) dm-2 SUN,Sun Storage 7410 (or 7310) [size=20g][features=1 queue_if_no_path][hwhandler=0][rw] \_ round-robin 0 [prio=50][active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [prio=0][enabled] \_ 2:0:1:0 sdd 8:48 [failed][faulty] VMware ESX VMware ESX HCL http://www.vmware.com/resources/compatibility/search.php (http://www.vmware.com/ resources/compatibility/search.php) ESX 4.1 : ALUA. ESX 4.0 : 1. esx nmp device list SATP. # esxcli nmp device list naa.600144f0ed81720500004bb3c1f60002 Device Display Name: SUN Fibre Channel Disk (naa.600144f0ed81720500004bb3c1f60002) Storage Array Type: VMW_SATP_DEFAULT_AA Storage Array Type Device Config: Path Selection Policy: VMW_PSP_FIXED Path Selection Policy Device Config: {preferred=vmhba0:c0:t1:l0;current=vmhba0:c0:t1:l0} Working Paths: vmhba0:c0:t1:l0 VMW_SATP_DEFAULT_AA. ALUA. 2. esxcli nmp satp addrule ZFS Storage 7000 ALUA. # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7110" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7210" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7310" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7410" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7120" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7320" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7420" -c "tpgs_on" 3.. # esxcli nmp satp listrules grep SUN VMW_SATP_ALUA SUN Sun Storage 7110 tpgs_on Sun Storage 7000 VMW_SATP_ALUA SUN Sun Storage 7310 tpgs_on Sun Storage 7000 VMW_SATP_ALUA SUN Sun Storage 7410 tpgs_on Sun Storage 7000 VMW_SATP_ALUA SUN Sun Storage 7210 tpgs_on Sun Storage 3 Configuration 129

FCMPxIO 7000 VMW_SATP_ALUA SUN ZFS Storage 7120 tpgs_on ZFS Storage 7000 VMW_SATP_ALUA SUN ZFS Storage 7320 tpgs_on ZFS Storage 7000 VMW_SATP_ALUA SUN ZFS Storage 7420 tpgs_on ZFS Storage 7000 4. VMware ESX Server. esxcli nmp device list. # esxcli nmp device list naa.600144f0ed81720500004bb3c1f60002 Device Display Name: SUN Fibre Channel Disk (naa.600144f0ed81720500004bb3c1f60002) Storage Array Type: VMW_SATP_ALUA Storage Array Type Device Config: {implicit_support=on;explicit_support=off; explicit_allow=on;alua_followover=on;{tpg_id=0,tpg_state=stby}{tpg_id=1,tpg_state=ao}} Path Selection Policy: VMW_PSP_MRU Path Selection Policy Device Config: Current Path=vmhba1:C0:T1:L0 Working Paths: vmhba1:c0:t1:l0. Multipath-tools version 0.4.7 bundled in OEL 5.4 is unable to recognize paths in ALUA standby access state SCSI Test Unit Ready multipath. 2009 4 21 (0.4.8 ). http://christophe.varoqui.free.fr/ (http://christophe.varoqui.free.fr/). git. multipath-tools-0.4.8.tar.bz2 tarball., multipath I/O /. fcinfo man http://docs.sun.com/app/docs/doc/816-5166/fcinfo-1m?l=en&a=view&q=fcinfo (http://docs.sun.com/app/docs/doc/816-5166/fcinfo-1m) Solaris Fibre Channel and Storage Multipathing Administration Guide http://docs.sun.com/source/819-0139-12/ (http://docs.sun.com/source/819-0139-12/) Windows Server High Availability with Microsoft MPIO 130 Sun ZFS Storage 7000 2012 12 E38995-01

iscsi http://www.microsoft.com/downloads/ details.aspx?familyid=cbd27a84-23a1-4e88-b198-6233623582f3&displaylang=en (http://www.microsoft.com/downloads/details.aspx) Using Device-Mapper Multipath - Red Hat http://www.redhat.com/docs/manuals/csgfs/browse/4.6/dm_multipath/index.html (http://www.redhat.com/docs/manuals/csgfs/browse/4.6/dm_multipath/index.html) iscsi SCSI SCSI. iscsi, IP TCP. iscsi,..,. iscsi. IQN(iSCSI Qualified Name) iscsi. iscsi IQN iqn.date.authority:uniqueid. iqn.1986-03.com.sun:02:c7824a5b-f3ea-6038-c79d-ca443337d92c IQN iscsi. 1986 3 iscsi. DNS ( "com.sun"). DNS Sun ID. Target IQN Alias Authentication mode CHAP name IQN.IQN.,. None, CHAP RADIUS. CHAP CHAP. 3 Configuration 131

iscsi CHAP secret Network interfaces CHAP CHAP.. BUI.... LUN. / iscsi. IPMP IPMP..LACP.LACP IPMP IPMP. iscsi. Initiator IQN Alias Use CHAP CHAP name CHAP secret IQN.,. CHAP /. CHAP CHAP. CHAP CHAP. 132 Sun ZFS Storage 7000 2012 12 E38995-01

iscsi iscsi. SAN IQN CHAP CHAP iscsi (LUN) LUN RADIUS CHAP. RADIUS RADIUS. RADIUS. RADIUS CHAP CHAP. IQN CHAP ( ) RADIUS Initiator Initiator. CHAP IQN CHAP Initiator. Initiator CHAP. Solaris iscsi/iser MPxIO MPxIO iscsi (MS/T) Solaris iscsi. NIC IPMP. iscsi iscsi NIC 2.NIC IPMP. iscsi NIC. Solaris iscsi MS/T IPMP. *IPMP NIC IPMP. MPxIO. IPMP MPxIO. *IPMP MPxIO. IPMP MPxIO. 3 Configuration 133

iscsi Solaris iscsi MS/T IPMP SunSolve Infodoc 207607, Understanding an iscsi MS/T multi-path configuration. How to Enable Multiple iscsi Sessions for a Target. http://download.oracle.com/docs/cd/ E19253-01/817-5093/gcawf (http://download.oracle.com/docs/cd/e19253-01/ 817-5093/gcawf) IPMP 6, IP Services. http://download.oracle.com/docs/cd/e19253-01/816-4554/ipmptm-1 (http://download.oracle.com/docs/cd/e19253-01/816-4554/ipmptm-1) iscsi iscsi. iscsi analytics., LUN. BUI Analytics analytics. 1. Analytics. 2. Add Statistic.. 3. Protocols iscsi operations > Broken down by initiator.. 4. analytics. analytics. 134 Sun ZFS Storage 7000 2012 12 E38995-01

iscsi CLI IQN iscsi ahi:configuration san targets iscsi> create ahi:configuration san targets iscsi target (uncommitted)> set alias="target 0" ahi:configuration san targets iscsi target (uncommitted)> set auth=none ahi:configuration san targets iscsi target (uncommitted)> set interfaces=nge1 ahi:configuration san targets iscsi target (uncommitted)> commit ahi:configuration san targets iscsi> list TARGET ALIAS target-000 Target 0 +-> IQN iqn.1986-03.com.sun:02:daf0161f-9f5d-e01a-b5c5-e1efa9578416 IQN RADIUS iscsi ahi:configuration san targets iscsi> create ahi:configuration san targets iscsi target (uncommitted)> set alias="target 1" ahi:configuration san targets iscsi target (uncommitted)> set iqn=iqn.2001-02.com.acme:12345 ahi:configuration san targets iscsi target (uncommitted)> set auth=radius ahi:configuration san targets iscsi target (uncommitted)> set interfaces=nge1 ahi:configuration san targets iscsi target (uncommitted)> commit ahi:configuration san targets iscsi> list TARGET ALIAS target-000 Target 0 +-> IQN iqn.1986-03.com.sun:02:daf0161f-9f5d-e01a-b5c5-e1efa9578416 target-001 Target 1 +-> IQN iqn.2001-02.com.acme:12345 CHAP iscsi ahi:configuration san initiators iscsi> create ahi:configuration san initiators iscsi initiator (uncommitted)> set initiator=iqn.2001-02.com.acme:initiator12345 ahi:configuration san initiators iscsi initiator (uncommitted)> set alias="init 0" ahi:configuration san initiators iscsi initiator (uncommitted)> set chapuser=thisismychapuser ahi:configuration san initiators iscsi initiator (uncommitted)> set chapsecret=123456789012abc ahi:configuration san initiators iscsi initiator (uncommitted)> commit ahi:configuration san initiators iscsi> list NAME ALIAS initiator-000 Init 0 +-> INITIATOR iqn.2001-02.com.acme:initiator12345 3 Configuration 135

SRP iscsi ahi:configuration san targets iscsi groups> create ahi:configuration san targets iscsi group (uncommitted)> set name=tg0 ahi:configuration san targets iscsi group (uncommitted)> set targets=iqn.2001-02.com.acme:12345, iqn.1986-03.com.sun:02:daf0161f-9f5d-e01a-b5c5-e1efa9578416 ahi:configuration san targets iscsi group (uncommitted)> commit ahi:configuration san targets iscsi groups> list GROUP NAME group-000 tg0 +-> TARGETS iqn.2001-02.com.acme:12345 iqn.1986-03.com.sun:02:daf0161f-9f5d-e01a-b5c5-e1efa9578416 iscsi ahi:configuration san initiators iscsi groups> create ahi:configuration san initiators iscsi group (uncommitted)> set name=ig0 ahi:configuration san initiators iscsi group (uncommitted)> set initiators=iqn.2001-02.com.acme:initiator12345 ahi:configuration san initiators iscsi group (uncommitted)> commit ahi:configuration san initiators iscsi groups> list GROUP NAME group-000 ig0 +-> INITIATORS iqn.2001-02.com.acme:initiator12345 SRP SCSI RDMA RDMA (, InfiniBand) SCSI. SRP IPoIB RDMA IB. SRP. SRP. Target EUI Alias EUI(Extended Unique Identifier). EUI SRP HCA GUID.,. 136 Sun ZFS Storage 7000 2012 12 E38995-01

SRP BUI. ( ). SRP I/O. SRP. Initiator EUI Alias EUI(Extended Unique Identifier).,. SRP analytics. analytics. {{Server}}/wiki/images/cfg_san_srp.png. OFED SRP Linux OFED. 1. Linux OFED. http://www.openfabrics.org/ download_linux.htm (http://www.openfabrics.org/download_linux.htm) 2. --all install.pl. all SRP., 'gcc'. 3. SRP. 3 Configuration 137

SRP http://www.openfabrics.org/downloads/ofed/ofed-1.4/ofed-1.4-docs/ srp_release_notes.txt (http://www.openfabrics.org/ downloads/ofed/ofed-1.4/ofed-1.4-docs/srp_release_notes.txt) 4. srp_daemon -n. SRP run_srp_daemon. srp_daemon. /usr/local/ofed/sbin/ <prefix>/sbin/. run_srp_daemon. SRP srp_daemon.sh(/usr/local/ofed/sbin/ <prefix>/sbin/ ). srp_daemon.sh/var/log/srp_daemon.log. 5. InfiniBand. /etc/infiniband/openib.conf SRP_DAEMON_ENABLE "yes". /etc/infiniband/openib.conf SRPHA_ENABLE "yes". SRP High Availability. 6. High-Availability - Automatic. SRPHA_ENABLE "yes" /etc/infiniband/openib.conf. multipathd OpenIB. service restart multipathd /etc/init.d/openibd stop /etc/init.d/openibd start 7. IB HCA. ibstat 8. SRP IO. srp_daemon -a -o -v ibsrpdm 9. SCSI. 138 Sun ZFS Storage 7000 2012 12 E38995-01

SRP cd /sys/class/infiniband_srp/srp-mthca0-1 10. add_target IO. -n. srp_daemon -o -c ibsrpdm -c 11. add_target. echo id_ext=0003ba0001002eac,ioc_guid=0003ba0001002eac,\ dgid=fe800000000000000003ba0001002ead,\ pkey=ffff,service_id=0003ba0001002eac > add_target 12. /var/log/messages SCSI. Linux isscsi.-h -v.. cat /proc/scsi/scsi 13.. mkfs /dev/sdd Linux. OFED 1.5. I/O. : I/O. I/O. Linux SRP. I/O srp_daemon srp_daemon.. srp_daemon -o -c -n -i <ib-device> -p <port-num> srp_daemon. srp_daemon. 3 Configuration 139

SRP : srp_daemon.. /var/log/messages SRP IO. Jan 27 11:57:03 ib-client-2 kernel: host11: SRP abort called Jan 27 11:57:37 ib-client-2 kernel: host11: ib_srp: failed send status 12 Jan 27 11:57:37 ib-client-2 kernel: ib_srp: host11: add qp_in_err timer Jan 27 11:57:37 ib-client-2 kernel: host11: ib_srp: failed send status 5 Jan 27 11:57:38 ib-client-2 kernel: host11: SRP abort called Jan 27 11:57:38 ib-client-2 kernel: host11: SRP reset_device called Jan 27 11:57:38 ib-client-2 kernel: host11: ib_srp: SRP reset_host called state 0 qp_err 1 Jan 27 11:57:58 ib-client-2 kernel: host11: SRP abort called Jan 27 11:57:58 ib-client-2 kernel: host11: SRP reset_device called Jan 27 11:57:58 ib-client-2 kernel: host11: ib_srp: SRP reset_host called state 0 qp_err 1 Jan 27 11:58:02 ib-client-2 kernel: host11: ib_srp: srp_qp_in_err_timer called Jan 27 11:58:02 ib-client-2 kernel: host11: ib_srp: srp_qp_in_err_timer flushed reset - done Jan 27 11:58:02 ib-client-2 kernel: host11: ib_srp: Got failed path rec status -22 Jan 27 11:58:02 ib-client-2 kernel: host11: ib_srp: Path record query failed Jan 27 11:58:02 ib-client-2 kernel: host11: ib_srp: reconnect failed (-22), removing target port. Jan 27 11:58:08 ib-client-2 kernel: scsi 11:0:0:0: scsi: Device offlined - not ready after error recovery Jan 27 11:58:08 ib-client-2 multipathd: sdc: tur checker reports path is down Jan 27 11:58:08 ib-client-2 multipathd: checker failed path 8:32 in map mpath148 Jan 27 11:58:08 ib-client-2 multipathd: mpath148: Entering recovery mode: max_retries=200 Jan 27 11:58:08 ib-client-2 multipathd: mpath148: remaining active paths: 0 Jan 27 11:58:08 ib-client-2 multipathd: sdc: remove path (uevent) Jan 27 11:58:08 ib-client-2 multipathd: mpath148: map in use Jan 27 11:58:08 ib-client-2 multipathd: mpath148: can t flush Jan 27 11:58:08 ib-client-2 multipathd: mpath148: Entering recovery mode: max_retries=200 Jan 27 11:58:08 ib-client-2 multipathd: dm-2: add map (uevent) Jan 27 11:58:08 ib-client-2 multipathd: dm-2: devmap already registered Jan 27 11:58:08 ib-client-2 kernel: scsi 11:0:0:0: scsi: Device offlined - not ready after error recovery Jan 27 11:58:08 ib-client-2 last message repeated 49 times Jan 27 11:58:08 ib-client-2 kernel: scsi 11:0:0:0: rejecting I/O to dead device Jan 27 11:58:08 ib-client-2 kernel: device-mapper: multipath: Failing path 8:32. Jan 27 11:58:08 ib-client-2 kernel: scsi 11:0:0:0: rejecting I/O to dead device : SRP.. /var/log/messages. device-mapper: table: 253:2: multipath: error getting device device-mapper: ioctl: error adding target to table multipath. ib-client-1:~ # multipath -d reload: maguro2lun (3600144f08068363800004b6075db0001) n/a SUN,Sun Storage 7310 140 Sun ZFS Storage 7000 2012 12 E38995-01

SRP [size=40g][features=0][hwhandler=0][n/a] \_ round-robin 0 [prio=50][undef] \_ 18:0:0:1 sde 8:64 [undef][ready] \_ round-robin 0 [prio=1][undef] \_ 17:0:0:1 sdc 8:32 [failed][ghost] :. SRP.. SRP.,,,,. /var/log/messages. Jan 26 17:42:12 mysystem kernel: sd 13:0:0:0: [sdd] Device not ready: Sense Key : Not Ready [current] Jan 26 17:42:12 mysystem kernel: sd 13:0:0:0: [sdd] Device not ready: Add. Sense: Logical unit not accessible, target port in standby state Jan 26 17:42:12 mysystem kernel: end_request: I/O error, dev sdd, sector 512248 Jan 26 17:42:12 mysystem kernel: scsi host13: SRP abort called Jan 26 17:42:12 mysystem kernel: scsi host13: SRP reset_device called Jan 26 17:42:12 mysystem kernel: scsi host13: ib_srp: SRP reset_host called state 0 qp_err 0 Jan 26 17:42:21 mysystem multipathd: 8:48: mark as failed :. VMWare 4.0 VMware Native MultiPath Plugin(nmp), 2. (psp) I/O. # esxcli nmp psp list Name Description VMW_PSP_MRU Most Recently Used Path Selection VMW_PSP_RR Round Robin Path Selection VMW_PSP_FIXED Fixed Path Selection (satp). / ALUA SATP. Sun Storage 7000 /. ESX CLI ALUA 7000 LUN. 3 Configuration 141

SRP # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7110" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7210" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7310" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7410" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7120" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7320" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7420" -c "tpgs_on" options are: -s VMW_SATP_ALUA - for the ALUA SATP -e description of the rule -V Vendor -M Model -c claim option for Target Portal Group (7000 seems to support implicit) LUN LUN. LUN ALUA.LUN ESX. ESX VMW_SATP_ALUE LUN. # esxcli nmp device list naa.600144f096bb823800004b707f2d0001 Device Display Name: Local SUN Disk (naa.600144f096bb823800004b707f2d0001) Storage Array Type: VMW_SATP_ALUA Storage Array Type Device Config: {implicit_support=on;explicit_support=off;explicit_allow=on; alua_followover=on; {TPG_id=0,TPG_state=AO}{TPG_id=1,TPG_state=STBY}} Path Selection Policy: VMW_PSP_MRU Path Selection Policy Device Config: Current Path=vmhba_mlx4_1.1.1:C0:T1:L0 Working Paths: vmhba_mlx4_1.1.1:c0:t1:l0 LUN Active Standby. # esxcli nmp path list gsan.80fe53553e0100282100-gsan.80fe8f583e0100282100 -naa.600144f096bb823800004b707f2d0001 Runtime Name: vmhba_mlx4_1.1.1:c0:t2:l0 Device: naa.600144f096bb823800004b707f2d0001 Device Display Name: Local SUN Disk (naa.600144f096bb823800004b707f2d0001) Group State: standby Storage Array Type Path Config: {TPG_id=1,TPG_state=STBY,RTP_id=256,RTP_health=UP} Path Selection Policy Path Config: {non-current path} gsan.80fe53553e0100282100-gsan.80fe73583e0100282100 -naa.600144f096bb823800004b707f2d0001 Runtime Name: vmhba_mlx4_1.1.1:c0:t1:l0 Device: naa.600144f096bb823800004b707f2d0001 Device Display Name: Local SUN Disk (naa.600144f096bb823800004b707f2d0001) Group State: active Storage Array Type Path Config: 142 Sun ZFS Storage 7000 2012 12 E38995-01

SRP {TPG_id=0,TPG_state=AO,RTP_id=2,RTP_health=UP} Path Selection Policy Path Config: {current path} VMWare ESX 4.0. esxcl nmp path list SRP. [root@ib-client-5 vmware]# esxcli nmp path list gsan.80fe53553e0100282100-gsan.80fe8f583e0100282100- naa.600144f096bb823800004b707f2d0001 Runtime Name: vmhba_mlx4_1.1.1:c0:t2:l0 Device: naa.600144f096bb823800004b707f2d0001 Device Display Name: Local SUN Disk (naa.600144f096bb823800004b707f2d0001) Group State: standby Storage Array Type Path Config: {TPG_id=1,TPG_state=STBY,RTP_id=256,RTP_health=UP} Path Selection Policy Path Config: {non-current path} gsan.80fe53553e0100282100-gsan.80fe73583e0100282100- naa.600144f096bb823800004b707f2d0001 Runtime Name: vmhba_mlx4_1.1.1:c0:t1:l0 Device: naa.600144f096bb823800004b707f2d0001 Device Display Name: Local SUN Disk (naa.600144f096bb823800004b707f2d0001) Group State: active Storage Array Type Path Config: {TPG_id=0,TPG_state=AO,RTP_id=2,RTP_health=UP} Path Selection Policy Path Config: {current path} esxcli nmp path list. : VMWare VM Linux. Linux /var/log/messages. Feb 10 16:10:00 ib-client-5 vmkernel: 1:21:41:36.385 cpu3:4421)<3>ib_srp: Send tsk_mgmt target[vmhba_mlx4_1.1.1:2] out of TX_IU head 769313 tail 769313 lim 0 : VM. BUI BUI iser SRP. 3 Configuration 143

SRP iser BUI Configuration > SAN iser iscsi. 1. ibd Configuration > Network ibd ( ipmp) Datalinks. Datalink Interfaces. 2. iser Configuration > SAN. iscsi Targets iser. 3. iscsi Target Group. 144 Sun ZFS Storage 7000 2012 12 E38995-01

SRP 4. Initiator iscsi initiators.. Initiator IQN OK. LUN. iscsi Initiator Groups. 5. LUN Shares LUN. Target Group Initiator Groups LUN. 3 Configuration 145

SRP 6. RedHat 4 SUSE 11 2. iser LUN. SRP SRP. 1. HCA IB.. 2. Configuration > SAN. 3. Target SRP. SRP. 4. Target Groups. 5. ( ) Initiator. 6. LUN SRP Shares. 7. LUN LUN. Create LUN Target Group Initiator Group LUN SRP. SRP. VMWare ESX RedHat 5.4 SUSE11 146 Sun ZFS Storage 7000 2012 12 E38995-01

CLI CLI configuration san targets srp groups targetsrpgroup SRP. swallower:configuration san targets srp groups> create swallower:configuration san targets srp group (uncommitted)> set name=targetsrpgroup name = targetsrpgroup (uncommitted) swallower:configuration san targets srp group (uncommitted)> set targets=eui.0002c903000489a4 targets = eui.0002c903000489a4 (uncommitted) swallower:configuration san targets srp group (uncommitted)> commit swallower:configuration san targets srp groups> list GROUP NAME group-000 targetsrpgroup +-> TARGETS eui.0002c903000489a4 CLI shares LUN targetsrpgroup. swallower:shares default> lun mylun swallower:shares default/mylun (uncommitted)> set targetgroup=targetsrpgroup targetgroup = targetsrpgroup (uncommitted) swallower:shares default/mylun (uncommitted)> set volsize=10 volsize = 10 (uncommitted) swallower:shares default/mylun (uncommitted)> commit swallower:shares default> list Filesystems: NAME SIZE MOUNTPOINT test 38K /export/test LUNs: NAME SIZE GUID mylun 10G 600144F0E9D19FFB00004B82DF490001,,BUI CLI.. Local -. Directory - NIS LDAP. NIS LDAP. 3 Configuration 147

.. administrator operator.. ( : root ).. "Basic administration".,,.. HTTP. ( ). Active Directory Active Directory Alerts. Analytics Clustering. Datasets Analytics Hardware /. Networking,. Projects and shares,, Roles Services 148 Sun ZFS Storage 7000 2012 12 E38995-01

Shares property schema. System Update. Users Worksheet BUI. 50... Type Username Full Name Password/Confirm Require session annotation Kiosk user Kiosk screen Roles Directory(NIS LDAP ) Local( )... Local.,. ID.. "Kiosk screen". Dashboard. CLI. "Kiosk user".. 3 Configuration 149

Exceptions. Roles. Name Description Authorizations... BUI BUI Users.,... /... Enter /. "Show All". /. /. /. //.,. CLI BUI CLI., help. 150 Sun ZFS Storage 7000 2012 12 E38995-01

CLI "brendan" NIS HTTP.. "webadmin". caji:> configuration roles caji:configuration roles> role webadmin caji:configuration roles webadmin (uncommitted)> set description="web server administrator" description = web server administrator (uncommitted) caji:configuration roles webadmin (uncommitted)> commit caji:configuration roles> show Roles: NAME basic webadmin DESCRIPTION Basic administration web server administrator webadmin HTTP.. caji:configuration roles> select webadmin caji:configuration roles webadmin> authorizations caji:configuration roles webadmin authorizations> create caji:configuration roles webadmin auth (uncommitted)> set scope=tab ad cluster net schema update alert hardware replication stat user appliance nas role svc worksheet caji:configuration roles webadmin auth (uncommitted)> set scope=svc scope = svc caji:configuration roles webadmin auth (uncommitted)> show Properties: scope = svc service = * allow_administer = false allow_configure = false allow_restart = false caji:configuration roles webadmin auth (uncommitted)> set service=tab * ftp ipmp nis ssh ad http iscsi ntp tags smb identity ldap routing vscan datalink:nge0 idmap ndmp scrk dns interface:nge0 nfs snmp caji:configuration roles webadmin auth (uncommitted)> set service=http service = http (uncommitted) caji:configuration roles webadmin auth (uncommitted)> set allow_restart=true allow_restart = true (uncommitted) caji:configuration roles webadmin auth (uncommitted)> commit caji:configuration roles webadmin authorizations> list NAME OBJECT PERMISSIONS auth-000 svc.http restart 3 Configuration 151

"brendan" "webadmin". caji:configuration roles webadmin authorizations> cd../../.. caji:configuration> users caji:configuration users> netuser brendan caji:configuration users> show Users: NAME USERNAME UID TYPE Brendan Gregg brendan 130948 Dir Super-User root 0 Loc caji:configuration users> select brendan caji:configuration users brendan> show Properties: logname = brendan fullname = Brendan Gregg initial_password = ************* require_annotation = false roles = basic kiosk_mode = false kiosk_screen = status/dashboard Children: exceptions => Configure this user s exceptions preferences => Configure user preferences caji:configuration users brendan> set roles=basic,webadmin roles = basic,webadmin (uncommitted) caji:configuration users brendan> commit brendan NIS HTTP.. CLI BUI.BUI. BUI 1 Roles. ( ). 2 Users. 3. 152 Sun ZFS Storage 7000 2012 12 E38995-01

4 5. Add. Users. 1 Roles. 2 3 4. ( ). Add. Roles. 1 2 3 4 "Scope". Scope... Authorization Add.. 1 Roles. 2 3. Apply. CLI 1 configuration roles. 3 Configuration 153

2 3 4 5 6 1 2 3 4 1 2 3 4 5 6 7 show. select authorizations show. ( ). configuration users. (NIS, LDAP) netuser. user. show commit.. select. show.. configuration roles. role.. ( ). configuration roles. select. authorizations. create. set scope=.. show. true ( ).. 154 Sun ZFS Storage 7000 2012 12 E38995-01

Preferences 8 commit.. 1 2 3 4 5 configuration roles. select. authorizations. show. destroy ( : "auth-001").. 1 2 3 ( ). true "status/dashboard".. Preferences, SSH. Initial login screen BUI. Status Dashboard. 3 Configuration 155

Preferences Locality Session timeout Current session annotation Advanced analytics statistics SSH Public Keys C.C POSIX ASCII. ISO 8859-1,,,,,,,,,,,,,,,,. BUI.. Analytics. RSA/DSA..BUI. CLI. BUI BUI,. CLI CLIconfiguration users. "brendan" analytics. caji:> configuration users caji:configuration users> select brendan caji:configuration users brendan> preferences caji:configuration users brendan preferences> show Properties: locale = C login_screen = status/dashboard session_timeout = 15 advanced_analytics = false Children: keys => Manage SSH public keys caji:configuration users brendan preferences> set advanced_analytics=true advanced_analytics = true (uncommitted) caji:configuration users brendan preferences> commit CLIconfiguration preferences.. 156 Sun ZFS Storage 7000 2012 12 E38995-01

twofish:> configuration preferences twofish:configuration preferences> show Properties: locale = C login_screen = status/dashboard session_timeout = 15 session_annotation = advanced_analytics = false Children: keys => Manage SSH public keys twofish:configuration preferences> set session_annotation="editing my user preferences" session_annotation = Editing my user preferences (uncommitted) twofish:configuration preferences> commit SSH Public Keys CLI. CLI SSH. caji:> configuration preferences keys caji:configuration preferences keys> create caji:configuration preferences key (uncommitted)> set type=dsa caji:configuration preferences key (uncommitted)> set key="...dsa key text..." key =...DSA key text...== (uncommitted) caji:configuration preferences key (uncommitted)> set comment="fw-log1" comment = fw-log1 (uncommitted) caji:configuration preferences key (uncommitted)> commit caji:configuration preferences keys> show Keys: NAME MODIFIED TYPE COMMENT key-000 10/12/2009 10:54:58 DSA fw-log1 ( ). Alerts,. Analytics... Maintenance Logs,. 3 Configuration 157

. Cluster Custom Hardware Events Hardware Faults NDMP operations Network Phone home Remote replication Service failures Thresholds ZFS pool,. "NDMP: " "NDMP: ".,,IP. " : " " : ". Service Analytics.... From aknobody@caji.com Mon Oct 13 15:24:47 2009 Date: Mon, 13 Oct 2009 15:24:21 +0000 (GMT) From: Appliance on caji <noreply@caji.com> Subject: High CPU on caji To: admin@hostname.com SUNW-MSG-ID: AK-8000-TT, TYPE: Alert, VER: 1, SEVERITY: Minor EVENT-TIME: Mon Oct 13 15:24:12 2009 PLATFORM: i86pc, CSN: 0809QAU005, HOSTNAME: caji SOURCE: svc:/appliance/kit/akd:default, REV: 1.0 EVENT-ID: 15a53214-c4e7-eae4-dae6-a652a51ea29b DESC: cpu.utilization threshold of 90 is violated. AUTO-RESPONSE: None. IMPACT: The impact depends on what statistic is being monitored. 158 Sun ZFS Storage 7000 2012 12 E38995-01

REC-ACTION: The suggested action depends on what statistic is being monitored. SEE: https://192.168.2.80:215/#maintenance/alert=15a53214-c4e7-eae4-dae6-a652a51ea29b SMTP. SNMP SNMP SNMP SNMP. Net-SNMP snmptrapd -P SNMP. # /usr/sfw/sbin/snmptrapd -P 2009-10-13 15:31:15 NET-SNMP version 5.0.9 Started. 2009-10-13 15:31:34 caji.com [192.168.2.80]: iso.3.6.1.2.1.1.3.0 = Timeticks: (2132104431) 246 days, 18:30:44.31 iso.3.6.1.6.3.1.1.4.1.0 = OID: iso.3.6.1.4.1.42.2.225.1.3.0.1 iso.3.6.1.4.1.42.2.225.1.2.1.2.36.55.99.102.48.97.99.100.52.45.51.48. 99.49.45.52.99.49.57.45.101.57.99.98.45.97.99.50.55.102.55.49.50.54. 98.55.57 = STRING: "7cf0acd4-30c1-4c19-e9cb-ac27f7126b79" iso.3.6.1.4.1.42.2.225.1.2.1.3.36.55.99.102.48.97.99.100.52.45.51.48. 99.49.45.52.99.49.57.45.101.57.99.98.45.97.99.50.55.102.55.49.50.54. 98.55.57 = STRING: "alert.ak.xmlrpc.threshold.violated" iso.3.6.1.4.1.42.2.225.1.2.1.4.36.55.99.102.48.97.99.100.52.45.51. 48.99.49.45.52.99.49.57.45.101.57.99.98.45.97.99.50.55.102.55.49.50. 54.98.55.57 = STRING: "cpu.utilization threshold of 90 is violated." Syslog Syslog syslog. syslog syslog Syslog. / Analytics Datasets. 24., CPU analytics NFS.. NFS. 24. /. CPU NFS. 3 Configuration 159

.. / Analytics... alert true.. Threshold Alerts Analytics.. Threshold exceeds/falls below Timing: for at least only between/only during Repost alert every... this condition persists Also post alert when this condition clears for at least... Analytics ( : "Protocol: NFSv4 operations/ ").. /.. ( : ).. "Add Threshold Alert".. Threshold CPU: percent utilization exceeds 95 percent Timing for at least 5 minutes only between 0:00 and 0:00 only during weekdays Repost alert every 5 minutes while this condition persists 160 Sun ZFS Storage 7000 2012 12 E38995-01

Also post alert when this condition clears for at least 5 minutes BUI Configuration->Alerts "Alert Actions" "Threshold Alerts".BUI. CLI CLI. configuration alerts help. BUI 1 2 3 4 5 6 7 1 2 "Alert Actions". Category "All Events". All Events Subset of Events.. "Alert Actions".. "TEST". SNMP. "Alert Actions". "ADD". "Threshold Alerts".. Analytics. 3 Configuration 161

3 4 5 6 7 exceeds/falls below.. 5,5, 5.. "Alert Actions". "Apply". Sun ZFS Storage 7000.,, /, Sun ZFS Storage 7000. Sun ZFS Storage 7310, 7320, 7410 7420.. 7420(2GHz 2.40GHz CPU ) 7420(1.86GHz 2.00GHz CPU )..,. Sun ZFS Storage 7000. ''. 2,. 2,., 2. - 2 162 Sun ZFS Storage 7000 2012 12 E38995-01

, -. Sun ZFS Storage 7000 System..,.... Sun ZFS Storage 7000.,,,.. Sun ZFS Storage 7000. Remote replication - - iscsi LUN - NFS, ( ) -, -,, -LACP IPMP RAID(Redundant Storage Device). Sun ZFS Storage 7000.IT. 3 Configuration 163

,....... /., Sun ZFS Storage 7000... / Sun ZFS Storage 7000 SLA,,,.,...... : : AKCS_OWNER AKCS_CLUSTERED ( ) : : : : : AKCS_CLUSTERED AKCS_STRIPPED AKCS_OWNER 164 Sun ZFS Storage 7000 2012 12 E38995-01

Sun ZFS Storage 7000 3 ( 1 ). I/O.,.,,,.. I/O CLUSTRON 3 I/O ( ). 2 1..... Sun ZFS Storage 7000. ( : ). 3 EIA/TIA-568B(8, Gigabit Ethernet).2 2.. 2 3 Configuration 165

. 3...,. 128 1-68 20. 9 115200bps, 1Gbps. 12.2ms..,. 3 I/O 50ms. 200ms( ) 500ms( ).3.,.. 550ms - 30ms -. Sun ZFS Storage System...,,.. ( )..,. 3,. ( 4 )... 166 Sun ZFS Storage 7000 2012 12 E38995-01

...,. IP,. 4 PrimaryA pool-0, PrimaryB pool-1....... ZFS.. ( ) ZFS. ATA ATA.... 3 Configuration 167

. AKCS_OWNER...,,,ID, SMB iscsi.... LUN. "ID" ( : )...... SSH. CLI.BUI SSL.... 168 Sun ZFS Storage 7000 2012 12 E38995-01

Takeover Failback. CLI/BUI UNCONFIGURED.. OWNER ( ),. ( : ).. STRIPPED ( ),.,, STRIPPED.. CLUSTERED,. ZFS CLUSTERED -. -... - ( )..,,..,. CLI BUI...... (, )..,,... A B. A 3 Configuration 169

B... OWNER, STRIPPED. STRIPPED CLUSTERED. OWNER, STRIPPED. ( ). STRIPPED. STRIPPED. /LUN. LUN,....... 1.. 2. ().... Sun ZFS Storage 7000.. 170 Sun ZFS Storage 7000 2012 12 E38995-01

..... Sun ZFS Storage 7000...,.... ReadZilla., ReadZillas., - ReadZillas. LogZilla. ( ) CPU 50%, DRAM 50% 50%..,. CPU DRAM. 50%.. (). 100%. 40%-100%. 3 Configuration 171

I/O () ReadZilla.. ReadZilla.... LUN. LUN... 7410C 3 4... NSPF(No Single Point of Failure). JBOD. NSPF JBOD. RAID NSPF JBOD.,. IPMP / LACP.. IP.DHCP DHCP. 172 Sun ZFS Storage 7000 2012 12 E38995-01

.,. 5...,. Sun... ( ). 2. ( : ). : e1000g3 'alice' e1000g4 'bob'. 172.16.27.0/24 172.16.27.1 172.16.64.0/22. 172.16.27.1 172.16.64.0/22 e1000g3, e1000g4. IP.,. AKCS_STRIPPED. LDAP Active Directory. /.., ( ). ( : LUN )... 3 Configuration 173

Infiniband Infiniband. HCA, IPMP. HCA. HCA (pkey). Infiniband. IP. IB... Sun. 7110 HCA. IB. 7410. 174 Sun ZFS Storage 7000 2012 12 E38995-01

. 2 HCA. ' ',... Sun ZFS Storage 7000 I/O. 3( ). 3 Configuration 175

. (SCSI ). ATA Sun ZFS Storage 7000. SAS SASZONELOCK.. I/O.. SAS AKCS_OWNER 10. SAS. 2 SAS..,2 SAS JBOD.. /. 4. Sun ZFS Storage 7000 NSPF(No Single Point of Failure). RAID /. 176 Sun ZFS Storage 7000 2012 12 E38995-01

. 3 I/O ( ). JBOD( ).,. JBOD..... NFS( ) I/O. NFSv2 NFSv3 stateless. NFSv4 I/O. Sun ZFS Storage 7000 ( ). 3 Configuration 177

/. iscsi.. iscsi. SMB, FTP HTTP/WebDAV... 15-20., 1/2 JBOD JBOD 1/2...... LUN. LUN. 178 Sun ZFS Storage 7000 2012 12 E38995-01

... 2.. A B A B.. LUN.. : / 2009.04.10,1-0.. Sun ZFS Storage 7000. 2. 1.. 2... 3. Sun ZFS Storage JBOD HBA JBOD. 4..... tty. : tty. 5. BUI CLI... Configuration->Cluster Setup. 6.. 3..3 Commit. 3 Configuration 179

7... Commit. 8.. :.. 9. # DHCP. IP. tty DHCP Static. 10. # (BUI CLI). 11. #.. 12...2,.... : JBOD 1/2. JBOD 2 1/2 JBOD. JBOD 1/2 2.2.. 13.... 14... Configuration/Cluster.. 180 Sun ZFS Storage 7000 2012 12 E38995-01

. Sun Storage 7310 ZFS Storage 7320 PCIe0. Sun Storage 7410 PCIe5 ZFS Storage 7420 PCIeC. 3. 2 ( 2 ) ( ). Cat 5 ( 3 1m ).,.. 3. JBOD JBOD. :. BUI Configuration->Cluster,. 3 Configuration 181

..,., /. ( ).., CLUSTERED. OWNER Apply.. OWNER CLUSTERED Apply () ()... Setup Unconfig Apply Revert Failback.... ( ). ( ). () OWNER CLUSTERED (-). 182 Sun ZFS Storage 7000 2012 12 E38995-01

Takeover () CLUSTERED STRIPPED OWNER.. 1. 2. 2.... 1.. 3. 2.. 3.. (, ). 4. ( ). 5. Configuration -> Clustering Unconfig..,.. : 2. 2010.Q1.0.0.,, 3 Configuration 183

. 2010.Q1.0.0. 184 Sun ZFS Storage 7000 2012 12 E38995-01

4 4 Services Services Services.. 185

Services NFS iscsi SMB SMB-over-TCP 445 NetBIOS Datagram 138 NetBIOS Name Service 137 FTP HTTP NDMP NFSv3 NFSv4 iscsi LUN SMB FTP HTTP NDMP 111 2049 3260 3205 SMB-over-NetBIOS 139 21 80 10000 Remote Replication 216 Shadow Migration SFTP SRP TFTP Virus Scan SFTP SRP TFTP 218 186 Sun ZFS Storage 7000 2012 12 E38995-01

Services : 0-99 UID GID.. NIS LDAP Active Directory Identity Mapping NIS LDAP Microsoft Active Directory Server Windows Unix ID 389 DNS Dynamic Routing IPMP NTP Phone Home RIP RIPng IP IP Network Time Protocol 53 443 Service Tags 443 SMTP SNMP SNMP 4 Services 187

Services Syslog System Identity syslog syslog Syslog SSH BUI CLI SSH 22 215 Kerberos Kerberos V 88 Kerberos V Change & Set Password (SET_CHANGE) Kerberos V Change & Set Password (RPCSEC_GSS) 464 749 BUI BUI Services.... 188 Sun ZFS Storage 7000 2012 12 E38995-01

Services... /.. /...... : Services ( )..... APPLY. REVERT. 4 Services 189

Services. Logs..,.. Executing start method Method "start" exited with status 0 Method "refresh" exited with status 0 Executing stop method Enabled Disabled. (0 == )..... NTP. [ Oct 11 21:05:31 Enabled. ] [ Oct 11 21:07:37 Executing start method (...). ] [ Oct 11 21:13:38 Method "start" exited with status 0. ] 21:05. 21:07:37 21:13:38. NTP. CLI CLI configuration services. show. 190 Sun ZFS Storage 7000 2012 12 E38995-01

Services caji:> configuration services caji:configuration services> show Services: ad => disabled smb => disabled dns => online dynrouting => online ftp => disabled http => disabled identity => online idmap => online ipmp => online iscsi => online ldap => disabled ndmp => online nfs => online nis => disabled ntp => disabled replication => online scrk => disabled sftp => disabled shadow => online smtp => online snmp => disabled ssh => online syslog => disabled tags => online tftp => disabled vscan => disabled Children: ad => Configure Active Directory smb => Configure SMB dns => Configure DNS dynrouting => Configure Dynamic Routing ftp => Configure FTP http => Configure HTTP identity => Configure System Identity idmap => Configure Identity Mapping ipmp => Configure IPMP iscsi => Configure iscsi ldap => Configure LDAP ndmp => Configure NDMP nfs => Configure NFS nis => Configure NIS ntp => Configure NTP replication => Configure Remote Replication scrk => Configure Phone Home sftp => Configure SFTP shadow => Configure Shadow Migration smtp => Configure SMTP snmp => Configure SNMP srp => Configure SRP ssh => Configure SSH syslog => Configure Syslog tags => Configure Service Tags tftp => Configure TFTP vscan => Configure Virus Scan routing => Configure Routing Table 4 Services 191

Services., nis. caji:configuration services> nis caji:configuration services nis>, /,. show. caji:configuration services nis> show Properties: <status> = online domain = fishworks broadcast = true ypservers = enable. caji:configuration services nis> enable disable. caji:configuration services nis> disable set. commit. caji:configuration services nis> set domain="mydomain" domain = mydomain (uncommitted) caji:configuration services nis> commit caji:configuration services nis> show Properties: <status> = online domain = mydomain broadcast = true ypservers = : BUI CLI. 192 Sun ZFS Storage 7000 2012 12 E38995-01

NFS CLI. BUI. help. caji:configuration services nis> help Subcommands that are valid in this context: help [topic] show commit done enable disable get [prop] set [prop] => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help", "script" or "properties". => Show information pertinent to the current context => Commit current state, including any changes => Finish operating on "nis" => Enable the nis service => Disable the nis service => Get value for property [prop]. ("help properties" for valid properties.) If [prop] is not specified, returns values for all properties. => Set property [prop] to [value]. ("help properties" for valid properties.) For properties taking list values, [value] should be a comma-separated list of values. NFS NFS( ). Sun ZFS Storage Appliance NFS 2, 3 4.. 4 Services 193

NFS Minimum supported version Maximum supported version Maximum # of server threads Grace period Custom NFSv4 identity domain Enable NFSv4 delegation Kerberos realm Kerberos master KDC Kerberos slave KDC Kerberos admin principal Kerberos admin password NFS. NFS. NFS (20-1000). NFS. ( ) (15-600). NFS. ( : 90). NFS. NFSv4 ID. DNS ID, _nfsv4idmapdomain DNS DNS..... NFSv4. KDC. ASCII. DNS. Kerberos DNS.. KDC KDC KDC., KDC.. KDC KDC..,. joe, joe/admin joe/admin@eng.example.com.. 194 Sun ZFS Storage 7000 2012 12 E38995-01

NFS Services BUI CLI. NFS. NFS NFS,. Kerberos Kerberos keytab. NTP Kerberos NFS. Kerberos NFS. host/node1.example.com@example.com nfs/node1.example.com@example.com. host/node1.example.com@example.com nfs/node1.example.com@example.com host/node2.example.com@example.com nfs/node2.example.com@example.com. Active Directory Kerberos. KDC Kerberos http://download.oracle.com/ docs/cd/e19253-01/816-4557/setup-8/index.html (http://download.oracle.com/docs/cd/ E19253-01/816-4557/setup-8/index.html.). Kerberos NFS Shares->Filesystem->Protocols Kerberos. Kerberos. Kerberos V :88 Kerberos V SET_CHANGE : 464 Kerberos V RPCSEC_GSS : 749 NFS. 4 Services 195

NFS network-nfs-server:default NFS appliance-kit-nfsconf:default NFS network-nfs-cbd: default network-nfs-mapid:default network-nfs-status:default network-nfs-nlockmgr: default NFSv4 NFSv4 mapid - NFSv4 NFS statd - NFS NFS lockd - Services Logs. Analytics NFS Analytics.. NFS... (//...)............ : NFS unknown. Analytics unknown. CLI CLI BUI. CLI version_min version_max nfsd_servers grace_period mapid_domain enable_delegation BUI Minimum supported version Maximum supported version Maximum # of server threads Grace period Custom NFSv4 identity domain Enable NFSv4 delegation 196 Sun ZFS Storage 7000 2012 12 E38995-01

iscsi CLI krb5_realm krb5_kdc krb5_kdc2 krb5_admin BUI Kerberos Realm Kerberos master KDC Kerberos slave KDC Kerberos admin principal NFS 1 2 3 4 NFS Configuration->Services. NFS.. Shares. Protocols NFS. NFS (/+). iscsi LUN iscsi(internet Small Computer System Interface). iscsi iscsi iscsi. isns,. iscsi CHAP ( ) ( ). RADIUS CHAP. 2. 4 Services 197

iscsi Use isns isns Server Use RADIUS RADIUS Server RADIUS Server Secret isns isns RADIUS RADIUS RADIUS Services BUI CLI. CLI. CHAP CHAP. CHAP. iscsi. iscsi SAN. CLI iscsi SAN. 198 Sun ZFS Storage 7000 2012 12 E38995-01

SMB. IQN IQN. isns IP isns. IP. CHAP. CHAP. RADIUS IP RADIUS. LUN LUN. LUN. LUN. LUN. / Red Hat iser Reduced Copy I/O. /etc/iscsi/iscsid.conf node.session.timeo.replacement_timeout 300. SMB SMB SMB. Shares SMB. LAN Manager compatibility level Preferred domain controller (LM, NTLM, LMv2, NTLMv2). Solaris Express Reference Manual Collection for smb. Active Directory. Active Directory DNS SRV Active Directory. 4 Services 199

SMB Active Directory site Maximum # of server threads Enable Dynamic DNS Enable Oplocks Restrict anonymous access to share list System Comment Idle Session Timeout Primary WINS server Secondary WINS server Excluded IP addreses from WINS SMB Signing Enabled SMB Signing Required Active Directory.,. Active Directory. ( ). 1024. DNS Active Directory DNS. off. SMB.. on. SMB oplock. oplock. SMB.... TCP/IP WINS. TCP/IP WINS. WINS IP. SMB SMB... SMB. SMB SMB.. Services BUI CLI. CLI. SMB. 200 Sun ZFS Storage 7000 2012 12 E38995-01

SMB Case sensitivity Reject non UTF-8 Non-Blocking Mandatory Locking Resource name Share-level ACL SMB "mixed'" "'insensitive". UTF-8 SMB.... ACL ACL.. Case sensitivity Reject non UTF-8. NFS/SMB NFS SMB. NFS/SMB. 1. Active Directory. 2. ID. 3. SMB. 4.,ACL ACL. SMB NFSv3. SMB SMB ACL. ACL ACL inheritance behavior. DFS DFS( ) SMB MSRPC. DFS DFS.DFS.,,... 4 Services 201

SMB DFS. DFS. DFS. SMB DFS DFS MMC. Active Directory DFS. SMB DFS. SMB dfsadmin DFS. 1. dfsadmin. Windows. 2. dfsadmin SMB. 3. DFS Windows dfsadmin. SMB. SMB. 3. NIS LDAP SMB.. "&".... Administrators Administrators. 202 Sun ZFS Storage 7000 2012 12 E38995-01

SMB Backup Operators Backup Operators. ID Windows ID. guest /. MMC MMC(Microsoft Management Console),. Microsoft Management Console, /,. Sun ZFS Storage 7000 MMC Windows Administrators. MMC Access is denied. Sun ZFS Storage 7000. MMC,. Sun ZFS Storage 7000,.. 4 Services 203

SMB. ACL MMC.. 204 Sun ZFS Storage 7000 2012 12 E38995-01

SMB,. SMB ( ) ( ). 4 Services 205

SMB. Services Sun ZFS Storage 7000. MMC. vscan. 206 Sun ZFS Storage 7000 2012 12 E38995-01

SMB MMC. Administrators Power Users Administrators..,,. 4 Services 207

SMB CLI CLI SMB. create list. "Bill". twofish:> configuration services smb twofish:configuration services smb> create twofish:configuration services rule (uncommitted)> set use_nss=false twofish:configuration services rule (uncommitted)> set user=bill twofish:configuration services rule (uncommitted)> set directory=/export/wdp twofish:configuration services rule (uncommitted)> set container="dc=com,dc=fishworks, ou=engineering,cn=myhome" twofish:configuration services rule (uncommitted)> commit twofish:configuration services smb> list RULE NSS USER DIRECTORY CONTAINER rule-000 false Bill /export/wdp dc=com,dc=fishworks, ou=engineering,cn=myhome. &?.. twofish:configuration services smb> create twofish:configuration services rule (uncommitted)> set use_nss=false twofish:configuration services rule (uncommitted)> set user=* twofish:configuration services rule (uncommitted)> set directory=/export/?/& twofish:configuration services rule (uncommitted)> set container="dc=com,dc=fishworks, ou=engineering,cn=myhome" twofish:configuration services rule (uncommitted)> commit twofish:configuration services smb> list RULE NSS USER DIRECTORY CONTAINER rule-000 false Bill /export/wdp dc=com,dc=fishworks, ou=engineering,cn=myhome. twofish:configuration services smb> create twofish:configuration services rule (uncommitted)> set use_nss=true twofish:configuration services rule (uncommitted)> set container="dc=com,dc=fishworks, ou=engineering,cn=myhome" twofish:configuration services rule (uncommitted)> commit twofish:configuration services smb> list RULE NSS USER DIRECTORY CONTAINER rule-000 true dc=com,dc=fishworks, ou=engineering,cn=myhome twofish:configuration services smb> groups twofish:configuration services smb groups> create twofish:configuration services smb member (uncommitted)> set user=bill 208 Sun ZFS Storage 7000 2012 12 E38995-01

SMB twofish:configuration services smb member (uncommitted)> set group="backup Operators" twofish:configuration services smb member (uncommitted)> commit twofish:configuration services smb groups> list MEMBER USER GROUP member-000 WINDOMAIN\Bill Backup Operators BUI SMB Sun ZFS Storage 7000. SMB 1 2 3 4 5 6 7 8 9 10 11 BUI CLI20. BUI CLI Maintenance > System. BUI.,.. Apply.. Apply. DNS.. Active Directory IP. NTP..,. NTP. 4 Services 209

SMB 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 1 NTP. NTP SMB. Active Directory.. Apply.... "Allocate and verify storage" JBOD. JBOD none, half all. JBOD JBOD. Commit. "Configure Added Storage".,.. Commit.. Oracle HTTP.... '$'.. Active Directory Active Directory. Active Directory. 210 Sun ZFS Storage 7000 2012 12 E38995-01

SMB 2 3 1 Configuration > Services > Active Directory Join Domain. Active Directory,, Apply.. 2 Shares Projects. 3 4 5 6 Add..... Projects.. 7. 8 9 10 11 12 General Inherit. SMB /export. Protocols. sharesmb ACL. Apply. SMB 1 Configuration > Services > SMB. 2 3 SMB Apply. Configuration > Services > SMB Autohome SMB Apply. 4 Services 211

FTP 4 Configuration > Services > SMB Local Groups Administrators Backup Operators Apply. FTP FTP( ) FTP.. Services. FTP Port (for incoming connections) Maximum # of connections ("0" for unlimited) Turn on delay engine to prevent timing attacks Default login root Logging level Permissions to mask from newly created files and dirs FTP. 21. FTP.. ( ) DoS 30... FTP. "/". FTP. proftpd... 212 Sun ZFS Storage 7000 2012 12 E38995-01

FTP Enable SSL/TLS Port for incoming SSL/TLS connections Permit root login Maximum # of allowable login attempts SSL/TLS FTP. FTP.. SSL/TLS FTP. 21. FTP. FTP. FTP,. 3. Services BUI CLI. CLI. proftpd proftpd_xfer proftpd_tls FTP.. SSL/TLS FTP. Services Logs. FTP 1 2 FTP Configuration->Services. FTP.. 4 Services 213

HTTP 3 4 Shares. "Protocols" FTP. (/+). HTTP HTTP HTTP, HTTPS HTTP WebDAV(Web based Distributed Authoring and Versioning).. HTTP HTTPS URL. http://hostname/shares/mountpoint/share_name https://hostname/shares/mountpoint/share_name HTTPS. Require client login Protocols HTTP Port (for incoming connections) HTTPS Port (for incoming secure connections),. HTTP "nobody".. (HTTP, HTTPS ). HTTP, 80. HTTP, 443. Services BUI CLI. 214 Sun ZFS Storage 7000 2012 12 E38995-01

HTTP "Require client login", NIS LDAP. Active Directory. HTTP. HTTPS. HTTP NFS FTP.. (uid 100 ) "nobody". "nobody". "Require client login". "nobody" "nobody".. Unix 666( / ) Unix 777( // ). network-http:apache22 HTTP Services Logs. HTTP 1 2 HTTP Configuration->Services. HTTP.. 4 Services 215

NDMP 3 4 Shares. "Protocols" HTTP. (/+). NDMP NDMP( ) DMA( ) NDMP NDMP.NDMP ( ). DMA. NDMP. [[Maintenance:System:ConfigurationBackup /]].. DMA. () SCSI FC( ). NDMP.DMA NDMP DMA.DMA. NDMP. DMA. ( ) DMA ( ).DMA "3 ". IP. 216 Sun ZFS Storage 7000 2012 12 E38995-01

NDMP NDMP. 3.DMA NDMP "TYPE". dump tar zfs. DAR(Direct Access Recovery).. DAR(Direct Access Recovery).. DAR(Direct Access Recovery). NDMPv4. NDMP 7000.., 2010.Q3 "zfs" 2010.Q1 2010.Q3 "zfs". "dump" "tar" "dump" "tar"., /export/home. /export/code...... ( : /export/code) ( : /export/code/src).. ( : /export/code/.zfs/snapshot/mysnap). 4 Services 217

NDMP "dump" "tar"., Shares. NDMP ( : )., /export/proj /export/proj. /export/someshare/somedir /export/someshare.., proj1 share1 share2 /export/foo.... "ndmp-".,.. "zfs" "zfs". BUI. CLI canonical_name. '/' '/'. 218 Sun ZFS Storage 7000 2012 12 E38995-01

NDMP "zfs".. '@snapshot',.. pool-0/local/default pool-0/local/ default@yesterday pool-0/local/default/code pool-0/local/ default/code@yesterday "default". "default" "yesterday" "yesterday". "default" "code". "code". "default" "code" "yesterday ". "code". "zfs". UPDATE=n DMA.. " ". ( : ) "zfs". "zfs".. "zfs".. " ".. DMA 3. LEVEL DMP_NAME UPDATE 0-9.. DMP_NAME.. 4 Services 219

NDMP -N N LEVEL, ("DMP_NAME" ). -0.UPDATE "y"( ) N. DMA.. 1 7,14,21-0.. -1. (). -2. -1. 24 1-0, 21-1 24-2.. "tar" "dump"... 24-2.. "zfs"... NDMP.., -2-1.21-1 24-2. 220 Sun ZFS Storage 7000 2012 12 E38995-01

NDMP "zfs" NDMP.NDMP "ZFS_FORCE" "y" NDMP " " NDMP ( ). NDMP. DMA username and password Enable DAR Ignore file metadata changes for incremental backups Restore full absolute path for partial restore (v3 only) NDMP version TCP port Default restore pool(s) DMA( )..... "tar" "dump"... DMA NDMP. NDMP 10000.NDMPv3. NDMPv4. "tar" "dump".ndmp..... 4 Services 221

SFTP Rollback datasets before restore (ZFS backups only) DMA tape mode (for locally attached drives) "zfs"... "ZFS_FORCE" DMA ( " " ) DMA.. ZFS_FORCE DMA. DMA SystemV BSD. DMA SystemV. NDMP.DMA DMA. DMA DMA. Services BUI CLI. system-ndmpd:default NDMP Services Logs. SFTP SFTP(SSH File Transfer Protocol) SFTP.. Services. 222 Sun ZFS Storage 7000 2012 12 E38995-01

SFTP Port (for incoming connections) Permit root login Logging level SFTP Keys* SFTP. 218. SFTP. SFTP. SFTP. SFTP RSA/DSA.. 2011.1 SFTP. SFTP " ". SFTP SFTP. SFTP. Services BUI CLI. CLI. SFTP SFTP. 22 SSH. SFTP 218 SFTP., SFTP OpenSolaris. manta# sftp -o "Port 218" root@guppy network-sftp:default SFTP. Services Logs. 4 Services 223

Virus Scan SFTP 1 2 3 4 SFTP Configuration->Services. SFTP.. Shares. "Protocols" SFTP. (/+). Virus Scan.,.. NFSv4 NFS. Maximum file size to scan Allow access to files that exceed maximum file size. ( : ). 1GB.... 224 Sun ZFS Storage 7000 2012 12 E38995-01

Virus Scan Services BUI CLI. CLI.. "*" ( ).., zip "data-archive*.zip". exe com bat doc data-archive*.zip * zip " *".. ICAP(Internet Content Adaptation Protocol, RFC 3507). Enable Host Maximum Connections Port. IP.. 8.. 4 Services 225

NIS vscan. Services Logs.. BUI CLI. Virus Scan 1 2 3 4 5 6 7 Configuration->Services->Virus Scan.. /. Shares.. "General". "Virus scan". NIS NIS( ). NIS. NIS FTP HTTP/WebDAV. 226 Sun ZFS Storage 7000 2012 12 E38995-01

NIS NIS. NIS. 0-99 UID GID.. Domain Server(s): Search using broadcast Server(s): Use listed servers NIS. NIS NIS. NIS IP. Services BUI CLI. NIS. network-nis-client:default appliance-kit-nsswitch:default system-identity:domain NIS. NIS.. Services Logs.. BUI CLI. 4 Services 227

LDAP NIS 1 2 3 4 5 6 7 NIS NIS NIS. Configuration->Services->NIS. NIS. /. Configuration->Users. "directory". NIS. Users. LDAP LDAP(Lightweight Directory Access Protocol),, ( ). LDAP. LDAP FTP HTTP/WebDAV. LDAP ( ID ) ACL. LDAP. LDAP. 0-99 UID.. LDAP. 228 Sun ZFS Storage 7000 2012 12 E38995-01

LDAP Protect LDAP traffic with SSL/TLS Base search DN Search scope Authentication method Bind credential level Proxy DN Proxy Password Schema definition Servers TLS(, SSL ) LDAP.,. LDAP. ( ) ( ).. LDAP. (RFC 4513), SASL/DIGEST-MD5 SASL/GSSAPI. DN SSL/TLS. SASL/GSSAPI. LDAP.... SASL/GSSAPI....,. " ". LDAP. LDAP...LDAP. Services BUI CLI. LDAP,. RFC 2307(posixAccount posixgroup). DN dc=example,dc=com. 4 Services 229

LDAP users ou=people,base search DN ou=people,dc=example,dc=com groups ou=group,base search DN ou=group,dc=example,dc=com, Schema definition. entire DN. Base search DN Search scope. ("Users" "Groups") default = new. default new.. posixaccount unixaccount Users Object posixaccount = unixaccount. uid employeenumber Users Attribute uid = employeenumber. posixgroup unixgroup Groups Object posixgroup = unixgroup. cn groupaccount Groups Attribute cn = groupaccount. appliance-kit-nsswitch: default LDAP. Services Logs.. BUI CLI. 230 Sun ZFS Storage 7000 2012 12 E38995-01

Active Directory LDAP 1 2 3 4 5 6 7 LDAP LDAP LDAP. Configuration->Services->LDAP. LDAP. /. Configuration->Users. "directory". LDAP. Users. Active Directory Active Directory,, Microsoft Active Directory. SMB 2. SMB AD. SMB. Users. Active Directory. 4 Services 231

Active Directory Active Directory Domain Administrative User Administrative Password Additional DNS Search Path Active Directory. Active Directory AD.. DNS DNS Active Directory.. Windows Workgroup. Services BUI CLI. CLI.. Active Directory.,. Active Directory, Active Directory SMB. Kerberos Kerberos NFS Active Directory. LDAP LDAP.LDAP SMB SMB. 232 Sun ZFS Storage 7000 2012 12 E38995-01

Active Directory Windows Server 2008 Windows Windows Server 2003 Windows Server 2008 SP1 2009.Q2 3.1 KB957441.B. Windows Server 2008 SP1 2009.Q2 4.0 KB951191, KB957441.A B. Windows Server 2008 SP2 2009.Q2 4.0 C. Windows Server 2008 R2 2009.Q2 4.0 C. A : Kerberos (KB951191) Windows Server 2008 SP1. Windows Server 2008 SP1 Kerberos, KB951191(http://support.microsoft.com/ default.aspx/kb/951191 (http://support.microsoft.com/default.aspx/kb/951191)). Windows Server 2008 SP2 R2. 2009.Q2.4.0 Windows 2008 Windows Server 2008 SP2 R2. 2009.Q2.4.0 Windows 2008 Windows Server 2008 SP1 KB951191 Windows 2008 SP2. B : NTLMv2 (KB957441) Windows Server 2008 SP1 http://support.microsoft.com/kb/957441/ (http://support.microsoft.com/kb/957441/). LMCompatibilityLevel NTLMv2. Windows 2008 SP1 LMCompatibilityLevel 5. KB957441. C : NTLMv2 Windows Server 2008 SP2 R2 KB957441. 4 Services 233

Active Directory BUI "JOIN DOMAIN", "JOIN WORKGROUP". CLI CLI. twofish:> configuration services ad twofish:configuration services ad> show Properties: <status> = online mode = domain domain = eng.fishworks.com Children: domain => Join an Active Directory domain workgroup => Join a Windows workgroup "eng.fishworks.com".. twofish:configuration services ad> workgroup twofish:configuration services ad workgroup> set workgroup=workgroup twofish:configuration services ad workgroup> commit twofish:configuration services ad workgroup> done twofish:configuration services ad> show Properties: <status> = disabled mode = workgroup workgroup = WORKGROUP. twofish:configuration services ad> done twofish:> configuration services smb twofish:configuration services smb> set ads_site=sf twofish:configuration services smb> set pdc=192.168.3.21 twofish:configuration services smb> commit twofish:configuration services smb> show Properties: <status> = online lmauth_level = 4 pdc = 192.168.3.21 ads_site = sf twofish:configuration services smb> done. 234 Sun ZFS Storage 7000 2012 12 E38995-01

Active Directory twofish:> configuration services ad twofish:configuration services ad> domain twofish:configuration services ad domain> set domain=fishworks.com twofish:configuration services ad domain> set user=administrator twofish:configuration services ad domain> set password=******* twofish:configuration services ad domain> set searchdomain=it.fishworks.com twofish:configuration services ad domain> commit twofish:configuration services ad domain> done twofish:configuration services ad> show Properties: <status> = online mode = domain domain = fishworks.com BUI CLI. Active Directory 1 2 3 4 5 6 SMB Active Directory ( ). SMB ( ). NTP, 5. DNS Active Directory, IP DNS. Active Directory,. /. 1 2. /. 4 Services 235

Identity Mapping Identity Mapping ID Unix UID/GID Windows SID Windows Unix ID. SMB ID Windows Unix ID. SMB ID Windows ID Unix ID. Windows Unix ID UID GID ID. SMB NFS. Windows Unix ID NFS SMB ID. Windows,. Active Directory SAM Windows. SID. SID Windows. Unix. ( : NIS LDAP) Unix. Unix UID GID. UID GID Unix.. ID SID, UID GID. 3. ID IDMU IDMU Active Directory. Active Directory LDAP... 236 Sun ZFS Storage 7000 2012 12 E38995-01

Identity Mapping IDMU.. ID. ID. IDMU Microsoft "Unix ID " IDMU. Windows Server 2003 Windows Server 2003 R2. "Unix ". IDMUWindows NIS/NFS.IDMU UNIX ( UID, GID,, ) Active Directory Users and Computers "UNIX Attribute". RFC2307 NIS AD. IDMU ID Unix Windows Unix ID. ID IDMU.. ID ID LDAP Active Directory.. ID Directory-Based Mapping AD Attribute - Unix User Name AD Attribute - Unix Group Name. Unix AD. Unix AD. 4 Services 237

Identity Mapping Native LDAP Attribute - Windows User Name Windows ID LDAP. Services BUI CLI. CLI. Active Directory LDAP Solaris CIFS Administration Guide Managing Directory-Based Identity Mapping for Users and Groups (Task Map). ID. Windows ID Unix ID. ID. Mapping type Mapping direction Windows domain Windows entity Unix entity Unix type.. Windows Unix Unix Windows. Windows ID Active Directory. Windows ID. Unix ID. Unix ID,. Case Sensitivity Windows Unix. JSMITH, JSmith jsmith Windows Unix 3.. 238 Sun ZFS Storage 7000 2012 12 E38995-01

Identity Mapping Windows-to-Unix Windows Unix., Windows "jsmith" Unix "jsmith". Windows "Jsmith". Windows-to-Unix "*".ID Windows *@some.domain Unix "*" Windows Unix. Windows Unix., Windows "JSmith@some.domain" Unix "jsmith". Windows.., Windows "JSmith@sun.com" Unix "jsmith". ID Windows. Unix-to-Windows., Unix "jsmith" "JSMITH" Windows. ID 10. 10ID. SMB. 10.. Windows Unix. ( )., "mydomain.com" Windows Unix. "mydomain.com" "Engineering" Windows Unix.. ID ID. Windows Unix 4 Services 239

Identity Mapping., "guest" Unix SMB... Windows ID Unix ID, Unix ID Windows ID. Windows ID Unix ID. Unix ID Windows ID. Windows ID. Unix ID. (,,,, ) Unix ID.. Unix Windows SMB Windows Windows ID. NFS "nobody". ID NFS SMB. NFS SMB ID. ID SMB., 10.. ID.,. 240 Sun ZFS Storage 7000 2012 12 E38995-01

Identity Mapping Windows Unix ID Windows. Windows Unix. IDMU. BUI Mappings ID. Windows Unix ID. User Properties Group Properties ID. CLI 2. Windows Unix. twofish:> configuration services idmap twofish:configuration services idmap> create twofish:configuration services idmap (uncommitted)> set windomain=eng.fishworks.com twofish:configuration services idmap (uncommitted)> set winname=bill twofish:configuration services idmap (uncommitted)> set direction=bi twofish:configuration services idmap (uncommitted)> set unixname=wdp twofish:configuration services idmap (uncommitted)> set unixtype=user twofish:configuration services idmap (uncommitted)> commit twofish:configuration services idmap> list MAPPING WINDOWS ENTITY DIRECTION UNIX ENTITY idmap-000 Bill@eng.fishworks.com (U) == wdp (U) Windows. twofish:configuration services idmap> create twofish:configuration services idmap (uncommitted)> list Properties: windomain = (unset) winname = (unset) direction = (unset) unixname = (unset) unixtype = (unset) twofish:configuration services idmap (uncommitted)> set windomain=guest.fishworks.com twofish:configuration services idmap (uncommitted)> set winname=* twofish:configuration services idmap (uncommitted)> set direction=win2unix twofish:configuration services idmap (uncommitted)> set unixname= twofish:configuration services idmap (uncommitted)> set unixtype=user twofish:configuration services idmap (uncommitted)> commit 4 Services 241

DNS twofish:configuration services idmap> list MAPPING WINDOWS ENTITY DIRECTION UNIX ENTITY idmap-000 Bill@eng.fishworks.com (U) == wdp (U) idmap-001 *@guest.fishworks.com (U) => "" (U). BUI CLI. ID 1 2 3 4 1 2 3 4 5 ID Active Directory. ( )..... Windows ID.. Unix ID. /. DNS DNS( ) IP IP. NIS / LDAP 242 Sun ZFS Storage 7000 2012 12 E38995-01

DNS DNS., Analytics. DNS IP.. NS(NameServer).DNS. DNS Domain DNS Server(s) Allow IPv4 non-dns resolution Allow IPv6 non-dns resolution. DNS.IP. NIS / LDAP IPv4, IPv4. NIS / LDAP IPv4 IPv6, IPv4 IPv6. Services BUI CLI. CLI CLI nslookup getent hosts. caji:> nslookup deimos 192.168.1.109 deimos.sf.fishworks.com caji:> getent hosts deimos 192.168.1.109 deimos.sf.fishworks.com network-dns-client:default DNS. 4 Services 243

DNS Services Logs. Active Directory DNS Active Directory Active Directory., example.com Active Directory redmond.example.com example.com redmond.example.com Active Directory. Active Directory. Active Directory. DNS DNS IP. DNS. NIS LDAP. DNS. LDAP ou=hosts( DN) DN. NFS. NFS. DNS DNS.DNS NIS( ) LDAP( ).. DNS DNS( ) NIS LDAP. 'getent' CLI.. DNS DNS 127.0.0.1 DNS.. Analytics. 244 Sun ZFS Storage 7000 2012 12 E38995-01

IPMP Active Directory ( ). SSL LDAP. IP... " DNS ". IPMP IPMP(Internet Protocol Network Multipathing) ( ).. IPMP Network. Failure detection latency Enable fail-back IPMP IP.. Services BUI CLI. network-initial:default. Services Logs. 4 Services 245

NTP IPMP Network. NTP NTP(Network Time Protocol).. UTC.BUI. multicast address NTP server(s) NTP Authentication Keys NTP. NTP ( ). NTP NTP.. 224.0.1.1 0.pool.ntp.org : 10, : ASCII, : SUN7000 Services BUI CLI... NTP NTP NTP 246 Sun ZFS Storage 7000 2012 12 E38995-01

NTP NTP NTP ID NTP.. NTP NTP (, ). NTP... DES DES 64 16 0101010101010101 NTP NTP 64 16 8080808080808080 ASCII 1-8 ASCII topsecret MD5 MD5 1-8 ASCII md5secret NTP., NTP. BUI BUI NTP,. NTP. CLI configuration services ntp authkey. clownfish:configuration services ntp> authkey clownfish:configuration services ntp authkey> create. clownfish:configuration services ntp authkey> create clownfish:configuration services ntp authkey-000 (uncommitted)> get keyno = (unset) type = (unset) key = (unset) clownfish:configuration services ntp authkey-000 (uncommitted)> set keyno=1 keyno = 1 (uncommitted) clownfish:configuration services ntp authkey-000 (uncommitted)> set type=a type = A (uncommitted) clownfish:configuration services ntp authkey-000 (uncommitted)> set key=coconuts key = ******** (uncommitted) 4 Services 247

NTP clownfish:configuration services ntp authkey-000 (uncommitted)> commit clownfish:configuration services ntp authkey> CLI serverkeys. servers. 0., "gefilte" "carp". clownfish:configuration services ntp> set servers=gefilte,carp servers = gefilte,carp (uncommitted) clownfish:configuration services ntp> set serverkeys=1,1 serverkeys = 1,1 (uncommitted) clownfish:configuration services ntp> commit clownfish:configuration services ntp> "gefilte"1, "carp"2, "dory"3. clownfish:configuration services ntp> set servers=gefilte,carp,dory servers = gefilte,carp,dory (uncommitted) clownfish:configuration services ntp> set serverkeys=1,2,3 serverkeys = 1,2,3 (uncommitted) clownfish:configuration services ntp> commit clownfish:configuration services ntp> "gefilte" "carp"1 NTP "dory". clownfish:configuration services ntp> set servers=gefilte,carp,dory servers = gefilte,carp,dory (uncommitted) clownfish:configuration services ntp> set serverkeys=1,1,0 serverkeys = 1,1,0 (uncommitted) clownfish:configuration services ntp> commit clownfish:configuration services ntp> BUI BUI ( ) ( ). NTP "SYNC". SMB 5. NTP SMB. 248 Sun ZFS Storage 7000 2012 12 E38995-01

Remote Replication network-ntp:default NTP Services Logs. NTP 1 2 BUI. NTP. "SYNC". Remote Replication Sun Storage 7000.... (Sources ) (Targets ).. 4 Services 249

Dynamic Routing Dynamic Routing RIP RIPng RIP(Routing Information Protocol) RIP - ( ) -. IPv4 RIPv1 RIPv2 IPv6 RIPng. "".. RIP RIPng UDP 520 521. network-routing-route:default network-routing-ripng:quagga RIP. RIPng. Phone Home Phone Home Phone Home. Oracle Oracle. Phone Home. Phone Home Oracle. * - Oracle... * - Oracle. Oracle. 250 Sun ZFS Storage 7000 2012 12 E38995-01

Phone Home * - Oracle.. Oracle Single Sign-On Phone Home Oracle Single Sign-On. http://support.oracle.com (http://support.oracle.com) Register. Services BUI CLI. CLI Phone Home scrk. Phone Home Oracle HTTP... Use proxy Host/port Username Password. IP... Oracle Single Sign-On. http://support.oracle.com (http://support.oracle.com) Register.BUI. 1. Oracle Single Sign-On.. BUI CLI. 2.. '$'. 3.. 4 Services 251

SNMP CLI. 1. soa_id soa_password Oracle Single Sign-On. 2.. 3. domain_name. 4... 'Change account...' Oracle Single Sign-On..... Last heartbeat sent at Oracle. Oracle Single Sign-On Phone Home. Phone Home Oracle Single Sign-On. Maintenance->Logs->Phone Home Phone Home. SNMP SNMP(Simple Network Management Protocol) 2. SNMP. MIB. 252 Sun ZFS Storage 7000 2012 12 E38995-01

SNMP SNMP. SNMP 1 2c. SNMP community name Authorized network Appliance contact Trap destinations SNMP. SNMP CIDR. 127.0.0.1/8(localhost ), 0.0.0.0/0. MIB-II OID (.1.3.6.1.2.1.1.4.0). SNMP. SNMP IP. SNMP. 127.0.0.1. Services BUI CLI. CLI. SNMP. SNMP MIB-II. ID. MIB SNMP MIB(Management Information Base). MIB.1.3.6.1.2.1.1 MIB-II -,.1.3.6.1.2.1.2 MIB-II -.1.3.6.1.2.1.4 MIB-II IP - IP.1.3.6.1.4.1.42 Sun Enterprise MIB(SUN-MIB.mib.txt).1.3.6.1.4.1.42.2.195 Sun FM - ( MIB ).1.3.6.1.4.1.42.2.225 Sun AK - ( MIB ) 4 Services 253

SNMP SunFMMIB Sun FM MIB(SUN-FM-MIB.mib) SUN Fault Manager. Fault Manager Fault Manager 4 OID. OID.1.3.6.1.4.1.42.2.195.1.1.1.3.6.1.4.1.42.2.195.1.2.1.3.6.1.4.1.42.2.195.1.3.1.3.6.1.4.1.42.2.195.1.5 MIB. SunAKMIB Sun AK MIB(SUN-AK-MIB.mib)., -, 3 OID. OID.1.3.6.1.4.1.42.2.225.1.4.1.3.6.1.4.1.42.2.225.1.5.1.3.6.1.4.1.42.2.225.1.6 MIB. 254 Sun ZFS Storage 7000 2012 12 E38995-01

SMTP SNMP. BUI CLI. SNMP 1 2 3 4 1 2 3 4 SNMP,. SNMP. 127.0.0.1. /.. SNMP,. SNMP. 127.0.0.1/8. /.. SMTP SMTP Alerts. SMTP. SMTP DNS(MX ). DNS 4 Services 255

SMTP DNS MX smarthost. Send mail through smarthost Smarthost hostname Allow customized from address Custom from address. DNS.. From Custom. From. From. Services BUI CLI.. DNS,. DNS smarthost. network-smtp:sendmail mail SMTP. SMTP ( ). Services Logs. 256 Sun ZFS Storage 7000 2012 12 E38995-01

System Identity Service Tags. Oracle Oracle.. Discovery Port Listener Port UDP. 6481. TCP. 6481. Services BUI CLI. System Identity.. 4 Services 257

SSH System Name System Location. DNS ( DNS )... SNMP MIB-II syslocation. Services BUI CLI. system-identity:node ID. Services Logs. SSH SSH(Secure Shell) CLI BUI. SSH Analytics. Server key length. 768 Key regeneration interval ( ). 3600 258 Sun ZFS Storage 7000 2012 12 E38995-01

Shadow Migration Login grace period ( ) SSH. 120 Permit root login SSH. yes Services BUI CLI. CLI. network-ssh:default SSH Services Logs.. BUI CLI. SSH 1 2 SSH false. /. Shadow Migration.... 4 Services 259

Syslog... Number of Threads. (, I/O CPU). Services BUI CLI.. Syslog Syslog 2. Syslog. Syslog Syslog. Syslog ( printf ).. facility -. severity -. timestamp - UTC. hostname -. 260 Sun ZFS Storage 7000 2012 12 E38995-01

Syslog tag -.. message -.. Solaris Linux Syslog. Syslog. Syslog. Syslog RFC 3164 "" RFC 5424. Syslog UDP... Protocol Version Destinations Syslog, Classic Modern. IPv4 IPv6. Services BUI CLI. CLI. Syslog. Syslog: RFC 3164 Syslog facility level, timestamp, hostname, tag. tag. hostname System Identity. 4 Services 261

Syslog Syslog: RFC 5424 Syslog facility level, (1), timestamp, hostname, app-name. Sun Storage Syslog RFC 5424 procid, msgid structured-data nil (-). app-name. hostname System Identity. Syslog. Sun Storage Syslog ak., Sun Storage. ak. ak SUNW-MSG-ID. idmap POSIX Windows ID ID. smbd SMB. Send Syslog Message Syslog.. SUNW-MSG-ID Sun. Sun,., sun.com/msg/. 262 Sun ZFS Storage 7000 2012 12 E38995-01

Syslog TYPE VER SEVERITY EVENT-TIME PLATFORM CSN HOSTNAME SOURCE REV EVENT-ID DESC AUTO-RESPONSE REC-ACTION.. Fault, Defect, Alert.. SUNW-MSG-ID "1". "1" VER. 1.... UTC "Day Mon DD HH:MM:SS YYYY". : Fri Aug 14 21:34:22 2009. Oracle.. System Identity.. Oracle.. Oracle. UUID(Universally Unique Identifier). Oracle, UUID. Oracle EVENT-ID Oracle..., DRAM... SEVERITY. Syslog Minor LOG_WARNING. 4 Services 263

Syslog Syslog Major LOG_ERR. Critical LOG_CRIT. Syslog.. Syslog. Solaris Solaris Syslog syslogd(1m). Syslog Solaris svccfg svcadm Syslog. # svccfg -s system/system-log setprop config/log_from_remote = true # svcadm refresh system/system-log Solaris syslogd Syslog. Solaris syslog.conf(4). Solaris syslogd /var/adm/messages. Aug 14 21:34:22 poptart.sf.fishpong.com poptart ak: SUNW-MSG-ID: AK-8000-LM, \ TYPE: alert, VER: 1, SEVERITY: Minor\nEVENT-TIME: Fri Aug 14 21:34:22 2009\n\ PLATFORM: i86pc, CSN: 12345678, HOSTNAME: poptart\n\ SOURCE: jsui.359, REV: 1.0\n\ EVENT-ID: 92dfeb39-6e15-e2d5-a7d9-dc3e221becea\n\ DESC: A test alert has been posted.\n\ AUTO-RESPONSE: None.\nIMPACT: None.\nREC-ACTION: None. Linux Linux Syslog sysklogd(8). Syslog Linux -r ( ) /etc/sysconfig/syslog. SYSLOGD_OPTIONS="-r -m 0". 264 Sun ZFS Storage 7000 2012 12 E38995-01

Syslog # /etc/init.d/syslog stop # /etc/init.d/syslog start Linux Syslog UDP ipfilter. Syslog UDP INPUT. # iptables -I INPUT 1 -p udp --sport 514 --dport 514 -j ACCEPT Linux syslogd /var/log/messages. Aug 12 22:03:15 192.168.1.105 poptart ak: SUNW-MSG-ID: AK-8000-LM, \ TYPE: alert, VER: 1, SEVERITY: Minor EVENT-TIME: Wed Aug 12 22:03:14 2009 \ PLATFORM: i86pc, CSN: 12345678, HOSTNAME: poptart SOURCE: jsui.3775, REV: 1.0 \ EVENT-ID: 9d40db07-8078-4b21-e64e-86e5cac90912 \ DESC: A test alert has been posted. AUTO-RESPONSE: None. IMPACT: None. \ REC-ACTION: None. 4 Services 265

266

5 5 Shares Shares.. shares.. -. Shadow Migration -. -. 267

-. Shares - LUN. -. - (NFS, SMB, iscsi ). -. -. -. -. -. -. -. -. -. Schema -. ZFS. ZFS LUN. LUN.... RAID-Z. BUI. CLI 'Pool'.. UI. 268 Sun ZFS Storage 7000 2012 12 E38995-01

LUN..,. ( : )... Shares LUN. SMB, NFS, HTTP/WebDav FTP.LUN iscsi FC( ). project/share.. LUN... Inherited Read-only Space Management,.... SMB.... ( ). 100G. 5 Shares 269

Create time Project default Filesystem local LUN local Custom LUN.,,.. LUN,.,.,... LUN... schema. LUN...,..zfs/snapshot..zfs..LUN.,.,.... 270 Sun ZFS Storage 7000 2012 12 E38995-01

Shadow Migration :.. Oracle. "Oracle ("SLA") ".,.... Shadow Migration.. Sun Storage 7000 2.,.,. NAS. Sun Storage 7000.. 5 Shares 271

Shadow Migration X X Y.... rsync.......,. X Y M. M,. ( : ). (,, ).. ( ).. Shadow Migration 272 Sun ZFS Storage 7000 2012 12 E38995-01

Shadow Migration. NFS " ".., X Sun Storage 7000. /.....,.. ID. ACL( ) ID... 5 Shares 273

Shadow Migration * *.,. NFS. NFSv4. NFSv2 NFSv3 ACL, NFSv2. SMB. LUN...,... (UNIX stat st_nblocks). du(1) 0..... SUNWshadow,..SUNWshadow.. 274 Sun ZFS Storage 7000 2012 12 E38995-01

Shadow Migration.. NFSv4 (" " ). ID ACL ACL ID.. NFSv4 mapid. NFSv4. NFSv3. ID () POSIX ACL.. "nobody" NFSv4 mapid. ( )' '.. BUI Filesystem Creation. CLI shadow.. Local - file:///<path> NFS - nfs://<host>/<path> BUI NFS <host>:/<path>. UNIX.BUI (file:// nfs://).. 5 Shares 275

Shadow Migration......... BUI....... ( : ).......,... 276 Sun ZFS Storage 7000 2012 12 E38995-01

Shadow Migration ( UI ) BUI CLI...,.....,..,..... ( : ) ''. BUI. CLI shadow cancel.. ( : SUNWshadow ). 5 Shares 277

Shadow Migration...... NDMP... ( )......NDMP.. analytics Analytics. analytics "Data Movement"... 278 Sun ZFS Storage 7000 2012 12 E38995-01

Shadow Migration.,,. ( ).. (,ACL ).. analytics.,...,,... file shadow... 1. Sun Storage 7000. NFS ( ). 5 Shares 279

2. CLI 'nfs://<host>/<snapshotpath>' BUI '<host>/<snapshotpath>' ( 'NFS' ) shadow.. /. 3. ID. 4.. NFS. 1.. 2. Sun Storage 7000. NFS ( ). 3... 4. CLI 'nfs://<host>/<path>' BUI '<host>/<path>' ( 'NFS' ) shadow. 5. SS7000... CLI. 7000 LUN. LUN.. 280 Sun ZFS Storage 7000 2012 12 E38995-01

LUN.,LUN............. Referenced Data... Snapshot Data.. 5 Shares 281

Quota.,,. Reservation....... ( ) "".. "".. 100%.... Sun Storage 7000. 2........, 282 Sun ZFS Storage 7000 2012 12 E38995-01

2 X Y,, Z, Z X Y. X X Y...... ( : )........ LUN....,.. 'include snapshots'.. 5 Shares 283

. ( )..,... Shares -> General -> Space Usage -> Data BUI. CLI quota quota_snap.... 'include snapshots'.. ( )., 50G 100G 50G 150G.. Shares -> General -> Space Usage -> Data BUI. CLI reservation reservation_snap.... 284 Sun ZFS Storage 7000 2012 12 E38995-01

BUI BUI "Space Usage -> Users and Groups" "Shares -> General".... User or Group -.. User -. Group -... "Show All". ( ). UNIX Windows ( ) ( ). CLI CLI users groups. show. get. clownfish:> shares select default clownfish:shares default> users clownfish:shares default users> list USER NAME USAGE user-000 root 325K user-001 ahl 9.94K user-002 eschrock 20.0G clownfish:shares default users> select name=eschrock clownfish:shares default user-002> get name = eschrock unixname = eschrock unixid = 132651 winname = (unset) winid = (unset) usage = 20.0G. POSIX Windows ID.. 5 Shares 285

..,...,...,.... NDMP.. BUI Space Usage -> Users & Groups general.. "".. 'Apply'.... CLI CLI 'users' 'groups'. 'set quota'.. 'quota'.. '0'. clownfish:> shares select default select eschrock clownfish:shares default/eschrock> users clownfish:shares default/eschrock users> list 286 Sun ZFS Storage 7000 2012 12 E38995-01

USER NAME USAGE QUOTA user-000 root 321K - user-001 ahl 9.94K - user-002 eschrock 20.0G - clownfish:shares default/eschrock users> select name=eschrock clownfish:shares default/eschrock user-002> get name = eschrock unixname = eschrock unixid = 132651 winname = (unset) winid = (unset) usage = 20.0G quota = (unset) clownfish:shares default/eschrock user-002> set quota=100g quota = 100G (uncommitted) clownfish:shares default/eschrock user-002> commit clownfish:shares default/eschrock user-002> done clownfish:shares default/eschrock users> quota clownfish:shares default/eschrock users quota (uncomitted)> set name=bmc name = bmc (uncommitted) clownfish:shares default/eschrock users quota (uncomitted)> set quota=200g quota = 200G (uncommitted) clownfish:shares default/eschrock users quota (uncomitted)> commit clownfish:shares default/eschrock users> list USER NAME USAGE QUOTA user-000 root 321K - user-001 ahl 9.94K - user-002 eschrock 20.0G 100G user-003 bmc - 200G ID ID. UNIX Windows ID. ID. UNIX Windows ID. UNIX UNIX ID.,IDUNIX ID. UNIX ID. Windows. ID ( : ). ID.. 5 Shares 287

... /export. /export...... 0755.,,.... ( )..... 288 Sun ZFS Storage 7000 2012 12 E38995-01

NFSv2/NFSv3 NFS MOUNT. NFSv2 NFSv3.. NFSv4. NFSv4 NFSv4 NFSv3.., /export/home /export. NFSv4 (Linux ) " "... SMB SMB.. ( ). FTP/FTPS/SFTP.... HTTP/HTTPS /shares /export/home HTTP/HTTPS /shares/export/home.. FTP HTTP. 5 Shares 289

Shares Shares BUI UI "Shares -> Shares"...,,. "Filesystems" "LUNs".. Name Size Mountpoint GUID.......LUN... NFS FTP HTTP. SMB. LUN SCSI GUID. iscsi....... 290 Sun ZFS Storage 7000 2012 12 E38995-01

Shares.... [[Shares:Projects ]]... ( ). ( ) ( ). 0. Available space... Referenced data. LUN.. 0. Snapshot data... 5 Shares 291

Shares Unused Reservation. LUN.... Total space,. ( ).,. Compression ratio..,2x 1/2.. Case sensitivity.. BUI CLI Mixed mixed. NFS, FTP HTTP. SMB.. SMB. SMB " ". 292 Sun ZFS Storage 7000 2012 12 E38995-01

Shares BUI CLI Insensitive insensitive ( : NFS). SMB. SMB 2. Sensitive sensitive SMB. SMB. Windows. Reject non UTF-8 UTF-8. UTF-8. NFSv3. NFSv4 UTF-8 SMB. "on". SMB UTF-8. NFSv3 "off". "off" SMB NFSv4 NFSv3 UTF-8. "none" "on". Normalization ( )... "none"( ) "Reject non UTF-8" "on". Wikipedia. BUI CLI None none. 5 Shares 293

Shares BUI CLI Form C formc NFC(Normalization Form Canonical Composition) -. Form D formd NFD(Normalization Form Canonical Decomposition) -. Form KC formkc NFKC(Normalization Form Compatibility Composition) -. Form KD formkd NFKD(Normalization Form Compatibility Decomposition) -. Volume block size LUN. 512 128K 2 8K. Origin. Data Migration Source NFS. Shadow Migration. BUI. "Projects".... 294 Sun ZFS Storage 7000 2012 12 E38995-01

Shares. "All" "Shares".... "shares". "Filesystems" "LUNs"... : User Group Permissions Mountpoint Reject non UTF-8( ) Case sensitivity( ) Normalization( ) LUN : Volume size Thin provisioned Volume block size( ) CLI CLI shares.. clownfish:> shares clownfish:shares> select default clownfish:shares default> select foo clownfish:shares default/foo> get Properties: aclinherit = restricted (inherited) aclmode = discard (inherited) atime = true (inherited) 5 Shares 295

Shares casesensitivity = mixed checksum = fletcher4 (inherited) compression = off (inherited) compressratio = 100 copies = 1 (inherited) creation = Mon Oct 13 2009 05:21:33 GMT+0000 (UTC) mountpoint = /export/foo (inherited) normalization = none quota = 0 quota_snap = true readonly = false (inherited) recordsize = 128K (inherited) reservation = 0 reservation_snap = true secondarycache = all (inherited) nbmand = false (inherited) sharesmb = off (inherited) sharenfs = on (inherited) snapdir = hidden (inherited) utf8only = true vscan = false (inherited) sharedav = off (inherited) shareftp = off (inherited) space_data = 43.9K space_unused_res = 0 space_snapshots = 0 space_available = 12.0T space_total = 43.9K root_group = other root_permissions = 700 root_user = nobody filesystem lun.. clownfish:shares default> filesystem foo clownfish:shares default/foo (uncommitted)> get aclinherit = restricted (inherited) aclmode = discard (inherited) atime = true (inherited) checksum = fletcher4 (inherited) compression = off (inherited) copies = 1 (inherited) mountpoint = /export/foo (inherited) quota = 0 (inherited) readonly = false (inherited) recordsize = 128K (inherited) reservation = 0 (inherited) secondarycache = all (inherited) nbmand = false (inherited) sharesmb = off (inherited) sharenfs = on (inherited) snapdir = hidden (inherited) vscan = false (inherited) sharedav = off (inherited) 296 Sun ZFS Storage 7000 2012 12 E38995-01

Shares shareftp = off (inherited) root_group = other (default) root_permissions = 700 (default) root_user = nobody (default) casesensitivity = (default) normalization = (default) utf8only = (default) quota_snap = (default) reservation_snap = (default) custom:int = (default) custom:string = (default) custom:email = (default) clownfish:shares default/foo (uncommitted)> set sharenfs=off sharenfs = off (uncommitted) clownfish:shares default/foo (uncommitted)> commit clownfish:shares default> destroy. clownfish:shares default/foo> destroy This will destroy all data in "foo"! Are you sure? (Y/N) clownfish:shares default> rename. clownfish:shares default> rename foo bar clownfish:shares default> move. clownfish:shares default> move foo home clownfish:shares default> users groups.. CLI BUI. CLI get set. unset. CLI BUI BUI aclinherit ACL inheritance behavior aclmode ACL behavior on mode change 5 Shares 297

Shares CLI BUI BUI atime Update access time on read casesensitivity Case sensitivity Static checksum Checksum compression Data compression compresratio Compression ratio Static copies Additional replication creation - - dedup Data deduplication exported, Export initiatorgroup LUN Initiator Group logbias Synchronous write bias lunumber LUN LU number lunguid,lun GUID mountpoint Mountpoint nbmand Non-blocking mandatory locking nodestroy Prevent destruction normalization Normalization Static origin Origin Static quota Quota quota_snap Quota/Include snapshots readonly Read-only recordsize Database record size reservation Reservation reservation_snap Reservation/Include snapshots root_group Group root_permissions Permissions 298 Sun ZFS Storage 7000 2012 12 E38995-01

CLI BUI BUI root_user User rstchown Restrict ownership change secondary cache Cache device usage shadow Data Migration Source Static sharedav Protocols / HTTP / Share mdoe shareftp Protocols / FTP / Share mode sharenfs Protocols / NFS / Share mode sharesmb Protocols / SMB / Resource name snapdir.zfs/snapshot visibility space_available Available space Usage space_data Referenced data Usage space_snapshots Snapshot data Usage space_total Total space Usage space_unused_res Unused reservation Usage sparse LUN Thin provisioned targetgroup LUN Target Group utf8only Reject non UTF-8 Static volblocksize Volume block size Static vscan Virus scan BUI. CLI. CLI Shares CLI. 5 Shares 299

..... Volume size iscsilun. LUN. LUN. LUN. Thin provisioned. LUN LUN. LUN. iscsi. Thin provisioned. LUN. LUN... LUN LUN. LUN " "... Reservation..BUI CLI. 300 Sun ZFS Storage 7000 2012 12 E38995-01

... /export... ('/')., "home" /export/home "home/bob" /export/home/bob. SMB. SMB......,. Access control. Update access time on read.. POSIX.... Non-blocking mandatory locking POSIX SMB.. 5 Shares 301

POSIX. SMB. SMB.. Data deduplication.,,. Data Deduplication. Status Dashboard.. SHA-256..... Data Deduplication ( CPU ). Data Deduplication ( )..., 1GB 1GB 1GB 2x. :. DRAM... DRAM. analytics analytics "DMU ZFS DMU " ZFS DDT ( ). I/O I/O. 302 Sun ZFS Storage 7000 2012 12 E38995-01

Data compression.. CPU.... BUI CLI Off off. LZJB( ) lzjb CPU. GZIP-2() gzip-2 gzip. GZIP( ) gzip gzip. GZIP-9( ) gzip-9 gzip. CPU. Checksum. ( ).. (fletcher4) CPU. ( LUN ). BUI CLI Fletcher 2( ) fletcher2 16 fletcher Fletcher 4() fletcher4 32 fletcher SHA-256( ) sha256 SHA-256 Cache device usage. 5 Shares 303

...., "all".. (DRAM). BUI CLI All data and metadata all LUN. Metadata only metadata.. Do not use cache devices.. Synchronous write bias...... 'latency' 'throughput'.. BUI CLI Latency latency ( ). Throughput throughput.. 304 Sun ZFS Storage 7000 2012 12 E38995-01

Database record size.. 128K., 128K 128K. 128K. ( : ). 8K. 512 128K 2. Additional replication... (RAID, ).. BUI CLI Normal( ) 1.. Two Copies 2 2. Three Copies 3 3. Virus scan..... Virus Scan. 5 Shares 305

Prevent destruction.,.... off. Restrict ownership change.. "". setuid setgid.. schema... NFS, SMB, HTTP FTP iscsi LUN. BUI., "twofish""fs0". 306 Sun ZFS Storage 7000 2012 12 E38995-01

NFS SMB HTTP FTP SFTP twofish:/export/fs0 \\twofish\fs0 http://twofish/shares/export/fs0/ (http://twofish/shares/export/fs0/) ftp://twofish/export/fs0/ /export/fs0/ iscsi SAN. NFS BUI CLI Share mode off/ro/rw,. CLI "on" "rw". Disable setuid/setgid file creation Prevent clients from mounting subdirectories Anonymous user mapping nosuid nosub anon setuid(s_isuid) setgid(s_isgid) chmod(2)... : NFSv4 NFSv2 NFSv3. "root" uid. ACL. Character encoding See below.. Security mode See below..... 5 Shares 307

CLI Host(FQDN) or Netgroup none IP IP caji.sf.example.com DNS Domain. IP Network @ IP CIDR IP sf.example.com 192.168.20.0/22 / ( ). NIS( ) LDAP( ).LDAP ou=netgroup,(base DN). NFS.. DNS., NFS IP FQDN... DNS.. IP DNS. CLI CLI NFS "sharenfs".. BUI "ro", "rw", "off"., set sharenfs=ro. "nobody". "nosuid" "anon". CLI "=". uid 153762. 308 Sun ZFS Storage 7000 2012 12 E38995-01

set sharenfs="ro,anon=153762" "option=collection". "option" "ro", "rw", "root". DNS / CIDR., sf.example.com / 192.168.44.0/24. set sharenfs="ro,anon=153762,rw=.sf.example.com,root=@192.168.44.0/24"., "engineering" /. set sharenfs="ro,rw=engineering" "option=mode". option "sec" mode "sys", "krb5", "krb5:krb5i", "krb5:krb5i:krb5p". set sharenfs="sec=krb5". Kerberos. krb5 krb5i krb5p Kerberos V5 krb5 ( ) krb5 ( ) krb5p krb5i krb5 krb5i krb5. 5 Shares 309

. NFSv3 NFSv2. NFSv4 UTF-8. UTF-8.,. SMB UTF-8. SMB... euc-cn euc-jp euc-jpms euc-kr euc-tw iso8859-1 iso8859-2 iso8859-5 iso8859-6 iso8859-7 iso8859-8 iso8859-9 iso8859-13 iso8859-15 koi8-r (pass-through). BUI. CLI '*'., set sharenfs="rw,euc-kr=*" 'euc-kr'. set sharenfs="rw,euc-kr=host1.domain.com,euc-jp=host2.domain.com" 'euc-kr' 'euc-jp' 'host1' 'host2'. CLI NFS. NFS. NFS. 310 Sun ZFS Storage 7000 2012 12 E38995-01

SMB Resource name Enable Access-based Enumeration Is a DFS Namespace Share-level ACL SMB. "off" SMB "on"..... DFS. ACL ACL. ACL. ACL ACL. SMB. SMB ACL. 2 SMB... 80. " /\[]: <>+;,?*=.., / ACL. SCSI Target group Initiator group LUN. LUN. 5 Shares 311

LU (logical unit) number Assigned LU number Operational status Fix LU number Write cache behavior GUID LUN LUN. 0. 2 LUN. LUN LU. LUN. LUN. LU. LU. LU. LUN.... LUN GUID SCSI. GUID. HTTP Share mode HTTP. none, read only, read/write. FTP Share mode FTP. none, read only, read/write. 312 Sun ZFS Storage 7000 2012 12 E38995-01

SFTP Share mode SFTP. none, read only, read/write. ACL...... ID. Unix Windows ID. Unix NFS chown.. ID. Unix Windows ID. Unix NFS chgrp. Permissions Unix. Unix NFS chmod.. 5 Shares 313

User Other.... Read R. Write W.* Execute X.. 2011.1 "write". (ACL D ).,. (ACL A ). (ACL W ). BUI. ("user", "group" "other"). CLI Unix 8. user, group other. read(4), write(2) execute(1). 743 user RWX, group R, other WX. POSIX "Use Windows Default Permissions". ACL ACL. Windows Windows. ACL ACL ACL ACL. 314 Sun ZFS Storage 7000 2012 12 E38995-01

ACL behavior on mode change Unix // chmod(2) ACL ACL. BUI CLI Discard ACL discard ACL.. Mask ACL with mode mask UID. ACL. ACL ACL. Do not change ACL passthrough ACL ACL. ACL inheritance behavior ACL.. ACL.. ACL SMB. ACL Unix owner/group/other. BUI CLI Do not inherit entries discard ACL.. Only inherit deny entries Inherit all but "write ACL" and "change owner" noallow restricted "deny" ACL. ACL "write_acl" "write_owner" ACL.. Inherit all entries passthrough ACL. "passthrough" "data". 0664 0666 ACL. 5 Shares 315

BUI CLI Inherit all but "execute" when not specified passthrough-x owner, group everyone ACL 'passthrough'. "passthrough" ACL. "cc" "gcc". ACL chmod(1). SMB ACL ACL.. SMB NFS. SMB ACL. NFS. SMB ACL. NFS ACL. ACE( ) ACLACL ACL. SMB ACL ACL.. ACL. ACL ( ). SMB NFSv4 ACL. NFSv3 POSIX ACL. ACL NFSv3 ACL NFSv3..ACL BUI ACL. CLI ACL.BUI. ACL.ACL. SMB ACL ACL. 316 Sun ZFS Storage 7000 2012 12 E38995-01

ACLACE( ). ACE /,,.ACE ACL. ACL.ACL BUI. Owner Group Everyone Named User Named Group. ACE.. ACE. 'target'. ID. 'target'. ID. Allow Deny ACE. ACE. (r) Read Read Data/List Directory.. (x) Execute File/Traverse Directory (p) Append Data/Add Subdirectory ().... ( ). (a) Read Attributes ( ACL). stat ls stat. (R) Read Extended Attributes. 5 Shares 317

Write (w) Write Data/Add File (d) Delete (D) Delete Child (A) Write Attributes (W) Write Extended Attributes..... 2011.1... (c) Admin Read ACL/Permissions ACL. (C) Write ACL/Permissions (o) Change Owner ACL.. Inheritance (f) Apply to Files. (d) Apply to Directories. (i) Do not apply to self ACE. "Apply to Files" "Apply to Directories". (n) Do not apply past children ACE. "Apply to Files" "Apply to Directories". Windows 3 ACL. Owner Allow Full Control Group Allow Read and Execute 318 Sun ZFS Storage 7000 2012 12 E38995-01

Everyone Allow Read and Execute...zfs/snapshot visibility.zfs/snapshot. ( ). '.zfs'.. BUI CLI Hidden hidden.zfs.. Visible visible.zfs. BUI "snapshots".. "Snapshots". "Schedules". "Snapshots",,.. 5 Shares 319

Name Creation Unique Total Clones... ".auto-<timestamp>"..rr ".ndmp". NDMP... LUN....... 0 "Show...".. "Snapshots".. "apply". (, )........ 320 Sun ZFS Storage 7000 2012 12 E38995-01

LUN....,....... Project Name Preserve Local Properties Mountpoint...... "Preserve local properties".. '.auto-<timestamp>' 30,,,.. "Schedules".. 5 Shares 321

Frequency Offset Keep at most half hour, "hour", "day", "week", "month"...,.,.. ( 48 24 30 )........... 2010.Q3. 30... CLI snapshots. clownfish:> shares select default select builds clownfish:shares default/builds> snapshots clownfish:shares default/builds snapshots> CLI. clownfish:shares default/builds snapshots> list today 322 Sun ZFS Storage 7000 2012 12 E38995-01

yesterday clownfish:shares default/builds snapshots> snapshot. clownfish:shares default/builds snapshots> snapshot test clownfish:shares default/builds snapshots> rename. clownfish:shares default/builds snapshots> rename test test2 clownfish:shares default/builds snapshots> destroy. clownfish:shares default/builds snapshots> select test2 clownfish:shares default/builds@test2> destroy This will destroy this snapshot. Are you sure? (Y/N) clownfish:shares default/builds snapshots> destroy. clownfish:shares default/builds snapshots> destroy test2 This will destroy this snapshot. Are you sure? (Y/N) clownfish:shares default/builds snapshots> rollback. clownfish:shares default/builds snapshots> select today clownfish:shares default/builds@today> rollback Rolling back will revert data to snapshot, destroying newer data. Active initiators will be disconnected. Continue? (Y/N) clownfish:shares default/builds@today> clone... 5 Shares 323

clownfish:shares default/builds snapshots> select today clownfish:shares default/builds@today> clone testbed clownfish:shares default/testbed (uncommitted clone)> get aclinherit = restricted (inherited) aclmode = discard (inherited) atime = true (inherited) checksum = fletcher4 (inherited) compression = off (inherited) copies = 1 (inherited) mountpoint = /export/testbed (inherited) quota = 0 (default) readonly = false (inherited) recordsize = 128K (inherited) reservation = 0 (default) secondarycache = all (inherited) nbmand = false (inherited) sharesmb = off (inherited) sharenfs = on (inherited) snapdir = hidden (inherited) vscan = false (inherited) sharedav = off (inherited) shareftp = off (inherited) root_group = other (default) root_permissions = 777 (default) root_user = nobody (default) quota_snap = true (default) reservation_snap = true (default) clownfish:shares default/testbed (uncommitted clone)> set quota=10g quota = 10G (uncommitted) clownfish:shares default/testbed (uncommitted clone)> commit clownfish:shares default/builds@today>.. automatic. create destroy., BUI. clownfish:shares default/builds snapshots> automatic clownfish:shares default/builds snapshots automatic> create clownfish:shares default/builds snapshots automatic (uncommitted)> set frequency=day frequency = day (uncommitted) clownfish:shares default/builds snapshots automatic (uncommitted)> set hour=14 hour = 14 (uncommitted) clownfish:shares default/builds snapshots automatic (uncommitted)> set minute=30 minute = 30 (uncommitted) clownfish:shares default/builds snapshots automatic (uncommitted)> set keep=7 keep = 7 (uncommitted) clownfish:shares default/builds snapshots automatic (uncommitted)> get frequency = day (uncommitted) day = (unset) hour = 14 (uncommitted) 324 Sun ZFS Storage 7000 2012 12 E38995-01

minute = 30 (uncommitted) keep = 7 (uncommitted) clownfish:shares default/builds snapshots automatic (uncommitted)> commit clownfish:shares default/builds snapshots automatic> list NAME FREQUENCY DAY HH:MM KEEP automatic-000 day - 14:30 7 clownfish:shares default/builds snapshots automatic> done clownfish:shares default/builds snapshots> BUI UI "Shares -> Projects"....... Name Size.......... 5 Shares 325

,,...... ( ). 0.. Available space shares. Referenced data. shares. Snapshot data. shares. Unused Reservation... 326 Sun ZFS Storage 7000 2012 12 E38995-01

Unused Reservation of shares. shares. Total space,,... Compression ratio shares.. "Add...".. CLI CLI shares. select. clownfish:> shares clownfish:shares> select default clownfish:shares default> get aclinherit = restricted aclmode = discard atime = true checksum = fletcher4 compression = off compressratio = 100 copies = 1 creation = Thu Oct 23 2009 17:30:55 GMT+0000 (UTC) mountpoint = /export quota = 0 readonly = false 5 Shares 327

recordsize = 128K reservation = 0 secondarycache = all nbmand = false sharesmb = off sharenfs = on snapdir = hidden vscan = false sharedav = off shareftp = off default_group = other default_permissions = 700 default_sparse = false default_user = nobody default_volblocksize = 8K default_volsize = 0 space_data = 43.9K space_unused_res = 0 space_unused_res_shares = 0 space_snapshots = 0 space_available = 12.0T space_total = 43.9K clownfish:shares default> project.. clownfish:shares> project home clownfish:shares home (uncommitted)> get mountpoint = /export (default) quota = 0 (default) reservation = 0 (default) sharesmb = off (default) sharenfs = on (default) sharedav = off (default) shareftp = off (default) default_group = other (default) default_permissions = 700 (default) default_sparse = true (default) default_user = nobody (default) default_volblocksize = 8K (default) default_volsize = 0 (default) aclinherit = (default) aclmode = (default) atime = (default) checksum = (default) compression = (default) copies = (default) readonly = (default) recordsize = (default) secondarycache = (default) nbmand = (default) snapdir = (default) vscan = (default) custom:contact = (default) 328 Sun ZFS Storage 7000 2012 12 E38995-01

custom:department = (default) clownfish:shares home (uncommitted)> set sharenfs=off sharenfs = off (uncommitted) clownfish:shares home (uncommitted)> commit clownfish:shares> destroy. clownfish:shares> destroy home This will destroy all data in "home"! Are you sure? (Y/N) clownfish:shares>. rename. clownfish:shares> rename default home clownfish:shares> /. CLI. set. clownfish:shares (pool-0)> set pool=pool-1 clownfish:shares (pool-1)> CLI. CLI BUI. CLI get set. unset. CLI BUI BUI aclinherit ACL inheritance behavior aclmode ACL behavior on mode change atime Update access time on read checksum Checksum compression Data compression 5 Shares 329

CLI BUI BUI compressratio Compression ratio Static copies Additional replication creation - - dedup Data deduplication default_group creation default Group default_permissions creation default Permissions default_sparse creation default Thin provisioned default_user creation default User default_volblocksize creation default Volume block size default_volsize creation default Volume size mountpoint Mountpoint nbmand Non-blocking mandatory locking quota Quota readonly Read-only recordsize Database record size reservation Reservation secondary cache Cache device usage sharedav Protocols / HTTP / Share mdoe shareftp Protocols / FTP / Share mode sharenfs Protocols / NFS / Share mode sharesmb Protocols / SMB / Resource name snapdir.zfs/snapshot visibility space_available Available space Usage space_data Referenced data Usage space_snapshots Snapshot data Usage space_total Total space Usage 330 Sun ZFS Storage 7000 2012 12 E38995-01

CLI BUI BUI space_unused_res Unused reservation Usage space_unused_res_shares Unused reservation of shares Usage vscan Virus scan BUI. CLI. CLI CLI.... Quota LUN. shares.. Reservation LUN. shares..... 5 Shares 331

Mountpoint Read only Update access time on read Non-blocking mandatory locking Data compression Data deduplication Checksum Cache device usage Database record size Additional replication Virus scan. schema.... User Group Permissions LUN LUN. LUN.. Volume size Thin provisioned Volume block size 332 Sun ZFS Storage 7000 2012 12 E38995-01

. shares... NFS NFS shares. SMB Resource name Enable Access-based Enumeration SMB..... 2 SMB.. "off" "on" off "on" SMB. SMB. <project's resource name>_<filesystem name>. iscsi iscsi. 5 Shares 333

HTTP HTTP shares. FTP FTP shares. ACL. ACL.. ACL behavior on mode change ACL inheritance behavior.. LUN ( ).. ( )..zfs/snapshot visibility. 334 Sun ZFS Storage 7000 2012 12 E38995-01

BUI... CLI snapshots. clownfish:> shares select default clownfish:shares default> snapshots clownfish:shares default snapshots>... 5 Shares 335

Remote Replication :.. Oracle. "Oracle ("SLA") ". Sun Storage 7000,.. ( "")... ( )..... ( )...... 7000...... 336 Sun ZFS Storage 7000 2012 12 E38995-01

.... NAS SAN.. Shares ZFS.., NFS... Sun Storage 7000 SSL. SSL... ( ): Sun Storage 7000. ( ): ().. ( ): ().. ( ): ( ).. ( ):, (, ). :... /. 5 Shares 337

( ):. : ( ). 2. Configuration > Services > Remote Replication.. ( BUI CLI ) ( ) ( ).... ID ( ) SSL.. A B B A. B AB A ( ),..,... ( ) ( ) (, ) 338 Sun ZFS Storage 7000 2012 12 E38995-01

( )........... ( ). /....BUI CLI -. -... ( ).. 2010.Q1.. 2010.Q1.. 5 Shares 339

.... IP IP. NAS (NFS SMB). IP. IP. IP.. Shares....... ( )...,. home bill, cindi dave home. home/bill home/cindi. home/dave. 340 Sun ZFS Storage 7000 2012 12 E38995-01

.. home, home/bill home/cindi home home/dave...,. Configuration > Services > Remote Replication.BUI Targets. CLI targets hostname, root_password label. knife:> configuration services replication targets. ( ) 5 Shares 341

( )... BUI Replication CLI "replication".. BUI CLI. 342 Sun ZFS Storage 7000 2012 12 E38995-01

(CLI ) Target Pool Enabled Mode(CLI: continuous) and schedule Include Snapshots Limit bandwidth Use SSL State Last sync.....,.... ( ). SSL..,,... 5 Shares 343

(CLI ) Last attempt Next update... ( ), "manual" "continuous". : Manual, Scheduled Continuous,.... ( )... Sun Storage.. "Include Snapshots" true ( )..., BUI CLI sendupdate. (.).. BUI sending. cancelupdate.. 344 Sun ZFS Storage 7000 2012 12 E38995-01

..BUI CLI,,,..,,... 2009.Q3. 2010.Q1 ( )...,,,.,,. ( : sharenfs, SMB ). NAS,.. ( ).. BUI CLI.,. 5 Shares 345

2010.Q1. 2010.Q1. " " (, ). BUI BUI "Replica". Shares.. ( ) Replication ( ).. 346 Sun ZFS Storage 7000 2012 12 E38995-01

BUI.. CLI CLI shares replication sources.. ( ),. : loader:> shares replication sources loader:shares replication sources> show Sources: source-000 ayu PROJECT STATE LAST UPDATE package-000 oldproj idle unknown package-001 aproj1 receiving Sun Feb 21 2010 22:04:35 GMT+0000 (UTC) loader:shares replication sources> select source-000 loader:shares replication source-000> select package-001 loader:shares replication source-000 package-001> show Properties: enabled = true state = receiving state_description = Receiving update last_sync = Sun Feb 21 2010 22:04:40 GMT+0000 (UTC) last_try = Sun Feb 21 2010 22:04:40 GMT+0000 (UTC) Projects: aproj1 loader:shares replication source-000 package-001> select aproj1 loader:shares replication source-000 package-001 aproj1> get mountpoint mountpoint = /export loader:shares replication source-000 package-001 aproj1> get sharenfs sharenfs = on BUI ( ) Replication. ( ). 5 Shares 347

. CLI ( ) cancelupdate.... BUI ( ) Replication. (, ). CLI. CLI ( ) enabled... ( ).,. BUI clone CLI ( ). CLI BUI,. 348 Sun ZFS Storage 7000 2012 12 E38995-01

2009.Q3. 2010.Q1 ( )..,.. (, ).. ( recv-xxx). BUI CLI. NAS. ( )..,. " "......BUI CLI " ". 5 Shares 349

.. ( ). LUN.. (, ). ( )... BUI ( ), Replication,.. CLI ( ) sever... ( )....,., 350 Sun ZFS Storage 7000 2012 12 E38995-01

. /,IP,. ( ), /., /.... "None". DR ( ). " "... 1.. 2..,.. 3.., ( )...,.. 5 Shares 351

, ( ). BUI ( ), Replication,.. CLI ( ) reverse... ( )... DR.. BUI. CLI shares replication sources destroy...,. perch:> shares perch:shares> replication perch:shares replication> sources perch:shares replication sources> select source-000 perch:shares replication source-000> select package-000 perch:shares replication source-000 package-000> clone perch:shares replication source-000 package-000 clone> set target_project=my_clone target_project = my_clone perch:shares replication source-000 package-000 clone> list CLONE PARAMETERS target_project = my_clone original_mountpoint = /export override_mountpoint = false 352 Sun ZFS Storage 7000 2012 12 E38995-01

mountpoint = SHARE MOUNTPOINT bob (inherited) myfs1 (inherited) perch:shares replication source-000 package-000 clone> set override_mountpoint=true override_mountpoint = true perch:shares replication source-000 package-000 clone> set mountpoint=/export/my_clone mountpoint = /export/my_clone perch:shares replication source-000 package-000 clone bob> select bob perch:shares replication source-000 package-000 clone bob> set override_mountpoint=true override_mountpoint = true perch:shares replication source-000 package-000 clone bob> set mountpoint=/export/bob mountpoint = /export/bob perch:shares replication source-000 package-000 clone bob> done perch:shares replication source-000 package-000 clone> commit CLONE PARAMETERS target_project = my_clone original_mountpoint = /export override_mountpoint = true mountpoint = /export/my_clone SHARE MOUNTPOINT bob /export/bob (overridden) myfs1 (inherited) Are you sure you want to clone this project? There are no conflicts. perch:shares replication source-000 package-000 clone>. M11,M5. 1.Production System M11 Configuration > SERVICES > Remote Replication. Targets,. Name=M5 Host name=192.168.1.17 Root password=pppp$1234 2. Pool=Pool1. 3. Shares > PROJECTS. Projects. =P1 4. Shares > PROJECTS > P1 > Protocols. 5 Shares 353

SMB Resource Name=on. 5. Shares > PROJECTS > P1 > Shares. Filesystems. Name=S1 User=root Group=other Permissions=RWX RWX RWX 6. Shares > PROJECTS > P1 > Shares > S1 > Protocols. SMB \\192.168.1.7\S1 SMB S1. 7. Shares > PROJECTS > P1 > Replication. Actions target pool. Target=M5 Pool=Pool1 Schedule frequency. Frequency= 00 30 8. \\192.168.1.7\S1(user=root, password=pppp$1234). 9. F1.txt. 10. Shares > PROJECTS > P1 > Replication. 11. Action TARGET=M5 Update Now. Disable. STATUS.,... 1. M5 Pool=Pool1. 2. Shares > PROJECTS > REPLICA. s7000-0:p1. 354 Sun ZFS Storage 7000 2012 12 E38995-01

3. Shares > PROJECTS > REPLICA > s7000-0:p1 > Replication. Idle. 4. Reverse the direction of replication. New Project Name=P1 5. Shares > PROJECTS > REPLICA. s7000:p1. 6. Shares > PROJECTS > LOCAL. P1. 7. Shares > PROJECTS > P1 > Protocols. SMB Resource Name=on. 8. Shares > PROJECTS > P1 > Shares > S1 > Protocols. SMB \\192.168.1.17\S1 SMB S1. 9. \\192.168.1.17\S1( =root, =pppp$1234). F1.txt F2.txt. 10.. 11. Shares > PROJECTS > P1 > Replication. 12. Action TARGET=s7000-0 Update Now. Disable.... 1. M11 Shares > PROJECTS > LOCAL > P1. P1... 5 Shares 355

2. Shares > PROJECTS > REPLICA > s7000-1:p1 > Replication. Idle. 3. P1... 4. Reverse the direction of replication. =P1 5. \\192.168.1.7\S1(user=root, password=pppp$1234). F1.txt F2.txt. Remote Replication, Services Remote Replication Projects and Shares 2. rrsource rrtarget,,.,,,. ( : ). Projects and Shares. rrsource Remote Replication... 356 Sun ZFS Storage 7000 2012 12 E38995-01

( ). ( ). ( ). ZFS Storage 7000 ZFS Storage 7000., 2 IP. IP. CLUSTERED.... IP. IP. IP. ( ) IP ( ). IP.... 5 Shares 357

( ) ( ). ( ). ( ) ( ). ( ). ( )....,..,.. 1.. 2.. 3..,., (). 4., ( ).. 358 Sun ZFS Storage 7000 2012 12 E38995-01

.. 2. ( : )... ( ). ( ).,.... iscsi Shares. LUN., LUN... LUN. LUN SCSI GUID LUN. LUN LUN SCSI GUID. LUN GUID( LUN GUID). 2009.Q3. 2010.Q1. 5 Shares 359

.,..... "include snapshots" true.. analytics TCP analytics. 216. BUI CLI. " ".. Replication....,..,. 360 Sun ZFS Storage 7000 2012 12 E38995-01

RPC...... "2009.Q3 ".......,.,,........,.... 5 Shares 361

.,.,.... " ". ( ).,. ( )...,... 2009.Q3 2009.Q3 2010.Q1. 2009.Q3.. 2010.Q1.. 2009.Q3 (2010.Q1 ), 2010.Q1 2010.Q1... 2009.Q3 2010.Q1. 362 Sun ZFS Storage 7000 2012 12 E38995-01

*, 2009.Q3. BUI CLI. *.,,. Projects and Shares. 2009.Q3 2010.Q1...,.,.. BUI "Shares -> Schema".,.. NAME DESCRIPTION TYPE CLI. ".:_\". BUI. CLI... 5 Shares 363

BUI CLI String String.. Integer Integer Positive Integer PositiveInteger Boolean Boolean /.BUI, CLI "true" "false". Email Address EmailAddress.. Hostname or IP Host DNS IP(v4 v6).. CLI,.... CLI "shares -> schema". carp:> shares schema carp:shares schema> show Properties: NAME TYPE DESCRIPTION owner EmailAddress Owner Contact,. create. carp:shares schema> create department carp:shares schema department (uncommitted)> get type = String description = department carp:shares schema department (uncommitted)> set description="department Code" description = Department Code (uncommitted) carp:shares schema department (uncommitted)> commit carp:shares schema> CLI. 364 Sun ZFS Storage 7000 2012 12 E38995-01

carp:shares schema> select owner carp:shares schema owner> get type = EmailAddress description = Owner Contact carp:shares schema owner> set description="owner Contact Email" description = Owner Contact Email (uncommitted) carp:shares schema owner> commit "custom:<property>". carp:shares default> get... custom:department = 123-45-6789 custom:owner =... carp:shares default> set custom:owner=bob@corp custom:owner = bob@corp (uncommitted) carp:shares default> commit BUI 1. "Shares -> Schema". 2. '+'. 3. ("contact"). 4. ("Owner Contact"). 5. ("Email Address"). 6. "Apply". 7.. 8. "Custom Properties" "Owner Contact". CLI 1. (shares schema). 2. "contact" (create contact). 3. (set description="owner Contact"). 4. (set type=emailaddress). 5 Shares 365

5. (commit). 6.. 7. "custom:contact". 366 Sun ZFS Storage 7000 2012 12 E38995-01

6 6 Analytics Analytics analytics CPU NFSv3 367

Sun ZFS Storage Appliance analytics DTrace. Analytics.. - analytics - - Open Worksheets - analytics Saved Worksheets - analytics Datasets - analytics - Analytics Analytics.. DTrace. Analytics. Analytics...,. / NFSv3 / / CPU /, CPU. NFSv3 /, NFS "read". "NFS operations/sec of type read",. 368 Sun ZFS Storage 7000 2012 12 E38995-01

Analytics. " NFSv3 / "., NFS.(.) "kiowa" NFS. Analytics. "NFSv3 operations/sec of type read for client kiowa broken down by filename". kiowa NFS. kiowa. Analytics.. "NFSv3 / " " NFSv3 / " " NFSv3 / " " kiowa NFSv3 / " Analytics. Analytics. Analytics.,,,,, NFS,,,,, SMB,,, CPU-id, CPU Open Worksheets, Analytics Preferences.... 6 Analytics 369

Datasets. /. ( ). Open Worksheets....., Datasets (0 "on disk" ). 24x7.,,..,,..,.. "Older than",,,., 3 "Older than" "3" "weeks".... BUI.,..,. Open Worksheets, Saved Worksheets. 370 Sun ZFS Storage 7000 2012 12 E38995-01

Analytics. Analytics... Analytics CPU: Percent utilization * Cache: ARC accesses * Cache: L2ARC I/O bytes Cache: L2ARC accesses Data Movement: NDMP bytes transferred to/from disk Data Movement: NDMP bytes transferred to/from tape Data Movement: Shadow migration bytes Data Movement: Shadow migration ops Data Movement: Shadow migration requests Disk: Disks * Disk: I/O bytes * Disk: I/O operations * Network: Device bytes Network: Interface bytes Protocol: SMB operations Protocol: Fibre Channel bytes Protocol: Fibre Channel operations Protocol: FTP bytes Protocol: HTTP/WebDAV requests Protocol: iscsi bytes Protocol: iscsi operations Protocol: NFSv2 operations Protocol: NFSv3 operations Protocol: NFSv4 operations Protocol: SFTP bytes Protocol: SRP bytes 6 Analytics 371

Protocol: SRP operations * Advanced Analytics Preferences Analytics... CPU: CPUs CPU: Kernel spins Cache: ARC adaptive parameter Cache: ARC evicted bytes Cache: ARC size Cache: ARC target size Cache: DNLC accesses Cache: DNLC entries Cache: L2ARC errors Cache: L2ARC size Data Movement: NDMP file system operations Data Movement: NDMP jobs Disk: Percent utilization Disk: ZFS DMU operations Disk: ZFS logical I/O bytes Disk: ZFS logical I/O operations Memory: Dynamic memory usage Memory: Kernel memory Memory: Kernel memory in use Memory: Kernel memory lost to fragmentation Network: IP bytes Network: IP packets Network: TCP bytes Network: TCP packets System: NSCD backend requests System: NSCD operations,. Datasets 30. 372 Sun ZFS Storage 7000 2012 12 E38995-01

CPU CPU Cache Cache Cache Cache Cache Cache Data Movement Disk Disk Disk Disk Disk Disk Network Network Network Protocol Protocol Protocol Protocol Protocol Protocol Protocol Protocol Protocol CPU / ARC ARC ARC / DNLC / L2ARC L2ARC NDMP 95% I/O I/O I/O I/O I/O SMB SMB FTP HTTP/WebDAV NFSv2 NFSv2 NFSv3 6 Analytics 373

Protocol Protocol Protocol Protocol Protocol Protocol NFSv3 NFSv4 / NFSv4 SFTP iscsi iscsi,.. 1 2 3 4 5 6 7 " Protocol: NFSv3 operations. OpenWorksheets. : " Protocol: NFSv3 operations".. NFSv3. : " Protocol: NFSv3 operations".. 60.. 60. 374 Sun ZFS Storage 7000 2012 12 E38995-01

CPU 8 9 " Protocol: NFSv3 operations".?..,. CPU CPU: Percent utilization CPU. CPU,., 4 - CPU. 16 CPU. CPU. CPU 100%,. 100% CPU. CPU CPU: Percent utilization, NFSv3 2Gbytes/sec. 82% 2Gbytes/sec.( 81%. 1%.) CPU NFS. Protocol: NFSv3 operations( ). CPU.. CPU CPU. CPU : 6 Analytics 375

ARC CPU CPU. CPU. CPU. CPU. ID(PID). CPU. CPU. CPU user kernel user-land. CPU user-land akd. CPU. NFS, SMB. CPU CPU 100%.. Advanced Analytic CPU: CPUs( ). CPU CPU 100%. CPU CPU., I/O. ARC Cache: ARC accesses ARC Adaptive Replacement Cache DRAM. ARC,. 376 Sun ZFS Storage 7000 2012 12 E38995-01

ARC ARC. ARC. / ARC. /. ARC.. L2ARC ARC L2ARC. L2ARC ARC. LUN ARC. ARC. ARC LUN.. /. / ARC DRAM. ARC DRAM. L2ARC ( ). ARC DRAM... ARC DRAM. L2ARC ( ). 6 Analytics 377

ARC / / / ARC.. : ZFS uberblock. I/O,. ARC ARC. ARC.,., ARC ARC. ""., ARC., ARC ARC. ARC "" ( ARC ).. ARC.. 128Kbyte I/O, 8Kbyte I/O.,. : 368 :23 378 Sun ZFS Storage 7000 2012 12 E38995-01

L2ARC 128KByte I/O 23 x 128 = 2944Kbyte, 8Kbyte I/O 368x8=2944 Kbyte. ARC Cache: ARC size ARCDRAM. L2ARC IO Cache: L2ARC I/O bytes L2ARC 2nd Level Adaptive Replacement Cache, SSD. L2ARC. L2ARC.. L2ARC.... L2ARC Cache: L2ARC accesses. Cache: L2ARC accesses L2ARC 2nd Level Adaptive Replacement Cache, SSD. L2ARC. L2ARC L2ARC,. 6 Analytics 379

L2ARC L2ARC. / L2ARC. /. L2ARC.. L2ARC L2ARC L2ARC. LUN L2ARC. L2ARC. L2ARC LUN.. L2ARC Advanced Analytic Cache: L2ARC size L2ARC. L2ARC Gbyte. Cache: L2ARC I/O bytes. Advanced Analytic Cache: L2ARC errors L2ARC. Cache: ARC accesses(l2arc ) L2ARC. L2ARC. 380 Sun ZFS Storage 7000 2012 12 E38995-01

NDMP NDMP Data Movement: NDMP bytes transferred to/from disk NDMP. NDMP.NDMP 0. NDMP.. NDMP.. Data Movement: NDMP bytes transferred to/from tape. NDMP Data Movement: NDMP bytes transferred to/from tape NDMP.NDMP 0. NDMP. 6 Analytics 381

. Data Movement: NDMP bytes transferred to/from disk. Data Movement: Shadow migration bytes. (,ACL ).. analytics...... Data Movement: Shadow migration ops Data Movement: Shadow migration requests. 382 Sun ZFS Storage 7000 2012 12 E38995-01

Data Movement: Shadow migration ops Shadow Migration.... latency... Data Movement: Shadow migration bytes Data Movement: Shadow migration requests. Data Movement: Shadow migration requests Shadow Migration.. ( ).. 6 Analytics 383

.. latency... Data Movement: Shadow migration ops Data Movement: Shadow migration bytes. Disk: Disks.... Y Y.,. IOPS. ( ). 100% 384 Sun ZFS Storage 7000 2012 12 E38995-01

IO I/O. I/O., 75%.. 100% ( )..,., I/O.. 100%. : 75%. 100%. I/O Disk: I/O operations( ). IOPS,, I/O I/O Disk: I/O operations Disk: I/O bytes. IO,.. 100%. Disk: I/O bytes. I/O I/O Storage RAID. 6 Analytics 385

IO, NFSv3 8Kbyte 128Kbyte 256Kbyte.. 8Kbyte NFSv3 128Kbyte. ( )..,. Network: Device bytes - ( ) Disk: ZFS logical I/O bytes - ( ) Disk: I/O bytes ( ) I/O. I/O. 50Mbytes/sec( I/O) 5Mbytes/sec( I/O). JBOD I/O.,... I/O vs I/O, I/O. Disk: Disks( ). Disk: I/O operations / /. 386 Sun ZFS Storage 7000 2012 12 E38995-01

IO IO Disk: I/O operations I/O( IOPS). I/O I/O Storage RAID.,16 8Kbyte NFSv3 ARC DRAM 128Kbyte RAID ( : 2 ). I/O.,. Protocol: NFSv3 operations - NFSv3 ( ) Disk: ZFS logical I/O operations - I/O( ) Disk: I/O operations I/O( ) I/O. I/O,. IOPS. 400 IOPS ( I/O DRAM ) 110 IOPS ( I/O ).. Disk: I/O operations I/O ( ). I/O ( ), I/O. I/O.. I/O I/O I/O (IOPS). IOPS IOPS ( ) I/O ( ).. Disk: I/O operations Disk: I/O operations 6 Analytics 387

JBOD IOPS. I/O I/O IOPS.. latency. I/O I/O I/O. I/O. I/O I/O. I/O. IOPS. /. Disk: Disks( ). Disk: I/O bytes //. Network: Device bytes /. Network. (, IP, TCP, NFS/SMB/ ).. 1Gbps.. 1Gb/s : ~120MB/s 10Gb/s : ~1.16GB/s 388 Sun ZFS Storage 7000 2012 12 E38995-01

.., NFS ( ). Network. Network: Interface bytes. Network: Interface bytes /. Network. (, IP, TCP, NFS/SMB/ ). Network: Device bytes... LACP. 6 Analytics 389

SMB., NFS ( ). Network. SMB Network: Device bytes. Protocol: SMB operations SMB / (SMB IOPS). SMB I/O,. Protocol: NFSv3 operations. SMB / SMB Dashboard. SMB. I/O. SMB, CPU., CPU... 390 Sun ZFS Storage 7000 2012 12 E38995-01

SMB.. SMB (//readx/writex/...) SMB IP. latency SMB I/O. "<unknown>". SMB I/O. SMB I/O. SMB SMB I/O. SMB I/O. SMB I/O. SMB I/O. SMB IOPS. I/O RAID SMB IOPSIOPS.. : " Protocol: SMB operations/ "( ) " '/export/fs4/10ga' Protocol: SMB operations/ "( ) " 'phobos.sf.fishpong.com' Protocol: SMB operations/ "( ) SMB Network: Device bytes, SMB / Cache: ARC accesses, I/O Disk: I/O operations. 6 Analytics 391

Protocol: Fibre Channel bytes Fibre Channel bytes/. Protocol: iscsi bytes. / FC. FC Protocol: Fibre Channel operations. LUN SCSI FC FC LUN SAN. FC Protocol: Fibre Channel operations,fc / Cache: ARC accesses, I/O Disk: I/O operations. 392 Sun ZFS Storage 7000 2012 12 E38995-01

Protocol: Fibre Channel operations / (FC IOPS). FC I/O,,. Protocol: iscsi operations. / FC Dashboard. FC. I/O.FC, LUN, CPU., CPU.., LUN. LUN SCSI FC FC LUN FC. SCSI FC I/O. FC SCSI. I/O (read/write/sync-cache/...). 6 Analytics 393

FTP latency FC FC I/O. FC I/O. FC I/O. FC IOPS. I/O LUN RAID FC IOPSIOPS. FC I/O.. : " Protocol: Fibre Channel operations/ "(SCSI ) FC I/O Protocol: Fibre Channel bytes,fc / Cache: ARC accesses, I/O Disk: I/O operations. FTP Protocol: FTP bytes FTP /. FTP,. FTP FTP / FTP Dashboard..,. 394 Sun ZFS Storage 7000 2012 12 E38995-01

HTTPWebDAV. FTP. FTP (get/put/...) FTP. "<unknown>". FTP. FTP. FTP IP.. : " 'phobos.sf.fishpong.com' Protocol: FTP bytes/ "( ) FTP / Cache: ARC accesses, I/O Disk: I/O operations. HTTPWebDAV Protocol: HTTP/WebDAV requests HTTP HTTP/WebDAV /. HTTP,. HTTP/WebDAV / HTTP Dashboard. 6 Analytics 395

HTTPWebDAV HTTP.. HTTP HTTP,, CPU., CPU..,. HTTP (get/post). HTTP (200/404/...). IP latency HTTP. HTTP HTTP. HTTP I/O. HTTP.. : ": get HTTP/WebDAV / "(HTTP GET ) " '404' Protocol: HTTP/WebDAV requests/ "( ) " 'deimos.sf.fishpong.com' Protocol: HTTP/WebDAV requests/ "( ) HTTP Network: Device bytes, HTTP / Cache: ARC accesses I/O Disk: I/O operations. 396 Sun ZFS Storage 7000 2012 12 E38995-01

iscsi iscsi Protocol: iscsi bytes iscsi /. iscsi / iscsi. iscsi Protocol: iscsi operations. LUN iscsi SCSI iscsi iscsi LUN iscsi IP SAN. iscsi iscsi Protocol: iscsi operations iscsi / Cache: ARC accesses, I/O Disk: I/O operations. Protocol: iscsi operations iscsi / (iscsi IOPS). iscsi I/O,,. 6 Analytics 397

iscsi iscsi / iscsi Dashboard. iscsi. I/O. iscsi, LUN, CPU., CPU.., LUN. LUN iscsi SCSI iscsi iscsi LUN iscsi. SCSI iscsi I/O. latency iscsi SCSI. I/O (read/write/sync-cache/...). iscsi iscsi I/O. iscsi I/O. iscsi I/O. iscsi IOPS. I/O LUN RAID iscsi IOPSIOPS. iscsi I/O.. : " Protocol: iscsi operations/ "(SCSI ) 398 Sun ZFS Storage 7000 2012 12 E38995-01

NFSv2 iscsi I/O Protocol: iscsi bytes, iscsi / Cache: ARC accesses, I/O Disk: I/O operations. NFSv2 Protocol: NFSv2 operations NFSv2 / (NFS IOPS). NFS I/O,. Protocol: NFSv3 operations. NFS / NFS Dashboard. NFS. I/O. NFS, CPU., CPU..... 6 Analytics 399

NFSv2 NFS (read/write/getattr/setattr/lookup/...). NFS IP. latency NFS I/O. NFS. "<unknown>". NFS I/O. NFS I/O. NFS NFS I/O. NFS I/O. NFS I/O. NFS I/O. NFS IOPS. I/O RAID NFS IOPSIOPS.. : " Protocol: NFSv2 operations/ "( ) " '/export/fs4/10ga' Protocol: NFSv2 operations/ "( ) " 'phobos.sf.fishpong.com' Protocol: NFSv2 operations/ "( ) NFS Network: Device bytes, NFS / Cache: ARC accesses, I/O Disk: I/O operations. 400 Sun ZFS Storage 7000 2012 12 E38995-01

NFSv3 NFSv3 Protocol: NFSv3 operations NFSv3 / (NFS IOPS). NFS I/O,. NFS / NFS Dashboard. NFS. I/O. NFS, CPU., CPU..... NFS (read/write/getattr/setattr/lookup/...). NFS IP. NFS I/O. NFS. "<unknown>". NFS I/O. NFS I/O. 6 Analytics 401

NFSv4 latency NFS NFS I/O. NFS I/O. NFS I/O. NFS I/O. NFS IOPS. I/O RAID NFS IOPSIOPS.. : " Protocol: NFSv3 operations/ "( ) " '/export/fs4/10ga' Protocol: NFSv3 operations/ "( ) " 'phobos.sf.fishpong.com' Protocol: NFSv3 operations/ "( ) NFS Network: Device bytes, NFS / Cache: ARC accesses, I/O Disk: I/O operations. NFSv4 Protocol: NFSv4 operations NFSv4 / (NFS IOPS). NFS I/O,. Protocol: NFSv3 operations. NFS / NFS Dashboard. 402 Sun ZFS Storage 7000 2012 12 E38995-01

NFSv4 NFS. I/O. NFS, CPU., CPU..... NFS (read/write/getattr/setattr/lookup/...). NFS IP. latency NFS I/O. NFS. "<unknown>". NFS I/O. NFS I/O. NFS NFS I/O. NFS I/O. NFS I/O. NFS I/O. NFS IOPS. I/O RAID NFS IOPSIOPS.. : " Protocol: NFSv4 operations/ "( ) 6 Analytics 403

SFTP " '/export/fs4/10ga' Protocol: NFSv4 operations/ "( ) " 'phobos.sf.fishpong.com' Protocol: NFSv4 operations/ "( ) NFS Network: Device bytes, NFS / Cache: ARC accesses, I/O Disk: I/O operations. SFTP Protocol: SFTP bytes SFTP /. SFTP,. Protocol: FTP bytes. SFTP / SFTP Dashboard..,.. SFTP. SFTP (get/put/...). 404 Sun ZFS Storage 7000 2012 12 E38995-01

SRP SFTP. "<unknown>". SFTP. SFTP. SFTP IP.. : " 'phobos.sf.fishpong.com' Protocol: SFTP bytes/ "( ) SFTP / Cache: ARC accesses, I/O Disk: I/O operations. SFTP SSH FTP CPU. CPU CPU: Percent utilization. SRP Protocol: SRP bytes SRP /. Protocol: iscsi bytes. SRP / SRP. SRP Protocol: SRP operations. 6 Analytics 405

SRP LUN SRP SCSI SRP SRP LUN SAN. SRP SRP Protocol: SRP operations, SRP / Cache: ARC accesses, I/O Disk: I/O operations. Protocol: SRP operations SRP / (SRP IOPS). SRP I/O,,. Protocol: iscsi operations. SRP / SRP. SRP. I/O. SRP, LUN, CPU., CPU. 406 Sun ZFS Storage 7000 2012 12 E38995-01

SRP., LUN. LUN SRP SCSI SRP SRP LUN SRP. SCSI SRP I/O. latency SRP SCSI. I/O (read/write/sync-cache/...). SRP SRP I/O. SRP I/O. SRP I/O. SRP IOPS. I/O LUN RAID SRP IOPSIOPS. SRP I/O.. : " Protocol: SRP operations/ "(SCSI ) SRP I/O Protocol: SRP bytes, SRP / Cache: ARC accesses, I/O Disk: I/O operations. 6 Analytics 407

CPU CPU CPU CPU CPU: CPUs CPU CPU CPU. CPU CPU: Percent utilization. CPU. CPU. CPU CPU. 100% CPU CPU. Y Y CPU () ( ). CPU ( ). CPU. ( ). CPU. 408 Sun ZFS Storage 7000 2012 12 E38995-01

ARC CPU CPU: Kernel spins CPU.. CPU, CPU: Percent utilization CPU: CPUs... CPU (mutex/...) CPU (0/1/2/3/...) ARC Cache: ARC adaptive parameter ZFS ARC arc_p. ARC MRU MFU. ZFS ARC. ARC. Cache: ARC accesses ARC. ARC Analytics Cache: ARC size Cache: ARC evicted bytes. 6 Analytics 409

ARC ARC Cache: ARC evicted bytes ZFS ARC. L2ARC. ZFS ARC. L2ARC (L2ARC). L2ARC ARC.. L2ARC. ARC Cache: ARC accesses ARC. ARC Analytics Cache: ARC size. L2ARC L2ARC, L2ARC. ARC Cache: ARC size, DRAM ZFS ARC. ZFS ARC. 410 Sun ZFS Storage 7000 2012 12 E38995-01

ARC ARC. ARC DRAM. ARC. ARC L2ARC DRAM (L2ARC). ARC Cache: ARC accesses ARC. : ARC. ARC : ARC ARC ARC L2ARC. ARC. ZFS. ARC. ARC. L2ARC. L2ARC ARC DRAM "ARC ". ARC Cache: ARC target size ZFS ARC arc_c. ARC. Advanced Analytic Cache: ARC size. ZFS ARC. 6 Analytics 411

DNLC ARC. Cache: ARC accesses ARC. ARC Analytics Cache: ARC size Cache: ARC evicted bytes. DNLC Cache: DNLC accesses DNLC( ). inode DNLC... DNLC. Cache: ARC accesses ARC. Analytic Cache: ARC size ARC. / DNLC /. 412 Sun ZFS Storage 7000 2012 12 E38995-01

L2ARC DNLC Cache: DNLC entries DNLC( ). inode DNLC... DNLC. Cache: ARC accesses ARC. Analytic Cache: ARC size ARC. L2ARC Cache: L2ARC errors L2ARC. L2ARC. : L2ARC. 6 Analytics 413

L2ARC L2ARC : io L2ARC L2ARC (DRAM) 1. L2ARC. ZFS ARC.... L2ARC Cache: L2ARC size L2ARC. L2ARC. L2ARC. L2ARC Cache: ARC evicted bytes L2ARC I/O. I/O L2ARC. Cache: L2ARC errors. L2ARC. 414 Sun ZFS Storage 7000 2012 12 E38995-01

NDMP NDMP Data Movement: NDMP file system operations NDMP /. ZFS.. Analytics Data Movement: NDMP bytes transferred to/from disk Data Movement: NDMP bytes transferred to/from tape. read/write/... NDMP Data Movement: NDMP jobs NDMP. NDMP NDMP. Analytics Data Movement: NDMP bytes transferred to/from disk Data Movement: NDMP bytes transferred to/from tape. : backup/restore 6 Analytics 415

Disk: Percent utilization... Disk: Disks.... 100 2. 50% 0%. (Disk: Disks) Advanced. Disk: Disks. ZFS DMU Disk: ZFS DMU operations ZFS DMU(Data Management Unit) /. ZFS. 416 Sun ZFS Storage 7000 2012 12 E38995-01

ZFS IO, Analytics. DMU DDT(Data Deduplication Table). Data Deduplication. DMU DMU read/write/... ZFS /ZFS /DMU dnode/spa /... ZFS IO Disk: ZFS logical I/O bytes ZFS /. I/O I/O, NFS.. I/O. read/write/... 6 Analytics 417

ZFS IO ZFS IO Disk: ZFS logical I/O operations ZFS /. I/O I/O, NFS.. I/O. read/write/... Memory: Dynamic memory usage 1 (DRAM).. :. : 418 Sun ZFS Storage 7000 2012 12 E38995-01

cache kernel mgmt unused ZFS (ARC)..... Memory: Kernel memory (kmem )... Dashboard DRAM ( : ). Memory: Kernel memory in use Memory: Kernel memory lost to fragmentation. kmem Memory: Kernel memory in use ( ) (kmem ).. 6 Analytics 419

. Dashboard DRAM ( : ). Memory: Kernel memory lost to fragmentation. kmem Memory: Kernel memory lost to fragmentation (kmem cache). ( : )... Dashboard DRAM ( : ). Memory: Kernel memory in use. kmem 420 Sun ZFS Storage 7000 2012 12 E38995-01

IP IP Network: IP bytes /IB IP IP /.. Analytics Network: Device bytes.., Protocol: iscsi bytes.. IP IP : tcp/udp IP Network: IP packets IP /..., Protocol: iscsi operations. 6 Analytics 421

TCP IP IP : tcp/udp TCP Network:TCP bytes /IB, IP TCP TCP /.. Analytics Network: Device bytes.., Protocol: iscsi bytes.. IP TCP : http/ssh/215()/... TCP Network:TCP packets TCP /. 422 Sun ZFS Storage 7000 2012 12 E38995-01

NSCD.., Protocol: iscsi operations. IP TCP : http/ssh/215()/... NSCD System: NSCD backend requests NSCD( ) DNS, NIS.... latency / NSCD (DNS/NIS/...) IP 6 Analytics 423

NSCD NSCD System: NSCD operations NSCD( ).. / NSCD. System: NSCD backend requests. latency / / NSCD (DNS/NIS/...) : / OpenWorksheets 424 Sun ZFS Storage 7000 2012 12 E38995-01

OpenWorksheets Analytics CPU NFSv3 Analytics. Analytics... CPU: percent utilization broken down by CPU identifier - Protocol: NFSv3 operations per second broken down by latency -. Analytics. CPU.. 6 Analytics 425

OpenWorksheets ( ). "... broken down by CPU identifier" CPU. ( ).. Shift. CPU 4.. 10 "...". "..."... 15:52:26. Zoom Pause... "At 15:52:26:". "Range average:". Y. 100%. Line Graph.. Line Graph.. Line Graph. NFS... ( ). 1ms NFSv3 9ms (y ).. 0. (0-9+ ms ) DRAM. I/O, I/O I/O.,. 426 Sun ZFS Storage 7000 2012 12 E38995-01

OpenWorksheets. y. Crop Outliers.. /: Shift. Crop Outliers /.. "Show hierarchy"... ( )... "+" "-".. 6 Analytics 427