Copyright 2009, 2011, 2012, Oracle and/or its affiliates. All rights reserved.,.,,,,,,,,,,,,.,...,. U.S. GOVERNMENT END USERS: Oracle programs, includ

Transcription

1 Sun ZFS Storage 7000 : E E

2 Copyright 2009, 2011, 2012, Oracle and/or its affiliates. All rights reserved.,.,,,,,,,,,,,,.,...,. U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government..., -,,. Oracle Corporation. Oracle Java Oracle Corporation /.. Intel Intel Xeon Intel Corporation. SPARC SPARC International, Inc.. AMD, Opteron, AMD, AMD Opteron Advanced Micro Devices. UNIX The Open Group. ( )3,. Oracle Corporation 3,. Oracle Corporation 3,, @25097

3 BUI( ) CLI( ) BUI CLI CLI CLI

4 CLI BUI CLI Status...69 Status Dashboard CLI Settings BUI CLI NDMP BUI CLI Configuration...81 Configuration Sun ZFS Storage E

5 BUI CLI Network Network BUI CLI Storage SAN SAN BUI CLI SAN FC BUI CLI FCMPxIO FC iscsi BUI CLI SRP BUI CLI

6 BUI CLI Preferences BUI CLI SSH Public Keys Threshold Alerts BUI CLI Infiniband ' ' JBOD BUI Services Services Sun ZFS Storage E

7 BUI CLI NFS Kerberos Analytics CLI iscsi CLI SMB NFS/SMB DFS MMC CLI FTP

8 HTTP NDMP SFTP Virus Scan NIS LDAP Active Directory LDAP Windows Server Sun ZFS Storage E

9 BUI CLI Identity Mapping IDMU DNS CLI Active Directory DNS DNS DNS IPMP NTP BUI Remote Replication Dynamic Routing RIP RIPng

10 Phone Home SNMP MIB Sun FM MIB Sun AK MIB SMTP Service Tags System Identity SSH Shadow Migration Syslog Sun ZFS Storage E

11 Syslog: RFC Syslog: RFC Shares Shares Shares Shadow Migration Shares BUI CLI

12 NFS SMB SCSI HTTP FTP SFTP ACL ACL BUI CLI BUI CLI LUN NFS SMB iscsi HTTP Sun ZFS Storage E

13 FTP ACL BUI CLI Remote Replication Remote Replication BUI CLI Analytics Analytics Analytics

14 CPU CPU: Percent utilization ARC Cache: ARC accesses L2ARC IO Cache: L2ARC I/O bytes L2ARC Cache: L2ARC accesses NDMP Data Movement: NDMP bytes transferred to/from disk NDMP Data Movement: NDMP bytes transferred to/from tape Data Movement: Shadow migration bytes Data Movement: Shadow migration ops Data Movement: Shadow migration requests Disk: Disks IO Disk: I/O bytes IO Disk: I/O operations Network: Device bytes Network: Interface bytes SMB Protocol: SMB operations Protocol: Fibre Channel bytes Protocol: Fibre Channel operations FTP Sun ZFS Storage E

15 Protocol: FTP bytes HTTPWebDAV Protocol: HTTP/WebDAV requests iscsi Protocol: iscsi bytes iscsi Protocol: iscsi operations NFSv Protocol: NFSv2 operations NFSv Protocol: NFSv3 operations NFSv Protocol: NFSv4 operations SFTP Protocol: SFTP bytes SRP Protocol: SRP bytes SRP Protocol: SRP operations CPU CPU CPU: CPUs CPU CPU: Kernel spins ARC Cache: ARC adaptive parameter ARC Cache: ARC evicted bytes ARC Cache: ARC size ARC Cache: ARC target size DNLC Cache: DNLC accesses DNLC Cache: DNLC entries L2ARC

16 Cache: L2ARC errors L2ARC Cache: L2ARC size NDMP Data Movement: NDMP file system operations NDMP Data Movement: NDMP jobs Disk: Percent utilization ZFS DMU Disk: ZFS DMU operations ZFS IO Disk: ZFS logical I/O bytes ZFS IO Disk: ZFS logical I/O operations Memory: Dynamic memory usage Memory: Kernel memory Memory: Kernel memory in use Memory: Kernel memory lost to fragmentation IP Network: IP bytes IP Network: IP packets TCP Network: TCP bytes TCP Network: TCP packets NSCD System: NSCD backend requests NSCD System: NSCD operations Open Worksheets Sun ZFS Storage E

17 CLI Saved Worksheets BUI CLI BUI CLI Symantec Symantec 'DMP'/Storage Foundation FC Oracle Solaris Cluster Oracle Solaris Cluster Sun ZFS Storage Appliance 441 Microsoft Volume Shadow Copy Service Software Sun ZFS Storage Appliance Provider Oracle Grid Control Oracle Enterprise Manager Grid Control Sun ZFS Storage Oracle Storage Connect Sun ZFS Storage Appliance Oracle Virtual Machine VMware Site Recovery Manager VMware Site Recovery Manager Sun ZFS Storage Exadata Oracle Exadata Sun ZFS Storage Appliance

18 Exadata Sun ZFS Storage Appliance Oracle Exadata Sun ZFS Storage E

19 Sun ZFS Storage 7000 NAS Oracle Sun ZFS Storage Help.. Sun ZFS Storage 7000.,,. Installation Guide, Analytics Guide and Service Manual ( technetwork/documentation/) URL. ( )3,. Oracle Corporation 3,. Oracle Corporation 3,,. 19

20 Oracle Oracle My Oracle Support. ). 20 Sun ZFS Storage E

21 1 Sun ZFS Storage : Disk Shelf : J4400/J

22 Sun ZFS Storage. SMB NFS HTTP HTTPS WebDAV iscsi FC SRP iser FTP SFTP Sun ZFS Storage /. Analytics - ZFS -,, DRAM Sun ZFS Storage. :.. Oracle. "Oracle ("SLA") ". RAID-Z(RAID-5 RAID-6), /, - (7310, 7320, 7410, 7420) 22 Sun ZFS Storage E

23 iscsi LUN NDMP Sun ZFS Storage.. (CPU, DRAM, I/O,,, ) ZFS RAID-6() Disk Shelf RAID-6 - (7310, 7320, 7410, 7420) IP Disk Shelf I/O Phone-Home Lights Out BUI( ) BUI.BUI,,. 1 23

24 . NET-0 IP ( : BUI.BUI Help. CLI( ) CLI BUI. CLI. CLI. Tab. Tab. Tab 'Tab'.. help. ( : help commands). help help topics.. dory:> help tab builtins commands general help properties script 24 Sun ZFS Storage E

25 BUI( ) BUI.BUI,,. BUI. -BUI - - CLI( ) CLI BUI. CLI. CLI

26 BUI BUI 1. Firefox 3.x 5.x Internet Explorer 7, 8, 9 Safari 3.1 WebKit BUI 2,.. 2. Firefox 2.x Solaris 10 Mozilla 1.7 Opera 9 Internet Explorer Sun ZFS Storage E

27 Projects BUI.. Sun,,.,,,. BUI Configuration, Maintenance, Shares, Status Analytics

28 . Dashboard Maintenance: Logs. ID.. Configuration: Users.,.,ID. ID ID ID Properties, Rules Logs Service Project. 28 Sun ZFS Storage E

29 Projects Add.... Shares Projects Shares. : ( ). Share Share. BUI BUI. BUI... Apply. 1 29

30 BUI Revert. ( )..... BUI Oracle oracle.com.. Let us know. Permissions Permissions RWX. User, Group, Other. Share Share Inherit from project. 30 Sun ZFS Storage E

31

32 .....,. * * -- ( ) Sun ZFS Storage E

33 * * / ( ) () / -- / *.. InfiniBand InfiniBand (IB ) VLAN 1 33

34 VLAN Settings Analytics Analytics. 34 Sun ZFS Storage E

35 ( ) 1 ID Windows Unix. * * Windows Unix Windows Unix Unix Windows Unix Windows *.. SAS SAS 1 35

36 CLI CLI CLI. SSH CLI CLI ssh. root. CLI,,. % ssh root@dory Password: Last login: Mon Oct 13 15:43: from kiowa.sf.fishpo dory:> CLI. - Tab. Tab 'Tab'. - help. help ( : help commands). help help topics.. dory:> help tab builtins commands general help properties script 36 Sun ZFS Storage E

37 CLI CLI.. BUI...., Configuration CLI configuration.. dory:> configuration dory:configuration>,. show., configuration. dory:configuration> show Children: net => Configure networking services => Configure services version => Display system version users => Configure administrative users roles => Configure administrative roles preferences => Configure user preferences alerts => Configure alerts storage => Configure Storage Network, Services, Users, Preferences Configuration.. dory:configuration> preferences dory:configuration preferences> ( )., configuration preferences. 1 37

38 dory:> configuration preferences dory:configuration preferences>. select,. list., users. dory:> configuration users dory:configuration users> list NAME USERNAME UID TYPE John Doe bmc Dir Super-User root 0 Loc bmc select bmc. dory:configuration users> select bmc dory:configuration users bmc> select destroy., maintenance logs system reboot. dory:maintenance logs system> select module=reboot dory:maintenance logs system entry-034> show Properties: timestamp = :24:41 module = reboot priority = crit text = initiated by root on /dev/console syslogd: going down on signal 15 select., bmc. dory:> configuration users select bmc dory:configuration users bmc> done. dory:configuration> done dory:>. 38 Sun ZFS Storage E

39 dory:> configuration users select bmc dory:configuration users bmc> done dory:> done. dory:> configuration dory:configuration> users dory:configuration users> select bmc dory:configuration users bmc> done dory:configuration users> done dory:configuration> done dory:> cd. UNIX cd "..". dory:> configuration users select bmc dory:configuration users bmc> cd.. dory:configuration users> UNIX "cd /". dory:> configuration dory:configuration> users dory:configuration users> select bmc dory:configuration users bmc> cd / dory:> UNIX "cd../..". dory:> configuration dory:configuration> users dory:configuration users> select bmc dory:configuration users bmc> cd../.. dory:configuration> (, select ). 15 bmc. 31. dory:> configtab dory:> configuration utab dory:> configuration users setab 1 39

40 dory:> configuration users select tab bmc root dory:> configuration users select btab dory:> configuration users select bmcenter dory:configuration users bmc>., configuration preferences get. dory:configuration preferences> get locale = C login_screen = status/dashboard session_timeout = 15 session_annotation = advanced_analytics = false., get. dory:> configuration preferences get locale = C login_screen = status/dashboard session_timeout = 15 session_annotation = advanced_analytics = false., configuration alerts threshold create threshold alert. dory:> configuration alerts thresholds create dory:configuration alerts threshold (uncommitted)> (uncommitted). commit.. dory:configuration alerts threshold (uncommitted)> cd / Leaving will abort creation of "threshold". Are you sure? (Y/N).,.. 40 Sun ZFS Storage E

41 dory:configuration alerts threshold (uncommitted)> commit error: missing value for property "statname". dory:configuration alerts threshold (uncommitted)> set statname=cpu.utilization statname = cpu.utilization (uncommitted) dory:configuration alerts threshold (uncommitted)> commit error: missing value for property "limit" dory:configuration alerts threshold (uncommitted)> set limit=90 limit = 90 (uncommitted) dory:configuration alerts threshold (uncommitted)> commit dory:configuration alerts thresholds> list THRESHOLD LIMIT TYPE STATNAME threshold normal cpu.utilization CLI /. "help properties".. dory:configuration preferences> help properties Properties that are valid in this context: locale login_screen session_timeout session_annotation advanced_analytics => Locality => Initial login screen => Session timeout => Current session annotation => Make available advanced analytics statistics get. get. dory:configuration preferences> get locale = C login_screen = status/dashboard session_timeout = 15 session_annotation = advanced_analytics = false 1 41

42 get., login_screen. dory:configuration preferences> get login_screen login_screen = status/dashboard get., iscsi. dory:> configuration services iscsi get tab <status> isns_server radius_secret target_chap_name isns_access radius_access radius_server target_chap_secret set.., login_screen "shares". dory:configuration preferences> set login_screen=shares login_screen = shares (uncommitted).. commit. dory:configuration preferences> get login_screen login_screen = shares (uncommitted) dory:configuration preferences> commit dory:configuration preferences> get login_screen login_screen = shares. : dory:configuration preferences> set login_screen=maintenance/hardware login_screen = maintenance/hardware (uncommitted) dory:configuration preferences> done You have uncommitted changes that will be discarded. Are you sure? (Y/N) 42 Sun ZFS Storage E

43 , set. : dory:> configuration preferences set login_screen=analytics/worksheets login_screen = analytics/worksheets dory:>.., NTP servers NTP. dory:configuration services ntp> set servers=0.pool.ntp.org,1.pool.ntp.org servers = 0.pool.ntp.org,1.pool.ntp.org (uncommitted) dory:configuration services ntp> commit,,., sharenfs shares "kiowa" /. dory:> shares select default dory:shares default> set sharenfs="ro,rw=kiowa" sharenfs = ro,rw=kiowa (uncommitted) dory:shares default> commit.,.., space_available. dory:> shares select default dory:shares default> get space_available space_available = 1.15T dory:shares default> set space_available=100p error: cannot set immutable property "space_available". set., bmc fullname. dory:> configuration users select bmc set fullname="rembrandt Q. Einstein" error: cannot set immutable property "fullname" 1 43

44 ., "myproj" "myfs" "newsnap". shares select myproj select myfs snapshots snapshot newsnap ssh. % ssh root@dory < myfile.txt "here file".. here file. % ssh root@dory << EOF shares select myproj select myfs snapshots snapshot newsnap EOF.., CLI.,. ECMAScript 3. ECMAScript C,. (if/else) (while, for ) Perl(split(), join() ) 44 Sun ZFS Storage E

45 CLI script. dory:> script ("." to run)>, ".". dory:> script ("." to run)> for(i=10;i>0;i--) ("." to run)> printf("%d... ", i); ("." to run)> printf("blastoff!\n"); ("." to run)> Blastoff! script. dory:> script print("it is now " + new Date()) It is now Tue Oct :33:01 GMT+0000 (UTC),.. get list run props set. ( : Date ) Run "run".. : 1 45

46 dory:> configuration version script dump(run( get boot_time )) boot_time = :02:17\n dump. ECMAScript.,. dory:> configuration version script dump(run( get boot_time ).split(/\s+/)) [&#39;, boot_time, =, , 07:02:17, &#39; ] Get run.. "get". boot_time ECMAScript Date., boot_time. script run( configuration version ); now = new Date(); uptime = (now.valueof() - get( boot_time ).valueof()) / 1000; printf( up %d day%s, %d hour%s, %d minute%s, %d second%s\n, d = uptime / 86400, d<1 d>=2? s :, h = (uptime / 3600) % 24, h<1 h>=2? s :, m = (uptime / 60) % 60, m<1 m>=2? s :, s = uptime % 60, s<1 s>=2? s : ); "uptime.aksh". % ssh root@dory < uptime.aksh Pseudo-terminal will not be allocated because stdin is not a terminal. Password: up 2 days, 10 hours, 47 minutes, 48 seconds SSH. SSH "-T". List. list.,. script run( shares ); projects = list(); 46 Sun ZFS Storage E

47 for (i = 0; i < projects.length; i++) { run( select + projects[i]); shares = list(); } for (j = 0; j < shares.length; j++) { run( select + shares[j]); printf("%s/%s %1.64g %1.64g\n", projects[i], shares[j], get( space_data ), get( space_available )); run( cd.. ); } run( cd.. ); "space.aksh". % ssh root@koi < space.aksh Password: admin/accounts admin/exports admin/primary admin/traffic admin/workflow aleventhal/hw_eng bcantrill/analytx bgregg/dashbd bgregg/filesys bpijewski/access_ctrl " "( ) get. script run( shares ); projects = list(); printf( %-40s %-10s %-10s\n, SHARE, USED, AVAILABLE ); for (i = 0; i < projects.length; i++) { run( select + projects[i]); shares = list(); for (j = 0; j < shares.length; j++) { run( select + shares[j]); share = projects[i] + / + shares[j]; used = run( get space_data ).split(/\s+/)[3]; avail = run( get space_available ).split(/\s+/)[3]; } } run( cd.. ); printf( %-40s %-10s %-10s\n, share, used, avail); run( cd.. ); 1 47

48 "prettyspace.aksh". % ssh root@koi < prettyspace.aksh Password: SHARE USED AVAILABLE admin/accounts 18K 248G admin/exports 18K 248G admin/primary 18K 248G admin/traffic 18K 248G admin/workflow 18K 248G aleventhal/hw_eng 18K 248G bcantrill/analytx 1.00G 248G bgregg/dashbd 18K 248G bgregg/filesys K 100G bpijewski/access_ctrl 18K 248G... Children. children.,. configuration services script var svcs = children(); for(vari=0; i<svcs.length; ++i) { run(svcs[i]); if (props().length!== 0) printf("%-10s %s\n", svcs[i], get( <status> )); run("done"); }. "svcinfo.aksh". % ssh root@koi < space.aksh Password: cifs disabled dns online ftp disabled http disabled identity online idmap online ipmp online iscsi online ldap disabled ndmp online nfs online nis online ntp online scrk online 48 Sun ZFS Storage E

49 sftp smtp snmp ssh tags vscan disabled online disabled online online disabled.. dump print printf. JSON... tostring. C printf(3c)... code - message -. CLI. : dory:> script run( not a cmd ) error: uncaught error exception (code EAKSH_BADCMD) in script: invalid command "not a cmd" (encountered while attempting to run command "not a cmd"). dory:> script try { run( not a cmd ) } catch (err) { dump(err); } { tostring: <function>, code: 10004, message: invalid command "not a cmd" (encountered while attempting to run command "not a cmd") }. #!/usr/bin/ksh -p ssh -T root@dory <<EOF 1 49

50 script try { run( shares select default select $1 ); } catch (err) { if (err.code == EAKSH_ENTITY_BADSELECT) { printf( error: "$1" is not a share in the + default project\n ); exit(1); } } throw (err); EOF printf( "default/$1": compression is %s\n, get( compression )); exit(0); "share.ksh". % ksh./share.ksh bogus error: "bogus" is not a share in the default project,.,, SSH....,. workflow ECMAScript Sun ZFS Storage E

51 name description execute. var workflow = { name: Hello world, description: Bids a greeting to the world, execute: function () { return ( hello world! ) } }; "Hello world". "hello world!"..,.,,,. run CPU. var workflow = { name: CPU utilization, description: Displays the current CPU utilization, execute: function () { run( analytics datasets select name=cpu.utilization ); cpu = run( csv 1 ).split( \n )[1].split(, ); return ( At + cpu[0] +, utilization is + cpu[1] + % ); } };.. workflow parameters. parameters. parameters. label type 1 51

52 type. Boolean ChooseOne Address File Host HostName HostPort Integer NetAddress NodeName NonNegativeInteger Number Password Permissions Port Size String StringList ( ) 0 ( ) POSIX., 2, ( ) ( ). var workflow = { name: New share, description: Creates a new share in a business unit, parameters: { name: { label: Name of new share, type: String }, unit: { label: Business unit, type: String 52 Sun ZFS Storage E

53 }; } }, execute: function (params) { run( shares select + params.unit); run( filesystem + params.name); run( commit ); return ( Created new share " + params.name + " ); }... ChooseOne,. options optionlabels options ChooseOne. var workflow = { name: Create share, description: Creates a new share in a business unit, parameters: { name: { label: Name of new share, type: String }, unit: { label: Business unit, type: ChooseOne, options: [ development, finance, qa, sales ], optionlabels: [ Development, Finance, Quality Assurance, Sales/Administrative ], } }, execute: function (params) { run( shares select + params.unit); run( filesystem + params.name); run( commit ); return ( Created new share " + params.name + " ); } }; unit. 1 53

54 UI. parameters optional. optional true., UI. execute undefined....,.. var workflow = { name: Create share, description: Creates a new share in a business unit, parameters: { name: { label: Name of new share, type: String }, unit: { label: Business unit, type: ChooseOne, options: [ development, finance, qa, sales ], optionlabels: [ Development, Finance, Quality Assurance, Sales/Administrative ], } }, execute: function (params) { try { run( shares select + params.unit); } catch (err) { if (err.code!= EAKSH_ENTITY_BADSELECT) throw (err); /* * We haven t yet created a project that corresponds to * this business unit; create it now. */ 54 Sun ZFS Storage E

55 } run( shares project + params.unit); run( commit ); run( shares select + params.unit); }; } run( filesystem + params.name); run( commit ); return ( Created new share " + params.name + " ); validate. validate.. var workflow = { name: Create share, description: Creates a new share in a business unit, parameters: { name: { label: Name of new share, type: String }, unit: { label: Business unit, type: ChooseOne, options: [ development, finance, qa, sales ], optionlabels: [ Development, Finance, Quality Assurance, Sales/Administrative ], } }, validate: function (params) { try { run( shares select + params.unit); run( select + params.name); } catch (err) { if (err.code == EAKSH_ENTITY_BADSELECT) return; } return ({ name: share already exists }); }, execute: function (params) { try { run( shares select + params.unit); } catch (err) { if (err.code!= EAKSH_ENTITY_BADSELECT) throw (err); /* * We haven t yet created a project that corresponds to * this business unit; create it now. 1 55

56 } */ run( shares project + params.unit); set( mountpoint, /export/ + params.unit); run( commit ); run( shares select + params.unit); }; } run( filesystem + params.name); run( commit ); return ( Created new share " + params.name + " ); audit. audit.. execute.. step execute execute execute.,. var steps =[{ step: Checking for associated project, execute: function (params) { try { run( shares select + params.unit); } catch (err) { if (err.code!= EAKSH_ENTITY_BADSELECT) throw (err); /* * We haven t yet created a project that corresponds to * this business unit; create it now. */ run( shares project + params.unit); set( mountpoint, /export/ + params.unit); run( commit ); 56 Sun ZFS Storage E

57 run( shares select + params.unit); } } }, { step: Creating share, execute: function (params) { run( filesystem + params.name); run( commit ); } }, { step: Creating audit record, execute: function (params) { audit( created " + params.name + " in " + params.unit); } }]; var workflow = { name: Create share, description: Creates a new share in a business unit, parameters: { name: { label: Name of new share, type: String }, unit: { label: Business unit, type: ChooseOne, options: [ development, finance, qa, sales ], optionlabels: [ Development, Finance, Quality Assurance, Sales/Administrative ], } }, validate: function (params) { try { run( shares select + params.unit); run( select + params.name); } catch (err) { if (err.code == EAKSH_ENTITY_BADSELECT) return; } }; return ({ name: share already exists }); }, execute: function (params) { return (steps); }.,

58 required (,,, ). version (major.minor.micro). required.,, "year.month.day.build-branch"., " ,12-0" "configuration version get version" CLI BUI "Version". required. var workflow = { name: Configure FC, description: Configures fibre channel target groups, required: ,1-0,.... version.,,..,. "2.1" "2.0" "2.0.1" "1.2" "0.1".. alert true. 58 Sun ZFS Storage E

59 ID. setid true.. class. code. items. timestamp. items. url URL. action. impact. description. severity. audit. audit.,. var workflow = { name: Failover, description: Fail the node over to its clustered peer, alert: true, setid: true, execute: function (params) { /* * To failover, we first confirm that clustering is configured * and that we are in the clustered state. We then reboot, * which will force our peer to takeover. Note that we re * being very conservative by only rebooting if in the * AKCS_CLUSTERED state: there are other states in which it 1 59

60 }; } * may well be valid to failback (e.g., we are in AKCS_OWNER, * and our peer is AKCS_STRIPPED), but those states may also * indicate aberrent operation, and we therefore refuse to * failback. (Even in an active/passive clustered config, a * FAILBACK should always be performed to transition the * cluster peers from OWNER/STRIPPED to CLUSTERED/CLUSTERED.) */ var uuid = params.uuid; var clustered = AKCS_CLUSTERED ; audit( attempting failover in response to alert + uuid); try { run( configuration cluster ); } catch (err) { audit( could not get clustered state; aborting ); return; } if ((state = get( state ))!= clustered) { audit( state is + state + ; aborting ); return; } if ((state = get( peer_state ))!= clustered) { audit( peer state is + state + ; aborting ); return; } run( cd / ); run( confirm maintenance system reboot );. scheduled Workflow true. CLI schedule Workflow. CLI CLI. dory:> maintenance workflows dory:maintenance workflows> "select workflow-002 dory:maintenance workflow-002> schedules dory:maintenance workflow-002 schedules>create dory:maintenance workflow-002 schedule (uncommitted)> set frequency=day frequency = day (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> set hour=10 hour = 10 (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> set minute=05 60 Sun ZFS Storage E

61 minute = 05 (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> commit dory:maintenance workflow-002 schedules> list NAME FREQUENCY DAY HH:MM schedule-001 day - 10:05 dory:maintenance workflow-002 schedules> create dory:maintenance workflow-002 schedule (uncommitted)> set frequency=week frequency = week (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> set day=monday day = Monday (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> set hour=13 hour = 13 (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> set minute=15 minute = 15 (uncommitted) dory:maintenance workflow-002 schedule (uncommitted)> commit dory:maintenance workflow-002 schedules> list NAME FREQUENCY DAY HH:MM schedule-001 day - 10:05 schedule-002 week Monday 13:15 dory:maintenance workflow-002 schedules>. NAME. frequency minute, halfhour, hour, day, week, month. day hour minute Monday, Tuesday, Wednesday, Thursday, Friday, Saturday Sunday. frequency week month frequency day, week month workflow. CLI.. offset. period. unit seconds month. 1 61

62 .. // Example of using Schedule definitions within a workflow var MyTextObject = { MyVersion: 1.0, MyName: Example 9, MyDescription: Example of use of Timer, Origin: Oracle }; var MySchedules = [ // half hr interval { offset: 0, period: 1800, units: "seconds" }, // offset 2 days, 4hr, 30min, week interval {offset: 2*24*60*60+4*60*60+30*60, period: ,units: "seconds" } ]; var workflow = { name: MyTextObject.MyName, description: MyTextObject.MyDescription, version: MyTextObject.MyVersion, alert: false, setid: true, schedules: MySchedules, scheduled: true, origin: MyTextObject.Origin, execute: function () { audit( workflow started for timer; ); } } }; MySchedules units offset period. seconds month. period offset., 4:30. schedules. 0. CLI. dory:> maintenance workflows dory:maintenance workflows>list WORKFLOW NAME OWNER SETID ORIGIN workflow-002 Example 9 root true Oracle dory:maintenance workflows> select workflow-002 dory:maintenance workflow-002> schedules dory:maintenance workflow-002 schedules> list NAME FREQUENCY DAY HH:MM schedule-000 halfhour - --:00 schedule-001 week Saturday 04:30 dory:maintenance workflow-002 schedules> :. 62 Sun ZFS Storage E

63 var steps =[{ step: Checking for existing worksheet, execute: function (params) { /* * In this step, we re going to see if the worksheet that * we re going to create already exists. If the worksheet * already exists, we blow it away if the user has indicated * that they desire this behavior. Note that we store our * derived worksheet name with the parameters, even though * it is not a parameter per se; this is explicitly allowed, * and it allows us to build state in one step that is * processed in another without requiring additional global * variables. */ params.worksheet = Drilling down on + params.type + disks ; try { run( analytics worksheets select name=" + params.worksheet + " ); if (params.overwrite) { run( confirm destroy ); return; } throw ( Worksheet called " + params.worksheet + " already exists! ); } catch (err) { if (err.code!= EAKSH_ENTITY_BADSELECT) throw (err); } } }, { step: Finding disks of specified type, execute: function (params) { /* * In this step, we will iterate over all chassis, and for * each chassis iterates over all disks in the chassis, * looking for disks that match the specified type. */ var chassis, name, disks; var i, j; run( cd / ); run( maintenance hardware ); chassis = list(); params.disks = []; for (i = 0; i < chassis.length; i++) { run( select + chassis[i]); name = get( name ); run( select disk ); disks = list(); for (j = 0; j < disks.length; j++) { run( select + disks[j]); 1 63

64 if (get( use ) == params.type) { params.disks.push(name + / + get( label )); } } run( cd.. ); } run( cd../.. ); if (params.disks.length === 0) throw ( No + params.type + disks found ); run( cd / ); } }, { step: Creating worksheet, execute: function (params) { /* * In this step, we re ready to actually create the worksheet * itself: we have the disks of the specified type and * we know that we can create the worksheet. Note that we * create several datasets: first, I/O bytes broken down * by disk, with each disk of the specified type highlighted * as a drilldown. Then, we create a separate dataset for * each disk of the specified type. Finally, note that we * aren t saving the datasets -- we ll let the user do that * from the created worksheet if they so desire. (It would * be straightforward to add a boolean parameter to this * workflow that allows that last behavior to be optionally * changed.) */ var disks = [], i; run( analytics worksheets ); run( create " + params.worksheet + " ); run( select name=" + params.worksheet + " ); run( dataset ); run( set name=io.bytes[disk] ); for (i = 0; i < params.disks.length; i++) disks.push( " + params.disks[i] + " ); run( set drilldowns= + disks.join(, )); run( commit ); } }]; for (i = 0; i < params.disks.length; i++) { run( dataset ); run( set name="io.bytes[disk= + params.disks[i] + ]" ); run( commit ); } var workflow = { name: Disk drilldown, description: Creates a worksheet that drills down on system, + cache, or log devices, 64 Sun ZFS Storage E

65 }; parameters: { type: { label: Create a new worksheet drilling down on, type: ChooseOne, options: [ cache, log, system ], optionlabels: [ Cache, Log, System ] }, overwrite: { label: Overwrite the worksheet if it exists, type: Boolean } }, execute: function (params) { return (steps); } BUI. CLI CLI maintenance workflows. download.. dory:maintenance workflows> download dory:maintenance workflows download (uncommitted)> get url = (unset) user = (unset) password = (unset) "url" URL.. URL HTTP(" ) FTP("ftp://" ). URL ( : "ftp://myusername:mypasswd@myserver/export/foo"), URL. dory:maintenance workflows download (uncommitted)> set url= ftp://foo/example1.akwf url = ftp://foo/example1.akwf dory:maintenance workflows download (uncommitted)> set user=bmc user = bmc dory:maintenance workflows download (uncommitted)> set password Enter password: password = ******** dory:maintenance workflows download (uncommitted)> commit Transferred 138 of 138 (100%)... done 1 65

66 maintenance workflows list. dory:maintenance workflows> list WORKFLOW NAME OWNER SETID ORIGIN workflow-000 Hello world root false <local> select. dory:maintenance workflows> select workflow-000 dory:maintenance workflow-000> get. dory:maintenance workflow-000> get name = Hello world description = Bids a greeting to the world owner = root origin = <local> setid = false alert = false scheduled = false execute.. dory:maintenance workflow-000> execute hello world!. dory:maintenance workflow-000> execute dory:maintenance workflow-000 execute (uncommitted)> get type = (unset) overwrite = (unset). dory:maintenance workflow-000 execute (uncommitted)> commit error: cannot execute workflow without setting property "type" commit. 66 Sun ZFS Storage E

67 dory:maintenance workflow-000 execute (uncommitted)> set type=system type = system dory:maintenance workflow-000 execute (uncommitted)> set overwrite=true overwrite = true dory:maintenance workflow-000 execute (uncommitted)> commit CLI. : dory:maintenance workflow-000 execute (uncommitted)> commit Checking for existing worksheet... done Finding disks of specified type... done Creating worksheet... done 1 67

68 68

69 2 2 Status Status Sun Storage 7000 Status > Dashboard Status.. Status > Dashboard,,,,. Status > Settings Dashboard Dashboard. 69

70 Dashboard Status > NDMP NDMP NDMP. Dashboard Dashboard Status Dashboard BUI( ). Dashboard 100 BUI,. Dashboard. Usage Dashboard Usage.. 70 Sun ZFS Storage E

71 Dashboard Storage Usage.. Shares Storage. Memory. Analytics Memory. Used Avail.. Shares. Compression. 1x. Dedup. 1x. (RAM) Cache. Unused.. Mgmt Other.. 2 Status 71

72 Dashboard (RAM) Kernel. analytics/component create+read. Dashboard. Services Dashboard.....,,, Logs Properties. Hardware Dashboard. 72 Sun ZFS Storage E

73 Dashboard. Hardware Maintenance. Dashboard 8. Disk /.. Analytics 4 (,,, )... Status Settings. 7 day graph (7d). 24 hour graph (24h) minute graph (60m) 1 (24 1 ). 1 second graph.. (7d, 24h 60m). 2 Status 73

74 Dashboard,.. 100% , 1000 NFSv3 / d.....BUI.. ( ). Dashboard / / Analytics Sun ZFS Storage E

75 Dashboard CLI CLI status dashboard Status > Dashboard. walu:> status dashboard Storage: pool_0: Used 10.0G bytes Avail 6.52T bytes State online Compression 1x Memory: Cache Unused Mgmt Other Kernel 550M bytes 121G bytes 272M bytes 4.10G bytes 1.90G bytes Services: ad disabled smb disabled dns online ftp disabled http online identity online idmap online ipmp online iscsi online ldap disabled ndmp online nfs online nis online ntp online routing online scrk maintenance snmp online ssh online tags online vscan online Hardware: CPU online Cards online Disks faulted Fans online Memory online PSU online Activity: CPU 1 %util Sunny Disk 32 ops/sec Sunny iscsi 0 ops/sec Sunny NDMP 0 bytes/sec Sunny NFSv3 0 ops/sec Sunny NFSv4 0 ops/sec Sunny Network 13K bytes/sec Sunny SMB 0 ops/sec Sunny Recent Alerts: :46: A cluster interconnect link has been restored. BUI. aalib. CLI BUI. status activity show. 2 Status 75

76 Settings caji:> status activity show Activity: CPU 10 %util Sunny Disk 478 ops/sec Partly Cloudy iscsi 0 ops/sec Sunny NDMP 0 bytes/sec Sunny NFSv3 681 ops/sec Partly Cloudy NFSv4 0 ops/sec Sunny Network 22.8M bytes/sec Partly Cloudy SMB 0 ops/sec Sunny caji:> Dashboard (24x7). ( ),.. Dashboard. Firefox about:config. "memory". browser.cache.memory.enable = false. Settings Status > Settings Status Dashboard. BUI 76 Sun ZFS Storage E

77 Settings Layout Layout Dashboard ( ). <empty> -. SMB / SMB. CPU CPU. CPU. Disk /. HTTP / HTTP. iscsi / iscsi. FC /. Network NDMP / /. / NDMP. NFSv2 / NFSv2. NFSv3 / NFSv3. NFSv4 / NFSv4. FTP SFTP / FTP. / SFTP. Dashboard "<empty>". 2 Status 77

78 Settings Thresholds Thresholds Dashboard.. Dashboard (60 )., CPU 41% Cloudy. 40%( ). Custom.. CLI CLI.BUI CLI. ( ). BUI 1 2 Status > Settings > Layout. Dashboard. 78 Sun ZFS Storage E

79 NDMP 3 Apply Status > Settings >Thresholds.. Custom.. Kilo/Mega/Giga Units. Apply. NDMP BUI NDMP NDMP.. NDMP. Type NDMP, Path NDMP /dev/rmt/0n Vendor STK Product SL500 NDMP. 2 Status 79

80 NDMP ID NDMP ID 49 Active Remote Client NDMP :4760 Authenticated, Data State,... Mover State Mover,... Current Operation NDMP,, Progress NDMP.. Active:. Idle:. Connected:. Halted:. Listen:. NDMP Mover NDMP.. Active:. Idle:. Paused:. Halted: /. Listen:. CLI CLI NDMP. 80 Sun ZFS Storage E

81 3 3 Configuration Configuration Configure Network,. Initial - Network - Services - SAN - SAN(Storage Area Network) Cluster - Users - Preferences - Alerts - Storage - 81

82 Shares. Maintenance System Backup DNS (NIS, LDAP, Active Directory) BUI....,.... Maintenance > System "INITIAL SETUP" CLI maintenance system setup. 82 Sun ZFS Storage E

83 BUI BUI.,, Start. Commit. Commit. NIC.BUI Allow Admin 215 BUI ssh 22 CLI. NIC. NIC. CLI CLI. help. done. 3 Configuration 83

84 CLI : caji console login: root Password: Last login: Sun Oct 19 02:55:31 on console To setup your system, you will be taken through a series of steps; as the setup process advances to each step, the help message for that step will be displayed. Press any key to begin initial configuration... (DHCP ) done. (, ) help... aksh: starting configuration with "net"... Configure Networking. Configure the appliance network interfaces. The first network interface has been configured for you, using the settings you provided at the serial console. Subcommands that are valid in this context: datalinks devices interfaces help [topic] show abort done => Manage datalinks => Manage devices => Manage interfaces => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help" or "script". => Show information pertinent to the current context => Abort this task (potentially resulting in a misconfigured system) => Finish operating on "net" caji:maintenance system setup net> devices show Devices: DEVICE UP MAC SPEED nge0 true 0:14:4f:8d:59:aa 1000 Mbit/s nge1 false 0:14:4f:8d:59:ab 0 Mbit/s nge2 false 0:14:4f:8d:59:ac 0 Mbit/s nge3 false 0:14:4f:8d:59:ad 0 Mbit/s caji:maintenance system setup net> datalinks show Datalinks: 84 Sun ZFS Storage E

85 DATALINK CLASS LINKS LABEL nge0 device nge0 Untitled Datalink caji:maintenance system setup net> interfaces show Interfaces: INTERFACE STATE CLASS LINKS ADDRS LABEL nge0 up ip nge /22 Untitled Interface caji:maintenance system setup net> done DNS DNS. Configure DNS. Configure the Domain Name Service. Subcommands that are valid in this context: help [topic] show commit abort done get [prop] set [prop] => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help", "script" or "properties". => Show information pertinent to the current context => Commit current state, including any changes => Abort this task (potentially resulting in a misconfigured system) => Finish operating on "dns" => Get value for property [prop]. ("help properties" for valid properties.) If [prop] is not specified, returns values for all properties. => Set property [prop] to [value]. ("help properties" for valid properties.) For properties taking list values, [value] should be a comma-separated list of values. caji:maintenance system setup dns> show Properties: <status> = online domain = sun.com servers = caji:maintenance system setup dns> set domain=sf.fishworks.com domain = sf.fishworks.com (uncommitted) caji:maintenance system setup dns> set servers= servers = (uncommitted) caji:maintenance system setup dns> commit caji:maintenance system setup dns> done aksh: done with "dns", advancing configuration to "ntp"... NTP(Network Time Protocol). NTP. 3 Configuration 85

86 Configure Time. Configure the Network Time Protocol. Subcommands that are valid in this context: help [topic] show commit abort done enable disable get [prop] set [prop] => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help", "script" or "properties". => Show information pertinent to the current context => Commit current state, including any changes => Abort this task (potentially resulting in a misconfigured system) => Finish operating on "ntp" => Enable the ntp service => Disable the ntp service => Get value for property [prop]. ("help properties" for valid properties.) If [prop] is not specified, returns values for all properties. => Set property [prop] to [value]. ("help properties" for valid properties.) For properties taking list values, [value] should be a comma-separated list of values. caji:maintenance system setup ntp> set servers=0.pool.ntp.org servers = 0.pool.ntp.org (uncommitted) caji:maintenance system setup ntp> commit caji:maintenance system setup ntp> done aksh: done with "ntp", advancing configuration to "directory"... NIS, LDAP Active Directory. Configure Name Services. Configure directory services for users and groups. You can configure and enable each directory service independently, and you can configure more than one directory service. Subcommands that are valid in this context: nis ldap ad help [topic] show abort => Configure NIS => Configure LDAP => Configure Active Directory => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help" or "script". => Show information pertinent to the current context => Abort this task (potentially resulting in a misconfigured system) 86 Sun ZFS Storage E

87 done => Finish operating on "directory" caji:maintenance system setup directory> nis caji:maintenance system setup directory nis> show Properties: <status> = online domain = sun.com broadcast = true ypservers = caji:maintenance system setup directory nis> set domain=fishworks domain = fishworks (uncommitted) caji:maintenance system setup directory nis> commit caji:maintenance system setup directory nis> done caji:maintenance system setup directory> done aksh: done with "directory", advancing configuration to "support"... LUN.. Configure Storage. Subcommands that are valid in this context: help [topic] show commit done config <pool> unconfig add import scrub <start stop> get [prop] set pool=[pool] => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help", "script" or "properties". => Show information pertinent to the current context => Commit current state, including any changes => Finish operating on "storage" => Configure the storage pool => Unconfigure the storage pool => Add additional storage to the storage pool => Search for existing or destroyed pools to import => Start or stop a scrub => Get value for property [prop]. ("help properties" for valid properties.) If [prop] is not specified, returns values for all properties. => Change current pool caji:maintenance system setup storage> show Properties: pool = pool-0 status = online profile = mirror log_profile = - cache_profile = - 3 Configuration 87

88 Network caji:maintenance system setup storage> done aksh: done with "storage", advancing configuration to "support"... (Phone Home). Remote Support. Register your appliance and configure remote monitoring. Subcommands that are valid in this context: tags scrk help [topic] show abort done => Configure service tags => Configure phone home => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help" or "script". => Show information pertinent to the current context => Abort this task (potentially resulting in a misconfigured system) => Finish operating on "support" caji:maintenance system setup support> done aksh: initial configuration complete! Network Network Networking Configuration, VLAN( LAN). IPv4 IPv6. 88 Sun ZFS Storage E

89 Network IPoIB(IP on InfiniBand). -. ( ) IB 1:1, VLAN. -IP. IP IPMP(IP ). -IP. IP.,. 2, LACP. IP 3. ( 2 3 ). IP ( ). nge0 datalink1 deimos( /22). nge1, nge2, nge3 aggr1(lacp ) phobos( /22) ("aggr1" ) (LACP ). ("phobos" ) IP. ( "nge1", "nge2" ).. 3 Configuration 89

90 Network InfiniBand,... VLAN - LAN. LACP -. ( ) ( ) LACP. IB - IB InfiniBand.. Name Speed Duplex VLAN VLAN ID. : "internal", "external", "adminnet". auto( ), 10( 10Mbit/ ), 100( 100Mbit/ ), 1000( 1Gbit/ ) 10000( 10Gbit/ ).. /.... CLI auto( ), half( ) full( ).. VLAN. VLAN ID 90 Sun ZFS Storage E

91 Network MTU LACP Aggregation LACP Policy LACP Mode LACP Timer IB Partition Partition Key MTU MTU( 9000 ). MTU MTU MTU. LAN MTU. MTU MTU ( ). MTU VLAN. MTU Q "". "" CLI...L2MAC L3IP L4.. LACP. LACP.. Cisco Etherchannel LACP. LACP LACP "". LACP. IB. ( ). (pkey). pkey GUID " ".HCA IPMP. IP.. IPv4 IPv6 IPMP - IP IP. 3 Configuration 91

92 Network Name Allow Administration Enable Interface IPv4 Configure with IPv4 Address/Mask IPv6 Configure with IPv6 Address/Mask IP MultiPathing Group. BUI CLI.. IP. IP / IP. IPMP IP. " " "DHCP". CIDR ( /24)IPv4. " " - - (IPv6 ) "IPv6 ". CIDR (1080::8:800:200C:417A/32)IPv6. IP. IPMP(IP ) IP IP ( : ) IP. IP IP. (LAN, IB VLAN)IP IPMP IPMP. IPMP IP <i>active</i> <i>standby</i>. Active: IPMP IP IP. Standby: IP. IP IPMP IP. IPMP., IPMP. 92 Sun ZFS Storage E

93 Network... IP,.,IP ( ) 5. ICMP (IPv4) ff02::1(ipv6) 5. IPMP ICMP. IPMP IPv4 IPv6.IPv IP., 10 IP. IPMP IPMP.., , IP.,. IPMP. IPMP. IPMP.. NIC. IPMP IPMP., IP IP NIC IPMP. 3 Configuration 93

94 Network,. IP. IP (, ). IP....,. Destination IP (CIDR ) /22 Gateway (IP )(" ", ) Family IPv4, IPv6 Type dhcp, static, system Interface IP nge0 "Destination" /0 ''.BUI. Kind, 94 Sun ZFS Storage E

95 Network "Interface" IP. IPMP IPMP IP, IP., IP IPMP "Interface".., "Type"... Static System DHCP Dynamic. IP. IP. "Gateway" IP. DHCP IP.DHCP DHCP. RIP RIPng ( ).. Inactive IP. Multihoming model IP IP. "loose", "adaptive" "strict".. IP IP 3 Configuration 95

96 Network., IP IP IP... Loose Adaptive Strict IP IP.1) IP IP IP.2)IP IP IP IP IP.. IP.1) IP IP IP.2)IP IP IP... IP IP.1) IP IP IP IP.2) IP IP IP IP... IP BUI., "Loose". "Adaptive" "Strict" IP. IP "Adaptive", "Strict". BUI BUI. 96 Sun ZFS Storage E

97 Network. IP. CLI. Configuration -> Network. / / / / / / I/O (,?) InfiniBand I/O InfiniBand InfiniBand (, ) InfiniBand InfiniBand ( ) VLAN VLAN IB ( ) 3 Configuration 97

98 Network ( ) IP Configuration, Addresses Routing,. Configuration Configuration Devices, Datalinks Interfaces.,. Devices.. IP. Datalink Interface Network Datalink datalink1(nge0 ) Network Interface IPv4 DHCP( 1 ) Network Addresses /22 Host Names caji.sf.example.com 98 Sun ZFS Storage E

99 Network IP.. CLI configuration net devices, datalinks, interfaces routing. show. caji:> configuration net caji:configuration net> devices show Devices: DEVICE UP SPEED MAC nge0 true 1000 Mbit/s 0:14:4f:9a:b9:0 nge1 true 1000 Mbit/s 0:14:4f:9a:b9:1 nge2 true 1000 Mbit/s 0:14:4f:9a:b8:fe nge3 true 1000 Mbit/s 0:14:4f:9a:b8:ff caji:configuration net> datalinks show Datalinks: DATALINK CLASS LINKS LABEL nge0 device nge0 datalink1 caji:configuration net> interfaces show Interfaces: INTERFACE STATE CLASS LINKS ADDRS LABEL nge0 up ip nge /22 caji caji:configuration net> routing show Properties: multihoming = loose Routes: ROUTE DESTINATION GATEWAY INTERFACE TYPE route / nge0 dhcp route / nge0 system, help. device ip. caji:configuration net> datalinks caji:configuration net datalinks> device caji:configuration net datalinks device (uncommitted)> set links=nge1 3 Configuration 99

100 Network links = nge1 (uncommitted) caji:configuration net datalinks device (uncommitted)> set label=datalink2 label = datalink2 (uncommitted) caji:configuration net datalinks device (uncommitted)> set mtu=9000 mtu = 9000 (uncommitted) caji:configuration net datalinks device (uncommitted)> commit caji:configuration net datalinks> show Datalinks: DATALINK CLASS LINKS LABEL nge0 device nge0 datalink1 nge1 device nge1 datalink2 caji:configuration net datalinks> cd.. caji:configuration net> interfaces caji:configuration net interfaces> ip caji:configuration net interfaces ip (uncommitted)> set label="caji2" label = caji2 (uncommitted) caji:configuration net interfaces ip (uncommitted)> set links=nge1 links = nge1 (uncommitted) caji:configuration net interfaces ip (uncommitted)> set v4addrs= /8 v4addrs = /8 (uncommitted) caji:configuration net interfaces ip (uncommitted)> commit caji:configuration net interfaces> show Interfaces: INTERFACE STATE CLASS LINKS ADDRS LABEL nge0 up ip nge /22 caji nge1 up ip nge /8 caji2 nge1 IP caji:configuration net routing> create caji:configuration net route (uncommitted)> set family=ipv4 family = IPv4 (uncommitted) caji:configuration net route (uncommitted)> set destination= destination = (uncommitted) caji:configuration net route (uncommitted)> set mask=0 mask = 0 (uncommitted) caji:configuration net route (uncommitted)> set interface=nge1 interface = nge1 (uncommitted) caji:configuration net route (uncommitted)> set gateway= gateway = (uncommitted) caji:configuration net route (uncommitted)> commit BUI 1 Datalink. 100 Sun ZFS Storage E

101 Network MTU ( 9000 ). Devices. "APPLY". Datalinks. 5 Interface "APPLY". Interfaces.. "APPLY" "APPLY". "APPLY"., 1 ( ) Datalink. MTU. "APPLY". Interfaces. "APPLY". "APPLY". 3 Configuration 101

102 Network LACP 1 Datalinks LACP Aggregation. Devices 2 "APPLY". 5 Interfaces. 6 7 Datalinks "APPLY". "APPLY". 1 IPMP IPMP "" IP. IP ( ). 2 Interface IP MultiPathing Group. UseIPv4Protocol / UseIPv6Protocol IPMP IP. Interfaces 1. "Active" "Standby". "APPLY". 1 IPMP IP /8 IPMP "" IP ( ). 102 Sun ZFS Storage E

103 Network 2 Interface IP MultiPathing Group. UseIPv4Protocol / UseIPv6Protocol IPMP IP. Interfaces 1. "Active" "Standby". "APPLY". LACP Devices.. "APPLY". IPMP Interfaces. IPMP. "APPLY". InfiniBand 1 Datalink IB Partition. Partition Devices. "APPLY". Datalinks. 3 Configuration 103

104 Network 6 Interface "APPLY". Interfaces.. "APPLY" Configuration->Network->Routing... "ADD" Configuration->Network->Routing.. CLI configuration net routing. create. show set. commit. 104 Sun ZFS Storage E

105 Storage configuration net routing. show ( : route-002). destroy route name multihoming strict configuration net routing. set multihoming=strict. commit. Q InfiniBand InfiniBand. Q InfiniBand. Q InfiniBand ( ). InfiniBand... Storage 3 Configuration 105

106 Storage., LUN. LUN Shares..,.. RAID-Z... Sun Storage 7000,. 106 Sun ZFS Storage E

107 Storage, LUN. 80% 90%. 20%. Shares UI..BUI.. CLI config JBOD " " ( ). ZFSSA , SAS (HBA ) JBOD.. 3--, ZFS.,... 3 Configuration 107

108 Storage.. JBOD JBOD. JBOD SAS-1 JBOD(, J4400 J4500). JBOD.,JBOD '' ''. JBOD NSPF JBOD. SAS-2 JBOD.,. JBOD Sun ZFS Storage E

109 Storage RAS... 3 RAID 3 ( : ).. 3 IOPS. 2 RAID.3. RAID, ( : ). RAID,. ( : ). 3 1 RAID., RAID.. RAID. RAID. 3 Configuration 109

110 Storage 3 RAID,... RAID. 3 RAID.. I/O. RAID.. 'NSPF'. NSPF ' ', JBOD RAID.. JBOD,. NSPF ( ) ( JBOD ). NSPF..,... ( ) : Sun ZFS Storage E

111 Storage....BUI 'IMPORT', CLI 'import'....,bui CLI 'name'..... RAID-Z NSPF config JBOD NSPF.JBOD JBOD RAID NSPF. LUN.,.. BUI.. 3 Configuration 111

112 SAN BUI Configuration->Storage "Allocate and verify storage" JBOD. JBOD none, half all. JBOD JBOD. "COMMIT". "Configure Added Storage".,. "COMMIT". SAN SAN SAN SAN(Storage Area Network). SAN Sun ZFS Storage E

113 SAN SAN.. LUN. LUN. LUN SCSI SCSI, SCSI. SCSI I/O.. 0..LUN. SCSI, SCSI I/O... LUN LUN. SCSI. FC iscsi iser SRP SAN(Storage Area Network) SCSI,,FC HBA. SCSI IP SCSI. RDMA iscsi,rdma (, InfiniBand) iscsi. iser IB iscsi. CLI BUI iser ( ) iscsi. SCSI RDMA RDMA (, InfiniBand) SCSI.. (FC, iscsi SRP). 3 Configuration 113

114 SAN LUN. LUN LUN.LUN LUN. LUN LUN.LUN LUN., LUN. BUI Configuration > SAN. Target Initiator. Fibre Channel Ports, iscsi Targets SRP Targets. 114 Sun ZFS Storage E

115 SAN Fibre Channel FC. Shares > Shares > Protocols. Target Group Initiator Group LUN. CLI CLI configuration san. shares CLI LUN. 3 Configuration 115

116 SAN SAN(Storage Area Network).. SCSI SCSI iscsi iser FC SRP IQN SCSI SCSI I/O. SCSI. SCSI 0. SCSI SCSI, SCSI I/O. SCSI (SCSI ).. LUN. LUN. LUNSCSI SCSI, SCSI. SCSI IP SCSI. RDMA iscsi,rdma (, InfiniBand) iscsi. iser IB iscsi. CLI BUI iser ( ) iscsi. SAN(Storage Area Network) SCSI,,FC HBA. SCSI RDMA RDMA (, InfiniBand) SCSI. iscsi Qualified Name iscsi. iscsi IQN iqn.date.authority:uniqueid. iqn com.sun:02:c7824a5b-f3ea-6038-c79d-ca443337d92c IQN iscsi iscsi. DNS ( "com.sun"). DNS Sun ID. 116 Sun ZFS Storage E

117 FC CHAP RADIUS iscsi, IP TCP. iscsi,..,. iscsi. Challenge-Handshake,. CHAP..LUN.. LUN LUN. LUN.. FC FC( ) SCSI.FC FC LUN, FC ( ). FC SAN.FC FC... root. FC WWN(World Wide Name), FC LUN LUN 3 Configuration 117

118 FC.FC. Sun ZFS Storage Appliance FC SAN Implementing Fibre Channel SAN Boot with Oracle's Sun ZFS Storage Appliance ( fbsanboot html ( technetwork/articles/servers-storage-admin/fbsanboot html)). LUN 2 ( ). ( ) LUN, ( ). ( ) ( ). I/O. ALUA( ) ALUA. WWN. WWN WWNFC..LUN LUN. FC SAN LUN LUN. ALUA. ALUA FC ALUA. FC analytics., LUN. 118 Sun ZFS Storage E

119 FC,, SCSI FC. LUN. FC. SCSI.. - FC FC LUN ( : - ) 3 Configuration 119

120 FC. (2048) LUN. LUN LUN ( FC 2 LUN 100, FC 4 LUN 28 ), LUN , LUN 32. Solaris ssd_max_throttle. FC.0. BUI FC FC BUI Configuration > SAN FC Target. root. Target.. Target Apply Sun ZFS Storage E

121 FC FC. FC Discovered Ports. Discovered Ports WWN. FC Initiators.. WWN Alias. 3 Configuration 121

122 FC Initiators FC Initiator Groups. Apply. LUN. FC LUN LUN LUN. Create LUN. Apply LUN. CLI FC dory:configuration san targets fc> set targets="wwn b32a11639" targets = wwn b32a11639 (uncommitted) dory:configuration san targets fc> commit 122 Sun ZFS Storage E

123 FC FC dory:configuration san targets fc> show Properties: targets = wwn b ,wwn b32a12239 Targets: 5: Port 1 target-001 initiator wwn b32a11639 PCIe 5: Port 2 0 Gbit/s target-002 initiator wwn b PCIe 2: Port 1 0 Gbit/s target-003 target wwn b32a12239 PCIe 2: Port 2 0 Gbit/s dory:configuration san targets fc> select target-000 dory:configuration san targets fc target-000> show Properties: wwn = wwn b port = PCIe 5: Port 1 mode = target NAME target-000 MODE target WWN wwn b PORT PCIe SPEED 4 Gbit/s speed = 4 Gbit/s discovered_ports = 6 link_failure_count = 0 loss_of_sync_count = 0 loss_of_signal_count = 0 protocol_error_count = 0 invalid_tx_word_count = 0 invalid_crc_count = 0 Ports: PORT WWN ALIAS MANUFACTURER port-000 wwn b3281a339 longjaw-1 QLogic Corporation port-001 wwn b32a1a339 longjaw-2 QLogic Corporation port-002 wwn b3281ac39 thicktail-1 QLogic Corporation port-003 wwn b32a1ac39 thicktail-2 QLogic Corporation port-004 wwn b3281e339 <none> QLogic Corporation port-005 wwn b32a1e339 <none> QLogic Corporation FC dory:configuration san initiators fc groups> create dory:configuration san initiators fc group (uncommitted)> set name=lefteye dory:configuration san initiators fc group (uncommitted)> set initiators=wwn b32a1ac39,wwn b3281ac39 dory:configuration san initiators fc group (uncommitted)> commit dory:configuration san initiators fc groups> list GROUP NAME group-001 lefteye +-> INITIATORS wwn b32a1ac39 wwn b3281ac39 FC LUN lefty LUN fera. dory:shares default> lun lefty dory:shares default/lefty (uncommitted)> set volsize=10 volsize = 10 (uncommitted) 3 Configuration 123

124 FC dory:shares default/lefty (uncommitted)> set initiatorgroup=fera initiatorgroup = default (uncommitted) dory:shares default/lefty (uncommitted)> commit CLI CLI. script /** This script creates both aliases for initiators and initiator * groups, as specified by the below data structure. In this * particular example, there are five initiator groups, each of * which is associated with a single host (thicktail, longjaw, etc.), * and each initiator group consists of two initiators, each of which * is associated with one of the two ports on the FC HBA. (Note that * there is nothing in the code that uses this data structure that * assumes the number of initiators per group.) */ groups = { thicktail: { thicktail-1 : wwn b3281ac39, thicktail-2 : wwn b32a1ac39 }, longjaw: { longjaw-1 : wwn b3281a339, longjaw-2 : wwn b32a1a339 }, tecopa: { tecopa-1 : wwn b3281e339, tecopa-2 : wwn b32a1e339 }, spinedace: { spinedace-1 : wwn b3281df39, spinedace-2 : wwn b32a1df39 }, fera: { fera-1 : wwn b , fera-2 : wwn b32a17939 } }; for (group in groups) { initiators = []; for (initiator in groups[group]) { printf( Adding %s for %s..., groups[group][initiator], initiator); try { run( select alias= + initiator); printf( (already exists)\n ); run( cd.. ); } catch (err) { if (err.code!= EAKSH_ENTITY_BADSELECT) throw err; run( create ); set( alias, initiator); set( initiator, groups[group][initiator]); 124 Sun ZFS Storage E

125 FCMPxIO } run( commit ); printf( done\n ); } run( select alias= + initiator); initiators.push(get( initiator )); run( cd.. ); } printf( Creating group for %s..., group); run( groups ); try { run( select name= + group); printf( (already exists)\n ); run( cd.. ); } catch (err) { if (err.code!= EAKSH_ENTITY_BADSELECT) throw err; run( create ); set( name, group); run( set initiators= + initiators); run( commit ); printf( done\n ); } run( cd.. ); FCMPxIO FC Sun ZFS Storage 7000 ALUA( ) FC. ( t10 SPC(SCSI Primary Commands). FC FC. Solaris MPxIO Solaris x86 SPARC. mpathadm show LU. luxadm display. stmsboot MPxIO /.. 1. MPxIO stmsboot -D fp -e, 2. MPxIO stmsboot -D fp -d. 3. mpathadm show LU. 3 Configuration 125

126 FCMPxIO Windows ALUA Windows 2008/R2 MPIO. R2. SP2. 1. FC HBA Windows HBA. 2. Windows Server 2008 MPIO. Windows mpclaim.exe -r -i -a "" SS MPIO /LUN. 3. Windows SS7000 LUN. Windows. GUI SS7000LUN. 4. SS7000 Microsoft DSM I/O. Multi-Path Disk Device DSM. Windows - Microsoft DSM Windows PathVerifyEnabled MPIO"N".N PathVerificationPeriod. Disabled Enabled PathVerificationPeriod MPIO ( ). PathVerifyEnabled = TRUE. 30 seconds 5 seconds RetryInterval (DSM IO RetryCount ). 1 second 5 seconds RetryCount PDORemovePeriod DSM IO. LUN ( ) seconds 1500 seconds : Emulex HBA Windows Server 2008: Emulex HBA Windows Server 2008 HBA. / I/O Emulex HBA NodeTimeout 126 Sun ZFS Storage E

127 FCMPxIO 0. ( OCManager Utility. Linux. OEL 5.4(Oracle Enterprise Linux 5.4) OEL 5.5(Oracle Enterprise Linux 5.5) SUSE Linux 11 SP multipathd. service multipathd stop Stopping multipathd daemon: [ OK ] 3. /etc/multipath.conf PRODUCT; CLI. run( cd / ); run( configuration version ); printf(get( product ).replace(/^sun /,"") + \n ); : spadefish:> script ("." to run)> run( cd / ); ("." to run)> run( configuration version ); ("." to run)> printf(get( product ).replace(/^sun /,"") + \n ); ("." to run)>. ZFS Storage 7420 OEL 5.4: device { vendor "SUN" } product "PRODUCT" getuid_callout "/sbin/scsi_id -g -u -s /block/%n" prio_callout "/sbin/mpath_prio_alua /dev/%n" hardware_handler "0" path_grouping_policy group_by_prio failback immediate no_path_retry queue rr_min_io 100 path_checker tur rr_weight uniform 3 Configuration 127

128 FCMPxIO OEL 5.5: defaults { find_multipaths user_friendly_names } yes yes devices { device { vendor "SUN" product "PRODUCT" getuid_callout "/sbin/scsi_id -g -u -s /block/%n" prio alua hardware_handler "1 alua" path_grouping_policy group_by_prio failback immediate no_path_retry 600 rr_min_io 100 path_checker tur rr_weight uniform features "0" } } SUSE Linux 11 SP 1: defaults { multipath_tool "/sbin/multipath -v0" udev_dir /dev polling_interval 100 default_selector "round-robin 0" default_path_grouping_policy group_by_prio default_getuid_callout "/lib/udev/scsi_id -g -u -d /dev/%n" default_prio_callout "/bin/true" prio "alua" default_features "0" rr_min_io 100 failback immediate user_friendly_names yes path_checker tur no_path_retry 1000 } device { vendor product } "SUN" "PRODUCT" 4. multipathd. #service multipathd start Starting multipathd daemon: [ OK ] 5. SCSI multipath I/O. Linux Sun ZFS Storage E

129 FCMPxIO #multipath ll sdd: checker msg is "tur checker reports path is down" mpath1 ( f094f0bd b31c88f0001) dm-2 SUN,Sun Storage 7410 (or 7310) [size=20g][features=1 queue_if_no_path][hwhandler=0][rw] \_ round-robin 0 [prio=50][active] \_ 2:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [prio=0][enabled] \_ 2:0:1:0 sdd 8:48 [failed][faulty] VMware ESX VMware ESX HCL ( resources/compatibility/search.php) ESX 4.1 : ALUA. ESX 4.0 : 1. esx nmp device list SATP. # esxcli nmp device list naa f0ed bb3c1f60002 Device Display Name: SUN Fibre Channel Disk (naa f0ed bb3c1f60002) Storage Array Type: VMW_SATP_DEFAULT_AA Storage Array Type Device Config: Path Selection Policy: VMW_PSP_FIXED Path Selection Policy Device Config: {preferred=vmhba0:c0:t1:l0;current=vmhba0:c0:t1:l0} Working Paths: vmhba0:c0:t1:l0 VMW_SATP_DEFAULT_AA. ALUA. 2. esxcli nmp satp addrule ZFS Storage 7000 ALUA. # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7110" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7210" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7310" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7410" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7120" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7320" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7420" -c "tpgs_on" 3.. # esxcli nmp satp listrules grep SUN VMW_SATP_ALUA SUN Sun Storage 7110 tpgs_on Sun Storage 7000 VMW_SATP_ALUA SUN Sun Storage 7310 tpgs_on Sun Storage 7000 VMW_SATP_ALUA SUN Sun Storage 7410 tpgs_on Sun Storage 7000 VMW_SATP_ALUA SUN Sun Storage 7210 tpgs_on Sun Storage 3 Configuration 129

130 FCMPxIO 7000 VMW_SATP_ALUA SUN ZFS Storage 7120 tpgs_on ZFS Storage 7000 VMW_SATP_ALUA SUN ZFS Storage 7320 tpgs_on ZFS Storage 7000 VMW_SATP_ALUA SUN ZFS Storage 7420 tpgs_on ZFS Storage VMware ESX Server. esxcli nmp device list. # esxcli nmp device list naa f0ed bb3c1f60002 Device Display Name: SUN Fibre Channel Disk (naa f0ed bb3c1f60002) Storage Array Type: VMW_SATP_ALUA Storage Array Type Device Config: {implicit_support=on;explicit_support=off; explicit_allow=on;alua_followover=on;{tpg_id=0,tpg_state=stby}{tpg_id=1,tpg_state=ao}} Path Selection Policy: VMW_PSP_MRU Path Selection Policy Device Config: Current Path=vmhba1:C0:T1:L0 Working Paths: vmhba1:c0:t1:l0. Multipath-tools version bundled in OEL 5.4 is unable to recognize paths in ALUA standby access state SCSI Test Unit Ready multipath (0.4.8 ). ( git. multipath-tools tar.bz2 tarball., multipath I/O /. fcinfo man ( Solaris Fibre Channel and Storage Multipathing Administration Guide ( Windows Server High Availability with Microsoft MPIO 130 Sun ZFS Storage E

131 iscsi details.aspx?familyid=cbd27a84-23a1-4e88-b f3&displaylang=en ( Using Device-Mapper Multipath - Red Hat ( iscsi SCSI SCSI. iscsi, IP TCP. iscsi,..,. iscsi. IQN(iSCSI Qualified Name) iscsi. iscsi IQN iqn.date.authority:uniqueid. iqn com.sun:02:c7824a5b-f3ea-6038-c79d-ca443337d92c IQN iscsi iscsi. DNS ( "com.sun"). DNS Sun ID. Target IQN Alias Authentication mode CHAP name IQN.IQN.,. None, CHAP RADIUS. CHAP CHAP. 3 Configuration 131

132 iscsi CHAP secret Network interfaces CHAP CHAP.. BUI.... LUN. / iscsi. IPMP IPMP..LACP.LACP IPMP IPMP. iscsi. Initiator IQN Alias Use CHAP CHAP name CHAP secret IQN.,. CHAP /. CHAP CHAP. CHAP CHAP. 132 Sun ZFS Storage E

133 iscsi iscsi. SAN IQN CHAP CHAP iscsi (LUN) LUN RADIUS CHAP. RADIUS RADIUS. RADIUS. RADIUS CHAP CHAP. IQN CHAP ( ) RADIUS Initiator Initiator. CHAP IQN CHAP Initiator. Initiator CHAP. Solaris iscsi/iser MPxIO MPxIO iscsi (MS/T) Solaris iscsi. NIC IPMP. iscsi iscsi NIC 2.NIC IPMP. iscsi NIC. Solaris iscsi MS/T IPMP. *IPMP NIC IPMP. MPxIO. IPMP MPxIO. *IPMP MPxIO. IPMP MPxIO. 3 Configuration 133

134 iscsi Solaris iscsi MS/T IPMP SunSolve Infodoc , Understanding an iscsi MS/T multi-path configuration. How to Enable Multiple iscsi Sessions for a Target. E / /gcawf ( /gcawf) IPMP 6, IP Services. ( iscsi iscsi. iscsi analytics., LUN. BUI Analytics analytics. 1. Analytics. 2. Add Statistic.. 3. Protocols iscsi operations > Broken down by initiator.. 4. analytics. analytics. 134 Sun ZFS Storage E

135 iscsi CLI IQN iscsi ahi:configuration san targets iscsi> create ahi:configuration san targets iscsi target (uncommitted)> set alias="target 0" ahi:configuration san targets iscsi target (uncommitted)> set auth=none ahi:configuration san targets iscsi target (uncommitted)> set interfaces=nge1 ahi:configuration san targets iscsi target (uncommitted)> commit ahi:configuration san targets iscsi> list TARGET ALIAS target-000 Target 0 +-> IQN iqn com.sun:02:daf0161f-9f5d-e01a-b5c5-e1efa IQN RADIUS iscsi ahi:configuration san targets iscsi> create ahi:configuration san targets iscsi target (uncommitted)> set alias="target 1" ahi:configuration san targets iscsi target (uncommitted)> set iqn=iqn com.acme:12345 ahi:configuration san targets iscsi target (uncommitted)> set auth=radius ahi:configuration san targets iscsi target (uncommitted)> set interfaces=nge1 ahi:configuration san targets iscsi target (uncommitted)> commit ahi:configuration san targets iscsi> list TARGET ALIAS target-000 Target 0 +-> IQN iqn com.sun:02:daf0161f-9f5d-e01a-b5c5-e1efa target-001 Target 1 +-> IQN iqn com.acme:12345 CHAP iscsi ahi:configuration san initiators iscsi> create ahi:configuration san initiators iscsi initiator (uncommitted)> set initiator=iqn com.acme:initiator12345 ahi:configuration san initiators iscsi initiator (uncommitted)> set alias="init 0" ahi:configuration san initiators iscsi initiator (uncommitted)> set chapuser=thisismychapuser ahi:configuration san initiators iscsi initiator (uncommitted)> set chapsecret= abc ahi:configuration san initiators iscsi initiator (uncommitted)> commit ahi:configuration san initiators iscsi> list NAME ALIAS initiator-000 Init 0 +-> INITIATOR iqn com.acme:initiator Configuration 135

136 SRP iscsi ahi:configuration san targets iscsi groups> create ahi:configuration san targets iscsi group (uncommitted)> set name=tg0 ahi:configuration san targets iscsi group (uncommitted)> set targets=iqn com.acme:12345, iqn com.sun:02:daf0161f-9f5d-e01a-b5c5-e1efa ahi:configuration san targets iscsi group (uncommitted)> commit ahi:configuration san targets iscsi groups> list GROUP NAME group-000 tg0 +-> TARGETS iqn com.acme:12345 iqn com.sun:02:daf0161f-9f5d-e01a-b5c5-e1efa iscsi ahi:configuration san initiators iscsi groups> create ahi:configuration san initiators iscsi group (uncommitted)> set name=ig0 ahi:configuration san initiators iscsi group (uncommitted)> set initiators=iqn com.acme:initiator12345 ahi:configuration san initiators iscsi group (uncommitted)> commit ahi:configuration san initiators iscsi groups> list GROUP NAME group-000 ig0 +-> INITIATORS iqn com.acme:initiator12345 SRP SCSI RDMA RDMA (, InfiniBand) SCSI. SRP IPoIB RDMA IB. SRP. SRP. Target EUI Alias EUI(Extended Unique Identifier). EUI SRP HCA GUID.,. 136 Sun ZFS Storage E

137 SRP BUI. ( ). SRP I/O. SRP. Initiator EUI Alias EUI(Extended Unique Identifier).,. SRP analytics. analytics. {{Server}}/wiki/images/cfg_san_srp.png. OFED SRP Linux OFED. 1. Linux OFED. download_linux.htm ( all install.pl. all SRP., 'gcc'. 3. SRP. 3 Configuration 137

138 SRP srp_release_notes.txt ( downloads/ofed/ofed-1.4/ofed-1.4-docs/srp_release_notes.txt) 4. srp_daemon -n. SRP run_srp_daemon. srp_daemon. /usr/local/ofed/sbin/ <prefix>/sbin/. run_srp_daemon. SRP srp_daemon.sh(/usr/local/ofed/sbin/ <prefix>/sbin/ ). srp_daemon.sh/var/log/srp_daemon.log. 5. InfiniBand. /etc/infiniband/openib.conf SRP_DAEMON_ENABLE "yes". /etc/infiniband/openib.conf SRPHA_ENABLE "yes". SRP High Availability. 6. High-Availability - Automatic. SRPHA_ENABLE "yes" /etc/infiniband/openib.conf. multipathd OpenIB. service restart multipathd /etc/init.d/openibd stop /etc/init.d/openibd start 7. IB HCA. ibstat 8. SRP IO. srp_daemon -a -o -v ibsrpdm 9. SCSI. 138 Sun ZFS Storage E

139 SRP cd /sys/class/infiniband_srp/srp-mthca add_target IO. -n. srp_daemon -o -c ibsrpdm -c 11. add_target. echo id_ext=0003ba eac,ioc_guid=0003ba eac,\ dgid=fe ba ead,\ pkey=ffff,service_id=0003ba eac > add_target 12. /var/log/messages SCSI. Linux isscsi.-h -v.. cat /proc/scsi/scsi 13.. mkfs /dev/sdd Linux. OFED 1.5. I/O. : I/O. I/O. Linux SRP. I/O srp_daemon srp_daemon.. srp_daemon -o -c -n -i <ib-device> -p <port-num> srp_daemon. srp_daemon. 3 Configuration 139

140 SRP : srp_daemon.. /var/log/messages SRP IO. Jan 27 11:57:03 ib-client-2 kernel: host11: SRP abort called Jan 27 11:57:37 ib-client-2 kernel: host11: ib_srp: failed send status 12 Jan 27 11:57:37 ib-client-2 kernel: ib_srp: host11: add qp_in_err timer Jan 27 11:57:37 ib-client-2 kernel: host11: ib_srp: failed send status 5 Jan 27 11:57:38 ib-client-2 kernel: host11: SRP abort called Jan 27 11:57:38 ib-client-2 kernel: host11: SRP reset_device called Jan 27 11:57:38 ib-client-2 kernel: host11: ib_srp: SRP reset_host called state 0 qp_err 1 Jan 27 11:57:58 ib-client-2 kernel: host11: SRP abort called Jan 27 11:57:58 ib-client-2 kernel: host11: SRP reset_device called Jan 27 11:57:58 ib-client-2 kernel: host11: ib_srp: SRP reset_host called state 0 qp_err 1 Jan 27 11:58:02 ib-client-2 kernel: host11: ib_srp: srp_qp_in_err_timer called Jan 27 11:58:02 ib-client-2 kernel: host11: ib_srp: srp_qp_in_err_timer flushed reset - done Jan 27 11:58:02 ib-client-2 kernel: host11: ib_srp: Got failed path rec status -22 Jan 27 11:58:02 ib-client-2 kernel: host11: ib_srp: Path record query failed Jan 27 11:58:02 ib-client-2 kernel: host11: ib_srp: reconnect failed (-22), removing target port. Jan 27 11:58:08 ib-client-2 kernel: scsi 11:0:0:0: scsi: Device offlined - not ready after error recovery Jan 27 11:58:08 ib-client-2 multipathd: sdc: tur checker reports path is down Jan 27 11:58:08 ib-client-2 multipathd: checker failed path 8:32 in map mpath148 Jan 27 11:58:08 ib-client-2 multipathd: mpath148: Entering recovery mode: max_retries=200 Jan 27 11:58:08 ib-client-2 multipathd: mpath148: remaining active paths: 0 Jan 27 11:58:08 ib-client-2 multipathd: sdc: remove path (uevent) Jan 27 11:58:08 ib-client-2 multipathd: mpath148: map in use Jan 27 11:58:08 ib-client-2 multipathd: mpath148: can t flush Jan 27 11:58:08 ib-client-2 multipathd: mpath148: Entering recovery mode: max_retries=200 Jan 27 11:58:08 ib-client-2 multipathd: dm-2: add map (uevent) Jan 27 11:58:08 ib-client-2 multipathd: dm-2: devmap already registered Jan 27 11:58:08 ib-client-2 kernel: scsi 11:0:0:0: scsi: Device offlined - not ready after error recovery Jan 27 11:58:08 ib-client-2 last message repeated 49 times Jan 27 11:58:08 ib-client-2 kernel: scsi 11:0:0:0: rejecting I/O to dead device Jan 27 11:58:08 ib-client-2 kernel: device-mapper: multipath: Failing path 8:32. Jan 27 11:58:08 ib-client-2 kernel: scsi 11:0:0:0: rejecting I/O to dead device : SRP.. /var/log/messages. device-mapper: table: 253:2: multipath: error getting device device-mapper: ioctl: error adding target to table multipath. ib-client-1:~ # multipath -d reload: maguro2lun ( f b6075db0001) n/a SUN,Sun Storage Sun ZFS Storage E

141 SRP [size=40g][features=0][hwhandler=0][n/a] \_ round-robin 0 [prio=50][undef] \_ 18:0:0:1 sde 8:64 [undef][ready] \_ round-robin 0 [prio=1][undef] \_ 17:0:0:1 sdc 8:32 [failed][ghost] :. SRP.. SRP.,,,,. /var/log/messages. Jan 26 17:42:12 mysystem kernel: sd 13:0:0:0: [sdd] Device not ready: Sense Key : Not Ready [current] Jan 26 17:42:12 mysystem kernel: sd 13:0:0:0: [sdd] Device not ready: Add. Sense: Logical unit not accessible, target port in standby state Jan 26 17:42:12 mysystem kernel: end_request: I/O error, dev sdd, sector Jan 26 17:42:12 mysystem kernel: scsi host13: SRP abort called Jan 26 17:42:12 mysystem kernel: scsi host13: SRP reset_device called Jan 26 17:42:12 mysystem kernel: scsi host13: ib_srp: SRP reset_host called state 0 qp_err 0 Jan 26 17:42:21 mysystem multipathd: 8:48: mark as failed :. VMWare 4.0 VMware Native MultiPath Plugin(nmp), 2. (psp) I/O. # esxcli nmp psp list Name Description VMW_PSP_MRU Most Recently Used Path Selection VMW_PSP_RR Round Robin Path Selection VMW_PSP_FIXED Fixed Path Selection (satp). / ALUA SATP. Sun Storage 7000 /. ESX CLI ALUA 7000 LUN. 3 Configuration 141

142 SRP # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7110" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7210" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7310" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "Sun Storage 7000" -V "SUN" -M "Sun Storage 7410" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7120" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7320" -c "tpgs_on" # esxcli nmp satp addrule -s VMW_SATP_ALUA -e "ZFS Storage 7000" -V "SUN" -M "ZFS Storage 7420" -c "tpgs_on" options are: -s VMW_SATP_ALUA - for the ALUA SATP -e description of the rule -V Vendor -M Model -c claim option for Target Portal Group (7000 seems to support implicit) LUN LUN. LUN ALUA.LUN ESX. ESX VMW_SATP_ALUE LUN. # esxcli nmp device list naa f096bb b707f2d0001 Device Display Name: Local SUN Disk (naa f096bb b707f2d0001) Storage Array Type: VMW_SATP_ALUA Storage Array Type Device Config: {implicit_support=on;explicit_support=off;explicit_allow=on; alua_followover=on; {TPG_id=0,TPG_state=AO}{TPG_id=1,TPG_state=STBY}} Path Selection Policy: VMW_PSP_MRU Path Selection Policy Device Config: Current Path=vmhba_mlx4_1.1.1:C0:T1:L0 Working Paths: vmhba_mlx4_1.1.1:c0:t1:l0 LUN Active Standby. # esxcli nmp path list gsan.80fe53553e gsan.80fe8f583e naa f096bb b707f2d0001 Runtime Name: vmhba_mlx4_1.1.1:c0:t2:l0 Device: naa f096bb b707f2d0001 Device Display Name: Local SUN Disk (naa f096bb b707f2d0001) Group State: standby Storage Array Type Path Config: {TPG_id=1,TPG_state=STBY,RTP_id=256,RTP_health=UP} Path Selection Policy Path Config: {non-current path} gsan.80fe53553e gsan.80fe73583e naa f096bb b707f2d0001 Runtime Name: vmhba_mlx4_1.1.1:c0:t1:l0 Device: naa f096bb b707f2d0001 Device Display Name: Local SUN Disk (naa f096bb b707f2d0001) Group State: active Storage Array Type Path Config: 142 Sun ZFS Storage E

143 SRP {TPG_id=0,TPG_state=AO,RTP_id=2,RTP_health=UP} Path Selection Policy Path Config: {current path} VMWare ESX 4.0. esxcl nmp path list SRP. vmware]# esxcli nmp path list gsan.80fe53553e gsan.80fe8f583e naa f096bb b707f2d0001 Runtime Name: vmhba_mlx4_1.1.1:c0:t2:l0 Device: naa f096bb b707f2d0001 Device Display Name: Local SUN Disk (naa f096bb b707f2d0001) Group State: standby Storage Array Type Path Config: {TPG_id=1,TPG_state=STBY,RTP_id=256,RTP_health=UP} Path Selection Policy Path Config: {non-current path} gsan.80fe53553e gsan.80fe73583e naa f096bb b707f2d0001 Runtime Name: vmhba_mlx4_1.1.1:c0:t1:l0 Device: naa f096bb b707f2d0001 Device Display Name: Local SUN Disk (naa f096bb b707f2d0001) Group State: active Storage Array Type Path Config: {TPG_id=0,TPG_state=AO,RTP_id=2,RTP_health=UP} Path Selection Policy Path Config: {current path} esxcli nmp path list. : VMWare VM Linux. Linux /var/log/messages. Feb 10 16:10:00 ib-client-5 vmkernel: 1:21:41: cpu3:4421)<3>ib_srp: Send tsk_mgmt target[vmhba_mlx4_1.1.1:2] out of TX_IU head tail lim 0 : VM. BUI BUI iser SRP. 3 Configuration 143

144 SRP iser BUI Configuration > SAN iser iscsi. 1. ibd Configuration > Network ibd ( ipmp) Datalinks. Datalink Interfaces. 2. iser Configuration > SAN. iscsi Targets iser. 3. iscsi Target Group. 144 Sun ZFS Storage E

145 SRP 4. Initiator iscsi initiators.. Initiator IQN OK. LUN. iscsi Initiator Groups. 5. LUN Shares LUN. Target Group Initiator Groups LUN. 3 Configuration 145

146 SRP 6. RedHat 4 SUSE iser LUN. SRP SRP. 1. HCA IB.. 2. Configuration > SAN. 3. Target SRP. SRP. 4. Target Groups. 5. ( ) Initiator. 6. LUN SRP Shares. 7. LUN LUN. Create LUN Target Group Initiator Group LUN SRP. SRP. VMWare ESX RedHat 5.4 SUSE Sun ZFS Storage E

147 CLI CLI configuration san targets srp groups targetsrpgroup SRP. swallower:configuration san targets srp groups> create swallower:configuration san targets srp group (uncommitted)> set name=targetsrpgroup name = targetsrpgroup (uncommitted) swallower:configuration san targets srp group (uncommitted)> set targets=eui.0002c a4 targets = eui.0002c a4 (uncommitted) swallower:configuration san targets srp group (uncommitted)> commit swallower:configuration san targets srp groups> list GROUP NAME group-000 targetsrpgroup +-> TARGETS eui.0002c a4 CLI shares LUN targetsrpgroup. swallower:shares default> lun mylun swallower:shares default/mylun (uncommitted)> set targetgroup=targetsrpgroup targetgroup = targetsrpgroup (uncommitted) swallower:shares default/mylun (uncommitted)> set volsize=10 volsize = 10 (uncommitted) swallower:shares default/mylun (uncommitted)> commit swallower:shares default> list Filesystems: NAME SIZE MOUNTPOINT test 38K /export/test LUNs: NAME SIZE GUID mylun 10G F0E9D19FFB00004B82DF490001,,BUI CLI.. Local -. Directory - NIS LDAP. NIS LDAP. 3 Configuration 147

148 .. administrator operator.. ( : root ).. "Basic administration".,,.. HTTP. ( ). Active Directory Active Directory Alerts. Analytics Clustering. Datasets Analytics Hardware /. Networking,. Projects and shares,, Roles Services 148 Sun ZFS Storage E

149 Shares property schema. System Update. Users Worksheet BUI Type Username Full Name Password/Confirm Require session annotation Kiosk user Kiosk screen Roles Directory(NIS LDAP ) Local( )... Local.,. ID.. "Kiosk screen". Dashboard. CLI. "Kiosk user".. 3 Configuration 149

150 Exceptions. Roles. Name Description Authorizations... BUI BUI Users.,... /... Enter /. "Show All". /. /. /. //.,. CLI BUI CLI., help. 150 Sun ZFS Storage E

151 CLI "brendan" NIS HTTP.. "webadmin". caji:> configuration roles caji:configuration roles> role webadmin caji:configuration roles webadmin (uncommitted)> set description="web server administrator" description = web server administrator (uncommitted) caji:configuration roles webadmin (uncommitted)> commit caji:configuration roles> show Roles: NAME basic webadmin DESCRIPTION Basic administration web server administrator webadmin HTTP.. caji:configuration roles> select webadmin caji:configuration roles webadmin> authorizations caji:configuration roles webadmin authorizations> create caji:configuration roles webadmin auth (uncommitted)> set scope=tab ad cluster net schema update alert hardware replication stat user appliance nas role svc worksheet caji:configuration roles webadmin auth (uncommitted)> set scope=svc scope = svc caji:configuration roles webadmin auth (uncommitted)> show Properties: scope = svc service = * allow_administer = false allow_configure = false allow_restart = false caji:configuration roles webadmin auth (uncommitted)> set service=tab * ftp ipmp nis ssh ad http iscsi ntp tags smb identity ldap routing vscan datalink:nge0 idmap ndmp scrk dns interface:nge0 nfs snmp caji:configuration roles webadmin auth (uncommitted)> set service=http service = http (uncommitted) caji:configuration roles webadmin auth (uncommitted)> set allow_restart=true allow_restart = true (uncommitted) caji:configuration roles webadmin auth (uncommitted)> commit caji:configuration roles webadmin authorizations> list NAME OBJECT PERMISSIONS auth-000 svc.http restart 3 Configuration 151

152 "brendan" "webadmin". caji:configuration roles webadmin authorizations> cd../../.. caji:configuration> users caji:configuration users> netuser brendan caji:configuration users> show Users: NAME USERNAME UID TYPE Brendan Gregg brendan Dir Super-User root 0 Loc caji:configuration users> select brendan caji:configuration users brendan> show Properties: logname = brendan fullname = Brendan Gregg initial_password = ************* require_annotation = false roles = basic kiosk_mode = false kiosk_screen = status/dashboard Children: exceptions => Configure this user s exceptions preferences => Configure user preferences caji:configuration users brendan> set roles=basic,webadmin roles = basic,webadmin (uncommitted) caji:configuration users brendan> commit brendan NIS HTTP.. CLI BUI.BUI. BUI 1 Roles. ( ). 2 Users Sun ZFS Storage E

153 4 5. Add. Users. 1 Roles ( ). Add. Roles "Scope". Scope... Authorization Add.. 1 Roles Apply. CLI 1 configuration roles. 3 Configuration 153

154 show. select authorizations show. ( ). configuration users. (NIS, LDAP) netuser. user. show commit.. select. show.. configuration roles. role.. ( ). configuration roles. select. authorizations. create. set scope=.. show. true ( ) Sun ZFS Storage E

155 Preferences 8 commit configuration roles. select. authorizations. show. destroy ( : "auth-001") ( ). true "status/dashboard".. Preferences, SSH. Initial login screen BUI. Status Dashboard. 3 Configuration 155

156 Preferences Locality Session timeout Current session annotation Advanced analytics statistics SSH Public Keys C.C POSIX ASCII. ISO ,,,,,,,,,,,,,,,,. BUI.. Analytics. RSA/DSA..BUI. CLI. BUI BUI,. CLI CLIconfiguration users. "brendan" analytics. caji:> configuration users caji:configuration users> select brendan caji:configuration users brendan> preferences caji:configuration users brendan preferences> show Properties: locale = C login_screen = status/dashboard session_timeout = 15 advanced_analytics = false Children: keys => Manage SSH public keys caji:configuration users brendan preferences> set advanced_analytics=true advanced_analytics = true (uncommitted) caji:configuration users brendan preferences> commit CLIconfiguration preferences Sun ZFS Storage E

157 twofish:> configuration preferences twofish:configuration preferences> show Properties: locale = C login_screen = status/dashboard session_timeout = 15 session_annotation = advanced_analytics = false Children: keys => Manage SSH public keys twofish:configuration preferences> set session_annotation="editing my user preferences" session_annotation = Editing my user preferences (uncommitted) twofish:configuration preferences> commit SSH Public Keys CLI. CLI SSH. caji:> configuration preferences keys caji:configuration preferences keys> create caji:configuration preferences key (uncommitted)> set type=dsa caji:configuration preferences key (uncommitted)> set key="...dsa key text..." key =...DSA key text...== (uncommitted) caji:configuration preferences key (uncommitted)> set comment="fw-log1" comment = fw-log1 (uncommitted) caji:configuration preferences key (uncommitted)> commit caji:configuration preferences keys> show Keys: NAME MODIFIED TYPE COMMENT key /12/ :54:58 DSA fw-log1 ( ). Alerts,. Analytics... Maintenance Logs,. 3 Configuration 157

158 . Cluster Custom Hardware Events Hardware Faults NDMP operations Network Phone home Remote replication Service failures Thresholds ZFS pool,. "NDMP: " "NDMP: ".,,IP. " : " " : ". Service Analytics.... From aknobody@caji.com Mon Oct 13 15:24: Date: Mon, 13 Oct :24: (GMT) From: Appliance on caji <noreply@caji.com> Subject: High CPU on caji To: admin@hostname.com SUNW-MSG-ID: AK-8000-TT, TYPE: Alert, VER: 1, SEVERITY: Minor EVENT-TIME: Mon Oct 13 15:24: PLATFORM: i86pc, CSN: 0809QAU005, HOSTNAME: caji SOURCE: svc:/appliance/kit/akd:default, REV: 1.0 EVENT-ID: 15a53214-c4e7-eae4-dae6-a652a51ea29b DESC: cpu.utilization threshold of 90 is violated. AUTO-RESPONSE: None. IMPACT: The impact depends on what statistic is being monitored. 158 Sun ZFS Storage E

159 REC-ACTION: The suggested action depends on what statistic is being monitored. SEE: SMTP. SNMP SNMP SNMP SNMP. Net-SNMP snmptrapd -P SNMP. # /usr/sfw/sbin/snmptrapd -P :31:15 NET-SNMP version Started :31:34 caji.com [ ]: iso = Timeticks: ( ) 246 days, 18:30:44.31 iso = OID: iso iso = STRING: "7cf0acd4-30c1-4c19-e9cb-ac27f7126b79" iso = STRING: "alert.ak.xmlrpc.threshold.violated" iso = STRING: "cpu.utilization threshold of 90 is violated." Syslog Syslog syslog. syslog syslog Syslog. / Analytics Datasets. 24., CPU analytics NFS.. NFS. 24. /. CPU NFS. 3 Configuration 159

160 .. / Analytics... alert true.. Threshold Alerts Analytics.. Threshold exceeds/falls below Timing: for at least only between/only during Repost alert every... this condition persists Also post alert when this condition clears for at least... Analytics ( : "Protocol: NFSv4 operations/ ").. /.. ( : ).. "Add Threshold Alert".. Threshold CPU: percent utilization exceeds 95 percent Timing for at least 5 minutes only between 0:00 and 0:00 only during weekdays Repost alert every 5 minutes while this condition persists 160 Sun ZFS Storage E

161 Also post alert when this condition clears for at least 5 minutes BUI Configuration->Alerts "Alert Actions" "Threshold Alerts".BUI. CLI CLI. configuration alerts help. BUI "Alert Actions". Category "All Events". All Events Subset of Events.. "Alert Actions".. "TEST". SNMP. "Alert Actions". "ADD". "Threshold Alerts".. Analytics. 3 Configuration 161

162 exceeds/falls below.. 5,5, 5.. "Alert Actions". "Apply". Sun ZFS Storage 7000.,, /, Sun ZFS Storage Sun ZFS Storage 7310, 7320, (2GHz 2.40GHz CPU ) 7420(1.86GHz 2.00GHz CPU )..,. Sun ZFS Storage ''. 2,. 2,., Sun ZFS Storage E

163 , -. Sun ZFS Storage 7000 System..,.... Sun ZFS Storage 7000.,,,.. Sun ZFS Storage Remote replication - - iscsi LUN - NFS, ( ) -, -,, -LACP IPMP RAID(Redundant Storage Device). Sun ZFS Storage 7000.IT. 3 Configuration 163

164 , /., Sun ZFS Storage / Sun ZFS Storage 7000 SLA,,,., : : AKCS_OWNER AKCS_CLUSTERED ( ) : : : : : AKCS_CLUSTERED AKCS_STRIPPED AKCS_OWNER 164 Sun ZFS Storage E

165 Sun ZFS Storage ( 1 ). I/O.,.,,,.. I/O CLUSTRON 3 I/O ( ) Sun ZFS Storage ( : ). 3 EIA/TIA-568B(8, Gigabit Ethernet) Configuration 165

166 . 3..., bps, 1Gbps. 12.2ms..,. 3 I/O 50ms. 200ms( ) 500ms( ).3.,.. 550ms - 30ms -. Sun ZFS Storage System...,,.. ( )..,. 3,. ( 4 ) Sun ZFS Storage E

167 ...,. IP,. 4 PrimaryA pool-0, PrimaryB pool ZFS.. ( ) ZFS. ATA ATA Configuration 167

168 . AKCS_OWNER...,,,ID, SMB iscsi.... LUN. "ID" ( : ) SSH. CLI.BUI SSL Sun ZFS Storage E

169 Takeover Failback. CLI/BUI UNCONFIGURED.. OWNER ( ),. ( : ).. STRIPPED ( ),.,, STRIPPED.. CLUSTERED,. ZFS CLUSTERED ( )..,,..,. CLI BUI (, )..,,... A B. A 3 Configuration 169

170 B... OWNER, STRIPPED. STRIPPED CLUSTERED. OWNER, STRIPPED. ( ). STRIPPED. STRIPPED. /LUN. LUN, ().... Sun ZFS Storage Sun ZFS Storage E

171 ..... Sun ZFS Storage ,.... ReadZilla., ReadZillas., - ReadZillas. LogZilla. ( ) CPU 50%, DRAM 50% 50%..,. CPU DRAM. 50%.. (). 100%. 40%-100%. 3 Configuration 171

172 I/O () ReadZilla.. ReadZilla.... LUN. LUN C NSPF(No Single Point of Failure). JBOD. NSPF JBOD. RAID NSPF JBOD.,. IPMP / LACP.. IP.DHCP DHCP. 172 Sun ZFS Storage E

173 .,. 5...,. Sun... ( ). 2. ( : ). : e1000g3 'alice' e1000g4 'bob' / / /22 e1000g3, e1000g4. IP.,. AKCS_STRIPPED. LDAP Active Directory. /.., ( ). ( : LUN )... 3 Configuration 173

174 Infiniband Infiniband. HCA, IPMP. HCA. HCA (pkey). Infiniband. IP. IB... Sun HCA. IB Sun ZFS Storage E

175 . 2 HCA. ' ',... Sun ZFS Storage 7000 I/O. 3( ). 3 Configuration 175

176 . (SCSI ). ATA Sun ZFS Storage SAS SASZONELOCK.. I/O.. SAS AKCS_OWNER 10. SAS. 2 SAS..,2 SAS JBOD.. /. 4. Sun ZFS Storage 7000 NSPF(No Single Point of Failure). RAID /. 176 Sun ZFS Storage E

177 . 3 I/O ( ). JBOD( ).,. JBOD..... NFS( ) I/O. NFSv2 NFSv3 stateless. NFSv4 I/O. Sun ZFS Storage 7000 ( ). 3 Configuration 177

178 /. iscsi.. iscsi. SMB, FTP HTTP/WebDAV , 1/2 JBOD JBOD 1/ LUN. LUN. 178 Sun ZFS Storage E

179 A B A B.. LUN.. : / ,1-0.. Sun ZFS Storage Sun ZFS Storage JBOD HBA JBOD tty. : tty. 5. BUI CLI... Configuration->Cluster Setup Commit. 3 Configuration 179

180 7... Commit. 8.. :.. 9. # DHCP. IP. tty DHCP Static. 10. # (BUI CLI). 11. # ,.... : JBOD 1/2. JBOD 2 1/2 JBOD. JBOD 1/ Configuration/Cluster Sun ZFS Storage E

181 . Sun Storage 7310 ZFS Storage 7320 PCIe0. Sun Storage 7410 PCIe5 ZFS Storage 7420 PCIeC ( 2 ) ( ). Cat 5 ( 3 1m ).,.. 3. JBOD JBOD. :. BUI Configuration->Cluster,. 3 Configuration 181

182 ..,., /. ( ).., CLUSTERED. OWNER Apply.. OWNER CLUSTERED Apply () ()... Setup Unconfig Apply Revert Failback.... ( ). ( ). () OWNER CLUSTERED (-). 182 Sun ZFS Storage E

183 Takeover () CLUSTERED STRIPPED OWNER (, ). 4. ( ). 5. Configuration -> Clustering Unconfig..,.. : Q1.0.0.,, 3 Configuration 183

184 Q Sun ZFS Storage E

185 4 4 Services Services Services.. 185

186 Services NFS iscsi SMB SMB-over-TCP 445 NetBIOS Datagram 138 NetBIOS Name Service 137 FTP HTTP NDMP NFSv3 NFSv4 iscsi LUN SMB FTP HTTP NDMP SMB-over-NetBIOS Remote Replication 216 Shadow Migration SFTP SRP TFTP Virus Scan SFTP SRP TFTP Sun ZFS Storage E

187 Services : 0-99 UID GID.. NIS LDAP Active Directory Identity Mapping NIS LDAP Microsoft Active Directory Server Windows Unix ID 389 DNS Dynamic Routing IPMP NTP Phone Home RIP RIPng IP IP Network Time Protocol Service Tags 443 SMTP SNMP SNMP 4 Services 187

188 Services Syslog System Identity syslog syslog Syslog SSH BUI CLI SSH Kerberos Kerberos V 88 Kerberos V Change & Set Password (SET_CHANGE) Kerberos V Change & Set Password (RPCSEC_GSS) BUI BUI Services Sun ZFS Storage E

189 Services... /.. / : Services ( )..... APPLY. REVERT. 4 Services 189

190 Services. Logs..,.. Executing start method Method "start" exited with status 0 Method "refresh" exited with status 0 Executing stop method Enabled Disabled. (0 == )..... NTP. [ Oct 11 21:05:31 Enabled. ] [ Oct 11 21:07:37 Executing start method (...). ] [ Oct 11 21:13:38 Method "start" exited with status 0. ] 21:05. 21:07:37 21:13:38. NTP. CLI CLI configuration services. show. 190 Sun ZFS Storage E

191 Services caji:> configuration services caji:configuration services> show Services: ad => disabled smb => disabled dns => online dynrouting => online ftp => disabled http => disabled identity => online idmap => online ipmp => online iscsi => online ldap => disabled ndmp => online nfs => online nis => disabled ntp => disabled replication => online scrk => disabled sftp => disabled shadow => online smtp => online snmp => disabled ssh => online syslog => disabled tags => online tftp => disabled vscan => disabled Children: ad => Configure Active Directory smb => Configure SMB dns => Configure DNS dynrouting => Configure Dynamic Routing ftp => Configure FTP http => Configure HTTP identity => Configure System Identity idmap => Configure Identity Mapping ipmp => Configure IPMP iscsi => Configure iscsi ldap => Configure LDAP ndmp => Configure NDMP nfs => Configure NFS nis => Configure NIS ntp => Configure NTP replication => Configure Remote Replication scrk => Configure Phone Home sftp => Configure SFTP shadow => Configure Shadow Migration smtp => Configure SMTP snmp => Configure SNMP srp => Configure SRP ssh => Configure SSH syslog => Configure Syslog tags => Configure Service Tags tftp => Configure TFTP vscan => Configure Virus Scan routing => Configure Routing Table 4 Services 191

192 Services., nis. caji:configuration services> nis caji:configuration services nis>, /,. show. caji:configuration services nis> show Properties: <status> = online domain = fishworks broadcast = true ypservers = enable. caji:configuration services nis> enable disable. caji:configuration services nis> disable set. commit. caji:configuration services nis> set domain="mydomain" domain = mydomain (uncommitted) caji:configuration services nis> commit caji:configuration services nis> show Properties: <status> = online domain = mydomain broadcast = true ypservers = : BUI CLI. 192 Sun ZFS Storage E

193 NFS CLI. BUI. help. caji:configuration services nis> help Subcommands that are valid in this context: help [topic] show commit done enable disable get [prop] set [prop] => Get context-sensitive help. If [topic] is specified, it must be one of "builtins", "commands", "general", "help", "script" or "properties". => Show information pertinent to the current context => Commit current state, including any changes => Finish operating on "nis" => Enable the nis service => Disable the nis service => Get value for property [prop]. ("help properties" for valid properties.) If [prop] is not specified, returns values for all properties. => Set property [prop] to [value]. ("help properties" for valid properties.) For properties taking list values, [value] should be a comma-separated list of values. NFS NFS( ). Sun ZFS Storage Appliance NFS 2, Services 193

194 NFS Minimum supported version Maximum supported version Maximum # of server threads Grace period Custom NFSv4 identity domain Enable NFSv4 delegation Kerberos realm Kerberos master KDC Kerberos slave KDC Kerberos admin principal Kerberos admin password NFS. NFS. NFS ( ). NFS. ( ) (15-600). NFS. ( : 90). NFS. NFSv4 ID. DNS ID, _nfsv4idmapdomain DNS DNS..... NFSv4. KDC. ASCII. DNS. Kerberos DNS.. KDC KDC KDC., KDC.. KDC KDC..,. joe, joe/admin joe/admin@eng.example.com Sun ZFS Storage E

195 NFS Services BUI CLI. NFS. NFS NFS,. Kerberos Kerberos keytab. NTP Kerberos NFS. Kerberos NFS. Active Directory Kerberos. KDC Kerberos docs/cd/e / /setup-8/index.html ( E / /setup-8/index.html.). Kerberos NFS Shares->Filesystem->Protocols Kerberos. Kerberos. Kerberos V :88 Kerberos V SET_CHANGE : 464 Kerberos V RPCSEC_GSS : 749 NFS. 4 Services 195

196 NFS network-nfs-server:default NFS appliance-kit-nfsconf:default NFS network-nfs-cbd: default network-nfs-mapid:default network-nfs-status:default network-nfs-nlockmgr: default NFSv4 NFSv4 mapid - NFSv4 NFS statd - NFS NFS lockd - Services Logs. Analytics NFS Analytics.. NFS... (//...) : NFS unknown. Analytics unknown. CLI CLI BUI. CLI version_min version_max nfsd_servers grace_period mapid_domain enable_delegation BUI Minimum supported version Maximum supported version Maximum # of server threads Grace period Custom NFSv4 identity domain Enable NFSv4 delegation 196 Sun ZFS Storage E

197 iscsi CLI krb5_realm krb5_kdc krb5_kdc2 krb5_admin BUI Kerberos Realm Kerberos master KDC Kerberos slave KDC Kerberos admin principal NFS NFS Configuration->Services. NFS.. Shares. Protocols NFS. NFS (/+). iscsi LUN iscsi(internet Small Computer System Interface). iscsi iscsi iscsi. isns,. iscsi CHAP ( ) ( ). RADIUS CHAP Services 197

198 iscsi Use isns isns Server Use RADIUS RADIUS Server RADIUS Server Secret isns isns RADIUS RADIUS RADIUS Services BUI CLI. CLI. CHAP CHAP. CHAP. iscsi. iscsi SAN. CLI iscsi SAN. 198 Sun ZFS Storage E

199 SMB. IQN IQN. isns IP isns. IP. CHAP. CHAP. RADIUS IP RADIUS. LUN LUN. LUN. LUN. LUN. / Red Hat iser Reduced Copy I/O. /etc/iscsi/iscsid.conf node.session.timeo.replacement_timeout 300. SMB SMB SMB. Shares SMB. LAN Manager compatibility level Preferred domain controller (LM, NTLM, LMv2, NTLMv2). Solaris Express Reference Manual Collection for smb. Active Directory. Active Directory DNS SRV Active Directory. 4 Services 199

200 SMB Active Directory site Maximum # of server threads Enable Dynamic DNS Enable Oplocks Restrict anonymous access to share list System Comment Idle Session Timeout Primary WINS server Secondary WINS server Excluded IP addreses from WINS SMB Signing Enabled SMB Signing Required Active Directory.,. Active Directory. ( ) DNS Active Directory DNS. off. SMB.. on. SMB oplock. oplock. SMB.... TCP/IP WINS. TCP/IP WINS. WINS IP. SMB SMB... SMB. SMB SMB.. Services BUI CLI. CLI. SMB. 200 Sun ZFS Storage E

201 SMB Case sensitivity Reject non UTF-8 Non-Blocking Mandatory Locking Resource name Share-level ACL SMB "mixed'" "'insensitive". UTF-8 SMB.... ACL ACL.. Case sensitivity Reject non UTF-8. NFS/SMB NFS SMB. NFS/SMB. 1. Active Directory. 2. ID. 3. SMB. 4.,ACL ACL. SMB NFSv3. SMB SMB ACL. ACL ACL inheritance behavior. DFS DFS( ) SMB MSRPC. DFS DFS.DFS.,,... 4 Services 201

202 SMB DFS. DFS. DFS. SMB DFS DFS MMC. Active Directory DFS. SMB DFS. SMB dfsadmin DFS. 1. dfsadmin. Windows. 2. dfsadmin SMB. 3. DFS Windows dfsadmin. SMB. SMB. 3. NIS LDAP SMB.. "&".... Administrators Administrators. 202 Sun ZFS Storage E

203 SMB Backup Operators Backup Operators. ID Windows ID. guest /. MMC MMC(Microsoft Management Console),. Microsoft Management Console, /,. Sun ZFS Storage 7000 MMC Windows Administrators. MMC Access is denied. Sun ZFS Storage MMC,. Sun ZFS Storage 7000,.. 4 Services 203

204 SMB. ACL MMC Sun ZFS Storage E

205 SMB,. SMB ( ) ( ). 4 Services 205

206 SMB. Services Sun ZFS Storage MMC. vscan. 206 Sun ZFS Storage E

207 SMB MMC. Administrators Power Users Administrators..,,. 4 Services 207

208 SMB CLI CLI SMB. create list. "Bill". twofish:> configuration services smb twofish:configuration services smb> create twofish:configuration services rule (uncommitted)> set use_nss=false twofish:configuration services rule (uncommitted)> set user=bill twofish:configuration services rule (uncommitted)> set directory=/export/wdp twofish:configuration services rule (uncommitted)> set container="dc=com,dc=fishworks, ou=engineering,cn=myhome" twofish:configuration services rule (uncommitted)> commit twofish:configuration services smb> list RULE NSS USER DIRECTORY CONTAINER rule-000 false Bill /export/wdp dc=com,dc=fishworks, ou=engineering,cn=myhome. &?.. twofish:configuration services smb> create twofish:configuration services rule (uncommitted)> set use_nss=false twofish:configuration services rule (uncommitted)> set user=* twofish:configuration services rule (uncommitted)> set directory=/export/?/& twofish:configuration services rule (uncommitted)> set container="dc=com,dc=fishworks, ou=engineering,cn=myhome" twofish:configuration services rule (uncommitted)> commit twofish:configuration services smb> list RULE NSS USER DIRECTORY CONTAINER rule-000 false Bill /export/wdp dc=com,dc=fishworks, ou=engineering,cn=myhome. twofish:configuration services smb> create twofish:configuration services rule (uncommitted)> set use_nss=true twofish:configuration services rule (uncommitted)> set container="dc=com,dc=fishworks, ou=engineering,cn=myhome" twofish:configuration services rule (uncommitted)> commit twofish:configuration services smb> list RULE NSS USER DIRECTORY CONTAINER rule-000 true dc=com,dc=fishworks, ou=engineering,cn=myhome twofish:configuration services smb> groups twofish:configuration services smb groups> create twofish:configuration services smb member (uncommitted)> set user=bill 208 Sun ZFS Storage E

209 SMB twofish:configuration services smb member (uncommitted)> set group="backup Operators" twofish:configuration services smb member (uncommitted)> commit twofish:configuration services smb groups> list MEMBER USER GROUP member-000 WINDOMAIN\Bill Backup Operators BUI SMB Sun ZFS Storage SMB BUI CLI20. BUI CLI Maintenance > System. BUI.,.. Apply.. Apply. DNS.. Active Directory IP. NTP..,. NTP. 4 Services 209

210 SMB NTP. NTP SMB. Active Directory.. Apply.... "Allocate and verify storage" JBOD. JBOD none, half all. JBOD JBOD. Commit. "Configure Added Storage".,.. Commit.. Oracle HTTP.... '$'.. Active Directory Active Directory. Active Directory. 210 Sun ZFS Storage E

211 SMB Configuration > Services > Active Directory Join Domain. Active Directory,, Apply.. 2 Shares Projects Add..... Projects General Inherit. SMB /export. Protocols. sharesmb ACL. Apply. SMB 1 Configuration > Services > SMB. 2 3 SMB Apply. Configuration > Services > SMB Autohome SMB Apply. 4 Services 211

212 FTP 4 Configuration > Services > SMB Local Groups Administrators Backup Operators Apply. FTP FTP( ) FTP.. Services. FTP Port (for incoming connections) Maximum # of connections ("0" for unlimited) Turn on delay engine to prevent timing attacks Default login root Logging level Permissions to mask from newly created files and dirs FTP. 21. FTP.. ( ) DoS FTP. "/". FTP. proftpd Sun ZFS Storage E

213 FTP Enable SSL/TLS Port for incoming SSL/TLS connections Permit root login Maximum # of allowable login attempts SSL/TLS FTP. FTP.. SSL/TLS FTP. 21. FTP. FTP. FTP,. 3. Services BUI CLI. CLI. proftpd proftpd_xfer proftpd_tls FTP.. SSL/TLS FTP. Services Logs. FTP 1 2 FTP Configuration->Services. FTP.. 4 Services 213

214 HTTP 3 4 Shares. "Protocols" FTP. (/+). HTTP HTTP HTTP, HTTPS HTTP WebDAV(Web based Distributed Authoring and Versioning).. HTTP HTTPS URL. HTTPS. Require client login Protocols HTTP Port (for incoming connections) HTTPS Port (for incoming secure connections),. HTTP "nobody".. (HTTP, HTTPS ). HTTP, 80. HTTP, 443. Services BUI CLI. 214 Sun ZFS Storage E

215 HTTP "Require client login", NIS LDAP. Active Directory. HTTP. HTTPS. HTTP NFS FTP.. (uid 100 ) "nobody". "nobody". "Require client login". "nobody" "nobody".. Unix 666( / ) Unix 777( // ). network- HTTP Services Logs. HTTP 1 2 HTTP Configuration->Services. HTTP.. 4 Services 215

216 NDMP 3 4 Shares. "Protocols" HTTP. (/+). NDMP NDMP( ) DMA( ) NDMP NDMP.NDMP ( ). DMA. NDMP. [[Maintenance:System:ConfigurationBackup /]].. DMA. () SCSI FC( ). NDMP.DMA NDMP DMA.DMA. NDMP. DMA. ( ) DMA ( ).DMA "3 ". IP. 216 Sun ZFS Storage E

217 NDMP NDMP. 3.DMA NDMP "TYPE". dump tar zfs. DAR(Direct Access Recovery).. DAR(Direct Access Recovery).. DAR(Direct Access Recovery). NDMPv4. NDMP , 2010.Q3 "zfs" 2010.Q Q3 "zfs". "dump" "tar" "dump" "tar"., /export/home. /export/code ( : /export/code) ( : /export/code/src).. ( : /export/code/.zfs/snapshot/mysnap). 4 Services 217

218 NDMP "dump" "tar"., Shares. NDMP ( : )., /export/proj /export/proj. /export/someshare/somedir /export/someshare.., proj1 share1 share2 /export/foo.... "ndmp-".,.. "zfs" "zfs". BUI. CLI canonical_name. '/' '/'. 218 Sun ZFS Storage E

219 NDMP "zfs".. pool-0/local/default pool-0/local/ pool-0/local/default/code pool-0/local/ "default". "default" "yesterday" "yesterday". "default" "code". "code". "default" "code" "yesterday ". "code". "zfs". UPDATE=n DMA.. " ". ( : ) "zfs". "zfs".. "zfs".. " ".. DMA 3. LEVEL DMP_NAME UPDATE DMP_NAME.. 4 Services 219

220 NDMP -N N LEVEL, ("DMP_NAME" ). -0.UPDATE "y"( ) N. DMA.. 1 7,14, () , "tar" "dump" "zfs"... NDMP.., Sun ZFS Storage E

221 NDMP "zfs" NDMP.NDMP "ZFS_FORCE" "y" NDMP " " NDMP ( ). NDMP. DMA username and password Enable DAR Ignore file metadata changes for incremental backups Restore full absolute path for partial restore (v3 only) NDMP version TCP port Default restore pool(s) DMA( )..... "tar" "dump"... DMA NDMP. NDMP NDMPv3. NDMPv4. "tar" "dump".ndmp Services 221

222 SFTP Rollback datasets before restore (ZFS backups only) DMA tape mode (for locally attached drives) "zfs"... "ZFS_FORCE" DMA ( " " ) DMA.. ZFS_FORCE DMA. DMA SystemV BSD. DMA SystemV. NDMP.DMA DMA. DMA DMA. Services BUI CLI. system-ndmpd:default NDMP Services Logs. SFTP SFTP(SSH File Transfer Protocol) SFTP.. Services. 222 Sun ZFS Storage E

223 SFTP Port (for incoming connections) Permit root login Logging level SFTP Keys* SFTP SFTP. SFTP. SFTP. SFTP RSA/DSA SFTP. SFTP " ". SFTP SFTP. SFTP. Services BUI CLI. CLI. SFTP SFTP. 22 SSH. SFTP 218 SFTP., SFTP OpenSolaris. manta# sftp -o "Port 218" network-sftp:default SFTP. Services Logs. 4 Services 223

224 Virus Scan SFTP SFTP Configuration->Services. SFTP.. Shares. "Protocols" SFTP. (/+). Virus Scan.,.. NFSv4 NFS. Maximum file size to scan Allow access to files that exceed maximum file size. ( : ). 1GB Sun ZFS Storage E

225 Virus Scan Services BUI CLI. CLI.. "*" ( ).., zip "data-archive*.zip". exe com bat doc data-archive*.zip * zip " *".. ICAP(Internet Content Adaptation Protocol, RFC 3507). Enable Host Maximum Connections Port. IP Services 225

226 NIS vscan. Services Logs.. BUI CLI. Virus Scan Configuration->Services->Virus Scan.. /. Shares.. "General". "Virus scan". NIS NIS( ). NIS. NIS FTP HTTP/WebDAV. 226 Sun ZFS Storage E

227 NIS NIS. NIS UID GID.. Domain Server(s): Search using broadcast Server(s): Use listed servers NIS. NIS NIS. NIS IP. Services BUI CLI. NIS. network-nis-client:default appliance-kit-nsswitch:default system-identity:domain NIS. NIS.. Services Logs.. BUI CLI. 4 Services 227

228 LDAP NIS NIS NIS NIS. Configuration->Services->NIS. NIS. /. Configuration->Users. "directory". NIS. Users. LDAP LDAP(Lightweight Directory Access Protocol),, ( ). LDAP. LDAP FTP HTTP/WebDAV. LDAP ( ID ) ACL. LDAP. LDAP UID.. LDAP. 228 Sun ZFS Storage E

229 LDAP Protect LDAP traffic with SSL/TLS Base search DN Search scope Authentication method Bind credential level Proxy DN Proxy Password Schema definition Servers TLS(, SSL ) LDAP.,. LDAP. ( ) ( ).. LDAP. (RFC 4513), SASL/DIGEST-MD5 SASL/GSSAPI. DN SSL/TLS. SASL/GSSAPI. LDAP.... SASL/GSSAPI....,. " ". LDAP. LDAP...LDAP. Services BUI CLI. LDAP,. RFC 2307(posixAccount posixgroup). DN dc=example,dc=com. 4 Services 229

230 LDAP users ou=people,base search DN ou=people,dc=example,dc=com groups ou=group,base search DN ou=group,dc=example,dc=com, Schema definition. entire DN. Base search DN Search scope. ("Users" "Groups") default = new. default new.. posixaccount unixaccount Users Object posixaccount = unixaccount. uid employeenumber Users Attribute uid = employeenumber. posixgroup unixgroup Groups Object posixgroup = unixgroup. cn groupaccount Groups Attribute cn = groupaccount. appliance-kit-nsswitch: default LDAP. Services Logs.. BUI CLI. 230 Sun ZFS Storage E

231 Active Directory LDAP LDAP LDAP LDAP. Configuration->Services->LDAP. LDAP. /. Configuration->Users. "directory". LDAP. Users. Active Directory Active Directory,, Microsoft Active Directory. SMB 2. SMB AD. SMB. Users. Active Directory. 4 Services 231

232 Active Directory Active Directory Domain Administrative User Administrative Password Additional DNS Search Path Active Directory. Active Directory AD.. DNS DNS Active Directory.. Windows Workgroup. Services BUI CLI. CLI.. Active Directory.,. Active Directory, Active Directory SMB. Kerberos Kerberos NFS Active Directory. LDAP LDAP.LDAP SMB SMB. 232 Sun ZFS Storage E

233 Active Directory Windows Server 2008 Windows Windows Server 2003 Windows Server 2008 SP Q2 3.1 KB B. Windows Server 2008 SP Q2 4.0 KB951191, KB A B. Windows Server 2008 SP Q2 4.0 C. Windows Server 2008 R Q2 4.0 C. A : Kerberos (KB951191) Windows Server 2008 SP1. Windows Server 2008 SP1 Kerberos, KB951191( default.aspx/kb/ ( Windows Server 2008 SP2 R Q2.4.0 Windows 2008 Windows Server 2008 SP2 R Q2.4.0 Windows 2008 Windows Server 2008 SP1 KB Windows 2008 SP2. B : NTLMv2 (KB957441) Windows Server 2008 SP1 ( LMCompatibilityLevel NTLMv2. Windows 2008 SP1 LMCompatibilityLevel 5. KB C : NTLMv2 Windows Server 2008 SP2 R2 KB Services 233

234 Active Directory BUI "JOIN DOMAIN", "JOIN WORKGROUP". CLI CLI. twofish:> configuration services ad twofish:configuration services ad> show Properties: <status> = online mode = domain domain = eng.fishworks.com Children: domain => Join an Active Directory domain workgroup => Join a Windows workgroup "eng.fishworks.com".. twofish:configuration services ad> workgroup twofish:configuration services ad workgroup> set workgroup=workgroup twofish:configuration services ad workgroup> commit twofish:configuration services ad workgroup> done twofish:configuration services ad> show Properties: <status> = disabled mode = workgroup workgroup = WORKGROUP. twofish:configuration services ad> done twofish:> configuration services smb twofish:configuration services smb> set ads_site=sf twofish:configuration services smb> set pdc= twofish:configuration services smb> commit twofish:configuration services smb> show Properties: <status> = online lmauth_level = 4 pdc = ads_site = sf twofish:configuration services smb> done. 234 Sun ZFS Storage E

235 Active Directory twofish:> configuration services ad twofish:configuration services ad> domain twofish:configuration services ad domain> set domain=fishworks.com twofish:configuration services ad domain> set user=administrator twofish:configuration services ad domain> set password=******* twofish:configuration services ad domain> set searchdomain=it.fishworks.com twofish:configuration services ad domain> commit twofish:configuration services ad domain> done twofish:configuration services ad> show Properties: <status> = online mode = domain domain = fishworks.com BUI CLI. Active Directory SMB Active Directory ( ). SMB ( ). NTP, 5. DNS Active Directory, IP DNS. Active Directory,. / /. 4 Services 235

236 Identity Mapping Identity Mapping ID Unix UID/GID Windows SID Windows Unix ID. SMB ID Windows Unix ID. SMB ID Windows ID Unix ID. Windows Unix ID UID GID ID. SMB NFS. Windows Unix ID NFS SMB ID. Windows,. Active Directory SAM Windows. SID. SID Windows. Unix. ( : NIS LDAP) Unix. Unix UID GID. UID GID Unix.. ID SID, UID GID. 3. ID IDMU IDMU Active Directory. Active Directory LDAP Sun ZFS Storage E

237 Identity Mapping IDMU.. ID. ID. IDMU Microsoft "Unix ID " IDMU. Windows Server 2003 Windows Server 2003 R2. "Unix ". IDMUWindows NIS/NFS.IDMU UNIX ( UID, GID,, ) Active Directory Users and Computers "UNIX Attribute". RFC2307 NIS AD. IDMU ID Unix Windows Unix ID. ID IDMU.. ID ID LDAP Active Directory.. ID Directory-Based Mapping AD Attribute - Unix User Name AD Attribute - Unix Group Name. Unix AD. Unix AD. 4 Services 237

238 Identity Mapping Native LDAP Attribute - Windows User Name Windows ID LDAP. Services BUI CLI. CLI. Active Directory LDAP Solaris CIFS Administration Guide Managing Directory-Based Identity Mapping for Users and Groups (Task Map). ID. Windows ID Unix ID. ID. Mapping type Mapping direction Windows domain Windows entity Unix entity Unix type.. Windows Unix Unix Windows. Windows ID Active Directory. Windows ID. Unix ID. Unix ID,. Case Sensitivity Windows Unix. JSMITH, JSmith jsmith Windows Unix Sun ZFS Storage E

239 Identity Mapping Windows-to-Unix Windows Unix., Windows "jsmith" Unix "jsmith". Windows "Jsmith". Windows-to-Unix "*".ID Windows Unix "*" Windows Unix. Windows Unix., Windows Unix "jsmith". Windows.., Windows Unix "jsmith". ID Windows. Unix-to-Windows., Unix "jsmith" "JSMITH" Windows. ID ID. SMB Windows Unix. ( )., "mydomain.com" Windows Unix. "mydomain.com" "Engineering" Windows Unix.. ID ID. Windows Unix 4 Services 239

240 Identity Mapping., "guest" Unix SMB... Windows ID Unix ID, Unix ID Windows ID. Windows ID Unix ID. Unix ID Windows ID. Windows ID. Unix ID. (,,,, ) Unix ID.. Unix Windows SMB Windows Windows ID. NFS "nobody". ID NFS SMB. NFS SMB ID. ID SMB., 10.. ID.,. 240 Sun ZFS Storage E

241 Identity Mapping Windows Unix ID Windows. Windows Unix. IDMU. BUI Mappings ID. Windows Unix ID. User Properties Group Properties ID. CLI 2. Windows Unix. twofish:> configuration services idmap twofish:configuration services idmap> create twofish:configuration services idmap (uncommitted)> set windomain=eng.fishworks.com twofish:configuration services idmap (uncommitted)> set winname=bill twofish:configuration services idmap (uncommitted)> set direction=bi twofish:configuration services idmap (uncommitted)> set unixname=wdp twofish:configuration services idmap (uncommitted)> set unixtype=user twofish:configuration services idmap (uncommitted)> commit twofish:configuration services idmap> list MAPPING WINDOWS ENTITY DIRECTION UNIX ENTITY idmap-000 (U) == wdp (U) Windows. twofish:configuration services idmap> create twofish:configuration services idmap (uncommitted)> list Properties: windomain = (unset) winname = (unset) direction = (unset) unixname = (unset) unixtype = (unset) twofish:configuration services idmap (uncommitted)> set windomain=guest.fishworks.com twofish:configuration services idmap (uncommitted)> set winname=* twofish:configuration services idmap (uncommitted)> set direction=win2unix twofish:configuration services idmap (uncommitted)> set unixname= twofish:configuration services idmap (uncommitted)> set unixtype=user twofish:configuration services idmap (uncommitted)> commit 4 Services 241

242 DNS twofish:configuration services idmap> list MAPPING WINDOWS ENTITY DIRECTION UNIX ENTITY idmap-000 (U) == wdp (U) idmap-001 (U) => "" (U). BUI CLI. ID ID Active Directory. ( )..... Windows ID.. Unix ID. /. DNS DNS( ) IP IP. NIS / LDAP 242 Sun ZFS Storage E

243 DNS DNS., Analytics. DNS IP.. NS(NameServer).DNS. DNS Domain DNS Server(s) Allow IPv4 non-dns resolution Allow IPv6 non-dns resolution. DNS.IP. NIS / LDAP IPv4, IPv4. NIS / LDAP IPv4 IPv6, IPv4 IPv6. Services BUI CLI. CLI CLI nslookup getent hosts. caji:> nslookup deimos deimos.sf.fishworks.com caji:> getent hosts deimos deimos.sf.fishworks.com network-dns-client:default DNS. 4 Services 243

244 DNS Services Logs. Active Directory DNS Active Directory Active Directory., example.com Active Directory redmond.example.com example.com redmond.example.com Active Directory. Active Directory. Active Directory. DNS DNS IP. DNS. NIS LDAP. DNS. LDAP ou=hosts( DN) DN. NFS. NFS. DNS DNS.DNS NIS( ) LDAP( ).. DNS DNS( ) NIS LDAP. 'getent' CLI.. DNS DNS DNS.. Analytics. 244 Sun ZFS Storage E

245 IPMP Active Directory ( ). SSL LDAP. IP... " DNS ". IPMP IPMP(Internet Protocol Network Multipathing) ( ).. IPMP Network. Failure detection latency Enable fail-back IPMP IP.. Services BUI CLI. network-initial:default. Services Logs. 4 Services 245

246 NTP IPMP Network. NTP NTP(Network Time Protocol).. UTC.BUI. multicast address NTP server(s) NTP Authentication Keys NTP. NTP ( ). NTP NTP pool.ntp.org : 10, : ASCII, : SUN7000 Services BUI CLI... NTP NTP NTP 246 Sun ZFS Storage E

247 NTP NTP NTP ID NTP.. NTP NTP (, ). NTP... DES DES NTP NTP ASCII 1-8 ASCII topsecret MD5 MD5 1-8 ASCII md5secret NTP., NTP. BUI BUI NTP,. NTP. CLI configuration services ntp authkey. clownfish:configuration services ntp> authkey clownfish:configuration services ntp authkey> create. clownfish:configuration services ntp authkey> create clownfish:configuration services ntp authkey-000 (uncommitted)> get keyno = (unset) type = (unset) key = (unset) clownfish:configuration services ntp authkey-000 (uncommitted)> set keyno=1 keyno = 1 (uncommitted) clownfish:configuration services ntp authkey-000 (uncommitted)> set type=a type = A (uncommitted) clownfish:configuration services ntp authkey-000 (uncommitted)> set key=coconuts key = ******** (uncommitted) 4 Services 247

248 NTP clownfish:configuration services ntp authkey-000 (uncommitted)> commit clownfish:configuration services ntp authkey> CLI serverkeys. servers. 0., "gefilte" "carp". clownfish:configuration services ntp> set servers=gefilte,carp servers = gefilte,carp (uncommitted) clownfish:configuration services ntp> set serverkeys=1,1 serverkeys = 1,1 (uncommitted) clownfish:configuration services ntp> commit clownfish:configuration services ntp> "gefilte"1, "carp"2, "dory"3. clownfish:configuration services ntp> set servers=gefilte,carp,dory servers = gefilte,carp,dory (uncommitted) clownfish:configuration services ntp> set serverkeys=1,2,3 serverkeys = 1,2,3 (uncommitted) clownfish:configuration services ntp> commit clownfish:configuration services ntp> "gefilte" "carp"1 NTP "dory". clownfish:configuration services ntp> set servers=gefilte,carp,dory servers = gefilte,carp,dory (uncommitted) clownfish:configuration services ntp> set serverkeys=1,1,0 serverkeys = 1,1,0 (uncommitted) clownfish:configuration services ntp> commit clownfish:configuration services ntp> BUI BUI ( ) ( ). NTP "SYNC". SMB 5. NTP SMB. 248 Sun ZFS Storage E

249 Remote Replication network-ntp:default NTP Services Logs. NTP 1 2 BUI. NTP. "SYNC". Remote Replication Sun Storage (Sources ) (Targets ).. 4 Services 249

250 Dynamic Routing Dynamic Routing RIP RIPng RIP(Routing Information Protocol) RIP - ( ) -. IPv4 RIPv1 RIPv2 IPv6 RIPng. "".. RIP RIPng UDP network-routing-route:default network-routing-ripng:quagga RIP. RIPng. Phone Home Phone Home Phone Home. Oracle Oracle. Phone Home. Phone Home Oracle. * - Oracle... * - Oracle. Oracle. 250 Sun ZFS Storage E

251 Phone Home * - Oracle.. Oracle Single Sign-On Phone Home Oracle Single Sign-On. ( Register. Services BUI CLI. CLI Phone Home scrk. Phone Home Oracle HTTP... Use proxy Host/port Username Password. IP... Oracle Single Sign-On. ( Register.BUI. 1. Oracle Single Sign-On.. BUI CLI. 2.. '$' Services 251

252 SNMP CLI. 1. soa_id soa_password Oracle Single Sign-On domain_name 'Change account...' Oracle Single Sign-On..... Last heartbeat sent at Oracle. Oracle Single Sign-On Phone Home. Phone Home Oracle Single Sign-On. Maintenance->Logs->Phone Home Phone Home. SNMP SNMP(Simple Network Management Protocol) 2. SNMP. MIB. 252 Sun ZFS Storage E

253 SNMP SNMP. SNMP 1 2c. SNMP community name Authorized network Appliance contact Trap destinations SNMP. SNMP CIDR /8(localhost ), /0. MIB-II OID ( ). SNMP. SNMP IP. SNMP Services BUI CLI. CLI. SNMP. SNMP MIB-II. ID. MIB SNMP MIB(Management Information Base). MIB MIB-II -, MIB-II MIB-II IP - IP Sun Enterprise MIB(SUN-MIB.mib.txt) Sun FM - ( MIB ) Sun AK - ( MIB ) 4 Services 253

254 SNMP SunFMMIB Sun FM MIB(SUN-FM-MIB.mib) SUN Fault Manager. Fault Manager Fault Manager 4 OID. OID MIB. SunAKMIB Sun AK MIB(SUN-AK-MIB.mib)., -, 3 OID. OID MIB. 254 Sun ZFS Storage E

255 SMTP SNMP. BUI CLI. SNMP SNMP,. SNMP /.. SNMP,. SNMP /8. /.. SMTP SMTP Alerts. SMTP. SMTP DNS(MX ). DNS 4 Services 255

256 SMTP DNS MX smarthost. Send mail through smarthost Smarthost hostname Allow customized from address Custom from address. DNS.. From Custom. From. From. Services BUI CLI.. DNS,. DNS smarthost. network-smtp:sendmail mail SMTP. SMTP ( ). Services Logs. 256 Sun ZFS Storage E

257 System Identity Service Tags. Oracle Oracle.. Discovery Port Listener Port UDP TCP Services BUI CLI. System Identity.. 4 Services 257

258 SSH System Name System Location. DNS ( DNS )... SNMP MIB-II syslocation. Services BUI CLI. system-identity:node ID. Services Logs. SSH SSH(Secure Shell) CLI BUI. SSH Analytics. Server key length. 768 Key regeneration interval ( ) Sun ZFS Storage E

259 Shadow Migration Login grace period ( ) SSH. 120 Permit root login SSH. yes Services BUI CLI. CLI. network-ssh:default SSH Services Logs.. BUI CLI. SSH 1 2 SSH false. /. Shadow Migration Services 259

260 Syslog... Number of Threads. (, I/O CPU). Services BUI CLI.. Syslog Syslog 2. Syslog. Syslog Syslog. Syslog ( printf ).. facility -. severity -. timestamp - UTC. hostname Sun ZFS Storage E

261 Syslog tag -.. message -.. Solaris Linux Syslog. Syslog. Syslog. Syslog RFC 3164 "" RFC Syslog UDP... Protocol Version Destinations Syslog, Classic Modern. IPv4 IPv6. Services BUI CLI. CLI. Syslog. Syslog: RFC 3164 Syslog facility level, timestamp, hostname, tag. tag. hostname System Identity. 4 Services 261

262 Syslog Syslog: RFC 5424 Syslog facility level, (1), timestamp, hostname, app-name. Sun Storage Syslog RFC 5424 procid, msgid structured-data nil (-). app-name. hostname System Identity. Syslog. Sun Storage Syslog ak., Sun Storage. ak. ak SUNW-MSG-ID. idmap POSIX Windows ID ID. smbd SMB. Send Syslog Message Syslog.. SUNW-MSG-ID Sun. Sun,., sun.com/msg/. 262 Sun ZFS Storage E

263 Syslog TYPE VER SEVERITY EVENT-TIME PLATFORM CSN HOSTNAME SOURCE REV EVENT-ID DESC AUTO-RESPONSE REC-ACTION.. Fault, Defect, Alert.. SUNW-MSG-ID "1". "1" VER UTC "Day Mon DD HH:MM:SS YYYY". : Fri Aug 14 21:34: Oracle.. System Identity.. Oracle.. Oracle. UUID(Universally Unique Identifier). Oracle, UUID. Oracle EVENT-ID Oracle..., DRAM... SEVERITY. Syslog Minor LOG_WARNING. 4 Services 263

264 Syslog Syslog Major LOG_ERR. Critical LOG_CRIT. Syslog.. Syslog. Solaris Solaris Syslog syslogd(1m). Syslog Solaris svccfg svcadm Syslog. # svccfg -s system/system-log setprop config/log_from_remote = true # svcadm refresh system/system-log Solaris syslogd Syslog. Solaris syslog.conf(4). Solaris syslogd /var/adm/messages. Aug 14 21:34:22 poptart.sf.fishpong.com poptart ak: SUNW-MSG-ID: AK-8000-LM, \ TYPE: alert, VER: 1, SEVERITY: Minor\nEVENT-TIME: Fri Aug 14 21:34: \n\ PLATFORM: i86pc, CSN: , HOSTNAME: poptart\n\ SOURCE: jsui.359, REV: 1.0\n\ EVENT-ID: 92dfeb39-6e15-e2d5-a7d9-dc3e221becea\n\ DESC: A test alert has been posted.\n\ AUTO-RESPONSE: None.\nIMPACT: None.\nREC-ACTION: None. Linux Linux Syslog sysklogd(8). Syslog Linux -r ( ) /etc/sysconfig/syslog. SYSLOGD_OPTIONS="-r -m 0". 264 Sun ZFS Storage E

265 Syslog # /etc/init.d/syslog stop # /etc/init.d/syslog start Linux Syslog UDP ipfilter. Syslog UDP INPUT. # iptables -I INPUT 1 -p udp --sport dport 514 -j ACCEPT Linux syslogd /var/log/messages. Aug 12 22:03: poptart ak: SUNW-MSG-ID: AK-8000-LM, \ TYPE: alert, VER: 1, SEVERITY: Minor EVENT-TIME: Wed Aug 12 22:03: \ PLATFORM: i86pc, CSN: , HOSTNAME: poptart SOURCE: jsui.3775, REV: 1.0 \ EVENT-ID: 9d40db b21-e64e-86e5cac90912 \ DESC: A test alert has been posted. AUTO-RESPONSE: None. IMPACT: None. \ REC-ACTION: None. 4 Services 265

266 266

267 5 5 Shares Shares.. shares.. -. Shadow Migration

268 -. Shares - LUN (NFS, SMB, iscsi ) Schema -. ZFS. ZFS LUN. LUN.... RAID-Z. BUI. CLI 'Pool'.. UI. 268 Sun ZFS Storage E

269 LUN..,. ( : )... Shares LUN. SMB, NFS, HTTP/WebDav FTP.LUN iscsi FC( ). project/share.. LUN... Inherited Read-only Space Management,.... SMB.... ( ). 100G. 5 Shares 269

270 Create time Project default Filesystem local LUN local Custom LUN.,,.. LUN,.,.,... LUN... schema. LUN...,..zfs/snapshot..zfs..LUN.,., Sun ZFS Storage E

271 Shadow Migration :.. Oracle. "Oracle ("SLA") ".,.... Shadow Migration.. Sun Storage ,.,. NAS. Sun Storage Shares 271

272 Shadow Migration X X Y.... rsync ,. X Y M. M,. ( : ). (,, ).. ( ).. Shadow Migration 272 Sun ZFS Storage E

273 Shadow Migration. NFS " ".., X Sun Storage /.....,.. ID. ACL( ) ID... 5 Shares 273

274 Shadow Migration * *.,. NFS. NFSv4. NFSv2 NFSv3 ACL, NFSv2. SMB. LUN...,... (UNIX stat st_nblocks). du(1) SUNWshadow,..SUNWshadow Sun ZFS Storage E

275 Shadow Migration.. NFSv4 (" " ). ID ACL ACL ID.. NFSv4 mapid. NFSv4. NFSv3. ID () POSIX ACL.. "nobody" NFSv4 mapid. ( )' '.. BUI Filesystem Creation. CLI shadow.. Local - file:///<path> NFS - nfs://<host>/<path> BUI NFS <host>:/<path>. UNIX.BUI (file:// nfs://).. 5 Shares 275

276 Shadow Migration BUI ( : ) , Sun ZFS Storage E

277 Shadow Migration ( UI ) BUI CLI...,.....,..,..... ( : ) ''. BUI. CLI shadow cancel.. ( : SUNWshadow ). 5 Shares 277

278 Shadow Migration NDMP... ( )......NDMP.. analytics Analytics. analytics "Data Movement" Sun ZFS Storage E

279 Shadow Migration.,,. ( ).. (,ACL ).. analytics.,...,,... file shadow Sun Storage NFS ( ). 5 Shares 279

280 2. CLI 'nfs://<host>/<snapshotpath>' BUI '<host>/<snapshotpath>' ( 'NFS' ) shadow.. /. 3. ID. 4.. NFS Sun Storage NFS ( ) CLI 'nfs://<host>/<path>' BUI '<host>/<path>' ( 'NFS' ) shadow. 5. SS CLI LUN. LUN Sun ZFS Storage E

281 LUN.,LUN Referenced Data... Snapshot Data.. 5 Shares 281

282 Quota.,,. Reservation ( ) "".. "".. 100%.... Sun Storage , 282 Sun ZFS Storage E

283 2 X Y,, Z, Z X Y. X X Y ( : ) LUN....,.. 'include snapshots'.. 5 Shares 283

284 . ( )..,... Shares -> General -> Space Usage -> Data BUI. CLI quota quota_snap.... 'include snapshots'.. ( )., 50G 100G 50G 150G.. Shares -> General -> Space Usage -> Data BUI. CLI reservation reservation_snap Sun ZFS Storage E

285 BUI BUI "Space Usage -> Users and Groups" "Shares -> General".... User or Group -.. User -. Group -... "Show All". ( ). UNIX Windows ( ) ( ). CLI CLI users groups. show. get. clownfish:> shares select default clownfish:shares default> users clownfish:shares default users> list USER NAME USAGE user-000 root 325K user-001 ahl 9.94K user-002 eschrock 20.0G clownfish:shares default users> select name=eschrock clownfish:shares default user-002> get name = eschrock unixname = eschrock unixid = winname = (unset) winid = (unset) usage = 20.0G. POSIX Windows ID.. 5 Shares 285

286 ..,...,...,.... NDMP.. BUI Space Usage -> Users & Groups general.. "".. 'Apply'.... CLI CLI 'users' 'groups'. 'set quota'.. 'quota'.. '0'. clownfish:> shares select default select eschrock clownfish:shares default/eschrock> users clownfish:shares default/eschrock users> list 286 Sun ZFS Storage E

287 USER NAME USAGE QUOTA user-000 root 321K - user-001 ahl 9.94K - user-002 eschrock 20.0G - clownfish:shares default/eschrock users> select name=eschrock clownfish:shares default/eschrock user-002> get name = eschrock unixname = eschrock unixid = winname = (unset) winid = (unset) usage = 20.0G quota = (unset) clownfish:shares default/eschrock user-002> set quota=100g quota = 100G (uncommitted) clownfish:shares default/eschrock user-002> commit clownfish:shares default/eschrock user-002> done clownfish:shares default/eschrock users> quota clownfish:shares default/eschrock users quota (uncomitted)> set name=bmc name = bmc (uncommitted) clownfish:shares default/eschrock users quota (uncomitted)> set quota=200g quota = 200G (uncommitted) clownfish:shares default/eschrock users quota (uncomitted)> commit clownfish:shares default/eschrock users> list USER NAME USAGE QUOTA user-000 root 321K - user-001 ahl 9.94K - user-002 eschrock 20.0G 100G user-003 bmc - 200G ID ID. UNIX Windows ID. ID. UNIX Windows ID. UNIX UNIX ID.,IDUNIX ID. UNIX ID. Windows. ID ( : ). ID.. 5 Shares 287

288 ... /export. /export ,,.... ( ) Sun ZFS Storage E

289 NFSv2/NFSv3 NFS MOUNT. NFSv2 NFSv3.. NFSv4. NFSv4 NFSv4 NFSv3.., /export/home /export. NFSv4 (Linux ) " "... SMB SMB.. ( ). FTP/FTPS/SFTP.... HTTP/HTTPS /shares /export/home HTTP/HTTPS /shares/export/home.. FTP HTTP. 5 Shares 289

290 Shares Shares BUI UI "Shares -> Shares"...,,. "Filesystems" "LUNs".. Name Size Mountpoint GUID LUN... NFS FTP HTTP. SMB. LUN SCSI GUID. iscsi Sun ZFS Storage E

291 Shares.... [[Shares:Projects ]]... ( ). ( ) ( ). 0. Available space... Referenced data. LUN.. 0. Snapshot data... 5 Shares 291

292 Shares Unused Reservation. LUN.... Total space,. ( ).,. Compression ratio..,2x 1/2.. Case sensitivity.. BUI CLI Mixed mixed. NFS, FTP HTTP. SMB.. SMB. SMB " ". 292 Sun ZFS Storage E

293 Shares BUI CLI Insensitive insensitive ( : NFS). SMB. SMB 2. Sensitive sensitive SMB. SMB. Windows. Reject non UTF-8 UTF-8. UTF-8. NFSv3. NFSv4 UTF-8 SMB. "on". SMB UTF-8. NFSv3 "off". "off" SMB NFSv4 NFSv3 UTF-8. "none" "on". Normalization ( )... "none"( ) "Reject non UTF-8" "on". Wikipedia. BUI CLI None none. 5 Shares 293

294 Shares BUI CLI Form C formc NFC(Normalization Form Canonical Composition) -. Form D formd NFD(Normalization Form Canonical Decomposition) -. Form KC formkc NFKC(Normalization Form Compatibility Composition) -. Form KD formkd NFKD(Normalization Form Compatibility Decomposition) -. Volume block size LUN K 2 8K. Origin. Data Migration Source NFS. Shadow Migration. BUI. "Projects" Sun ZFS Storage E

295 Shares. "All" "Shares".... "shares". "Filesystems" "LUNs"... : User Group Permissions Mountpoint Reject non UTF-8( ) Case sensitivity( ) Normalization( ) LUN : Volume size Thin provisioned Volume block size( ) CLI CLI shares.. clownfish:> shares clownfish:shares> select default clownfish:shares default> select foo clownfish:shares default/foo> get Properties: aclinherit = restricted (inherited) aclmode = discard (inherited) atime = true (inherited) 5 Shares 295

296 Shares casesensitivity = mixed checksum = fletcher4 (inherited) compression = off (inherited) compressratio = 100 copies = 1 (inherited) creation = Mon Oct :21:33 GMT+0000 (UTC) mountpoint = /export/foo (inherited) normalization = none quota = 0 quota_snap = true readonly = false (inherited) recordsize = 128K (inherited) reservation = 0 reservation_snap = true secondarycache = all (inherited) nbmand = false (inherited) sharesmb = off (inherited) sharenfs = on (inherited) snapdir = hidden (inherited) utf8only = true vscan = false (inherited) sharedav = off (inherited) shareftp = off (inherited) space_data = 43.9K space_unused_res = 0 space_snapshots = 0 space_available = 12.0T space_total = 43.9K root_group = other root_permissions = 700 root_user = nobody filesystem lun.. clownfish:shares default> filesystem foo clownfish:shares default/foo (uncommitted)> get aclinherit = restricted (inherited) aclmode = discard (inherited) atime = true (inherited) checksum = fletcher4 (inherited) compression = off (inherited) copies = 1 (inherited) mountpoint = /export/foo (inherited) quota = 0 (inherited) readonly = false (inherited) recordsize = 128K (inherited) reservation = 0 (inherited) secondarycache = all (inherited) nbmand = false (inherited) sharesmb = off (inherited) sharenfs = on (inherited) snapdir = hidden (inherited) vscan = false (inherited) sharedav = off (inherited) 296 Sun ZFS Storage E

297 Shares shareftp = off (inherited) root_group = other (default) root_permissions = 700 (default) root_user = nobody (default) casesensitivity = (default) normalization = (default) utf8only = (default) quota_snap = (default) reservation_snap = (default) custom:int = (default) custom:string = (default) custom: = (default) clownfish:shares default/foo (uncommitted)> set sharenfs=off sharenfs = off (uncommitted) clownfish:shares default/foo (uncommitted)> commit clownfish:shares default> destroy. clownfish:shares default/foo> destroy This will destroy all data in "foo"! Are you sure? (Y/N) clownfish:shares default> rename. clownfish:shares default> rename foo bar clownfish:shares default> move. clownfish:shares default> move foo home clownfish:shares default> users groups.. CLI BUI. CLI get set. unset. CLI BUI BUI aclinherit ACL inheritance behavior aclmode ACL behavior on mode change 5 Shares 297

298 Shares CLI BUI BUI atime Update access time on read casesensitivity Case sensitivity Static checksum Checksum compression Data compression compresratio Compression ratio Static copies Additional replication creation - - dedup Data deduplication exported, Export initiatorgroup LUN Initiator Group logbias Synchronous write bias lunumber LUN LU number lunguid,lun GUID mountpoint Mountpoint nbmand Non-blocking mandatory locking nodestroy Prevent destruction normalization Normalization Static origin Origin Static quota Quota quota_snap Quota/Include snapshots readonly Read-only recordsize Database record size reservation Reservation reservation_snap Reservation/Include snapshots root_group Group root_permissions Permissions 298 Sun ZFS Storage E

299 CLI BUI BUI root_user User rstchown Restrict ownership change secondary cache Cache device usage shadow Data Migration Source Static sharedav Protocols / HTTP / Share mdoe shareftp Protocols / FTP / Share mode sharenfs Protocols / NFS / Share mode sharesmb Protocols / SMB / Resource name snapdir.zfs/snapshot visibility space_available Available space Usage space_data Referenced data Usage space_snapshots Snapshot data Usage space_total Total space Usage space_unused_res Unused reservation Usage sparse LUN Thin provisioned targetgroup LUN Target Group utf8only Reject non UTF-8 Static volblocksize Volume block size Static vscan Virus scan BUI. CLI. CLI Shares CLI. 5 Shares 299

300 ..... Volume size iscsilun. LUN. LUN. LUN. Thin provisioned. LUN LUN. LUN. iscsi. Thin provisioned. LUN. LUN... LUN LUN. LUN " "... Reservation..BUI CLI. 300 Sun ZFS Storage E

301 ... /export... ('/')., "home" /export/home "home/bob" /export/home/bob. SMB. SMB......,. Access control. Update access time on read.. POSIX.... Non-blocking mandatory locking POSIX SMB.. 5 Shares 301

302 POSIX. SMB. SMB.. Data deduplication.,,. Data Deduplication. Status Dashboard.. SHA Data Deduplication ( CPU ). Data Deduplication ( )..., 1GB 1GB 1GB 2x. :. DRAM... DRAM. analytics analytics "DMU ZFS DMU " ZFS DDT ( ). I/O I/O. 302 Sun ZFS Storage E

303 Data compression.. CPU.... BUI CLI Off off. LZJB( ) lzjb CPU. GZIP-2() gzip-2 gzip. GZIP( ) gzip gzip. GZIP-9( ) gzip-9 gzip. CPU. Checksum. ( ).. (fletcher4) CPU. ( LUN ). BUI CLI Fletcher 2( ) fletcher2 16 fletcher Fletcher 4() fletcher4 32 fletcher SHA-256( ) sha256 SHA-256 Cache device usage. 5 Shares 303

304 ...., "all".. (DRAM). BUI CLI All data and metadata all LUN. Metadata only metadata.. Do not use cache devices.. Synchronous write bias 'latency' 'throughput'.. BUI CLI Latency latency ( ). Throughput throughput Sun ZFS Storage E

305 Database record size.. 128K., 128K 128K. 128K. ( : ). 8K K 2. Additional replication... (RAID, ).. BUI CLI Normal( ) 1.. Two Copies 2 2. Three Copies 3 3. Virus scan..... Virus Scan. 5 Shares 305

306 Prevent destruction.,.... off. Restrict ownership change.. "". setuid setgid.. schema... NFS, SMB, HTTP FTP iscsi LUN. BUI., "twofish""fs0". 306 Sun ZFS Storage E

307 NFS SMB HTTP FTP SFTP twofish:/export/fs0 \\twofish\fs0 ( ftp://twofish/export/fs0/ /export/fs0/ iscsi SAN. NFS BUI CLI Share mode off/ro/rw,. CLI "on" "rw". Disable setuid/setgid file creation Prevent clients from mounting subdirectories Anonymous user mapping nosuid nosub anon setuid(s_isuid) setgid(s_isgid) chmod(2)... : NFSv4 NFSv2 NFSv3. "root" uid. ACL. Character encoding See below.. Security mode See below Shares 307

308 CLI Host(FQDN) or Netgroup none IP IP caji.sf.example.com DNS Domain. IP IP CIDR IP sf.example.com /22 / ( ). NIS( ) LDAP( ).LDAP ou=netgroup,(base DN). NFS.. DNS., NFS IP FQDN... DNS.. IP DNS. CLI CLI NFS "sharenfs".. BUI "ro", "rw", "off"., set sharenfs=ro. "nobody". "nosuid" "anon". CLI "=". uid Sun ZFS Storage E

309 set sharenfs="ro,anon=153762" "option=collection". "option" "ro", "rw", "root". DNS / CIDR., sf.example.com / /24. set sharenfs="ro,anon=153762,rw=.sf.example.com,root=@ /24"., "engineering" /. set sharenfs="ro,rw=engineering" "option=mode". option "sec" mode "sys", "krb5", "krb5:krb5i", "krb5:krb5i:krb5p". set sharenfs="sec=krb5". Kerberos. krb5 krb5i krb5p Kerberos V5 krb5 ( ) krb5 ( ) krb5p krb5i krb5 krb5i krb5. 5 Shares 309

310 . NFSv3 NFSv2. NFSv4 UTF-8. UTF-8.,. SMB UTF-8. SMB... euc-cn euc-jp euc-jpms euc-kr euc-tw iso iso iso iso iso iso iso iso iso koi8-r (pass-through). BUI. CLI '*'., set sharenfs="rw,euc-kr=*" 'euc-kr'. set sharenfs="rw,euc-kr=host1.domain.com,euc-jp=host2.domain.com" 'euc-kr' 'euc-jp' 'host1' 'host2'. CLI NFS. NFS. NFS. 310 Sun ZFS Storage E

311 SMB Resource name Enable Access-based Enumeration Is a DFS Namespace Share-level ACL SMB. "off" SMB "on"..... DFS. ACL ACL. ACL. ACL ACL. SMB. SMB ACL. 2 SMB " /\[]: <>+;,?*=.., / ACL. SCSI Target group Initiator group LUN. LUN. 5 Shares 311

312 LU (logical unit) number Assigned LU number Operational status Fix LU number Write cache behavior GUID LUN LUN LUN. LUN LU. LUN. LUN. LU. LU. LU. LUN.... LUN GUID SCSI. GUID. HTTP Share mode HTTP. none, read only, read/write. FTP Share mode FTP. none, read only, read/write. 312 Sun ZFS Storage E

313 SFTP Share mode SFTP. none, read only, read/write. ACL ID. Unix Windows ID. Unix NFS chown.. ID. Unix Windows ID. Unix NFS chgrp. Permissions Unix. Unix NFS chmod.. 5 Shares 313

314 User Other.... Read R. Write W.* Execute X "write". (ACL D ).,. (ACL A ). (ACL W ). BUI. ("user", "group" "other"). CLI Unix 8. user, group other. read(4), write(2) execute(1). 743 user RWX, group R, other WX. POSIX "Use Windows Default Permissions". ACL ACL. Windows Windows. ACL ACL ACL ACL. 314 Sun ZFS Storage E

315 ACL behavior on mode change Unix // chmod(2) ACL ACL. BUI CLI Discard ACL discard ACL.. Mask ACL with mode mask UID. ACL. ACL ACL. Do not change ACL passthrough ACL ACL. ACL inheritance behavior ACL.. ACL.. ACL SMB. ACL Unix owner/group/other. BUI CLI Do not inherit entries discard ACL.. Only inherit deny entries Inherit all but "write ACL" and "change owner" noallow restricted "deny" ACL. ACL "write_acl" "write_owner" ACL.. Inherit all entries passthrough ACL. "passthrough" "data" ACL. 5 Shares 315

316 BUI CLI Inherit all but "execute" when not specified passthrough-x owner, group everyone ACL 'passthrough'. "passthrough" ACL. "cc" "gcc". ACL chmod(1). SMB ACL ACL.. SMB NFS. SMB ACL. NFS. SMB ACL. NFS ACL. ACE( ) ACLACL ACL. SMB ACL ACL.. ACL. ACL ( ). SMB NFSv4 ACL. NFSv3 POSIX ACL. ACL NFSv3 ACL NFSv3..ACL BUI ACL. CLI ACL.BUI. ACL.ACL. SMB ACL ACL. 316 Sun ZFS Storage E

317 ACLACE( ). ACE /,,.ACE ACL. ACL.ACL BUI. Owner Group Everyone Named User Named Group. ACE.. ACE. 'target'. ID. 'target'. ID. Allow Deny ACE. ACE. (r) Read Read Data/List Directory.. (x) Execute File/Traverse Directory (p) Append Data/Add Subdirectory ().... ( ). (a) Read Attributes ( ACL). stat ls stat. (R) Read Extended Attributes. 5 Shares 317

318 Write (w) Write Data/Add File (d) Delete (D) Delete Child (A) Write Attributes (W) Write Extended Attributes (c) Admin Read ACL/Permissions ACL. (C) Write ACL/Permissions (o) Change Owner ACL.. Inheritance (f) Apply to Files. (d) Apply to Directories. (i) Do not apply to self ACE. "Apply to Files" "Apply to Directories". (n) Do not apply past children ACE. "Apply to Files" "Apply to Directories". Windows 3 ACL. Owner Allow Full Control Group Allow Read and Execute 318 Sun ZFS Storage E

319 Everyone Allow Read and Execute...zfs/snapshot visibility.zfs/snapshot. ( ). '.zfs'.. BUI CLI Hidden hidden.zfs.. Visible visible.zfs. BUI "snapshots".. "Snapshots". "Schedules". "Snapshots",,.. 5 Shares 319

320 Name Creation Unique Total Clones... ".auto-<timestamp>"..rr ".ndmp". NDMP... LUN "Show...".. "Snapshots".. "apply". (, ) Sun ZFS Storage E

321 LUN...., Project Name Preserve Local Properties Mountpoint "Preserve local properties".. '.auto-<timestamp>' 30,,,.. "Schedules".. 5 Shares 321

322 Frequency Offset Keep at most half hour, "hour", "day", "week", "month"...,.,.. ( ) Q CLI snapshots. clownfish:> shares select default select builds clownfish:shares default/builds> snapshots clownfish:shares default/builds snapshots> CLI. clownfish:shares default/builds snapshots> list today 322 Sun ZFS Storage E

323 yesterday clownfish:shares default/builds snapshots> snapshot. clownfish:shares default/builds snapshots> snapshot test clownfish:shares default/builds snapshots> rename. clownfish:shares default/builds snapshots> rename test test2 clownfish:shares default/builds snapshots> destroy. clownfish:shares default/builds snapshots> select test2 clownfish:shares destroy This will destroy this snapshot. Are you sure? (Y/N) clownfish:shares default/builds snapshots> destroy. clownfish:shares default/builds snapshots> destroy test2 This will destroy this snapshot. Are you sure? (Y/N) clownfish:shares default/builds snapshots> rollback. clownfish:shares default/builds snapshots> select today clownfish:shares rollback Rolling back will revert data to snapshot, destroying newer data. Active initiators will be disconnected. Continue? (Y/N) clownfish:shares clone... 5 Shares 323

324 clownfish:shares default/builds snapshots> select today clownfish:shares clone testbed clownfish:shares default/testbed (uncommitted clone)> get aclinherit = restricted (inherited) aclmode = discard (inherited) atime = true (inherited) checksum = fletcher4 (inherited) compression = off (inherited) copies = 1 (inherited) mountpoint = /export/testbed (inherited) quota = 0 (default) readonly = false (inherited) recordsize = 128K (inherited) reservation = 0 (default) secondarycache = all (inherited) nbmand = false (inherited) sharesmb = off (inherited) sharenfs = on (inherited) snapdir = hidden (inherited) vscan = false (inherited) sharedav = off (inherited) shareftp = off (inherited) root_group = other (default) root_permissions = 777 (default) root_user = nobody (default) quota_snap = true (default) reservation_snap = true (default) clownfish:shares default/testbed (uncommitted clone)> set quota=10g quota = 10G (uncommitted) clownfish:shares default/testbed (uncommitted clone)> commit clownfish:shares default/builds@today>.. automatic. create destroy., BUI. clownfish:shares default/builds snapshots> automatic clownfish:shares default/builds snapshots automatic> create clownfish:shares default/builds snapshots automatic (uncommitted)> set frequency=day frequency = day (uncommitted) clownfish:shares default/builds snapshots automatic (uncommitted)> set hour=14 hour = 14 (uncommitted) clownfish:shares default/builds snapshots automatic (uncommitted)> set minute=30 minute = 30 (uncommitted) clownfish:shares default/builds snapshots automatic (uncommitted)> set keep=7 keep = 7 (uncommitted) clownfish:shares default/builds snapshots automatic (uncommitted)> get frequency = day (uncommitted) day = (unset) hour = 14 (uncommitted) 324 Sun ZFS Storage E

325 minute = 30 (uncommitted) keep = 7 (uncommitted) clownfish:shares default/builds snapshots automatic (uncommitted)> commit clownfish:shares default/builds snapshots automatic> list NAME FREQUENCY DAY HH:MM KEEP automatic-000 day - 14:30 7 clownfish:shares default/builds snapshots automatic> done clownfish:shares default/builds snapshots> BUI UI "Shares -> Projects" Name Size Shares 325

326 ,, ( ). 0.. Available space shares. Referenced data. shares. Snapshot data. shares. Unused Reservation Sun ZFS Storage E

327 Unused Reservation of shares. shares. Total space,,... Compression ratio shares.. "Add...".. CLI CLI shares. select. clownfish:> shares clownfish:shares> select default clownfish:shares default> get aclinherit = restricted aclmode = discard atime = true checksum = fletcher4 compression = off compressratio = 100 copies = 1 creation = Thu Oct :30:55 GMT+0000 (UTC) mountpoint = /export quota = 0 readonly = false 5 Shares 327

328 recordsize = 128K reservation = 0 secondarycache = all nbmand = false sharesmb = off sharenfs = on snapdir = hidden vscan = false sharedav = off shareftp = off default_group = other default_permissions = 700 default_sparse = false default_user = nobody default_volblocksize = 8K default_volsize = 0 space_data = 43.9K space_unused_res = 0 space_unused_res_shares = 0 space_snapshots = 0 space_available = 12.0T space_total = 43.9K clownfish:shares default> project.. clownfish:shares> project home clownfish:shares home (uncommitted)> get mountpoint = /export (default) quota = 0 (default) reservation = 0 (default) sharesmb = off (default) sharenfs = on (default) sharedav = off (default) shareftp = off (default) default_group = other (default) default_permissions = 700 (default) default_sparse = true (default) default_user = nobody (default) default_volblocksize = 8K (default) default_volsize = 0 (default) aclinherit = (default) aclmode = (default) atime = (default) checksum = (default) compression = (default) copies = (default) readonly = (default) recordsize = (default) secondarycache = (default) nbmand = (default) snapdir = (default) vscan = (default) custom:contact = (default) 328 Sun ZFS Storage E

329 custom:department = (default) clownfish:shares home (uncommitted)> set sharenfs=off sharenfs = off (uncommitted) clownfish:shares home (uncommitted)> commit clownfish:shares> destroy. clownfish:shares> destroy home This will destroy all data in "home"! Are you sure? (Y/N) clownfish:shares>. rename. clownfish:shares> rename default home clownfish:shares> /. CLI. set. clownfish:shares (pool-0)> set pool=pool-1 clownfish:shares (pool-1)> CLI. CLI BUI. CLI get set. unset. CLI BUI BUI aclinherit ACL inheritance behavior aclmode ACL behavior on mode change atime Update access time on read checksum Checksum compression Data compression 5 Shares 329

330 CLI BUI BUI compressratio Compression ratio Static copies Additional replication creation - - dedup Data deduplication default_group creation default Group default_permissions creation default Permissions default_sparse creation default Thin provisioned default_user creation default User default_volblocksize creation default Volume block size default_volsize creation default Volume size mountpoint Mountpoint nbmand Non-blocking mandatory locking quota Quota readonly Read-only recordsize Database record size reservation Reservation secondary cache Cache device usage sharedav Protocols / HTTP / Share mdoe shareftp Protocols / FTP / Share mode sharenfs Protocols / NFS / Share mode sharesmb Protocols / SMB / Resource name snapdir.zfs/snapshot visibility space_available Available space Usage space_data Referenced data Usage space_snapshots Snapshot data Usage space_total Total space Usage 330 Sun ZFS Storage E

331 CLI BUI BUI space_unused_res Unused reservation Usage space_unused_res_shares Unused reservation of shares Usage vscan Virus scan BUI. CLI. CLI CLI.... Quota LUN. shares.. Reservation LUN. shares Shares 331

332 Mountpoint Read only Update access time on read Non-blocking mandatory locking Data compression Data deduplication Checksum Cache device usage Database record size Additional replication Virus scan. schema.... User Group Permissions LUN LUN. LUN.. Volume size Thin provisioned Volume block size 332 Sun ZFS Storage E

333 . shares... NFS NFS shares. SMB Resource name Enable Access-based Enumeration SMB SMB.. "off" "on" off "on" SMB. SMB. <project's resource name>_<filesystem name>. iscsi iscsi. 5 Shares 333

334 HTTP HTTP shares. FTP FTP shares. ACL. ACL.. ACL behavior on mode change ACL inheritance behavior.. LUN ( ).. ( )..zfs/snapshot visibility. 334 Sun ZFS Storage E

335 BUI... CLI snapshots. clownfish:> shares select default clownfish:shares default> snapshots clownfish:shares default snapshots>... 5 Shares 335

336 Remote Replication :.. Oracle. "Oracle ("SLA") ". Sun Storage 7000,.. ( "")... ( )..... ( ) Sun ZFS Storage E

337 .... NAS SAN.. Shares ZFS.., NFS... Sun Storage 7000 SSL. SSL... ( ): Sun Storage ( ): ().. ( ): ().. ( ): ( ).. ( ):, (, ). :... /. 5 Shares 337

338 ( ):. : ( ). 2. Configuration > Services > Remote Replication.. ( BUI CLI ) ( ) ( ).... ID ( ) SSL.. A B B A. B AB A ( ),..,... ( ) ( ) (, ) 338 Sun ZFS Storage E

339 ( ) ( ). /....BUI CLI ( ) Q Q1.. 5 Shares 339

340 .... IP IP. NAS (NFS SMB). IP. IP. IP.. Shares ( )...,. home bill, cindi dave home. home/bill home/cindi. home/dave. 340 Sun ZFS Storage E

341 .. home, home/bill home/cindi home home/dave...,. Configuration > Services > Remote Replication.BUI Targets. CLI targets hostname, root_password label. knife:> configuration services replication targets. ( ) 5 Shares 341

342 ( )... BUI Replication CLI "replication".. BUI CLI. 342 Sun ZFS Storage E

343 (CLI ) Target Pool Enabled Mode(CLI: continuous) and schedule Include Snapshots Limit bandwidth Use SSL State Last sync.....,.... ( ). SSL..,,... 5 Shares 343

344 (CLI ) Last attempt Next update... ( ), "manual" "continuous". : Manual, Scheduled Continuous,.... ( )... Sun Storage.. "Include Snapshots" true ( )..., BUI CLI sendupdate. (.).. BUI sending. cancelupdate Sun ZFS Storage E

345 ..BUI CLI,,,..,, Q Q1 ( )...,,,.,,. ( : sharenfs, SMB ). NAS,.. ( ).. BUI CLI.,. 5 Shares 345

346 2010.Q Q1. " " (, ). BUI BUI "Replica". Shares.. ( ) Replication ( ) Sun ZFS Storage E

347 BUI.. CLI CLI shares replication sources.. ( ),. : loader:> shares replication sources loader:shares replication sources> show Sources: source-000 ayu PROJECT STATE LAST UPDATE package-000 oldproj idle unknown package-001 aproj1 receiving Sun Feb :04:35 GMT+0000 (UTC) loader:shares replication sources> select source-000 loader:shares replication source-000> select package-001 loader:shares replication source-000 package-001> show Properties: enabled = true state = receiving state_description = Receiving update last_sync = Sun Feb :04:40 GMT+0000 (UTC) last_try = Sun Feb :04:40 GMT+0000 (UTC) Projects: aproj1 loader:shares replication source-000 package-001> select aproj1 loader:shares replication source-000 package-001 aproj1> get mountpoint mountpoint = /export loader:shares replication source-000 package-001 aproj1> get sharenfs sharenfs = on BUI ( ) Replication. ( ). 5 Shares 347

348 . CLI ( ) cancelupdate.... BUI ( ) Replication. (, ). CLI. CLI ( ) enabled... ( ).,. BUI clone CLI ( ). CLI BUI,. 348 Sun ZFS Storage E

349 2009.Q Q1 ( )..,.. (, ).. ( recv-xxx). BUI CLI. NAS. ( )..,. " "......BUI CLI " ". 5 Shares 349

350 .. ( ). LUN.. (, ). ( )... BUI ( ), Replication,.. CLI ( ) sever... ( )....,., 350 Sun ZFS Storage E

351 . /,IP,. ( ), /., /.... "None". DR ( ). " " ,.. 3.., ( )...,.. 5 Shares 351

352 , ( ). BUI ( ), Replication,.. CLI ( ) reverse... ( )... DR.. BUI. CLI shares replication sources destroy...,. perch:> shares perch:shares> replication perch:shares replication> sources perch:shares replication sources> select source-000 perch:shares replication source-000> select package-000 perch:shares replication source-000 package-000> clone perch:shares replication source-000 package-000 clone> set target_project=my_clone target_project = my_clone perch:shares replication source-000 package-000 clone> list CLONE PARAMETERS target_project = my_clone original_mountpoint = /export override_mountpoint = false 352 Sun ZFS Storage E

353 mountpoint = SHARE MOUNTPOINT bob (inherited) myfs1 (inherited) perch:shares replication source-000 package-000 clone> set override_mountpoint=true override_mountpoint = true perch:shares replication source-000 package-000 clone> set mountpoint=/export/my_clone mountpoint = /export/my_clone perch:shares replication source-000 package-000 clone bob> select bob perch:shares replication source-000 package-000 clone bob> set override_mountpoint=true override_mountpoint = true perch:shares replication source-000 package-000 clone bob> set mountpoint=/export/bob mountpoint = /export/bob perch:shares replication source-000 package-000 clone bob> done perch:shares replication source-000 package-000 clone> commit CLONE PARAMETERS target_project = my_clone original_mountpoint = /export override_mountpoint = true mountpoint = /export/my_clone SHARE MOUNTPOINT bob /export/bob (overridden) myfs1 (inherited) Are you sure you want to clone this project? There are no conflicts. perch:shares replication source-000 package-000 clone>. M11,M5. 1.Production System M11 Configuration > SERVICES > Remote Replication. Targets,. Name=M5 Host name= Root password=pppp$ Pool=Pool1. 3. Shares > PROJECTS. Projects. =P1 4. Shares > PROJECTS > P1 > Protocols. 5 Shares 353

354 SMB Resource Name=on. 5. Shares > PROJECTS > P1 > Shares. Filesystems. Name=S1 User=root Group=other Permissions=RWX RWX RWX 6. Shares > PROJECTS > P1 > Shares > S1 > Protocols. SMB \\ \S1 SMB S1. 7. Shares > PROJECTS > P1 > Replication. Actions target pool. Target=M5 Pool=Pool1 Schedule frequency. Frequency= \\ \S1(user=root, password=pppp$1234). 9. F1.txt. 10. Shares > PROJECTS > P1 > Replication. 11. Action TARGET=M5 Update Now. Disable. STATUS., M5 Pool=Pool1. 2. Shares > PROJECTS > REPLICA. s7000-0:p Sun ZFS Storage E

355 3. Shares > PROJECTS > REPLICA > s7000-0:p1 > Replication. Idle. 4. Reverse the direction of replication. New Project Name=P1 5. Shares > PROJECTS > REPLICA. s7000:p1. 6. Shares > PROJECTS > LOCAL. P1. 7. Shares > PROJECTS > P1 > Protocols. SMB Resource Name=on. 8. Shares > PROJECTS > P1 > Shares > S1 > Protocols. SMB \\ \S1 SMB S1. 9. \\ \S1( =root, =pppp$1234). F1.txt F2.txt Shares > PROJECTS > P1 > Replication. 12. Action TARGET=s Update Now. Disable M11 Shares > PROJECTS > LOCAL > P1. P Shares 355

356 2. Shares > PROJECTS > REPLICA > s7000-1:p1 > Replication. Idle. 3. P Reverse the direction of replication. =P1 5. \\ \S1(user=root, password=pppp$1234). F1.txt F2.txt. Remote Replication, Services Remote Replication Projects and Shares 2. rrsource rrtarget,,.,,,. ( : ). Projects and Shares. rrsource Remote Replication Sun ZFS Storage E

357 ( ). ( ). ( ). ZFS Storage 7000 ZFS Storage 7000., 2 IP. IP. CLUSTERED.... IP. IP. IP. ( ) IP ( ). IP Shares 357

358 ( ) ( ). ( ). ( ) ( ). ( ). ( )....,.., ,., (). 4., ( ) Sun ZFS Storage E

359 .. 2. ( : )... ( ). ( ).,.... iscsi Shares. LUN., LUN... LUN. LUN SCSI GUID LUN. LUN LUN SCSI GUID. LUN GUID( LUN GUID) Q Q1. 5 Shares 359

360 .,..... "include snapshots" true.. analytics TCP analytics BUI CLI. " ".. Replication....,..,. 360 Sun ZFS Storage E

361 RPC "2009.Q3 " ,.,, , Shares 361

362 .,.,.... " ". ( ).,. ( )..., Q Q Q Q Q Q3 (2010.Q1 ), 2010.Q Q Q Q Sun ZFS Storage E

363 *, 2009.Q3. BUI CLI. *.,,. Projects and Shares Q Q1...,.,.. BUI "Shares -> Schema".,.. NAME DESCRIPTION TYPE CLI. ".:_\". BUI. CLI... 5 Shares 363

364 BUI CLI String String.. Integer Integer Positive Integer PositiveInteger Boolean Boolean /.BUI, CLI "true" "false". Address Address.. Hostname or IP Host DNS IP(v4 v6).. CLI,.... CLI "shares -> schema". carp:> shares schema carp:shares schema> show Properties: NAME TYPE DESCRIPTION owner Address Owner Contact,. create. carp:shares schema> create department carp:shares schema department (uncommitted)> get type = String description = department carp:shares schema department (uncommitted)> set description="department Code" description = Department Code (uncommitted) carp:shares schema department (uncommitted)> commit carp:shares schema> CLI. 364 Sun ZFS Storage E

365 carp:shares schema> select owner carp:shares schema owner> get type = Address description = Owner Contact carp:shares schema owner> set description="owner Contact " description = Owner Contact (uncommitted) carp:shares schema owner> commit "custom:<property>". carp:shares default> get... custom:department = custom:owner =... carp:shares default> set custom:owner=bob@corp custom:owner = bob@corp (uncommitted) carp:shares default> commit BUI 1. "Shares -> Schema". 2. '+'. 3. ("contact"). 4. ("Owner Contact"). 5. (" Address"). 6. "Apply" "Custom Properties" "Owner Contact". CLI 1. (shares schema). 2. "contact" (create contact). 3. (set description="owner Contact"). 4. (set type= address). 5 Shares 365

366 5. (commit) "custom:contact". 366 Sun ZFS Storage E

367 6 6 Analytics Analytics analytics CPU NFSv3 367

368 Sun ZFS Storage Appliance analytics DTrace. Analytics.. - analytics - - Open Worksheets - analytics Saved Worksheets - analytics Datasets - analytics - Analytics Analytics.. DTrace. Analytics. Analytics...,. / NFSv3 / / CPU /, CPU. NFSv3 /, NFS "read". "NFS operations/sec of type read",. 368 Sun ZFS Storage E

369 Analytics. " NFSv3 / "., NFS.(.) "kiowa" NFS. Analytics. "NFSv3 operations/sec of type read for client kiowa broken down by filename". kiowa NFS. kiowa. Analytics.. "NFSv3 / " " NFSv3 / " " NFSv3 / " " kiowa NFSv3 / " Analytics. Analytics. Analytics.,,,,, NFS,,,,, SMB,,, CPU-id, CPU Open Worksheets, Analytics Preferences Analytics 369

370 Datasets. /. ( ). Open Worksheets....., Datasets (0 "on disk" ). 24x7.,,..,,..,.. "Older than",,,., 3 "Older than" "3" "weeks".... BUI.,..,. Open Worksheets, Saved Worksheets. 370 Sun ZFS Storage E

371 Analytics. Analytics... Analytics CPU: Percent utilization * Cache: ARC accesses * Cache: L2ARC I/O bytes Cache: L2ARC accesses Data Movement: NDMP bytes transferred to/from disk Data Movement: NDMP bytes transferred to/from tape Data Movement: Shadow migration bytes Data Movement: Shadow migration ops Data Movement: Shadow migration requests Disk: Disks * Disk: I/O bytes * Disk: I/O operations * Network: Device bytes Network: Interface bytes Protocol: SMB operations Protocol: Fibre Channel bytes Protocol: Fibre Channel operations Protocol: FTP bytes Protocol: HTTP/WebDAV requests Protocol: iscsi bytes Protocol: iscsi operations Protocol: NFSv2 operations Protocol: NFSv3 operations Protocol: NFSv4 operations Protocol: SFTP bytes Protocol: SRP bytes 6 Analytics 371

372 Protocol: SRP operations * Advanced Analytics Preferences Analytics... CPU: CPUs CPU: Kernel spins Cache: ARC adaptive parameter Cache: ARC evicted bytes Cache: ARC size Cache: ARC target size Cache: DNLC accesses Cache: DNLC entries Cache: L2ARC errors Cache: L2ARC size Data Movement: NDMP file system operations Data Movement: NDMP jobs Disk: Percent utilization Disk: ZFS DMU operations Disk: ZFS logical I/O bytes Disk: ZFS logical I/O operations Memory: Dynamic memory usage Memory: Kernel memory Memory: Kernel memory in use Memory: Kernel memory lost to fragmentation Network: IP bytes Network: IP packets Network: TCP bytes Network: TCP packets System: NSCD backend requests System: NSCD operations,. Datasets Sun ZFS Storage E

373 CPU CPU Cache Cache Cache Cache Cache Cache Data Movement Disk Disk Disk Disk Disk Disk Network Network Network Protocol Protocol Protocol Protocol Protocol Protocol Protocol Protocol Protocol CPU / ARC ARC ARC / DNLC / L2ARC L2ARC NDMP 95% I/O I/O I/O I/O I/O SMB SMB FTP HTTP/WebDAV NFSv2 NFSv2 NFSv3 6 Analytics 373

374 Protocol Protocol Protocol Protocol Protocol Protocol NFSv3 NFSv4 / NFSv4 SFTP iscsi iscsi, " Protocol: NFSv3 operations. OpenWorksheets. : " Protocol: NFSv3 operations".. NFSv3. : " Protocol: NFSv3 operations" Sun ZFS Storage E

375 CPU 8 9 " Protocol: NFSv3 operations".?..,. CPU CPU: Percent utilization CPU. CPU,., 4 - CPU. 16 CPU. CPU. CPU 100%,. 100% CPU. CPU CPU: Percent utilization, NFSv3 2Gbytes/sec. 82% 2Gbytes/sec.( 81%. 1%.) CPU NFS. Protocol: NFSv3 operations( ). CPU.. CPU CPU. CPU : 6 Analytics 375

376 ARC CPU CPU. CPU. CPU. CPU. ID(PID). CPU. CPU. CPU user kernel user-land. CPU user-land akd. CPU. NFS, SMB. CPU CPU 100%.. Advanced Analytic CPU: CPUs( ). CPU CPU 100%. CPU CPU., I/O. ARC Cache: ARC accesses ARC Adaptive Replacement Cache DRAM. ARC,. 376 Sun ZFS Storage E

377 ARC ARC. ARC. / ARC. /. ARC.. L2ARC ARC L2ARC. L2ARC ARC. LUN ARC. ARC. ARC LUN.. /. / ARC DRAM. ARC DRAM. L2ARC ( ). ARC DRAM... ARC DRAM. L2ARC ( ). 6 Analytics 377

378 ARC / / / ARC.. : ZFS uberblock. I/O,. ARC ARC. ARC.,., ARC ARC. ""., ARC., ARC ARC. ARC "" ( ARC ).. ARC.. 128Kbyte I/O, 8Kbyte I/O.,. : 368 : Sun ZFS Storage E

379 L2ARC 128KByte I/O 23 x 128 = 2944Kbyte, 8Kbyte I/O 368x8=2944 Kbyte. ARC Cache: ARC size ARCDRAM. L2ARC IO Cache: L2ARC I/O bytes L2ARC 2nd Level Adaptive Replacement Cache, SSD. L2ARC. L2ARC.. L2ARC.... L2ARC Cache: L2ARC accesses. Cache: L2ARC accesses L2ARC 2nd Level Adaptive Replacement Cache, SSD. L2ARC. L2ARC L2ARC,. 6 Analytics 379

380 L2ARC L2ARC. / L2ARC. /. L2ARC.. L2ARC L2ARC L2ARC. LUN L2ARC. L2ARC. L2ARC LUN.. L2ARC Advanced Analytic Cache: L2ARC size L2ARC. L2ARC Gbyte. Cache: L2ARC I/O bytes. Advanced Analytic Cache: L2ARC errors L2ARC. Cache: ARC accesses(l2arc ) L2ARC. L2ARC. 380 Sun ZFS Storage E

381 NDMP NDMP Data Movement: NDMP bytes transferred to/from disk NDMP. NDMP.NDMP 0. NDMP.. NDMP.. Data Movement: NDMP bytes transferred to/from tape. NDMP Data Movement: NDMP bytes transferred to/from tape NDMP.NDMP 0. NDMP. 6 Analytics 381

382 . Data Movement: NDMP bytes transferred to/from disk. Data Movement: Shadow migration bytes. (,ACL ).. analytics Data Movement: Shadow migration ops Data Movement: Shadow migration requests. 382 Sun ZFS Storage E

383 Data Movement: Shadow migration ops Shadow Migration.... latency... Data Movement: Shadow migration bytes Data Movement: Shadow migration requests. Data Movement: Shadow migration requests Shadow Migration.. ( ).. 6 Analytics 383

384 .. latency... Data Movement: Shadow migration ops Data Movement: Shadow migration bytes. Disk: Disks.... Y Y.,. IOPS. ( ). 100% 384 Sun ZFS Storage E

385 IO I/O. I/O., 75%.. 100% ( )..,., I/O.. 100%. : 75%. 100%. I/O Disk: I/O operations( ). IOPS,, I/O I/O Disk: I/O operations Disk: I/O bytes. IO,.. 100%. Disk: I/O bytes. I/O I/O Storage RAID. 6 Analytics 385

386 IO, NFSv3 8Kbyte 128Kbyte 256Kbyte.. 8Kbyte NFSv3 128Kbyte. ( )..,. Network: Device bytes - ( ) Disk: ZFS logical I/O bytes - ( ) Disk: I/O bytes ( ) I/O. I/O. 50Mbytes/sec( I/O) 5Mbytes/sec( I/O). JBOD I/O.,... I/O vs I/O, I/O. Disk: Disks( ). Disk: I/O operations / /. 386 Sun ZFS Storage E

387 IO IO Disk: I/O operations I/O( IOPS). I/O I/O Storage RAID.,16 8Kbyte NFSv3 ARC DRAM 128Kbyte RAID ( : 2 ). I/O.,. Protocol: NFSv3 operations - NFSv3 ( ) Disk: ZFS logical I/O operations - I/O( ) Disk: I/O operations I/O( ) I/O. I/O,. IOPS. 400 IOPS ( I/O DRAM ) 110 IOPS ( I/O ).. Disk: I/O operations I/O ( ). I/O ( ), I/O. I/O.. I/O I/O I/O (IOPS). IOPS IOPS ( ) I/O ( ).. Disk: I/O operations Disk: I/O operations 6 Analytics 387

388 JBOD IOPS. I/O I/O IOPS.. latency. I/O I/O I/O. I/O. I/O I/O. I/O. IOPS. /. Disk: Disks( ). Disk: I/O bytes //. Network: Device bytes /. Network. (, IP, TCP, NFS/SMB/ ).. 1Gbps.. 1Gb/s : ~120MB/s 10Gb/s : ~1.16GB/s 388 Sun ZFS Storage E

389 .., NFS ( ). Network. Network: Interface bytes. Network: Interface bytes /. Network. (, IP, TCP, NFS/SMB/ ). Network: Device bytes... LACP. 6 Analytics 389

390 SMB., NFS ( ). Network. SMB Network: Device bytes. Protocol: SMB operations SMB / (SMB IOPS). SMB I/O,. Protocol: NFSv3 operations. SMB / SMB Dashboard. SMB. I/O. SMB, CPU., CPU Sun ZFS Storage E

391 SMB.. SMB (//readx/writex/...) SMB IP. latency SMB I/O. "<unknown>". SMB I/O. SMB I/O. SMB SMB I/O. SMB I/O. SMB I/O. SMB I/O. SMB IOPS. I/O RAID SMB IOPSIOPS.. : " Protocol: SMB operations/ "( ) " '/export/fs4/10ga' Protocol: SMB operations/ "( ) " 'phobos.sf.fishpong.com' Protocol: SMB operations/ "( ) SMB Network: Device bytes, SMB / Cache: ARC accesses, I/O Disk: I/O operations. 6 Analytics 391

392 Protocol: Fibre Channel bytes Fibre Channel bytes/. Protocol: iscsi bytes. / FC. FC Protocol: Fibre Channel operations. LUN SCSI FC FC LUN SAN. FC Protocol: Fibre Channel operations,fc / Cache: ARC accesses, I/O Disk: I/O operations. 392 Sun ZFS Storage E

393 Protocol: Fibre Channel operations / (FC IOPS). FC I/O,,. Protocol: iscsi operations. / FC Dashboard. FC. I/O.FC, LUN, CPU., CPU.., LUN. LUN SCSI FC FC LUN FC. SCSI FC I/O. FC SCSI. I/O (read/write/sync-cache/...). 6 Analytics 393

394 FTP latency FC FC I/O. FC I/O. FC I/O. FC IOPS. I/O LUN RAID FC IOPSIOPS. FC I/O.. : " Protocol: Fibre Channel operations/ "(SCSI ) FC I/O Protocol: Fibre Channel bytes,fc / Cache: ARC accesses, I/O Disk: I/O operations. FTP Protocol: FTP bytes FTP /. FTP,. FTP FTP / FTP Dashboard..,. 394 Sun ZFS Storage E

395 HTTPWebDAV. FTP. FTP (get/put/...) FTP. "<unknown>". FTP. FTP. FTP IP.. : " 'phobos.sf.fishpong.com' Protocol: FTP bytes/ "( ) FTP / Cache: ARC accesses, I/O Disk: I/O operations. HTTPWebDAV Protocol: HTTP/WebDAV requests HTTP HTTP/WebDAV /. HTTP,. HTTP/WebDAV / HTTP Dashboard. 6 Analytics 395

396 HTTPWebDAV HTTP.. HTTP HTTP,, CPU., CPU..,. HTTP (get/post). HTTP (200/404/...). IP latency HTTP. HTTP HTTP. HTTP I/O. HTTP.. : ": get HTTP/WebDAV / "(HTTP GET ) " '404' Protocol: HTTP/WebDAV requests/ "( ) " 'deimos.sf.fishpong.com' Protocol: HTTP/WebDAV requests/ "( ) HTTP Network: Device bytes, HTTP / Cache: ARC accesses I/O Disk: I/O operations. 396 Sun ZFS Storage E

397 iscsi iscsi Protocol: iscsi bytes iscsi /. iscsi / iscsi. iscsi Protocol: iscsi operations. LUN iscsi SCSI iscsi iscsi LUN iscsi IP SAN. iscsi iscsi Protocol: iscsi operations iscsi / Cache: ARC accesses, I/O Disk: I/O operations. Protocol: iscsi operations iscsi / (iscsi IOPS). iscsi I/O,,. 6 Analytics 397

398 iscsi iscsi / iscsi Dashboard. iscsi. I/O. iscsi, LUN, CPU., CPU.., LUN. LUN iscsi SCSI iscsi iscsi LUN iscsi. SCSI iscsi I/O. latency iscsi SCSI. I/O (read/write/sync-cache/...). iscsi iscsi I/O. iscsi I/O. iscsi I/O. iscsi IOPS. I/O LUN RAID iscsi IOPSIOPS. iscsi I/O.. : " Protocol: iscsi operations/ "(SCSI ) 398 Sun ZFS Storage E

399 NFSv2 iscsi I/O Protocol: iscsi bytes, iscsi / Cache: ARC accesses, I/O Disk: I/O operations. NFSv2 Protocol: NFSv2 operations NFSv2 / (NFS IOPS). NFS I/O,. Protocol: NFSv3 operations. NFS / NFS Dashboard. NFS. I/O. NFS, CPU., CPU Analytics 399

400 NFSv2 NFS (read/write/getattr/setattr/lookup/...). NFS IP. latency NFS I/O. NFS. "<unknown>". NFS I/O. NFS I/O. NFS NFS I/O. NFS I/O. NFS I/O. NFS I/O. NFS IOPS. I/O RAID NFS IOPSIOPS.. : " Protocol: NFSv2 operations/ "( ) " '/export/fs4/10ga' Protocol: NFSv2 operations/ "( ) " 'phobos.sf.fishpong.com' Protocol: NFSv2 operations/ "( ) NFS Network: Device bytes, NFS / Cache: ARC accesses, I/O Disk: I/O operations. 400 Sun ZFS Storage E

401 NFSv3 NFSv3 Protocol: NFSv3 operations NFSv3 / (NFS IOPS). NFS I/O,. NFS / NFS Dashboard. NFS. I/O. NFS, CPU., CPU..... NFS (read/write/getattr/setattr/lookup/...). NFS IP. NFS I/O. NFS. "<unknown>". NFS I/O. NFS I/O. 6 Analytics 401

402 NFSv4 latency NFS NFS I/O. NFS I/O. NFS I/O. NFS I/O. NFS IOPS. I/O RAID NFS IOPSIOPS.. : " Protocol: NFSv3 operations/ "( ) " '/export/fs4/10ga' Protocol: NFSv3 operations/ "( ) " 'phobos.sf.fishpong.com' Protocol: NFSv3 operations/ "( ) NFS Network: Device bytes, NFS / Cache: ARC accesses, I/O Disk: I/O operations. NFSv4 Protocol: NFSv4 operations NFSv4 / (NFS IOPS). NFS I/O,. Protocol: NFSv3 operations. NFS / NFS Dashboard. 402 Sun ZFS Storage E

403 NFSv4 NFS. I/O. NFS, CPU., CPU..... NFS (read/write/getattr/setattr/lookup/...). NFS IP. latency NFS I/O. NFS. "<unknown>". NFS I/O. NFS I/O. NFS NFS I/O. NFS I/O. NFS I/O. NFS I/O. NFS IOPS. I/O RAID NFS IOPSIOPS.. : " Protocol: NFSv4 operations/ "( ) 6 Analytics 403

404 SFTP " '/export/fs4/10ga' Protocol: NFSv4 operations/ "( ) " 'phobos.sf.fishpong.com' Protocol: NFSv4 operations/ "( ) NFS Network: Device bytes, NFS / Cache: ARC accesses, I/O Disk: I/O operations. SFTP Protocol: SFTP bytes SFTP /. SFTP,. Protocol: FTP bytes. SFTP / SFTP Dashboard..,.. SFTP. SFTP (get/put/...). 404 Sun ZFS Storage E

405 SRP SFTP. "<unknown>". SFTP. SFTP. SFTP IP.. : " 'phobos.sf.fishpong.com' Protocol: SFTP bytes/ "( ) SFTP / Cache: ARC accesses, I/O Disk: I/O operations. SFTP SSH FTP CPU. CPU CPU: Percent utilization. SRP Protocol: SRP bytes SRP /. Protocol: iscsi bytes. SRP / SRP. SRP Protocol: SRP operations. 6 Analytics 405

406 SRP LUN SRP SCSI SRP SRP LUN SAN. SRP SRP Protocol: SRP operations, SRP / Cache: ARC accesses, I/O Disk: I/O operations. Protocol: SRP operations SRP / (SRP IOPS). SRP I/O,,. Protocol: iscsi operations. SRP / SRP. SRP. I/O. SRP, LUN, CPU., CPU. 406 Sun ZFS Storage E

407 SRP., LUN. LUN SRP SCSI SRP SRP LUN SRP. SCSI SRP I/O. latency SRP SCSI. I/O (read/write/sync-cache/...). SRP SRP I/O. SRP I/O. SRP I/O. SRP IOPS. I/O LUN RAID SRP IOPSIOPS. SRP I/O.. : " Protocol: SRP operations/ "(SCSI ) SRP I/O Protocol: SRP bytes, SRP / Cache: ARC accesses, I/O Disk: I/O operations. 6 Analytics 407

408 CPU CPU CPU CPU CPU: CPUs CPU CPU CPU. CPU CPU: Percent utilization. CPU. CPU. CPU CPU. 100% CPU CPU. Y Y CPU () ( ). CPU ( ). CPU. ( ). CPU. 408 Sun ZFS Storage E

409 ARC CPU CPU: Kernel spins CPU.. CPU, CPU: Percent utilization CPU: CPUs... CPU (mutex/...) CPU (0/1/2/3/...) ARC Cache: ARC adaptive parameter ZFS ARC arc_p. ARC MRU MFU. ZFS ARC. ARC. Cache: ARC accesses ARC. ARC Analytics Cache: ARC size Cache: ARC evicted bytes. 6 Analytics 409

410 ARC ARC Cache: ARC evicted bytes ZFS ARC. L2ARC. ZFS ARC. L2ARC (L2ARC). L2ARC ARC.. L2ARC. ARC Cache: ARC accesses ARC. ARC Analytics Cache: ARC size. L2ARC L2ARC, L2ARC. ARC Cache: ARC size, DRAM ZFS ARC. ZFS ARC. 410 Sun ZFS Storage E

411 ARC ARC. ARC DRAM. ARC. ARC L2ARC DRAM (L2ARC). ARC Cache: ARC accesses ARC. : ARC. ARC : ARC ARC ARC L2ARC. ARC. ZFS. ARC. ARC. L2ARC. L2ARC ARC DRAM "ARC ". ARC Cache: ARC target size ZFS ARC arc_c. ARC. Advanced Analytic Cache: ARC size. ZFS ARC. 6 Analytics 411

412 DNLC ARC. Cache: ARC accesses ARC. ARC Analytics Cache: ARC size Cache: ARC evicted bytes. DNLC Cache: DNLC accesses DNLC( ). inode DNLC... DNLC. Cache: ARC accesses ARC. Analytic Cache: ARC size ARC. / DNLC /. 412 Sun ZFS Storage E

413 L2ARC DNLC Cache: DNLC entries DNLC( ). inode DNLC... DNLC. Cache: ARC accesses ARC. Analytic Cache: ARC size ARC. L2ARC Cache: L2ARC errors L2ARC. L2ARC. : L2ARC. 6 Analytics 413

414 L2ARC L2ARC : io L2ARC L2ARC (DRAM) 1. L2ARC. ZFS ARC.... L2ARC Cache: L2ARC size L2ARC. L2ARC. L2ARC. L2ARC Cache: ARC evicted bytes L2ARC I/O. I/O L2ARC. Cache: L2ARC errors. L2ARC. 414 Sun ZFS Storage E

415 NDMP NDMP Data Movement: NDMP file system operations NDMP /. ZFS.. Analytics Data Movement: NDMP bytes transferred to/from disk Data Movement: NDMP bytes transferred to/from tape. read/write/... NDMP Data Movement: NDMP jobs NDMP. NDMP NDMP. Analytics Data Movement: NDMP bytes transferred to/from disk Data Movement: NDMP bytes transferred to/from tape. : backup/restore 6 Analytics 415

416 Disk: Percent utilization... Disk: Disks % 0%. (Disk: Disks) Advanced. Disk: Disks. ZFS DMU Disk: ZFS DMU operations ZFS DMU(Data Management Unit) /. ZFS. 416 Sun ZFS Storage E

417 ZFS IO, Analytics. DMU DDT(Data Deduplication Table). Data Deduplication. DMU DMU read/write/... ZFS /ZFS /DMU dnode/spa /... ZFS IO Disk: ZFS logical I/O bytes ZFS /. I/O I/O, NFS.. I/O. read/write/... 6 Analytics 417

418 ZFS IO ZFS IO Disk: ZFS logical I/O operations ZFS /. I/O I/O, NFS.. I/O. read/write/... Memory: Dynamic memory usage 1 (DRAM).. :. : 418 Sun ZFS Storage E

419 cache kernel mgmt unused ZFS (ARC)..... Memory: Kernel memory (kmem )... Dashboard DRAM ( : ). Memory: Kernel memory in use Memory: Kernel memory lost to fragmentation. kmem Memory: Kernel memory in use ( ) (kmem ).. 6 Analytics 419

420 . Dashboard DRAM ( : ). Memory: Kernel memory lost to fragmentation. kmem Memory: Kernel memory lost to fragmentation (kmem cache). ( : )... Dashboard DRAM ( : ). Memory: Kernel memory in use. kmem 420 Sun ZFS Storage E

421 IP IP Network: IP bytes /IB IP IP /.. Analytics Network: Device bytes.., Protocol: iscsi bytes.. IP IP : tcp/udp IP Network: IP packets IP /..., Protocol: iscsi operations. 6 Analytics 421

422 TCP IP IP : tcp/udp TCP Network:TCP bytes /IB, IP TCP TCP /.. Analytics Network: Device bytes.., Protocol: iscsi bytes.. IP TCP : http/ssh/215()/... TCP Network:TCP packets TCP /. 422 Sun ZFS Storage E

423 NSCD.., Protocol: iscsi operations. IP TCP : http/ssh/215()/... NSCD System: NSCD backend requests NSCD( ) DNS, NIS.... latency / NSCD (DNS/NIS/...) IP 6 Analytics 423

424 NSCD NSCD System: NSCD operations NSCD( ).. / NSCD. System: NSCD backend requests. latency / / NSCD (DNS/NIS/...) : / OpenWorksheets 424 Sun ZFS Storage E

425 OpenWorksheets Analytics CPU NFSv3 Analytics. Analytics... CPU: percent utilization broken down by CPU identifier - Protocol: NFSv3 operations per second broken down by latency -. Analytics. CPU.. 6 Analytics 425

426 OpenWorksheets ( ). "... broken down by CPU identifier" CPU. ( ).. Shift. CPU "...". "..."... 15:52:26. Zoom Pause... "At 15:52:26:". "Range average:". Y. 100%. Line Graph.. Line Graph.. Line Graph. NFS... ( ). 1ms NFSv3 9ms (y ).. 0. (0-9+ ms ) DRAM. I/O, I/O I/O.,. 426 Sun ZFS Storage E

427 OpenWorksheets. y. Crop Outliers.. /: Shift. Crop Outliers /.. "Show hierarchy"... ( )... "+" "-".. 6 Analytics 427