manualshive.com logo in svg

HP 3PAR StoreServ 7000, Руководство администратора

"HP 3PAR StoreServ 7000 - это надежное и мощное хранилище данных для вашего бизнеса. Вы можете бесплатно загрузить руководство администратора для этого устройства с нашего сайта. Получите максимум от вашего оборудования с помощью нашего практического руководства."

Поделиться
Скачать
background image

Table 20 Edit Rights

(continued)

Description of Right

Right

Set a user's own password.

password_setown

Start the remote copy service.

rcopy_start

Stop the remote copy service.

rcopy_stop

Synchronize remote copy groups.

rcopy_sync

Create a remote copy group.

rcopygroup_create

Remove a remote copy group.

rcopygroup_remove

Set remote copy group properties.

rcopygroup_set

Start remote copy group.

rcopygroup_start

Stop a remote copy group.

rcopygroup_stop

Add remote copy links.

rcopylink_admit

Dismiss remote copy links.

rcopylink_dismiss

Add a target to a remote copy volume.

rcopytarget_admit

Create a remote copy target.

rcopytarget_create

Dismiss a remote copy target from a group.

rcopytarget_dismiss

Remove a remote copy target.

rcopytarget_remove

Set remote copy target properties.

rcopytarget_set

Add a volume to a remote copy volume group.

rcopyvv_admit

Dismiss a virtual volume from a remote copy group.

rcopyvv_dismiss

Create a schedule for tasks.

sched_create

Set a user's own scheduled task properties.

sched_setown

Update snapshot space accounting information.

snapspace_update

Set the public SSH key for a user.

sshkey_set

Create a virtual copy volume.

sv_create

Promote a virtual copy volume.

sv_promote

Remove virtual copy of a volume.

sv_remove

Set properties of a virtual copy of a volume.

sv_set

Create a user.

user_create

Set a user's own properties.

user_setown

Export a virtual volume as a SCSI LUN.

vlun_create

Create a Virtual Volume.

vv_create

Grow a virtual volume.

vv_grow

Tune a virtual volume.

vv_tune

Create physical copy volume.

vvcopy_create

Promote a physical volume back to its base volume.

vvcopy_promote

Remove physical copy of a volume.

vvcopy_remove

Set properties of a physical copy of a volume.

vvcopy_set

Basic Edit Role

187

Содержание 3PAR StoreServ 7000

Страница 1: ... 2 Abstract This manual is for all levels of system and storage administrators This guide provides instructions for installing the HP 3PAR CLI and using the CLI to configure and manage HP 3PAR storage systems HP Part Number QR482 96525 Published September 2013 Edition 1 ...

Страница 2: ... for HP products and services are set forth in the express warranty statements accompanying such products and services Nothing herein should be construed as constituting an additional warranty HP shall not be liable for technical or editorial errors or omissions contained herein Acknowledgements Microsoft and Windows are U S registered trademarks of Microsoft Corporation Java Oracle and Oracle Sol...

Страница 3: ... User s Default Domain 20 Removing a User s Default Domain 20 Setting a User s Current Domain 21 Removing a User s Current Domain 21 Viewing User Connections 21 Removing User Connections 21 Configuring LDAP Connections 21 Active Directory LDAP Configuration with SASL Binding 22 Configuring Connection Parameters 23 Configuring Binding Parameters 24 Configuring Account Location Parameters 24 Configu...

Страница 4: ... Variable on Windows 47 Startup Files 48 Setting the TPDSTARTFILE Environment Variable on Solaris and Linux 48 Setting the TPDSTARTFILE Environment Variable on Windows 48 Comma Separated Values 48 Setting the TPDCSVTABLE Environment Variable on Solaris and Linux 49 Setting the TPDCSVTABLE Environment Variable on Windows 49 Using the csvtable Option 49 Listing Domains 49 Setting the TPDLISTDOM Envi...

Страница 5: ... Creating a Host without Assigning a Path 67 Modifying Hosts 68 Changing a Host Name 68 Adding Fibre Channel Path WWNs 68 Adding iSCSI Path iSCSI Names 68 Removing Fibre Channel Path WWNs 68 Removing iSCSI Path iSCSI Names 69 Configuring iSCSI CHAP Authentication Information 69 Removing iSCSI CHAP Authentication Information 69 Moving Removing and Disconnecting Hosts 69 Removing Host Paths 70 Manag...

Страница 6: ...utonomic Groups 87 Creating Virtual Volume Sets 87 Adding Virtual Volumes to Virtual Volume Sets 87 Modifying Virtual Volume Sets 87 Removing Virtual Volume Sets 87 Freeing Virtual Volume Snapshot Space 88 Setting Expiration Times for Virtual Volumes 88 Setting Retention Times for Virtual Volumes 88 Removing Virtual Volumes 89 Validating and Repairing Virtual Volumes 90 Exporting Virtual Volumes 9...

Страница 7: ...ers 104 Removing Templates 104 9 Monitoring System and Physical Disk Capacity 105 Overview 105 System Capacity 105 Determining Total System Capacity 105 Determining System Capacity by Physical Disk Type 106 Physical Disk Capacity 106 Determining Total Physical Disk Capacity 106 Determining Physical Disk Capacity by Disk Type 107 Determining the Capacity of a Specific Physical Disk 107 Spare Chunkl...

Страница 8: ...mory 122 Viewing Statistics for CPU Usage 122 Viewing Statistical Reports Using the On node System Reporter 123 Viewing Statistical Report for Region I O Density 123 Viewing Statistical Reports for Used Capacity Space Reports 123 Viewing Statistical Reports for Performance 123 Viewing Statistical Histogram Reports for Performance 124 Viewing Histograms 124 Viewing Histograms for Chunklets 124 View...

Страница 9: ... Requirements 147 Virtual Volume Sets 147 Applying Quality of Service Rules 151 Mode of Operation 152 QoS Rule Minimum and Maximum 152 QoS Rule Actions 153 Overlapping QoS rules 153 Minimum QoS Settings 153 QoS on Copied Volumes 153 QoS Metrics 154 Using HP 3PAR Priority Optimization 154 Creating a New QoS Rule 154 Creating a New QoS Rule with the HP 3PAR CLI 154 Creating a New QoS Rule with the H...

Страница 10: ... Volume Space Distribution 174 Tuning and Modifying Virtual Volumes 174 Changing Virtual Volume Layouts 174 Thinly Provisioned Virtual Volumes 175 Fully Provisioned Virtual Volumes 175 Changing Virtual Volume RAID Levels 175 Thinly Provisioned Virtual Volumes 175 Fully Provisioned Virtual Volumes 175 Changing Virtual Volume Fault Tolerance Levels 176 Thinly Provisioned Virtual Volumes 176 Fully Pr...

Страница 11: ...s and Rights 185 3PAR AO Role 185 3PAR RM Role 185 Basic Edit Role 186 Browse Role 188 Create Role 188 Edit Role 189 Service Role 191 Super Role 192 B Mapping Rights and CLI Commands 197 Mapping Rights and CLI Commands 197 Index 201 Contents 1 1 ...

Страница 12: ...for unsecure communications If you intend to keep the HP 3PAR CLI connected to your system for an extended period of time you must configure your firewall to allow the CLI to remain connected when there is no activity Before installing the HP 3PAR CLI on a remote computer verify that the system meets the following requirements 1 GHz or higher processor Minimum of 256 MB RAM 1 GB recommended CD ROM...

Страница 13: ...creen directions The default installation location is C Program Files 3PAR inform_cli_ version Command Line Installation on Solaris and Linux To install the HP 3PAR CLI on Solaris platforms 1 Log in as root 2 Insert the HP 3PAR CLI and SNMP CD into your drive 3 Start the installation using the setup bin file cdrom InFormCLI_and_SNMP solaris linux setup bin 4 Log out and log back in The default ins...

Страница 14: ... Start Programs 3PAR Uninstall HP 3PAR CLI version where version is the current release version for example 3 1 2 Result The Uninstaller window appears 3 Follow the instructions that appear in the Uninstaller window 4 When prompted click Finish to finish removing the CLI from your system On Solaris and Linux To remove the HP 3PAR CLI from a Solaris or Linux system 1 Bring up a terminal window and ...

Страница 15: ...ling a new version Multiple versions of the HP 3PAR CLI can reside on the same client system simultaneously By default the HP 3PAR CLI is installed in the following locations On Windows C Program Files 3PAR inform_cli_ version On Solaris or Linux opt 3PAR inform_cli_ version The version is the current release version for example 3 1 2 Scripting Considerations 15 ...

Страница 16: ...ol over your system assign CLI users roles with the minimum set of rights they need to perform their tasks For procedures to create and remove user accounts see Creating Users page 18 and Removing Users page 20 For information about viewing user roles and rights with the CLI see Viewing User Roles and Rights page 17 To view a list of roles and all of the rights assigned to the roles see Mapping Ro...

Страница 17: ...rage server the SP resets the default password to a randomized value Changing the password prevents the SP from performing monitoring operations If the SP is not being used for monitoring and is only used for maintenance activities the password can be changed When a maintenance activity takes place the password for 3parsvc should be set to a defined value after the maintenance the SP changes the p...

Страница 18: ...mmand where role_name is the name of the role To view a list of roles and all of the rights assigned to each role see Mapping Roles and Rights page 185 To view a list of the rights required to use each CLI command see Mapping Rights and CLI Commands page 197 Creating Users The first user created should be assigned the Super user role by specifying super for the role when issuing the createuser com...

Страница 19: ...haracter of the user name must be alphanumeric domain_name is the name of the domain to which the user will belong The domain name can be up to 31 characters long If you are not using Virtual Domains specify the all domain If you are using virtual domains specify the name of an existing domain in your system NOTE By default users in systems not using virtual domains are in the all domain Users in ...

Страница 20: ...a Domain To remove a user from a domain issue the setuser rmdomain domain_name user_name command where domain_name is the name of the domain from which the specified user is being removed user_name is the name of the user being removed from the specified domain You can specify all in place of a specific domain name to remove the user from all domains Setting a User s Default Domain A user s defaul...

Страница 21: ...and is issued when viewing system objects a Domain column is displayed Objects not belonging to any domain no domain are displayed with a dash under the Domain column Otherwise the domain to which the object belongs appears under the Domain column To view a user s connection to the system issue the showuserconn ID user_name IP_address command where ID is the user s numeric ID user_name is the user...

Страница 22: ...key from being used for authentication when operating in CC mode The user s public key must be stored using the setsshkey CLI command Storing the user s public key allows the user s private key to be validated when entered for login attempts after the first successful LDAP authentication Active Directory LDAP Configuration with SASL Binding To configure your system to use Active Directory with SAS...

Страница 23: ...ibutes for later use Run ldp exe as follows 1 Click Start Run 2 In the Open box enter ldp and click OK 3 The Ldp window opens 4 In the Ldp window click Connection Connect 5 In the Server box enter the Active Directory server s IP address and click OK The root DSE attributes and values are displayed in the right side pane NOTE In ldp exe the Kerberos realm is the portion of the ldapServiceName valu...

Страница 24: ...rresponds to Step 2 through Step 4 Configuring Binding Parameters After you have configured the connection parameters to your LDAP server you must configure the binding authentication parameters for users 1 Issue the setauthparam binding sasl command 2 Issue the setauthparam sasl mechanism SASL_type command where type is specified as PLAIN DIGEST MD5 or GSSAPI For information on binding types see ...

Страница 25: ...sponds to Step 2 through Step 5 The Users group within the 3par group is set as the basis for any user search when authenticating with the LDAP server The values user SAMAccountName and memberOf for the account obj account name attr and memberof attr parameters are typical of Active Directory configurations system1 cli checkpassword 3paruser password attempting authentication and authorization usi...

Страница 26: ...alue command where map param is one of the following super map provides Super user rights within the specified group service map provides Service user rights within the specified group edit map provides Edit user rights within the specified group browse map provides Browse user rights within the specified group create map provides Create user rights within the specified group basic_edit map provid...

Страница 27: ...le match browse mapped to by CN Eng CN Users DC 3par DC com user 3PARuser is authenticated and authorized In the example above 3PARuser is a member of the Software group and is assigned Edit rights within the system Although 3PARuser is also a member of the Eng group the Edit rights associated with the Software group supersede the Browse rights associated with the Eng group The mapping rules set f...

Страница 28: ... exe available as part of the downloadable Windows Support Tools from Microsoft You must know the LDAP server s IP address NOTE If you do not have access to the ldapsearch command use the ldp exe command Issue the ldapsearch command and record the displayed dnsHostName the value used for the ldap server hn parameter this is only necessary when setting CA certificates Run ldp exe as follows 1 Click...

Страница 29: ...cli setauthparam f ldap server hn domaincontroller 3par com The example above corresponds to Step 2 through Step 3 Configuring Binding Parameters After you have configured the connection parameters to your LDAP server you must configure the binding authentication parameters for users 1 Issue the setauthparam binding simple command Example system1 cli setauthparam f binding simple 2 Issue the setau...

Страница 30: ...e Configuring Account Location Parameters To configure the account location parameters 1 If you are unsure of the user s account information select a known user s full name to be entered as the cn value and run the ldapsearch command using the defaultNamingContext value previously displayed in Configuring Connection Parameters page 28 Make a note of the group information displayed in the command s...

Страница 31: ...3PARuser password attempting authentication and authorization using system local data authentication denied unknown username attempting authentication and authorization using LDAP using Kerberos configuration file domain_realm domaincontroller 3par com NTDOM1 3PAR COM realms NTDOM1 3PAR COM kdc 192 168 10 13 temporarily setting name to address mapping domaincontroller 3par com 192 168 10 13 attemp...

Страница 32: ...user rights within the specified group browse map provides Browse user rights within the specified group create map provides Create user rights within the specified group basic_edit map provides Basic Edit user rights within the specified group 3PAR_AO map provides 3PAR AO user rights within the specified group 3PAR_RM map provides 3PAR RM user rights within the specified group map_value is the gr...

Страница 33: ... authenticated and authorized In the example above 3PARuser is found to be a member of the Software group and is assigned Edit rights within the system Although 3PARuser is also a member of the Eng group the Edit rights associated with the Software group supersede the Browse rights associated with the Eng group The mapping rules set for 3PARuser are applied to all members of the Software group and...

Страница 34: ... server s IP address 2 Issue the setauthparam ldap ssl 1 command Example system1 cli setauthparam f ldap server 192 168 10 13 system1 cli setauthparam f ldap ssl 1 In the example above The OpenLDAP server s IP address is 192 168 10 13 SSL security is activated using the 1 parameter Configuring Binding Parameters After you have configured the connection parameters to your LDAP server you must confi...

Страница 35: ...bject group_object_class command 3 Issue the setauthparam group name attr attribute command where attribute is the group object s attribute that holds the group name 4 Issue the setauthparam member attr member_attribute command where member_attribute is attribute that holds the names of the users in the group Example system1 cli setauthparam f groups dn ou groups dc ldaptest dc 3par dc com In the ...

Страница 36: ...eate map provides Create user rights within the specified group basic_edit map provides Basic Edit user rights within the specified group 3PAR_AO map provides 3PAR AO user rights within the specified group 3PAR_RM map provides 3PAR RM user rights within the specified group map_value is the group to which the user has membership You can specify multiple groups with multiple map_value arguments 2 Re...

Страница 37: ...d 3paruser ou people dc ldaptest dc 3par dc com search result gidNumber 2345 searching LDAP using search base ou groups dc ldaptest dc 3par dc com filter objectClass posixGroup gidNumber 2345 memberUid 3paruser for attributes cn search result DN cn software ou groups dc ldaptest dc 3par dc com search result cn software search result DN cn engineering ou groups dc ldaptest dc 3par dc com search res...

Страница 38: ...igure binding authentication parameters using the following commands setauthparam binding sasl setauthparam sasl mechanism SASL_type Configure account location parameters using the following commands setauthparam accounts dn DN_path setauthparam account obj user setauthparam account name attr sAMAccount setauthparam memberof attr memberOf checkpassword user_name Configure group to role mapping par...

Страница 39: ... 3par DC com rule match edit mapped to by CN Software CN Users DC 3par DC com mapping rule browse mapped to by CN Eng CN Users DC 3par DC com rule match browse mapped to by CN Eng CN Users DC 3par DC com searching LDAP using search base CN Software Group CN Users DC 3par DC com filter objectClass group for attributes name search result DN CN Software Group CN Users DC 3par DC com search result nam...

Страница 40: ...ers DC 3par DC com search result memberOf CN Eng CN Users DC 3par DC com mapping rule edit mapped to by CN Software CN Users DC 3par DC com rule match edit mapped to by CN Software CN Users DC 3par DC com mapping rule browse mapped to by CN Eng CN Users DC 3par DC com rule match browse mapped to by CN Eng CN Users DC 3par DC com searching LDAP using search base CN Software CN Users DC 3par DC com ...

Страница 41: ...nt Variables Global options are provided to help configure the CLI and control the operation of the system Environment variables are provided to customize the CLI The global CLI options and environment variables are listed in Global CLI Options and Environment Variables page 42 You can use the global options with the CLI program and individual CLI commands When used with the CLI program global opt...

Страница 42: ... secure Using encrypted passwords in scripts using an encrypted pwfile or the encrypted password on the command line raises the risk of password exposure It is imperative that the script pwfile or command line history file be strictly maintained to avoid compromise of the encrypted password Failure to do so can leave the user open to impersonation by anyone with access to the encrypted string that...

Страница 43: ... topic allows you to search help search topic command help by topic Use this command when running in a system s native shell Lists all commands or details for the specified topic Allows you to search command help by topic clihelp search topic Displays help text for individual CLI commands Lists all help text for the specified command Use this command when running in a system s native shell help co...

Страница 44: ...ment variable in Windows 1 On the taskbar click the Start Settings Control Panel 2 In Control Panel double click the System icon 3 In the System Properties dialog box click the Advanced tab 4 Click Environment Variables 5 Enter TPDSOCKSSL in the Variable box 6 Enter 1 in the Value box 7 Click OK Using the sockssl Option A secure CLI connection is also available for single instances of a command On...

Страница 45: ...your system The environment variable remains in effect for that window until you exit that window To set the environment variable in Windows indefinitely and for all newly created Command Prompt windows 1 Perform Step 1 through Step 4 of the procedure for setting the TPDSOCKSSL environment variable in Windows as described in Using SSL page 44 2 In the Environment Variable dialog box enter TPDSYSNA...

Страница 46: ... six character minimum password separated by a space For example 3paruser PPTpdJPDHuwnU Additionally you can also store your password using the setpassword command Storing your password is useful when you need to Establish sessions when using the individual commands Otherwise you are prompted for user and password information each time you issue a command Allow unattended operations such as schedu...

Страница 47: ...e the createuser e command to enter the encrypted password createuser e user1 testdomain edit User created Caching Client Bytecode Client bytecode contains the majority of functionality of the HP 3PAR CLI and is version specific The CLI caches the code required for communication for each different version of the HP 3PAR OS running on the system The TPDCACHEDIR environment variable controls the loc...

Страница 48: ...edure for setting the TPDSOCKSSL environment variable in Windows as described in Using SSL page 44 2 In the Environment Variable dialog box enter TPDSTARTFILE in the Variable box 3 Enter the startup file name in the Value box 4 Click OK Comma Separated Values HP 3PAR CLI output can be configured to display comma separated values so that data can be easily input into spreadsheets To set the output ...

Страница 49: ...STDOM environment variable The setclienv listdom command if changing within the Tcl shell for details about this command see the HP 3PAR Command Line Interface Reference The following example displays CLI output with and without the domain column CLI showvv listdom Rsvd MB MB Id Name Domain Prov Type CopyOf BsId Rd Detailed_State Adm Snp Usr VSize_MB 0 admin Dom001 full base 0 RW normal 0 0 10240 ...

Страница 50: ...d environment variables including no indicate that headings and totals are not displayed To set the output display use The nohdtot or hafter global options The TPDNOHDTOT or TPDHAFTER environment variables The setclienv nohdtot or setclienv hafter commands if changing within the Tcl shell for details about this command see the HP 3PAR Command Line Interface Reference The following example displays...

Страница 51: ...ng you to bypass the confirmation and force the execution of the command The TPDFORCE environment variable automatically provides the functionality of the f option and alleviates the need of having to specify the f option when issuing commands that would otherwise require a confirmation The TPDFORCE environment variable is set differently in Solaris and Linux and Windows Setting the TPDFORCE Envir...

Страница 52: ... you to write Tcl procedures or source Tcl scripts that build on top of CLI commands NOTE Help is available to display information about CLI commands If you started the CLI with the cli command you can obtain help by typing either help or clihelp When you run individual CLI commands from a system s native shell use clihelp For details about HP 3PAR CLI commands see also the HP 3PAR Command Line In...

Страница 53: ...td sys sync man proxy list New Users When creating a new user specify a new user name that is not on the reserved name list Existing Users To delete a user issue the removeuser command cli removeuser pw1 User removed In the previous example a user with the reserved user name of pw1 is removed To create a new user issue the createuser command cli createuser ROOT1 all edit In the previous example ne...

Страница 54: ...ion Tools These keys can be installed with the administration tools or can be obtained directly from the HP 3PAR Administration Tools CD Consult the SSH client documentation for instructions on creating a key fingerprint from the public key 3 When prompted enter the default password If you do not have the password contact your service provider 4 After you enter the password the CLI prompt appears ...

Страница 55: ...SH 1 Create a CLI user and password using the createuser command ssh user1 system1 3paradm s password system1 cli createuser c testpw3 user3 all browse User created In the previous example user user1 logs into system system1 and creates user user3 with the password testpw3 2 Create a key pair using the ssh keygen utility Both rsa and dsa key formats are supported by the system and the recommended ...

Страница 56: ...ient refuses the key file 3 Log in to a system from any system with the SSH client installed ssh user3 system1 user3 system1 s password testpw3 4 Issue the setsshkey command system1 cli setsshkey setsshkey Please enter the SSH public key below When finished press enter twice The key is usually long It s better to copy it from inside an editor and paste it here Please make sure there is no extra bl...

Страница 57: ... running your script script_name in the following example ssh agent echo Agent pid 24216 ssh add Enter passphrase for home usr user3 ssh id_dsa Identity added home usr user3 ssh id_dsa home usr user3 ssh id_dsa script_name NOTE You can redirect the standard input or stdin to execute multiple CLI commands SSH 57 ...

Страница 58: ...ge 58 The user s default domain can be set and reset at any time by the administrator In the following example the default domain Domain A can be reassigned to Domain B for the current working session using the setclienv currentdomain name command When the currentdomain name is set you are able to view and work on objects in Domain B for a single CLI session When that session is over your next ses...

Страница 59: ...Name To change a domain s name issue the setdomain name name domain_name command where name is the new name you wish to assign the domain domain_name is the current name of the domain you are renaming Once changed users and comments previously belonging to the old domain name are displayed as belonging to the new domain name All previous associations objects and settings such as default settings a...

Страница 60: ...ct VV CPG host respectively from which you are removing the domain association object_name is the name of the VV CPG or host from which you are removing the domain association Managing Virtual Domain Autonomic Groups Virtual domains can be combined into autonomic groups A autonomic groups are sets of objects that can be managed as one object If you have a group of domains that require the same adm...

Страница 61: ...command where set_name is the name of the domain set that will include the added domains domain_names are the names of the domains being added to the domain set For a complete list of options available for the createdomainset command see the HP 3PAR Command Line Interface Reference and the HP 3PAR CLI Help Modifying Virtual Domain Sets To change the name of a domain set issue the setdomainset name...

Страница 62: ...To view domain sets containing domains matching a pattern issue the showdomainset domain pattern command 62 Managing HP 3PAR Virtual Domains ...

Страница 63: ...tem and reduces management time related to complex host connections For more information about ports hosts and host personas see the HP 3PAR StoreServ Storage Concepts Guide NOTE See the HP 3PAR Implementation Guides for recommended practices and detailed configuration information about using your specific host devices with the system Modifying Port Parameters Use the HP 3PAR CLI controlport comma...

Страница 64: ...nect the cable to the port on the system Set the port data rate in Gbps with the rate sub command followed by 1 2 or 4 and the port location in node slot port format Alternatively you can use the controlport rate auto command followed by the port location in node slot port format to allow the system to determine the data rate automatically The default value for this option is auto To set the port ...

Страница 65: ... Issue the controliscsiport gw gw_address node slot port command To set the TCP port number for the iSNS server use the isnsport sub command followed by the iSNS server port number then the port location in node slot port format To set the TCP port number for the iSNS server issue the controliscsiport isnsport isns_port node slot port command NOTE Other options are available for the controliscsipo...

Страница 66: ...because the host is offline When a host on a system port becomes inactive for any reason the following happens 1 The HP 3PAR OS recognizes that the host is missing on the port and changes the state of the host from active to inactive 2 The HP 3PAR OS remembers all volumes exported to the host before it became inactive It will not unexport the volumes on the port with the missing host 3 If and when...

Страница 67: ...d configuration information about using specific host devices with the system Creating a Host with a Fibre Channel Path To create a host with an FC path 1 Determine the HBA s WWN by issuing the showhost command 2 Issue the createhost persona persona_number host_name WWN command where persona_number is the host persona number assigned to the host host_name is the user designated name of the host be...

Страница 68: ... host_name command where new_host_name is the new name you wish to assign to the existing host host_name is the original name of the host you are modifying Adding Fibre Channel Path WWNs To add a WWN to an existing host 1 Determine the HBA s WWN by issuing the showhost command 2 Issue the createhost add host_name WWN command where host_name is the name of the host being modified WWN is the HBA s W...

Страница 69: ...mal number See the HP 3PAR Command Line Interface Reference for further information Removing iSCSI CHAP Authentication Information To remove CHAP authentication information from a host with an iSCSI path issue one of the following commands sethost removechap target host_name sethost removechap host_name For both commands listed above host_name is the name of the host you are modifying The target a...

Страница 70: ...emove node slot port WWN_or_iSCSI_name command where node slot port is the old port location WWN is the WWN path or paths to the host to be removed iscsi_name is the iSCSI path or paths to the host to be removed 3 Disconnect the host from the system and then connect it to the new port The system discovers the host on the new port automatically by matching the WWNs or iSCSI Names of the inactive ho...

Страница 71: ... range of the host set the host cannot see the LUN and must be assigned a new ID For more information about VLUN templates see Exporting Virtual Volumes page 90 To create a host set issue the createhostset set_name host_names command where set_name is the name of the host set being created host_names are the names of the hosts included in the host set For a complete list of options available for t...

Страница 72: ...nistrator s Manual or the HP 3PAR Management Console Online Help for instructions on displaying creating modifying and removing host personas Different host personas have different functions and support different host operating systems The specific host persona is designated by the host persona number Depending on the selected host persona number the following additional capabilities are supported...

Страница 73: ...es are presented to the system Without the Host Explorer agents running on the attached hosts the system is unable to determine which host the WWN or iSCSI names belongs to and you must manually assign each WWN or iSCSI name to a host With Host Explorer agents running the system automatically groups WWNs or iSCSI names for the host together assisting with creating the host The Host Explorer agent ...

Страница 74: ...ding iSCSI Path iSCSI Names page 68 Removing Fibre Channel Path WWNs page 68 Removing iSCSI Path iSCSI Names page 69 By adding or removing iSCSI CHAP authentication information This operation is restricted to users with Super or Edit rights in the all domain see the HP 3PAR StoreServ Storage Concepts Guide for details about this restriction See the following for instructions Configuring iSCSI CHAP...

Страница 75: ...configuration must support NPIV FC adapters on hosts used in this configuration do not require NPIV support Port failover is only supported on target mode point to point port configurations Persistent port failover is only supported on the following NPIV compliant HBAs HP 3PAR 4 port 4G FC Emulex 2 port 4G Emulex 4 port 8G FC QLogic CNA in FCoE mode QLogic 1G iSCSI QLogic CNA in 10G iSCSI mode Use...

Страница 76: ...020002AC0000AA disk FC 1 0 3 target loss_sync 2FF70002AC0000AA 21030002AC0000AA free FC 0 0 3 none 1 0 4 initiator ready 2FF70002AC0000AA 21040002AC0000AA host FC 1 3 1 target loss_sync 2FF70002AC0000AA 21310002AC0000AA free FC 0 3 1 none 1 3 2 initiator loss_sync 2FF70002AC0000AA 21320002AC0000AA free FC 1 5 1 initiator ready 2FF70002AC0000AA 21510002AC0000AA disk FC 1 5 2 target ready 2FF70002AC...

Страница 77: ...es which persistent port WWN each device is associated with For example in normal mode there is only a single native WWN hosted by the physical port root snodeb40 S170 showportdev all 1 5 1 PtId LpID Hadr Node_WWN Port_WWN ftrs svpm bbct flen vp_WWN 0x10300 0x01 0x00 2FF70002AC0000AA 21510002AC0000AA 0x8800 0x0012 n a 0x0800 21510002AC0000AA 0x10b00 0x06 n a 200000E08B041B50 210000E08B041B50 0x000...

Страница 78: ...n Provisioning Software license Creating virtual copies or snapshots requires the HP 3PAR Virtual Copy Software license Contact your HP representative for more information Common Provisioning Groups By default a CPG is configured to auto grow new LDs when the amount of available LD space falls below a configured threshold CPGs are initially empty The initial buffer pool of LDs starts off at a frac...

Страница 79: ...effectively disables these safety features See the HP 3PAR Messages and Operator s Guide for additional information about specific alerts Managing alerts is described in Managing Events and Alerts page 118 System Guidelines for Creating Common Provisioning Groups When you use the advanced parameters for LD creation use the following guidelines to ensure maximum performance and optimal reliability ...

Страница 80: ... you want to use RAID 5 for NL device types you must first change the setsys AllowR5OnNLDrives value to yes the default is no For a complete list of options available for the createcpg command see the HP 3PAR Command Line Interface Reference and the HP 3PAR CLI Help Modifying a Common Provisioning Group The following modifications are commonly performed on CPGs using variations of the setcpg comma...

Страница 81: ...irtual Volume Types There are three types of virtual volumes FPVVs TPVVs Administrative Volumes Administrative volumes are created by the system and are for system usage only CAUTION It is strongly recommended that you do not tamper with the admin volumes Virtual volumes have three separate data components User space is the area of the volume that corresponds to the LD regions in the CPG available...

Страница 82: ...on about fully provisioned virtual volumes see the HP 3PAR StoreServ Storage Concepts Guide NOTE Some fully provisioned virtual volumes may be designated as a CPVV in the HP 3PAR CLI Thinly Provisioned Virtual Volumes With an HP 3PAR Thin Provisioning Software license you can also create TPVVs A TPVV uses LDs that belong to an LD pool known as a CPG TPVVs associated with the same CPG draw user spa...

Страница 83: ...of user space that is allocated for user data The virtual volume s snapshot space is allocated on demand To create a group of fully provisioned virtual volumes issue the createvv cnt number snp_cpg CPG_name user_CPG VV_name VV_size g G t T command where number is the number of virtual volumes to create with the designated properties CPG_name is the name of the CPG from which the created volume wil...

Страница 84: ... modified For a complete list of options available for the setvv command see the HP 3PAR Command Line Interface Reference and the HP 3PAR CLI Help Growing Virtual Volumes You can increase the size of or grow an existing virtual volume s user space automatically when the volume reaches capacity To automatically increase the user space available to a virtual volume when the volume reaches capacity i...

Страница 85: ...e where usr_cpg cpg is the name of the CPG from which the converted virtual volume will draw its user space full indicates that you are converting the existing TPVV to an FPVV VV_name is the name of the virtual volume being converted For a complete list of options available for the tunevv command see the HP 3PAR Command Line Interface Reference and the HP 3PAR CLI Help Reducing Volume Size with HP...

Страница 86: ...e data on the converted destination TPVV you can use the removevv command to remove the base volume converted in Step 4 Reducing Volume Size with HP 3PAR Thin Persistence Software HP 3PAR Thin Persistence Software is an optional feature keeps system TPVVs and read write snapshots of TPVVs small by detecting pages of zeros during data transfers and not allocating space for the zeros in the destinat...

Страница 87: ...s the name of the virtual volume set being created VV_names are the names of the virtual volumes included in the virtual volume set For a complete list of options available for the createvvset command see the HP 3PAR Command Line Interface Reference and the HP 3PAR CLI Help Adding Virtual Volumes to Virtual Volume Sets To add a virtual volume to an existing virtual volume set issue the createvvset...

Страница 88: ...s are not automatically removed so you must use the removevv command or the System Scheduler to remove the expired volumes The expiration time can be set during the volume creation time or can be applied to an existing volume To set the volume expiration time for any volume issue the setvv exp time d D h H VV_name command where exp time d D h H specifies the amount of time from the current time in...

Страница 89: ... D h H CPG_name snp_cpg CPG_name user_CPG VV_name VV_size g G t T command where retain time d D h H specifies that the volume cannot be removed from the system for the specified number of hours or days The value for time must be a positive integer in the range of 1 hour to 43 800 hours 1 825 days The value for time can be specified in days with d D or hours with h H For example 14d specifies 14 da...

Страница 90: ...a gap between the virtual volume set export and any further exports to allow for expansion This example uses the next free VLUN from 100 onwards which this allows you to take advantage of the auto LUN functionality while retaining separation between virtual volume set ranges createvlun VV_name 100 host set host_set Creating VLUN Templates A VLUN template creates an association between a virtual vo...

Страница 91: ... is the name of the virtual volume being exported to the host set_name is the name of the virtual volume set being exported to the host LUN is the LUN to which the virtual volume is to be exported expressed as an integer ranging from 0 through 16383 N S P is the location of the target port used to export the virtual volume host is the name of the host to which the virtual volume is exported host_s...

Страница 92: ... being exported to the host set_name is the name of the virtual volume set being exported to the host LUN is the LUN that the virtual volume is to be exported expressed as an integer ranging from 0 through 16383 N S P is the location of the target port used to export the virtual volume host is the name of the host to which the virtual volume is exported host_set is the name of the host set to whic...

Страница 93: ...n in which you are creating the virtual volume Virtual volumes are created by allocating resources from CPGs within a domain A CPG must exist in the domain in which you wish to create a virtual volume before you can create a domain specific virtual volume For instructions on creating domain CPGs see Moving a Common Provisioning Group to a Domain page 94 For more information about virtual domains s...

Страница 94: ... in systems that do not use virtual domains If you are using virtual domains you can export virtual volumes in your domains The right to perform inter domain exports is determined by the user s role Users with Super and Edit rights who belong to the all domain can export virtual volumes in any domain existing in the system Users with Edit rights who belong to a specified domain can only export vir...

Страница 95: ... volume a virtual copy only records the changes to the original volume This allows an earlier state of the original volume to be recreated by starting with the current state and rolling back all of the changes that have been made since the virtual copy was created You can use the HP 3PAR CLI to perform the following virtual copy tasks Create Promote Modify Remove For additional information about v...

Страница 96: ...riority of the task if your virtual volumes are unavailable and restoring access to the volume is the highest priority However raising the priority of the task may impact host I O performance The priority of the task can also be changed after it has begun with the settask command For more information about changing the priority of a running task see Setting the Priority of a Running Task page 128 ...

Страница 97: ...hysical copies For online physical copies the destination volume is automatically created and can be exported immediately Offline physical copies require a destination volume that must have a user space size at least as large as the user space of the base volume being copied and cannot be exported The maximum number of physical copies that can be created on a system is determined by the system con...

Страница 98: ...e priority of the task may impact host I O performance This option cannot be used with halt option The priority of the task can also be changed after it is running by using the settask command For more information about changing the priority of a running task see Setting the Priority of a Running Task page 128 For a complete list of options available for the createvvcopy command see the HP 3PAR Co...

Страница 99: ...olume The destination volume must be equal to or greater in size than the parent volume must be a writeable base volume and must not be exported as a VLUN For a complete list of options available for the creategroupvvcopy command see the HP 3PAR Command Line Interface Reference and the HP 3PAR CLI Help NOTE If you wish to copy multiple volumes the parent_VV destination_VV argument can be repeated ...

Страница 100: ...g destination volume command where p parent_volume creates a snapshot of the parent volume as identified by parent_volume and copies the snapshot data to the destination volume online specifies that the copy is to be performed online The destination can be immediately exported and is automatically created snp_cpg snap_cpg specifies the snapshot CPG to be associated with the destination volume tpvv...

Страница 101: ...a Virtual Copy page 97 for instructions Physical copies can be Created See Creating an Offline Physical Copy page 98 for instructions Resynchronized See Resynchronizing a Physical Copy page 99 for instructions Promoted See Promoting a Physical Copy page 100 for instructions Moving Snapshots Snapshots can be moved in or out of a domain using the movetodomain command This action is restricted to Sup...

Страница 102: ...overrides of a property users can modify that property s value when applying the template When you do not allow overrides of a property it is not possible to modify the defined value system default or system calculated value for that property when applying the template Creating Templates You can create three types of templates that can then be applied to create LDs virtual volumes and CPGs To crea...

Страница 103: ...th new options at the time of creation However read only parameters in the template cannot be overwritten See the HP 3PAR Command Line Interface Reference for additional information Modifying Templates You can modify a template at any time by adding replacing or removing template parameters regardless of whether the template has been applied to create LDs virtual volumes or CPGs When you edit a te...

Страница 104: ... is the name of the template being modified Removing Templates Removing a template deletes the template from the system but does not delete objects created with that template To remove a template from the system issue the removetemplate template_name command where template_name is the name of the template being removed You can optionally remove templates matching a specific pattern by issuing the ...

Страница 105: ...y To determine system capacity issue the showsys space command The following is an example of the data returned by issuing this command cli showsys space System Capacity MB Total Capacity 557056 Allocated 359424 Volumes 179200 Base Volumes 27648 User 27648 Copy 0 Admin 0 CPGs TPVVs CPVVs 151552 Copy 86016 Used 1024 Unused 84992 Admin 65536 Used 512 Unused 65024 Unmapped 0 System 180224 Internal 11...

Страница 106: ...al speed For an SSD the number is to be treated as a relative performance benchmark that takes into account I O per second bandwidth and the access time Determining Total Physical Disk Capacity To view the total physical disk capacity of the system issue the showpd spacecommand The following is an example of the data returned by issuing this command cli showpd space MB Id CagePos Type State Size V...

Страница 107: ... display the total capacity for all physical disks of a specific type in a system issue the showpd space devtype FC NL SSD command NOTE The output of the showpd space devtype FC NL SSD command is displayed in the same format as the showpd space command output example in Determining Total Physical Disk Capacity page 106 Determining the Capacity of a Specific Physical Disk Physical disk capacity can...

Страница 108: ...ncludes the name and ID of the LD using it as well as the chunklet s position on the LD If the spare chunklet is not in use it is listed as available and the columns relating to the LD show only dashes The following example displays information about two spare chunklets 0 and 1 on physical disk 23 cli showspare PdId Chnk LdName LdCh State Usage Media Sp Cl From To 23 0 normal available valid Y Y 2...

Страница 109: ...t one minute per 1GB chunklet and 20 seconds per 256MB chunklet To see chunklets that are currently in the process of being initialized issue the showpd c command Chunklets that are uninitialized are listed in the Uninit column Recovering Failed RAID Sets Failed RAID sets can prevent chunklet relocation which in turn prevents servicemag operations from completing Use the checkld command to recover...

Страница 110: ...details the showinventory command also displays the following information Part number Serial number Assembly part number Assembly serial number Spare part number Model name Revision 1 10 Monitoring System and Physical Disk Capacity ...

Страница 111: ...er System features such as thin provisioning and dynamic optimization work independently of encryption Each band has a single authentication key that controls access to data on the band In the HP 3PAR StoreServ data encryption implementation the entire disk is in one band Access to data is controlled by setting the authentication key which locks and unlocks the drive The LKM which is part of the H...

Страница 112: ... HITECH Act but is not compliant with the Federal Information Processing Standard FIPS 140 2 Supported Configurations HP 3PAR StoreServ Storage The following HP 3PAR Storage systems support data encryption They can be ordered with an HP 3PAR Data Encryption license and SEDs that provide data encryption HP 3PAR StoreServ 10000 Storage HP 3PAR StoreServ 7450 Storage HP 3PAR StoreServ 7000 Storage NO...

Страница 113: ... and password contents are not For example Time 2013 05 28 13 52 20 PDT Severity Informational Type CLI command executed Message 3parsvc super all 0 8 1 127 0 0 1 9534 controlencryption enable_start password secret Message 3paradm super all 0 8 1 16 94 229 83 9706 controlencryption status_details Message 3paradm super all 0 8 1 16 94 229 83 30353 controlencryption rekey_finish A user with Super au...

Страница 114: ...st be performed from an HP 3PAR OS CLI Client running on a PC or server or through the HP 3PAR MC not from an SSH connection To enable encryption using the HP 3PAR CLI issue the following command controlencryption enable backup filename To enable encryption using the HP 3PAR MC 1 Under Command Actions select Systems 2 In the Encryption drop down menu on the toolbar select Enable Encryption To enab...

Страница 115: ... the event of a recovery action requiring restoration of the key file the correct key file must be available otherwise the data will be lost Showing Data Encryption Status To see the status of data encryption issue the following command controlencryption status Optionally issue the command with the d option to show disks that are failed or not SED capable controlencryption status Licensed Enabled ...

Страница 116: ...admitted into the system For more information about the servicemag command see the HP 3PAR Command Line Interface Reference Upgrading an SED with New Firmware Upgrades to the HP 3PAR OS function as they do for non SEDs except that SED drives have different firmware Typically the upgrade is performed automatically for users To upgrade a SED issue the upgradepd f skiptest a w WWN PD_ID command where...

Страница 117: ...s controlencryption enable Back up the authentication key file Back up the authentication key file controlencryption backup Restore the key file from an external source to the controller nodes in the StoreServ system This is needed only if there is a catastrophic problem and the key files on all nodes are destroyed or corrupted controlencryption restore Show the data encryption status See Showing ...

Страница 118: ...orage Troubleshooting Guide and HP 3PAR Messages and Operator s Guide Monitoring and Managing Alerts Alerts are system events that require your immediate attention An alert may be triggered by a single event a combination of events or by repeated occurrences of the same event The system administrator is responsible for responding to alerts For information about system alerts go to http www hp com ...

Страница 119: ... ssue the setsys RawSpaceAlertSSD threshold command For each command above threshold is an integer from 100 to 100 000 that represents the total available space on the system in gigabytes A value of 0 effectively disables the raw space alert safety feature Any alerts for system thresholds below the user defined thresholds are suppressed Monitoring and Managing the Event Log The event log has infor...

Страница 120: ...tartld command and the system manager will begin reconstructing the missing mirrored data The system automatically relocates the chunklets from the missing physical disks to free space or spare space For a complete list of options available for the showpd and showld commands see the HP 3PAR Command Line Interface Reference and the HP 3PAR CLI Help Preserved Data When the system is holding data in ...

Страница 121: ...isplay only physical disk statistics from a specific physical disk specified by either its ID or WWN specific nodes specific PCI slots specific ports Additional filters can be used to refine the data output For complete details about statpd command options and for sample command data output see the HP 3PAR Command Line Reference NOTE The device speed number does not represent a rotational speed fo...

Страница 122: ...ic virtual volume host combination To view virtual volume statistics issue the statvv command To view remote copy virtual volume statistics issue the statrcvv command By default statistics for virtual volume read and write operations are displayed together You can optionally display statistics for volume reads and writes separately by issuing the statvv rw command You can use additional filters to...

Страница 123: ...ter page 123 in addition to specifying a CPG name or an AO configuration name The report contains the distribution of IOP intensity for the LD region for a CPG or AO configuration For complete details about srrgiodensity command options and for sample command data output see the HP 3PAR Command Line Reference Viewing Statistical Reports for Used Capacity Space Reports To view statistical reports f...

Страница 124: ...iew histogram data for chunklets issue the histch command Several filters are provided to refine the data output to display only data from specific chunklets and LDs Additional filters are provided to customize the way the histogram data is displayed For complete details about histch command options and for sample command data output see the HP 3PAR Command Line Reference Viewing Histograms for Lo...

Страница 125: ...onnected target ports disk connected initiator ports FC ports for Remote Copy Ethernet ports for Remote Copy You can use additional filters to refine the histogram data output and specify how that data is displayed For complete details about histport command options and for sample command data output see the HP 3PAR Command Line Reference Viewing Histograms for VLUNs To view histogram data for VLU...

Страница 126: ...you can display cancel or remove tasks running on the system Tasks are created in one of two ways by a user or script issuing certain HP 3PAR CLI commands such as the tunevv command by the system initiating certain automated or scheduled processes such as the periodic resynchronization of remote copy volumes groups Task IDs Each task is identified by a task ID When a task is started with the HP 3P...

Страница 127: ...und_command task issue the starttask command_name command where command_name is the name of the CLI command to run as a background_command For a complete list of options available for the starttask command see the HP 3PAR Command Line Interface Reference and the HP 3PAR CLI Help Displaying Task Information Once a task is started you can use the task ID to obtain information about the task To see a...

Страница 128: ... Time For done cancelled and failed tasks indicates the time at which the task stopped due to completion cancellation or failure Use the showtask t option to see older tasks For a complete list of options available for the showtask command see the HP 3PAR Command Line Interface Reference and the HP 3PAR CLI Help Setting the Priority of a Running Task You can use the settask command to change the p...

Страница 129: ...o testvol usr 2 0MB Fri Apr 29 11 42 06 PDT 2005 Started region move of 256MB from testvol usr 0 0MB to testvol usr 3 0MB Fri Apr 29 11 42 06 PDT 2005 Started region move of 256MB from testvol usr 1 256MB to testvol usr 2 256MB Fri Apr 29 11 42 06 PDT 2005 Started region move of 256MB from testvol usr 0 256MB to testvol usr 3 256MB Fri Apr 29 11 42 19 PDT 2005 Completed region move of 256MB from t...

Страница 130: ...Cancelled Task Types See Table 1 1 page 130 for a list of all task types and their associated operations The table also includes information about the CLI commands associated with each operation For a complete list of options available for the CLI commands listed in Table 1 1 page 130 see the HP 3PAR Command Line Interface Reference and the HP 3PAR CLI Help Table 1 1 Task Types by Operation and Co...

Страница 131: ...ume s snapshot space to the specified CPG tunevv snp _cpg Change the layout of a virtual volume Dynamic Optimization tune_sd Analyzes and automatically corrects space usage imbalances tunesys Perform space usage based analysis and tuning Dynamic Optimization tune_sys Changes the layout of the user space of an existing virtual volume tunevv Tuning virtual volumes Dynamic Optimization tune_vv Restar...

Страница 132: ...ted by systems using Remote Copy for example when issuing the setrcopytargetreverse or setrcopygrouptarget commands on a primary storage system or a system with primary groups in order to reverse targets Use of Remote Copy requires an HP 3PAR Remote Copy Software license For promote_sv tasks there is only one phase The number of steps in that phase is equal to the number of Gigabytes being copied ...

Страница 133: ...heduler commands See Table 13 page 137 for a complete list of the CLI commands that can be initiated with the System Scheduler commands The scheduled_task task type has only one phase and one step snapspace_accounting The snapspace_accounting task type tracks the progress of the snapshot space usage accounting process and runs the updatesnapspace command The snapshot space usage displayed by showv...

Страница 134: ...itional operations You can restart the tunevv operation by using tunevv restart command This starts a tunevv_restart task You can completely reverse the tunevv operation by using tunevv rollback command This starts a tune_vv_rollback task For tune_vv tasks there are three phases Phase 1 Storing away region move information for future restarts and or rollbacks Phase 2 Performing the actual region m...

Страница 135: ... command see Performance Tuning page 173 NOTE The tune_vv task type and tunevv CLI command require the HP 3PAR Dynamic Optimization Software license tunevv_rollback The tune_vv_rollback task type is used to track tunevv rollback operations performed with the System Tuner tunevv rollback command The tunevv rollback command enables you to reverse any changes that resulted from a previously interrupt...

Страница 136: ...m and perform a consistency check on LDs to verify they match the characteristics of any CPG to which the LD belongs Any LDs that do not match are modified to match the characteristics of the CPG The tune_sys task output displays information about the tuning process including Summary of tunesys command parameters Physical disk analysis Internode balance analysis Intranode balance analysis For more...

Страница 137: ... tasks currently scheduled on the system showsched Create tasks to be initiated by the System Scheduler createsched Modify tasks started by the System Scheduler setsched Remove tasks initiated by the System Scheduler from the system removesched Only the following command operations can be scheduled with the System Scheduler commands Table 13 Commands Initiated by the System Scheduler Description C...

Страница 138: ... and system tasks issue the showsched all command To display specific scheduled tasks issue the showsched schedule_name pattern command where schedule_name is the name of the schedule to display pattern displays all schedules that match the pattern For a complete list of options available for the showsched command see the HP 3PAR Command Line Interface Reference and the HP 3PAR CLI Help Scheduling...

Страница 139: ... the schedule specified by the schedule name schedule_name is the name of the schedule to suspend or resume NOTE You can use the suspend_all and resume_all options in place of the suspend and resume options to suspend or resume all scheduled tasks in the system For a complete list of options available for the setsched command see the HP 3PAR Command Line Interface Reference and the HP 3PAR CLI Hel...

Страница 140: ...mization license Contact your local HP representative for information AO is built on top of the new version of System Reporter SR which also now runs as part of the HP 3PAR OS SR must have been actively gathering data on VV regions for a period of time a minimum of 3 hours but preferable for a longer time including any time periods where the storage system has had peak load usage The startao comma...

Страница 141: ...nt startao option createaocfg option External SR Specify a name for the AO configuration in the createaocfg command and use that name for the startao command as well AOCFG_name AOCFG_name Id The system name is not required because it is the system name itself on which the AO configuration is being created System name t0cpg Tier 0 CPG Name cpgname t1cpg Tier 1 CPG Name cpgname t0cpg Tier 2 CPG Name...

Страница 142: ...m command start time mode Mode specifies one of the following modes mode mode Mode Performance Balanced Cost NOTE The value of the mode is case sensitive Because the AO configuration itself does not execute if the startao command is not run Active there is no option to disable it Instead use setsched to suspend or resume the schedules that run startao on that AO configuration 1 It is recommended t...

Страница 143: ...g limits to limit the size of tier 0 and tier 1 leaving tier 2 unlimited In the following example the tier 0 limits are set to 64 g and the tier 1 limits to 256 g respectively cli setcpg sdgw 64g t0cpgname cli cli setcpg sdgw 256g t1cpgname 3 Create a schedule to run startao on the AO configuration periodically In the following example the AO configuration is set to run every day at 7 15 pm from M...

Страница 144: ...Friday completing by 1 15 am cli createsched startao btsecs 12h etsecs 4h compact trimonly maxrunh 6 AOCFG_name 15 19 1 5 run_AOCFG_name Displaying an Adaptive Optimization Setting To display AO configurations in the system use the showaocfg command cli showaocfg domain domain_name_or_pattern AOCFG_name pattern where domain domain_name_or_pattern shows only AO configurations that are in domains wi...

Страница 145: ...me where t0cpg is the tier 0 CPG tier0_R6_SSD is the tier 0 CPG name t1cpg is the tier 1 CPG tier1_R5_FC is the tier 1 CPG name mode mode specifies one of the following optimization biases for the AO configuration Performance Move more regions towards higher performance tier Balanced Balance between higher performance and lower cost Cost Move more regions towards lower cost tier name AOCFG_name sp...

Страница 146: ... proceed with the command unless the f option is specified This option must be used if the pattern specifier is used This specifier can be repeated to remove multiple AO configurations If this specifier is not used the AOCFG_name specifier must be used f specifies that the command is forced If this option is not used the command requires confirmation before proceeding with its operation AOCFG_name...

Страница 147: ...ion is required on the HP 3PAR StoreServ Storage system The HP 3PAR Priority Optimization has its own license key The license is spindle based available a la carte and as part of the Data Optimization Suite v2 available for HP 3PAR StorServ 7000 and HP 3PAR StorServ 10000 systems Consult your HP representative or authorized HP partner for more information on HP 3PAR Priority Optimization licensing...

Страница 148: ...L 1 MS SQL 2 and MS SQL 3 At creation time each volume member has the same provisioning type user and copy CPG allocation warnings and volume size Each of these settings can be changed afterwards Figure 2 VVset Creation at Time of Volume Creation To ends the operation of creating volumes and their VVset click Finish The end result of VVset creation is shown in Figure 3 page 149 148 HP Priority Opt...

Страница 149: ...Sets entry in the Provisioning tree of the MC and then select Create Virtual Volume set see Figure 4 page 149 Figure 4 Creating a VVset from the Provisioning Tree In the Create Virtual Volume Set dialog box supply the name and optionally the domain and comments for the new VVset see Figure 5 page 150 Virtual Volume Sets 149 ...

Страница 150: ...dding Existing Volumes to a New VVset Select the volumes you want to be a member of VVset Test in this example and click the yellow arrow pointing downward to add them to the VVset then click the Next button to proceed As shown in Figure 7 page 151 the VVset Test now has three member volumes 150 HP Priority Optimization ...

Страница 151: ...vels and sizes VVsets can contain volumes with different provisioning type RAID level size and CPGs All volumes in a VVset can be exported in one step to hosts or host sets by right clicking the name of the VVset in Figure 8 page 151 and then selecting the Export option in the menu that appears Applying Quality of Service Rules HP 3PAR Priority Optimization software introduces quality of service r...

Страница 152: ...e volumes contained in that VVset These delayed I O requests are pushed onto an outstanding I O queue for the VV s in the VVset experiencing the limit breach Every QoS rule maintains its own queue for delayed I Os These queues are constructed inside the each HP 3PAR StoreServ controller node that receives an I O request that needs to be delayed Only the I O request descriptions are queued not the ...

Страница 153: ...it values in a QoS rule activates those values instantly on the VVset there is no need to disable the QoS rule first An active QoS rule can be removed without first disabling it Overlapping QoS rules A VV can be a member of multiple VVsets each of which can have a QoS rule defined In such a case the I O to and from volumes in the VVset is governed by multiple possibly overlapping rules All active ...

Страница 154: ...ity Optimization You can manage HP 3PAR Priority Optimization either through the HP 3PAR MC or through the HP 3PAR CLI This section describes procedures and best practices Creating a New QoS Rule Creating a New QoS Rule with the HP 3PAR CLI To create a new rule for a VVset using the MC select and right click the VVset name in the Virtual Volume Sets tree node in the Provisioning pane A popup menu ...

Страница 155: ...e clear or disable the rule as shown in Figure 1 1 page 155 Figure 1 1 Configuration Options for Existing QoS on a VVset You can change the values and save the rule by clicking OK Changes made to an enabled QoS rule take effect immediately Creating a New QoS Rule with the HP 3PAR MC The Provisioning tree in the HP 3PAR MC for an HP 3PAR StoreServ Storage system that has a QoS license has a new tre...

Страница 156: ...becomes available With the QoS node selected in the Provisioning manager you can also initiate QoS rule configuration from the icons shown in the highlighted portion of Figure 13 page 156 Figure 13 QoS Configuration Icons With the Virtual Volume Sets node selected in the Provisioning tree you can click the QoS icon highlighted in Figure 14 page 157 to configure a particular QoS rule 156 HP Priorit...

Страница 157: ...vailable to all the volumes under this rule HP 3PAR Priority Optimization enforces a minimum value for the I O limit of 1000 IOs sec and 100 000 KBs sec for the bandwidth limit although these limits may be far too low for some environments If lower values are entered a popup message appears showing information about the minimum values and closes the configuration window If the System QoS rule is c...

Страница 158: ... commands and their syntax details are setqos options vvset name pattern sys all_others The seqqos command creates and updates QoS rules in a system showqos options vvset name pattern sys all_others The showqos command lists the QoS rules configured in a system statqos options vvset name pattern sys all_others The statqos command displays run time statistics of active QoS rules For details about c...

Страница 159: ... setqos statement If a VV is in a VVset for which no QoS rule has yet been defined a new rule is created with the limits in the setqos command The built in rule all_others is switched off by the command setqos off sys all_others The showqos and statqos command provide a centralized view of the QoS rules in the systems and how the workloads conform to them Executing the QoS CLI commands requires a ...

Страница 160: ...the Values for IOPS and Bandwidth for a System HP 3PAR Priority Optimization s QoS rules define limits for IOPS and bandwidth in absolute numbers not percentages Because of this the QoS administrator needs reasonably accurate data on the system s maximum I O and throughput capability He also requires an accurate understanding of actual I O and throughput workloads exercise to the volumes that will...

Страница 161: ...tem QoS rule should be enabled and a value specified for its maximum IOPS and bandwidth to control the I O traffic from VVs and VVsets not subject to a specific QoS rule This prevents any new volumes added to the HP 3PAR StoreServ or existing ones that are not subject to a QoS rule from negatively affecting the entire system overconsuming IOPS or bandwidth As explained in Creating a New QoS Rule w...

Страница 162: ...d in another VVset the named QoS rule takes precedence over the System rule even if the named QoS rule has a lower value for IOPS or bandwidth Application Interoperability HP 3PAR Priority Optimization sets and manages QoS rules defined on I O traffic that enters and leaves an HP 3PAR StoreServ meaning on the front end host ports of the array Software products for HP 3PAR StoreServ storage systems...

Страница 163: ...ntrol queue depth of datastore SAN LUN in VMkernel Control queue depth of datastore SAN LUN in VMkernel VM shares enforced Migrate VM to other datastore Limit IOPS and bandwidth I O control technique Queue Full or Device Busy at LUN or port level I O latency I O latency and space utilization None Reacts on All hosts using the SAN LUN for the All VMs in a single datastore Single VM All VMs in datas...

Страница 164: ...rule governs one or more LUNs that make up a datastore Storage DRS implies a source and a destination datastore each possibly subject to a QoS rule The QoS rule if any on the Storage DRS destination datastore should have enough headroom to accommodate the additional I O capacity of the migrated VMDK If the workload s I O characteristics are not very well known HP recommends manual Storage DRS migr...

Страница 165: ...OPS cap set by the user Qt Cur Current IOPS Avg Average IOPS over all iterations of the statqos command so far Max Maximum IOPS over all iterations of the statqos command so far Kbytes_per_second IOPS cap set by the user Qt Cur Current IOPS Avg Average IOPS over all iterations of the statqos command so far Max Maximum IOPS over all iterations of the statqos command so far Svt_ms Total service time...

Страница 166: ...ds srstatqos options mirror the options of other srstat commands Event Management There is no mechanism to alert storage administrators of rejected I O requests Instead monitor the QoS rules as described in Reporting page 164 The debug event logs will show QFULL messages for rejected I O requests if they occur To display the debug event log issue the following HP 3PAR CLI command showeventlog debu...

Страница 167: ...s All alerts generated by the system as well as all alert status change events are translated into alertNotify traps For information about system alerts go to http www hp com support hpgt 3par and select your server platform NOTE If you receive a trap with messageCode 1245186 this is to notify you that an alert has changed state In order to find out which alert has changed state you must extract t...

Страница 168: ...t this command shows the read write community string If the community string you are requesting does not exist you will get an error message To add or update a community string use the setsnmppw command followed by the new community string By default this command changes the read write community string To remove a community string use the removesnmppw command Once the community string is removed t...

Страница 169: ...owsnmpuser user issue the showuser user_name command where user_name is the name of the user NOTE Users with Browse or Service roles are limited to viewing the information for their own accounts For more information about the showsnmpuser command see the HP 3PAR Command Line Interface Reference Removing SNMPv3 Users After an SNMPv3 user is removed with the removesnmpuser command an SNMPv3 manager ...

Страница 170: ...ts are set with the HP 3PAR CLI setuseracl command In order to use this command you must have at least an Edit rights For additional details see Learning About User Accounts page 16 The following example shows how to set administrative rights for user testuser1 to allow testuser1 to update copies of virtual volumes test rw1 and test rw2 cli setuseracl testuser1 updatevv test rw1 test rw2 This comm...

Страница 171: ...est ro can have a VLUN associated with it The command will remember the VLUN remove the old snapshot create a new snapshot and recreate the VLUN with the new snapshot After the snapshot is replaced testuser1 must remount the filesystem on the host Replacing a Read Write Snapshot The following example shows how to replace a read write snapshot volume with a new read write snapshot volume In this ex...

Страница 172: ...CLI command cli updatevv ro test rw1 This command removes test rw1 and test ro1 and replaces them with a new read only snapshot of test called test ro1 and a new read write snapshot of test ro1 called test rw1 This gives the user an up to date read write copy of the original database by creating a new read only snapshot of the original in addition to the new read write copy NOTE In this example on...

Страница 173: ...sks to under used physical disks on a node The threshold diskpct percentage must be between 1 and 100 The default value is 10 This option cannot be used with the cpg option Other options for controlling the tunesys command cpg cpg_name pattern limits the tunesys operations to the specified CPGs The specified CPGs must all be in the same domain as the CLI user The cpg option cannot be used with the...

Страница 174: ...e tunevv command to improve the performance of a system by modifying the virtual volumes to use all available hardware resources You can also change the parameters RAID levels and set sizes for virtual volumes Modifying virtual volumes does not interrupt access to the virtual volumes A good time to use the tunevv command is after changing the hardware configuration of your system When a system is ...

Страница 175: ...tructions 2 Tune the user space of the volume by issuing the tunevv usr_cpg CPG_name volume_name command where usr_cpg CPG_name is the name of the CPG created previously in Step 1 indicating the user space of the new CPG will be used volume_name is the fully provisioned virtual volume you are modifying 3 Tune the snapshot space of the volume by issuing the tunevv snp_cpg CPG_name volume_name comma...

Страница 176: ...t tolerance level of a fully provisioned virtual volume perform the following 1 Create a new CPG with the desired fault tolerance level and RAID type See Creating a Common Provisioning Group page 80 for instructions 2 Change the fully provisioned virtual volume s fault tolerance level by issuing the tunevv usr_cpg CPG_name volume_name command where usr_cpg CPG_name is the name of the CPG created p...

Страница 177: ...es a volume to its state prior to the initiation of the task Essentially issuing a rollback command performs an undo of the operation You may choose to roll back a tunevv task if the operation was interrupted because of Lack of space User initiated cancellation of a task To perform a rollback operation perform the following 1 Issue the canceltask task_ID command where task_ID is the ID of the tune...

Страница 178: ... system performance statistics see Viewing Statistics and Histograms page 121 For more information about how the system manages space usage see the HP 3PAR StoreServ Storage Concepts Guide NOTE Running the tunepd command does not eliminate the need to run the tunesys command The tunesys command performs space based balancing The tunepd command performs service time and I O based balancing Compacti...

Страница 179: ... less efficient in space usage Compacting a CPG allows you to reclaim space from a CPG that has become less efficient in space usage from creating deleting and relocating volumes Compacting consolidates LD space in CPGs into as few LDs as possible compactcpg CPG_name or compactcpg pattern CPG_name is the name of the CPG you wish to compact pattern displays all CPGs that match the pattern See the H...

Страница 180: ...software platforms http www hp com storage spock The HP 3PAR StoreServ Storage site Locating HP 3PAR documents http www hp com go 3par To access HP 3PAR documents click the Support link for your product HP 3PAR storage system software HP 3PAR StoreServ Storage Concepts Guide Storage concepts and terminology HP 3PAR Management Console User s Guide Using the HP 3PAR Management Console GUI to configu...

Страница 181: ... Copy Software User s Guide Using the CLI to configure and manage HP 3PAR Remote Copy HP 3PAR Upgrade Pre Planning Guide Updating HP 3PAR operating systems HP 3PAR F Class T Class and StoreServ 10000 Storage Troubleshooting Guide Identifying storage system components troubleshooting information and detailed alert information HP 3PAR Policy Server Installation and Setup Guide Installing configuring...

Страница 182: ...ice Processor HP 3PAR Service Processor Onsite Customer Care SPOCC User s Guide HP 3PAR host application solutions HP 3PAR Recovery Manager Software for Oracle User s Guide Backing up Oracle databases and using backups for disaster recovery HP 3PAR Recovery Manager Software for Microsoft Exchange 2007 and 2010 User s Guide Backing up Exchange databases and using backups for disaster recovery HP 3P...

Страница 183: ...ible damage to data or to the operating system CAUTION Indicates that failure to follow directions could result in damage to equipment or data NOTE Provides additional information Required Indicates that a procedure must be followed as directed in order to achieve a functional and supported implementation based on testing at HP HP 3PAR branding information The server previously referred to as the ...

Страница 184: ...that meets your needs To help us improve the documentation send any errors suggestions or comments to Documentation Feedback docsfeedback hp com Include the document title and part number version number or the URL when submitting your feedback 184 Documentation feedback ...

Страница 185: ...ht Right Create a virtual volume group groupsv_create Set host properties host_set Remove an LD ld_remove Check a user s own password password_checkown Set a user s own password password_setown Start the remote copy service rcopy_start Stop the remote copy service rcopy_stop Synchronize remote copy groups rcopy_sync Create a remote copy group rcopygroup_create Set remote copy group properties rcop...

Страница 186: ... virtual volume set vvset_create Set virtual volume set properties vvset_set Basic Edit Role Table 20 Edit Rights Description of Right Right Set alert properties alert_set Set LDAP authentication properties authparam_set Consolidate space in a CPG cpg_compact Create CPG cpg_create Create a domain domain_create Move objects in or out of a domain domain_moveto Create a domain set domainset_create Se...

Страница 187: ...perties rcopytarget_set Add a volume to a remote copy volume group rcopyvv_admit Dismiss a virtual volume from a remote copy group rcopyvv_dismiss Create a schedule for tasks sched_create Set a user s own scheduled task properties sched_setown Update snapshot space accounting information snapspace_update Set the public SSH key for a user sshkey_set Create a virtual copy volume sv_create Promote a ...

Страница 188: ... domain set domainset_annotate Create a domain set domainset_create Create a virtual volume group groupsv_create Create a physical copy volume group groupvvcopy_create Check the current health of the system health_check Create a host host_create Annotate the comment of a host set hostset_annotate Create paths to a system host set hostset_create Check a user s own password password_checkown Set a u...

Страница 189: ...roperties host_set Annotate the comment of a host set hostset_annotate Create paths to a system host set hostset_create Set host set properties hostset_set Consolidate space in an LD ld_compact Create an LD ld_create Remove an LD ld_remove Check a user s own password password_checkown Set a user s own password password_setown Move used chunklets from a physical disk to spare space pd_movetospare R...

Страница 190: ...ume sv_remove Set properties of a virtual copy of a volume sv_set Cancel a task task_cancel Remove a task task_remove Edit tasks task_set Run commands in the background task_start Set a user s own properties user_setown Set user Access Control List properties useracl_set Export a virtual volume as a SCSI LUN vlun_create Remove a VLUN vlun_remove Create a Virtual Volume vv_create Grow a virtual vol...

Страница 191: ... and time properties date_set Check the current health of the system health_check Service a host host_service Create an LD ld_create Remove an LD ld_remove Start an LD ld_start Take magazines or disks on or off a loop mag_control Service a magazine or physical disk mag_service Locate a controller node by blinking its LEDs node_locate Service a controller node node_service Set controller node prope...

Страница 192: ...ackground task_start Virtual volume validity checks vv_check Start a virtual volume vv_start Start the Web Services API server wsapi_start Stop the Web Services API server wsapi_stop Super Role Table 25 Super Rights Description of Right Right Remove alerts alert_remove Set alert properties alert_set Start execution of an Adaptive Optimization configuration aocfg_start Set LDAP authentication prope...

Страница 193: ...y_create Check the current health of the system health_check Create a host host_create Remove a host host_remove Service a host host_service Set host properties host_set Annotate the comment of a host set hostset_annotate Create paths to a system host set hostset_create Set host set properties hostset_set Consolidate space in an LD ld_compact Create an LD ld_create Remove an LD ld_remove Start an ...

Страница 194: ...pygroup_set Start remote copy group rcopygroup_start Stop a remote copy group rcopygroup_stop Add remote copy links rcopylink_admit Dismiss remote copy links rcopylink_dismiss Add a target to a remote copy volume rcopytarget_admit Create a remote copy target rcopytarget_create Dismiss a remote copy target from a group rcopytarget_dismiss Remove a remote copy target rcopytarget_remove Set remote co...

Страница 195: ...G or LD template_create Remove template for a VV CPG or LD template_remove Set template properties template_set Create a user user_create Remove a user user_remove Set any user properties user_setany Set a user s own properties user_setown Set user Access Control List properties useracl_set Remove user connections userconn_remove Export a virtual volume as a SCSI LUN vlun_create Remove a VLUN vlun...

Страница 196: ...Remove physical copy of a volume vvcopy_remove Set properties of a physical copy of a volume vvcopy_set Annotate the comment of a virtual volume set vvset_annotate Create a virtual volume set vvset_create Set virtual volume set properties vvset_set Start the Web Services API server wsapi_start Stop the Web Services API server wsapi_stop 196 Mapping Roles and Rights ...

Страница 197: ... cpg_compact createcpg cpg_create setcpg cpg_set setdate date_set createdomain domain_create movetodomain domain_moveto removedomain domain_remove setdomain domain_set setdomainset domainset_annotate createdomainset domainset_create setdomainset removedomainset domainset_set controlencryption encryption_control controlencryption status subcommand encryption_status removeeventlog eventlog_remove cr...

Страница 198: ...password password_checkany checkpassword password_checkown setpassword password_setany setpassword password_setown admitpd pd_admit controlpd pd_control dismisspd pd_dismiss moverelocpd pd_movereloc movepdtospare pd_movetospare setpd pd_set upgradepd pd_upgrade checkport port_check controliscsiport controlport port_control checkrclink rclink_check startrcopy rcopy_start stoprcopy rcopy_stop syncrc...

Страница 199: ...te removesched sched_remove setsched sched_setany setsched sched_setown updatesnapspace snapspace_update addsnmpmgr snmpmgr_add removesnmpmgr snmpmgr_remove removesnmppw snmppw_remove setsnmppw snmppw_set createspare spare_create removesshkey sshkey_remove setsshkey sshkey_set createsv sv_create promotesv sv_promote removevv sv_remove setvv sv_set locatesys sys_locate setsys sys_set shutdownsys sy...

Страница 200: ...v vv_check createaldvv createavv createtpvv createvv vv_create growaldvv growavv growtpvv growvv vv_grow importvv vv_import removevv vv_remove setvv vv_set startvv vv_start tunealdvv tunevv tunetpvv vv_tune updatevv vv_update createvvcopy vvcopy_create promotevvcopy vvcopy_promote removevv vvcopy_remove setvv vvcopy_set setvvset vvset_annotate createvvset vvset_create setvvset removevvset vvset_se...

Страница 201: ...ng virtual volumes resynchronizing physical copy status 99 CPGs common provisioning groups 78 considerations 78 growth increment 78 removing 81 CPU usage statistics viewing 122 createaldvv command 103 createcpg command 80 createhost command 67 createtemplate command 102 createvlun command 90 createvv command 83 createvvcopy command 98 creating a host 67 creating domain CPGs 92 creating domains 58 ...

Страница 202: ...er 1 1 1 log files installation error log 14 logging logical disk allocated at set up 107 logical disk histograms viewing 124 LUN 90 M management consoles registered displayed using showsnmpmgr 168 registering for SNMP 168 unregistering with removesnmpmgr 168 managing domain objects 47 60 matched set 90 modifying domain hosts 74 modifying domain VVs 93 modifying domains 59 modifying templates 103 ...

Страница 203: ...5 specifying 45 Systems using domains LDAP configuration 38 T task removing 130 task information displaying 127 task management commands 126 Tcl shell global option limitations 41 template parameters adding and replacing 103 removing 104 template rules 102 templates applying 102 creating 102 modifying 103 removing 104 viewing parameters 103 templates allowing overrides 102 templates and default va...

Страница 204: ...tistics 122 virtual volume statistics viewing 122 virtual volumes exporting 90 growing 84 resynchronizing physical copy 99 snapshot space 88 unexporting 91 VLUN histograms viewing 125 VLUN template host sees 90 matched set 90 port presents 90 removing 91 types 90 204 Index ...

Отзывы:

Нет отзывов

Похожие инструкции для 3PAR StoreServ 7000

CallDirect CD Series At Command Manual preview
CD Series
Бренд: CallDirect Страницы: 33
NEC HX4500 Configuration Manual preview
HX4500
Бренд: NEC Страницы: 74
NEC Express5800/T120e Quick Start Manual preview
Express5800/T120e
Бренд: NEC Страницы: 6
NEC Express5800/T120e Getting Started preview
Express5800/T120e
Бренд: NEC Страницы: 2
NEC Express5800/T110i Getting Started preview
Express5800/T110i
Бренд: NEC Страницы: 2
NEC EXPRESS5800/R320d-E4 Configuration Manual preview
EXPRESS5800/R320d-E4
Бренд: NEC Страницы: 19
NEC Express5800/R320d-M4 Series User Manual preview
Express5800/R320d-M4 Series
Бренд: NEC Страницы: 98
NEC Express5800/R120h-2M Getting Started preview
Express5800/R120h-2M
Бренд: NEC Страницы: 2
NEC Express5800/R120f-1E Use Precautions preview
Express5800/R120f-1E
Бренд: NEC Страницы: 12
NEC Express5800/R120d-1E Hardware Specification preview
Express5800/R120d-1E
Бренд: NEC Страницы: 2
NEC Express5800/R110i-1 Getting Started preview
Express5800/R110i-1
Бренд: NEC Страницы: 2
NEC Express5800/R110h-1 System Configuration Manual preview
Express5800/R110h-1
Бренд: NEC Страницы: 40
NEC Express5800/A2040d Configuration Manual preview
Express5800/A2040d
Бренд: NEC Страницы: 83
NEC Express5800/320Lc Datasheet preview
Express5800/320Lc
Бренд: NEC Страницы: 4
NEC Express5800/120Rj-2 Datasheet preview
Express5800/120Rj-2
Бренд: NEC Страницы: 2
NEC NEAX2000 IVS2 Manual preview
NEAX2000 IVS2
Бренд: NEC Страницы: 106
NEC Express5800/T120d User Manual preview
Express5800/T120d
Бренд: NEC Страницы: 153
NEC Express5800/T110g-S User Manual preview
Express5800/T110g-S
Бренд: NEC Страницы: 128

Бренды по названию

Популярные бренды

Загрузить еще бренды