background image

Hitachi Compute Blade 2500 Series

Getting Started Guide

MK-99CB2500003-

12

Document Organization

Product Version

Getting Help

Contents

Содержание Compute Blade 2500

Страница 1: ...Hitachi Compute Blade 2500 Series Getting Started Guide MK 99CB2500003 12 Document Organization Product Version Getting Help Contents ...

Страница 2: ...erms and conditions of the applicable Hitachi Data Systems Corporation agreements The use of Hitachi Ltd products is governed by the terms of your agreements with Hitachi Data Systems Corporation Hitachi is a registered trademark of Hitachi Ltd in the United States and other countries Hitachi Data Systems is a registered trademark and service mark of Hitachi Ltd in the United States and other coun...

Страница 3: ...ormation xix Getting help xix Comments xix 1 CB 2500 Overview 1 1 Features of the CB 2500 1 2 Features of the server chassis 1 3 Features of server blades 1 4 Management module overview 1 5 I O board module 1 6 Switch module overview 1 6 Power supply module 1 6 Fan module 1 6 LPAR manager functionality 1 6 N M cold standby 1 7 Reducing server blade power consumption 1 7 Internal LAN and management...

Страница 4: ...5 3 Starting the CB 2500 and Performing Initial Setup 3 1 Connecting the system unit to the power supply 3 2 Preparing the system console used for configuring the settings of the CB 2500 3 2 Setting the Web browser used to display the Web console 3 3 Connecting the system console to the management module 3 3 Management LAN when LPAR manager is used 3 4 Setting the network information of the system...

Страница 5: ... Exiting the remote console 5 14 Screen messages displayed by the remote console 5 15 6 Troubleshooting 6 1 Troubleshooting on the remote console 6 2 Issuing an NMI for the operating system running on the server blade 6 6 A System Unit Specifications A 1 Specifications of the server chassis A 2 Specifications of server blades A 2 Specifications of switch modules A 12 Specifications of the power su...

Страница 6: ...he ventilation holes C 2 D Components that have specified lifetimes D 1 Components that have specified lifetimes D 2 E License E 1 End user license E 2 Glossary Index vi Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 7: ...Hitachi Data Systems products is governed by the terms of your agreement s with Hitachi Data Systems Safety Information Intended Audience Product Version Release Notes Document Organization Referenced Documents Document Conventions Conventions for storage capacity values Safety information Getting help Comments Preface vii Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 8: ...quipment or damage to surrounding property Example label 1 Electric Shock Hazard Icons represented by indicate that you need to take care A graphic that indicates why you need to take care such as Electric Shock Hazard is placed inside Example label 2 Disassembly Prohibited Icons represented by indicate prohibited actions A graphic that indicates the prohibited action such as Disassembly Prohibite...

Страница 9: ...ary statements and advice indicated on this product or in the manual were considered carefully However unexpected accidents or incidents might occur When performing operations in addition to following the instructions you need to personally pay attention and take care General safety precautions When handling this product always follow the precautions below Handling power cables Always use the powe...

Страница 10: ...ect the outlet s grounding electrode to a grounding terminal installed by a qualified electrician Without a connection to a grounding terminal an electric shock can occur if the power supply fails Plugging in and unplugging When inserting a power plug into or removing a power plug from an electrical outlet be sure to hold the plug part If you pull the cable the cable might partially break The brea...

Страница 11: ...immediately after the power is turned off Wait for about 5 minutes before adding or replacing internal parts unless otherwise specified in this manual Hot parts can cause burns Laser beam A Class 1 laser product is installed on this product Do not look directly at the laser beam Do not look at the laser beam by using an optical instrument Under the laser module cover a laser beam is being emitted ...

Страница 12: ...cause injuries Improper battery type There is a risk of explosion if a battery is replaced by an incorrect type Dispose of used batteries according to the instructions Handling batteries Maintenance personnel should replace batteries Do not replace batteries by yourself Follow the instructions described below Inappropriate handling can result in injury because a battery can overheat burst or catch...

Страница 13: ...nside the equipment Using the equipment with condensation present can cause an equipment failure Leave the equipment at a new location for several hours until the temperature of the equipment matches the temperature of the new environment and then start using the equipment For example if you move the equipment from an environment whose temperature is 5 C to one whose temperature is 25 C leave the ...

Страница 14: ...er cables can cause injury or failure of devices connected to the equipment In addition valuable data might be lost Do not place heavy items on the cables Do not route cables close to devices that become hot If you do so the cable covering might be damaged which can cause failures in connected devices Before turning off the power Follow the prescribed procedure for power operation If you turn on o...

Страница 15: ...e ratings Reliable grounding Maintain reliable grounding of rack mounted equipment Pay particular attention to non grounded power connections to branch circuits for example in power strips Intended Audience This document is intended for the personnel who are involved in planning managing and performing the tasks to prepare your site for Compute Blade installation and to install the same This docum...

Страница 16: ...bes the initial setup that is required from installation of the system unit to powering on the server blades Chapter 4 Stopping the CB 2500 Describes how to stop the system unit Chapter 5 Functionality and Operation of the Remote Console Provides an overview of the remote console The remote console is mainly used for configuring the operating environment of the server blade and creating virtual me...

Страница 17: ...ame has been changed to Hitachi logical partitioning manager LPAR manager or LP If you are using HVM based logical partitioning feature substitute references to Hitachi logical partitioning manager LPAR manager or LP with HVM This document uses the following typographic conventions Convention Description Regular text bold In text keyboard key parameter name property name hardware labels hardware b...

Страница 18: ...storage capacity values for example disk drive capacity are calculated based on the following values Physical capacity unit Value 1 kilobyte KB 1 000 103 bytes 1 megabyte MB 1 000 KB or 1 0002 bytes 1 gigabyte GB 1 000 MB or 1 0003 bytes 1 terabyte TB 1 000 GB or 1 0004 bytes 1 petabyte PB 1 000 TB or 1 0005 bytes 1 exabyte EB 1 000 PB or 1 0006 bytes Logical storage capacity values for example lo...

Страница 19: ...e Hitachi Data Systems Portal for contact information https portal hds com Comments Please send us your comments on this document doc comments hds com Include the document title and number including the revision level for example 07 and refer to specific sections and paragraphs whenever possible All comments become the property of Hitachi Data Systems Corporation Thank you Preface xix Hitachi Comp...

Страница 20: ...xx Preface Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 21: ...assis that make up the CB 2500 and the various modules to be installed in the CB 2500 Features of the CB 2500 Various consoles that manage the CB 2500 General procedure for setup of the CB 2500 CB 2500 Overview 1 1 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 22: ...that is used to expand I O adapter slots PCI expansion blades have a maximum of six I O adapter slots One PCI expansion blade is used in combination with one half wide blade Management module A management module manages the entire system unit You can use a management module to monitor the operating statuses of server blades and various modules and to set their power control or network information ...

Страница 23: ...eed complicated cable connections When you add a server blade you can connect the server blade by just inserting the server blade to a standby slot of the server chassis The CB 2500 can facilitate integration of traditional rack mount server systems such as system integration and data center operations The following shows the number of modules that can be installed in the CB 2500 server chassis A ...

Страница 24: ...backplane of a server chassis In addition a PCI expansion blade can be used in combination with a half wide blade to expand I O adapter slots A PCI expansion blade is an option for a half wide blade The following shows the main features of CB 2500 server blades Equipped with high performance Intel multicore processors that use QPI technology which improves throughput and reduces delays Provide hig...

Страница 25: ...supported CB 520H B4 Basic mode Not supported Supported U3a U3b Supported U1b U2 LP mode Not supported Note 1 Not supported 4 blade SMP configuration Related topics Manual Hitachi Compute Blade Series Hitachi Compute Rack Series OS Installation Guide for Windows Server Manual Hitachi Compute Blade Series OS Installation Guide for Red Hat Enterprise Linux Management module overview A management mod...

Страница 26: ...the system unit Fan module A fan module cools server blades and various modules There are two types of fan modules system fan modules that cool server blades and switch fan modules that cool switch modules Fan modules are mounted on the rear side of a server chassis Eight system fan modules and two switch fan modules are installed The operation status of fan modules are monitored by cooling fan co...

Страница 27: ...lowered to reduce power consumption The state in which LPAR manager functionality is used and no process to be dispatched exists is called a host idle loop Note If the power capping upper limit is set too low the CPU performance might become low and affect OS operation Thoroughly evaluate the value to be set to power capping during the operation test before starting the operation Internal LAN and ...

Страница 28: ... can select the console appropriate to your goals from some console types such as a Web console which enables you to perform operations by using a GUI and a CLI console which enables you to perform operations by entering commands This section describes the types of the consoles used to manage the CB 2500 and the hardware and software conditions required for the system console Types of consoles sup...

Страница 29: ...t is equipped with an LCD touch panel interface This LCD touch console is a device that directly connects to the system unit and therefore does not require a system console This console is suitable for specifying settings when installing a system because you can set up a system unit without needing to set up a network System console prerequisites This subsection describes the prerequisites of the ...

Страница 30: ... OS X 10 10 Yosemite Notes 1 When you operate server blades from a remote console a maximum of 256 MB of memory installed on the system console is used per server blade To operate multiple server blades at the same time install a sufficient amount of memory on the system console computer 2 Use a LAN cable of Category 5 or higher to connect the system console computer and the CB 2500 system unit 3 ...

Страница 31: ...s to disk check box For Internet Explorer 8 or later disable the SmartScreen Filter function In the Tools menu execute the Internet Options command In the displayed Internet Options dialog box select Advanced and Security and then clear the Turn On SmartScreen Filter check box 3 Do not install an unsupported version of Java VM to the system console If you install an unsupported version of Java VM ...

Страница 32: ...hat enables you to manage various settings of the system unit The Web console can graphically display the operating status of the entire system and failed parts and list setting information of server blades and switching modules Figure 1 4 Web console window The Hitachi Compute Blade 2500 Series Getting Started Guide explains the setting procedures that use the Web console for the basic settings r...

Страница 33: ...he virtual media functionality to install an OS or applications onto a server blade Note If the system console OS is Windows Server 2008 R2 and you want to access a physical drive of the system console use the built in user account to log in to the remote console If you use a different user account to log in you cannot access the physical drive Note that all user accounts can be used to access a v...

Страница 34: ... initial settings of a server blade Figure 1 6 General procedure for the initial settings Powering on a server blade and setting up the OS The following figure shows the general procedure from powering on a server blade to setting up the OS For details on the tasks in LP mode see the Hitachi Compute Blade 2500 Series Logical partitioning manager User Guide 1 14 CB 2500 Overview Hitachi Compute Bla...

Страница 35: ... general procedure for stopping the entire system unit for example for a planned power outage or to change the installation location of the system unit For details on the tasks in LP mode see the Hitachi Compute Blade 2500 Series Logical partitioning manager User Guide CB 2500 Overview 1 15 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 36: ...Figure 1 8 General procedure for stopping the system unit 1 16 CB 2500 Overview Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 37: ...that indicate operation status This chapter also describes these connectors and indicators Appearance of the server chassis Server blade appearance Appearance of the I O board module Appearance of the management module Appearance of the management LAN module Appearance of the switch module Appearance of the power supply module Appearance of the fan module CB 2500 Appearance 2 1 Hitachi Compute Bla...

Страница 38: ... are installed This section also describes the ID tag for identifying the server chassis Server chassis and the locations where modules are installed Install server blades and modules in predetermined slots on the front or rear of the server chassis The following figures show the locations where server blades and modules are installed for the front and rear sides of the server chassis Figure 2 1 S...

Страница 39: ...s in appropriate slots on the front of the server chassis Install I O board modules switch modules power supply modules fan modules and management LAN modules in appropriate slots on the rear of the server chassis The server blade chassis has location tags for identifying slots and slot numbers This section describes the slots in which server blades and modules are installed and the location tags ...

Страница 40: ...ide 1 Tip One PCI expansion blade is always used in combination with one half wide blade Therefore when a PCI expansion blade is used half wide blades are installed in odd numbered slots PCI expansion blades are installed in even numbered slots and then each blade is connected to a corresponding I O adapter 2 4 CB 2500 Appearance Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 41: ...d by the CB 2500 have an indicator that shows the operational status of connectors and server blades connected to external devices This section describes the appearance connectors and indicators of the server blade Appearance of a full wide blade A full wide blade has slots for installing HDDs and SSDs as well as connectors for connecting external devices It also has indicators to show the operati...

Страница 42: ...ll wide blade 1 HDD slot 0 1 These are slots for installing HDDs on the server blade Table 2 1 When an SAS HDD or SAS SSD is mounted Symbol Name State Description 1 Active lamp Green light on The drive is idle Flashing green The drive is running or is being accessed The drive is being rebuilt source drive 2 Fault lamp Orange light on An error occurred on the drive 1 2 6 CB 2500 Appearance Hitachi ...

Страница 43: ...ually power on or power off the server blade To force the operating system on the server blade to shut down press and hold this power switch for more than 4 seconds which turns the power off Never press the power switch when the operating system on the server blade is running correctly If you do so the operating system might not start or data might be corrupted This power switch has a green LED to...

Страница 44: ...s the LED to light up 4 ATN LED yellow LED This LED lights up in the following cases The PCI Express adapter is not compatible with the bus speed of the server blade Even though the ON switch was pressed no power is going to the server blade The server blade detected another failure with an unknown cause After correcting these failures you can use the management module to turn off this LED You can...

Страница 45: ...his LED lights up when an NMI is issued 5 TEMP LED orange LED This LED turns up when the temperature of the server blade exceeds the limit 6 MEM LED orange LED This LED lights up when a memory failure is detected 7 ADJ LED orange LED This LED lights up when a failure is detected on the server blade Appearance of a half wide blade A half wide blade has slots for installing HDDs and SSDs as well as ...

Страница 46: ... the model name serial number and relevant version information Figure 2 14 ID tags of half wide blades Figure 2 15 Switch and connector of a half wide 1 HDD slots 0 and 1 These are slots for installing HDDs on the server blade 2 10 CB 2500 Appearance Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 47: ...VGA output and a serial port and USB port are available 5 Power switch This is the power switch of the server blade Use this switch to manually power on or power off the server blade To force the operating system on the server blade to shut down press and hold this power switch for more than 4 seconds which turns the power off Never press the power switch when the operating system on the server bl...

Страница 48: ...rd is not compatible with the switch module The PCI Express adapter is not compatible with the bus speed of the server blade Even though the ON switch was pressed no power is going to the server blade The server blade detected another failure with an unknown cause After correcting these failures you can use the management module to turn off this LED You can also configure the management module to ...

Страница 49: ...s and connection destination switch modules The table below lists network devices mounted on the server blade and connected switch module ports Note that you cannot connect a server blade installed in slot 15 to a switch module Table 2 3 Onboard LAN or mezzanine card and 1 10Gb LAN switch module or Brocade 10Gb DCB switch module CB 520X B1 CB 520H B3 B4 Network devices mounted on the server blade ...

Страница 50: ... In an SMP configuration the PCI bus number changes as shown below Note that SMP configurations support only CB 520X B1 server blades 2 blade SMP configuration Node 1 8 08h Node 2 136 88h 4 blade SMP configuration Node 1 8 08h Node 2 72 48h Node 3 136 88h Node 4 200 C8h 4 In an SMP configuration the PCI bus number changes as shown below Note that SMP configurations support only CB 520X B1 server b...

Страница 51: ...N 1 9 09h 3 0 Switch module 1 2 1 Switch module 2 3 2 Switch module 1 4 3 Switch module 2 Notes 1 A physical port number that actually exists in the system 2 In an SMP configuration the PCI bus number changes as shown below Note that SMP configurations support only CB 520X B2 B3 server blades 2 blade SMP configuration Node 1 Mezzanine card slot 1 8 08h Mezzanine card slot 3 65 41h Node 2 Mezzanine...

Страница 52: ...Node 2 73 49h Node 3 137 89h Node 4 201 C9h Appearance of the I O board module An I O board module has a slot for mounting a PCI Express adapter The I O board module also has an indicator to show the operational status of the module If you wish to mount a PCI Express adapter on the I O board module the expansion board must support the Low Profile form factor The appearance of the I O board module ...

Страница 53: ...press adapter Mezzanine card installation locations and corresponding I O board modules PCI Express path through mezzanine cards installed in mezzanine card slots 2 and 4 of a full wide blade correspond to the PCI Express adapters mounted on an I O board module The table below lists the slot numbers of a full wide blade and the corresponding slot numbers of an I O module Table 2 5 Full wide blade ...

Страница 54: ...and xxB of an I O board module xx in xxA and xxB corresponds to the slot number on which the half wide blade is mounted Appearance of the management module The management module has a serial connector for directly connecting it to the system console as well as USB ports for connecting USB devices The management module also has an indicator to show its operational status The appearance of the manag...

Страница 55: ...emory device or DVD drive that draws 2A or less 4 Serial port This port connects the management module to the system console via a serial cable When the management LAN is unavailable this port can be used as an alternative connection 5 MM MSR LED This LED lights up when the management module is running as the Active system The standby management module remains switched off 6 MM ALM LED This LED li...

Страница 56: ...module is powered off 11 MM HB LED This LED flashes green when the management module is running Appearance of the management LAN module The management LAN module has connectors for connecting with the management LAN and indicators to show the status of the management LAN module itself The management LAN module and the management module are connected via the backplane The appearance of the manageme...

Страница 57: ...The lights of this LED signify the following statuses Green light ON The management LAN module is running correctly Flashing green A failure has occurred on the management LAN module OFF The management LAN module is powered off Appearance of the switch module The switch module has connectors for connecting the system unit to external networks and indicators to show the status of the switch module ...

Страница 58: ...hes when the module is communicating via the network 3 ALM LED This LED lights up when a failure occurs on the switch module 4 LID LED This LED identifies the switch module You can turn this LED on or off from the Web console You can use this LED to easily identify the location of the target switch module 5 PWR LED This LED lights up when the switch module is powered on This LED is off when the sw...

Страница 59: ...odule is powered off 4 SFP port Trunk group A B These are optical transceiver ports for connecting to an external network Ports that support SFP are classified into two Trunk groups SFP ports 43 to 50 belong to Trunk group A and SFP ports 51 to 56 belong to Trunk group B There are two LEDs next to each port The LED on the upper left TX RX lights up green when a link to the network is established T...

Страница 60: ... a link to the network is established The LED flashes green when the port is communicating via the network The LED on the upper right ALM lights up green when the communication speed of the port is 100 Mbps or 1000 Mbps This LED stays off when the communication speed of the port is 10 Mbps Appearance of the power supply module The power supply module has a connector for connecting the AC power cab...

Страница 61: ... The prescribed power is being correctly supplied from the power supply module to the system unit Fast flashing green The firmware is being updated Slow flashing green The power of the server chassis is being turned off OFF Power is not being supplied to the system unit 3 IN LED This LED lights up when the module is supplying sufficient power to run the system unit The LED is off if the power supp...

Страница 62: ... of the fan module are described below 1 PWR LED This LED lights up when the fan module is powered on 2 ARM LED This LED lights up when a failure occurs on the fan module 2 26 CB 2500 Appearance Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 63: ... the system console used for configuring the settings of the CB 2500 Setting the network for the server blades and modules Preparing the server blades so that they can operate in LP mode Powering on the server blade Resetting the power of the server blade Restarting the BMC of a server blade Setting the UEFI of a server blade Starting the CB 2500 and Performing Initial Setup 3 1 Hitachi Compute Bl...

Страница 64: ...ally starts first followed by the switch module 2 Ensure the management module and switch module start normally The indicators on the management module and the switch module show the startup status of each module 3 Press the power switch for the server blades Electricity is supplied to the server blades and they power on Check the indicators of the server blades to make sure electricity is being s...

Страница 65: ...ers depending on IPv4 address assign mode which determines whether IP address is assigned manually or via DHCP With management module firmware earlier than A0165 IPv4 address assign mode is Static which means IP address is set manually With management module firmware A0165 or later you can set IPv4 address assign mode to Static or DHCP The default setting is Static and it can be changed to DHCP wh...

Страница 66: ...anagement module 323GG RE4xxxxxx Yxxxxxxxx Server blade BMC 323GG RV3xxxxxxxx Yxxxxxxxx Use the serial port of the management module to log in to CLI console and run show mgmt module mgmt lan and show blade mgmt lan commands to show the IP addresses Connecting by using an RS232C cable Use a generic RS232C cable to connect the serial port of the management module to that of the system console Manag...

Страница 67: ... of the server blades and modules use the Web browser of the system console to access the management module The workflow for the configuration is as follows 1 Log in to the management module 2 Change the login password of the management module 3 Set the chassis ID of the server chassis 4 Change the network addresses of the management module server blades and switch modules After completing these s...

Страница 68: ...lowing values in the User ID and Password fields respectively and then click the Login button User ID administrator Password password When you have successfully logged in the menu window of the Web console is displayed Changing the password used to log in to the management module To ensure security change the login password of the login user administrator from the initial settings the settings at ...

Страница 69: ...ettings the settings at shipping time to ones that comply with the network of the management LAN To do this you must be logged in to the management module as a user with Administrator privileges such as administrator If IPv4 address assign mode is DHCP IP address and subnet mask and default gateway if configured of management module and server blades are assigned via DHCP and you do not need to ch...

Страница 70: ...it is recommended that you reset the default password for the user account of the switch default ID is admin to ensure the security for the switch To do this follow the procedure described below 1 Log in to the switch module 2 Reset the password for the initial admin account 3 Log out from the switch module To reset the default password for the switch use the terminal software to access the manage...

Страница 71: ...as the password and press Enter The following response is displayed to prompt you to retype the password for confirmation Re enter new admin password 6 Retype the string that you entered in step 5 and press Enter The following response is displayed New admin password accepted When the confirmation is displayed setting of the password is complete 7 Type disable and press Enter The prompt changes fr...

Страница 72: ...g time To do this you must be connected and logged in to the switch module using the admin account via the terminal software 1 At the prompt of the terminal software type configure terminal and press Enter The mode is changed to the global configuration mode 2 Type username admin password role admin enable true and press Enter The string represents the new password Enter a string of your choice Wh...

Страница 73: ... Hitachi Compute Blade 2500 Series Logical partitioning manager User Guide Related topics The manual Hitachi Compute Blade 2500 Series Logical partitioning manager User Guide Powering on the server blade To power on the server blade access the management module via the system module You can power on the server blade from the Web console CLI console or remote console During the initial configuratio...

Страница 74: ...n the server blade starts shut down the OS and restart the server blade Related topics The section on the poweron blade command in the Hitachi Compute Blade 2500 Series Management Module User Guide Starting the remote console on page 5 4 Functionality of the remote console on page 5 7 The manual Hitachi Compute Blade 2500 UEFI Setup Guide Resetting the power of the server blade To reset the power ...

Страница 75: ...ual Hitachi Compute Blade 2500 Series Management Module User Guide Functionality of the remote console on page 5 7 Restarting the BMC of a server blade If a failure occurs in the server blade restart the BMC of the server blade You can restart the BMC from the Web console or CLI console Restart the BMC only if a failure has occurred in the server blade Restarting the BMC from the Web console You m...

Страница 76: ...tionality of the CPU installed on the server blade Set the type and operational functionality of the DIMM installed on the server blade Set the options for storage network devices and the power supply unit installed on the server blade Set the RAID configuration of the storage installed on the server blade Set up the PXE boot environment or iSCSI boot environment For more information on using UEFI...

Страница 77: ...lains how to stop the system unit Powering off a server blade Forcibly powering off a server blade Powering off a switch module Powering off the server chassis Stopping the CB 2500 4 1 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 78: ...tree view select the desired server blade In the application area the Server Blade n Information view is displayed The variable n represents a unique number that identifies the server blade 3 In Server blade n information view select the State tab and click the Power Off button Using the CLI console to power off a server blade Execute the poweroff blade command For more information about the power...

Страница 79: ...enu select Forced Power off Using the CLI console to forcibly power off a server blade Execute the poweroff blade h command For more information about the poweroff blade command see the Hitachi Compute Blade 2500 Series Management Module User Guide Using the remote console to forcibly power off a server blade 1 Start the remote console and log in to the desired server blade 2 In the Power menu cli...

Страница 80: ...ser on the system console 1 From the global task bar of the Web console select the Resources tab In the navigation area the Modules tree view is displayed 2 In the Modules tree view select Chassis In the navigation area the Chassis Summary tree view is displayed 3 In the Chassis Summary view click the Action drop down list and select Shut down When you respond to the confirmation window the server...

Страница 81: ... 4 The Shutting Down Server Chassis window is displayed Related topics The section on the shutdown chassis command in the Hitachi Compute Blade 2500 Series Management Module User Guide Appearance of the management module on page 2 18 Stopping the CB 2500 4 5 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 82: ...4 6 Stopping the CB 2500 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 83: ...rom the server blade Notes on using the remote console Server blade image signals that can be displayed on the remote console Starting the remote console Remote console window Functionality of the remote console Creating a disk image for access from the server blade Exiting the remote console Screen messages displayed by the remote console Functionality and Operation of the Remote Console 5 1 Hita...

Страница 84: ...though you can connect the server blade to some devices via the system console such as FD and USB memory you cannot format these devices or media from the server blade Formatting must be performed from the operating system of the system console In addition FD or USB memory devices that are remotely connected to the server blade must be formatted using the following settings Remote FD The media mus...

Страница 85: ...lade window When the remote console window is active if you press Print Screen or Alt Print Screen the instruction is sent to the server blade and the operating system of the system console becomes inoperable In this case make the remote console window inactive and press Print Screen to capture the whole screen If you click anywhere other than the window of the software displaying the remote conso...

Страница 86: ...s logged in to the management module or a server blade If the system console is running on OS X you must install the certificate file and enable Java accessibility in advance Set up the System console that OS X is running If you want to use a PC running OS X as the system console you must set up the environment as follows in advance Install the certificate file Enable Java accessibility Installing...

Страница 87: ...ity Starting the remote console from the management module via a Web console This section describes how to start the remote console from a Web console that is logged in to the management module First log in to the management module 1 From the global taskbar in the Web console select the Resources tab In the navigation area the Modules tree view is displayed 2 In the Modules tree view select the ta...

Страница 88: ...or the system console and enter the URL https IP address of server blade The Web console starts and the login window to the server blade is displayed 2 In the Username field enter user01 In the Password field enter pass01 This user name and password combination is the initial setting the setting at shipping time If you have modified the settings enter the modified user name and password as necessa...

Страница 89: ... destination server blade Related topics Functionality of the remote console on page 5 7 Functionality of the remote console This section describes the functionality available on the remote console and the corresponding menu commands Table 5 1 List of remote console functionality Available functionality Menu Command Functionality related to the remote screen Refresh the remote screen View menu Ref...

Страница 90: ...rations conducted in the remote screen are linked to the operation of the keyboard on the system console Tools menu Session Options command When you execute this command the Session Options dialog box is displayed If the Pass all keystrokes to target check box is selected in the General tab in the dialog box keyboard pass through mode is enabled and keystrokes entered on the keyboard of the system...

Страница 91: ...or older or any version of Linux other than Red Hat Enterprise Select the Relative Linux Acceleration radio button Functionality related to virtual media Start the virtual media console Tools menu Launch Virtual Media command Functionality related to the power operation of the server blade4 Power on the server blade Power menu Power On command Power off the server blade Power menu Forced Power Off...

Страница 92: ...of the CD or DVD media on the system console and connect to the console remotely from the server blade We recommend creating a virtual disk image This is because the data can be read faster from a disk image than from a CD or DVD and a disk image can be handled more flexibly This chapter describes how to create a virtual disk by using the remote console functionality Starting the Virtual Media con...

Страница 93: ...e Drive column This column displays the devices as the names recognized by the system console Details list box This box displays the list of virtual media that are set to be available for remote connections from the server blade Target Drive column This column displays the types of devices that are recognized by the server blade when connected remotely Mapped To column This column displays the bus...

Страница 94: ...s section describes notes creating disk images from virtual media Compliance with license agreements Disk images of media subject to license agreements such as operating system installation media are subject to the same license agreement as the original media Take care not to violate any license agreements Disk images created using the image creation tools FDDUMP exe and MakeCDImg exe can only be ...

Страница 95: ...and Windows Vista Business a In the Start menu click Control Panel to display the Control Panel If the items are displayed by Category select Hardware and Sound b Select AutoPlay c Deselect the Use AutoPlay for all media and devices check box and click the Save button For Windows Server 2012 Windows Server 2012 R2 and Windows 8 a Type AutoPlay in the Windows search box click the Settings button an...

Страница 96: ...reated a confirmation message appears Creating a disk image from FD media Use the FD image creation tool FDDUMP exe to create a disk image from FD media 1 Start the FD disk image creation tool 2 Insert the FD media from which you wish to create the disk image into the FD drive of the system console 3 Under Drive in the FD image creation tool select the FD drive where you inserted the media 4 Click...

Страница 97: ...een Message Description No Signal For some reason for example because the operating system of the server blade has not started the remote console is not receiving any VGA signal from the server blade VideoStopped_OutOfRange The VGA signal sent from the server blade was interrupted or the VGA signal became unavailable for display on the remote console Functionality and Operation of the Remote Conso...

Страница 98: ...5 16 Functionality and Operation of the Remote Console Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 99: ...escribes troubleshooting procedures for the CB 2500 Troubleshooting on the remote console Issuing an NMI for the operating system running on the server blade Troubleshooting 6 1 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 100: ...ot match between the server blade and the system console Caps Lock key Num Lock key Scroll Lock key You can only change the key statuses of the system console by deactivating the remote console window and performing the key operation on the system console By doing this you can match the key statuses on the system console with those on the server blade 3 If the FD is changed when remotely connected...

Страница 101: ... the General tab and then under Temporary Internet Files click the Settings button In the displayed dialog box verify that the Keep temporary files on my computer check box is selected 12 When using the remote console keystrokes continue to be entered without stopping Alternatively keystrokes are not entered Use the mouse to deactivate the remote console window and then re activate the window If t...

Страница 102: ... connection the operation might fail 20 After using a remotely connected CD DVD drive the media cannot be ejected even when the eject button on the drive is pressed For an external CD DVD drive connected via USB remove the drive from the system console connect it again and then perform the operation to safely remove hardware For a built in CD DVD drive perform the following operations and then per...

Страница 103: ...certificate from the Java keystore because you are prompted to enter a password Enter changeit which is the default Java keystore password and then delete the certificate 28 In case using remote console on OS X you cannot connect to the remote console The OS X console window displays Connection failed In CA certificate in user level trusted store there is no certificate file for the Java program f...

Страница 104: ...ty can be used when the operating system of the server blade is not running correctly To issue an NMI use the management module If any failure occurs on an operating system running on a server blade issue an NMI from the Web console or CLI console Issuing an NMI from the Web console 1 In the global taskbar in the Web console select the Resources tab In the navigation area the Modules tree view is ...

Страница 105: ...Related topic Hitachi Compute Blade 2500 Series Management Module User Guide reset blade Troubleshooting 6 7 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 106: ...6 8 Troubleshooting Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 107: ...ystem unit components Specifications of the server chassis Specifications of server blades Specifications of switch modules Specifications of the power supply module Specifications of fan modules System Unit Specifications A 1 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 108: ...edundancy Chassis size Height 527 mm excluding projections Width 447 mm excluding projections Depth 830 mm excluding projections Front panel Embedded in the management module Weight Maximum 250 kg This is the value when all modules are installed and includes the weight of the rack rails Temperature 5 to 40 C When not operating 10 to 43 C Humidity 8 to 80 When not operating 8 to 90 Vibration Maximu...

Страница 109: ... Mode initial setting value Lockstep Mode Rank Sparing Mode Mirrored Memory Mode Number of slots 48 Memory capacity Maximum 1536 GB LR DIMM 32 GB x 48 Memory frequency 1066 MT s 1333 MT s 1600 MT s SMP connection 2 blade SMP connection 4 blade SMP connection HDD SSD Number of HDDs and SSDs Maximum 2 Capacity 1 Maximum 2 4 TB I O Internal disk interface Controller LSI SAS 3004 Interface SAS Number ...

Страница 110: ...d disk drive capacity 1 GB is calculated as 109 bytes 2 The mezzanine card slot 1 and slot 3 are used exclusively with an onboard LAN Mezzanine card slot 3 is not supported 3 In SMP configuration between blades VGA and COM are not available and only USB port is available on non primary server blades 4 This is the maximum weight including the weights of SMP connection boards Table A 4 CB 520X B2 CP...

Страница 111: ...ber of HDDs Maximum 2 Capacity1 Maximum 3 6 TB I O Internal disk I F Controller LSI SAS 3004 I F SAS Number of I F Maximum 2 RAID function RAID 0 1 VGA Resolution 640x400 1680x1050 Refresh rate 60 85 Hz Bit per pixel 8 16 32 Internal USB port USB2 0 x 2port For LPAR manager Onboard LAN2 Midplane I F 10Gbps 4 port converged network adapter Functions Wake on LAN PXE boot iSCSI FCoE Mezzanine card sl...

Страница 112: ...y server blades 4 The Weight of a server blade includes that of SMP connection board Table A 6 CB 520X B3 CPU specifications Items Specifications Supported CPUs E7 8880 v4 E7 8890v4 E7 8893v4 Number of cores 22 C 24C 4C CPU frequency 2 2 GHz 2 2 GHz 3 2 GHz QPI frequency 9 6 GT s 9 6 GT s 9 6 GT s L3 Cache 55 M 60 M 60 M TDP 150 W 165 W 140 W Number of CPUs Maximum 2 Table A 7 CB 520X B3 blade spe...

Страница 113: ...ted COM Serial redirection DC input supply voltage DC 12 V Front interface USB 1 USB 3 0 3 KVM connector 1 VGA COM USB2 0 2port 3 DCMI Data Center Management Interface Supported Power consumption Maximum 770 W including HDDs and mezzanine cards External dimensions width x depth x height 435 3 mm x 492 7 mm x 55 5 mm 2S Weight Maximum 13 8kg4 Note 1 Hard disk drive capacity is calculated assuming t...

Страница 114: ...es 8 C 12 C 14 C 16 C 18 C CPU frequency 3 2 GHz 2 6 GHz 2 6 GHz 2 3 GHz 2 3 GHz QPI frequency 9 6 GT s 9 6 GT s 9 6 GT s 9 6 GT s 9 6 GT s L3 Cache 20 MB 30 MB 35 MB 40 MB 45 MB TDP 135 W 135 W 145 W 135 W 145 W Number of CPUs Maximum 2 Table A 10 CB 520H B3 blade specifications Items Specifications Main Memory DIMM type per slot DDR4 RDIMM 8 16 32 GB LR DIMM 32 64 GB Operating mode Independent C...

Страница 115: ...V DC Front interface USB 1 USB 3 0 KVM connector 1 VGA COM 2 port USB 2 0 DCMI Data Center Management Interface Supported Power consumption 498 W External dimensions width x depth x height 215 4 mm x 500 6 mm x 51 1 mm Weight 6 7 kg Note 1 Hard disk drive capacity is calculated assuming that 1 GB is equal to 109 bytes 2 The mezzanine card slot 1 is used exclusively with an onboard LAN Table A 11 C...

Страница 116: ... W 145 W 135 W 145 W Number of CPUs Maximum 2 Notes 1 Inlet air temperature is restricted to 5 to 35 C Non operating 10 to 43 C Table A 13 CB 520H B4 blade specifications Items Specifications Main Memory DIMM type per slot DDR4 RDIMM 16 32 64 GB Operating mode Independent Channel Mode Rank Sparing Mode Mirrored Channel Mode Number of slots 24 Memory capacity Maximum 1536 GB 64 GB x 24 Memory frequ...

Страница 117: ... Supported Power consumption 510 W External dimensions width x depth x height 215 4 mm x 500 6 mm x 51 1 mm Weight 6 7 kg Note 1 Hard disk drive capacity is calculated assuming that 1 GB is equal to 109 bytes 2 The mezzanine card slot 1 is used exclusively with an onboard LAN Table A 14 PCI expansion blade specification Items Specifications I O I O adapter slot Interface PCI Express Number of slot...

Страница 118: ...lf is not included in external dimensions 3 This value includes the shelf for a PCI expansion blade Specifications of switch modules The following table shows the specifications of switch modules Table A 15 Specifications of the 1 10Gb LAN switch module Item Specifications Interfaces Internal port SerDes 1000Base X 28 port default 14 port External port RJ 45 10Base T 100Base Tx 1000Base T 20 port ...

Страница 119: ...o 128 static routes Routing protocol support OSPF v3 routing protocol IPv6 filtering with ACLs Network functionality IEEE 802 1p Class of Service CoS prioritization Traffic shaping and re marking based on defined policies Eight Weighted Round Robin WRR priority queues per port Layer 2 Trunk Failover to support active standby configurations of network adapter teaming Virtual Router Redundancy Proto...

Страница 120: ...x Flow Control IEEE 802 3ae 10GBASE SR short range fiber optics 10 Gb Ethernet External dimensions width x depth x height 401 mm x 317 mm x 30 mm Weight 3 7 kg Note 1 Using this port requires a dedicated serial management cable Table A 16 Specifications of the Brocade 10Gb DCB switch module Item Specifications Interfaces DCB switch component VDX2746 Internal port SerDes 10GBaseKR 428 port 1 Extern...

Страница 121: ...RA IPv6 Static Route MLD Multicast Listener Discovery Snooping VRRP VRRP E v3 OSPF v3 BGP4 IPv6 ACL Dynamic Route Leak IPv4 IPv6 Network functionality VCS functionality Automatic Fabric Formation Distributed Fabric Services Transparent LAN services Virtual Ling Aggregation Group vLAG spanning multiple physical switches Switch Beaconing Distributed Configuration Management Transparent Interconnecti...

Страница 122: ...EEE 802 3 Ethernet IEEE 802 3ab 1000BASE T copper twisted pair Gigabit Ethernet IEEE 802 3ad Link Aggregation Control Protocol IEEE 802 3ae 10GBASE SR short range fiber optics 10 Gb Ethernet IEEE 802 3ap 10GBASE KR backplane 10 Gb Ethernet IEEE 802 3ba 40GBASE SR4 short range fiber optics 40 Gb Ethernet IEEE 802 3x Full duplex Flow Control External dimensions width x depth x height 401 mm x 317 mm...

Страница 123: ...nlet standard IEC60320 C20 Specifications of fan modules The following table shows the specifications of fan modules Table A 18 Specifications of fan modules Item Specifications Cooling unit Mounted per module Cooling type Forced cooling by cooling fans Air flow Cooling from the front side towards the rear side of the server chassis to prevent exhaust from re entering Maintainability Maintenance c...

Страница 124: ...A 18 System Unit Specifications Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 125: ...in the physical memory on the hard drive It is saved to a file when a failure occurs in the OS that operates on a server blade The memory dump can then be used to investigate the state of the system or application that is deadlocked Taking a memory dump in Windows Taking a memory dump in Linux Taking a memory dump with VMware Taking a Memory Dump of the OS Running on a Server Blade B 1 Hitachi Com...

Страница 126: ...Set Control CrashControl Name NMICrashDump Type REG_DWORD Value 1 Virtual memory pagefile sys settings Set the virtual memory as follows 1 In the Start menu right click Computer and then select Properties 2 Select Advanced System Settings 3 In the System Properties window select the Advanced tab 4 In Performance click the Settings button 5 Select the Advanced tab 6 In Virtual memory click the Chan...

Страница 127: ... in advance Registry settings for taking a complete memory dump Use the registry editor to edit the key as follows Execute this operation only in Windows Server 2008 R2 This operation is not necessary in Windows Server 2012 and Windows Server 2012 R2 HKEY_LOCAL_MACHINE System CurrentControlSet Control CrashControl Name CrashDumpEnabled Type REG_DWORD Value 1 Settings for taking a complete memory d...

Страница 128: ...te if Windows does not restart automatically restart Windows manually After a memory dump is taken After the system restarts the memory dump that was taken is copied to the specified output destination The output destination of the memory dump is SystemRoot MEMORY DMP initial setting When the output of the memory dump is complete the following event is output to the system event log Event ID 1001 ...

Страница 129: ...riggers for taking a memory dump When an OS failure such as kernel panic occurs a memory dump is taken automatically Also when the system is hung up the management module issues an MNI to take a memory dump While a memory dump is being taken The progress is displayed on the screen as a percentage while a memory dump is being taken When the percentage reaches 100 the memory dump is complete When th...

Страница 130: ...urple diagnostic screen error message displayed on a purple screen is displayed on the local console a memory dump is taken automatically Also when VMware becomes unresponsive the management module issues an NMI to take a memory dump Perform the following procedure to issue an NMI 1 Write down the information in the VMware console window 2 Use another computer to check via the network whether the ...

Страница 131: ...aken The output destination of the memory dump is var core vmkernel zdump X initial setting X indicates a number There might be more than one vmkernel zdump X file Use the ls command to check the dates of the files and then look for the file that has the time stamp of the system restart This will be the dump file generated this time If you want to start the virtual machine after the system is rest...

Страница 132: ...B 8 Taking a Memory Dump of the OS Running on a Server Blade Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 133: ... Unit This appendix describes how to clean the system unit Rough guide for cleaning intervals Removing the dust from the ventilation holes Cleaning the System Unit C 1 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 134: ...the power module If you find any abnormality during or after cleaning contact your reseller or call maintenance personnel CAUTION Before cleaning the system unit turn off the power supply and disconnect the power cable Failure to do so may lead to an electric shock or an equipment failure Removing the dust from the ventilation holes Use a dry cloth to wipe the ventilation holes in the locations sh...

Страница 135: ...bes the service life of the hardware components making up the system unit and the components that have specified lifetimes Components that have specified lifetimes Components that have specified lifetimes D 1 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 136: ...onents that have specified lifetimes Preventive replacement of components that have specified lifetimes is a paid service Contact your maintenance company or the store where you purchased the product Compone nt name Applicable product type Remarks Not applicable Note The service life periods described here are only rough guides and do not guarantee that failures will not occur during these periods...

Страница 137: ...cense This appendix describes the license terms for the CB 2500 including the license terms for products of other vendors End user license License E 1 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 138: ... copy for back up purpose of the Licensed Products in connection with the end user s use of the Customer Products with which the Licensed Products is distributed ii all right title and ownership including all proprietary rights including trademarks copyrights and patents shall remain with Avocent iii the end user shall safeguard the confidentiality of the Licensed Products and iv that neither Avoc...

Страница 139: ...he APC function reduces power consumption by controlling the CPU clock rate of the system unit when power consumption exceeds a predetermined level B BMC Baseboard Management Controller A controller that monitors and controls the status of server blades The BMC monitors and controls server blades by connecting to the system console and the management module C CLI Command Line Interface CNA Converg...

Страница 140: ... up and restore the disk data of a server blade as an image file You can also use a backed up image file to replicate the environment of a managed resource on another managed resource DIMM Dual Inline Memory Module E EFI Extensible Firmware Interface ETS Enhanced Transmission Selection F FC Fibre Channel G GUI Graphical User Interface A B C D E F G H I J K L M N O P Q R S T U V W X Y Z Glossary 2 ...

Страница 141: ...e Institute of Electrical and Electronics Engineers Inc IPv4 Internet Protocol version 4 IPv6 Internet Protocol version 6 iSCSI Internet Small Computer System Interface J JRE Java Runtime Environment K KVM Keyboard Video and Mouse A B C D E F G H I J K L M N O P Q R S T U V W X Y Z Glossary 3 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 142: ...l PARtition When using logical partitioning an LPAR is the term for each logical partition that can accommodate a discrete server environment LPAR manager Logical PARtitioning manager A function of logical partitioning A component that manages LPARs on a blade server LR DIMM Load Reduced DIMM M MAC Media Access Control management module A module that monitors and configures the system unit as a wh...

Страница 143: ...applications is called the active blade The server blade that is in standby is called the standby blade NIC Network Interface Card NMI Non Maskable Interrupt A hardware interrupt issued to the CPU from an external device An NMI can be used for example to collect OS dump files NTP Network Time Protocol P PCI Peripheral Component Interconnect PCIe PCI Express PFC Priority Flow Control PXE Preboot eX...

Страница 144: ...n which server blades and modules are mounted SMP Symmetric Multi Processor SNMP Simple Network Management Protocol SSH Secure SHell SSL Secure Sockets Layer standby blade When using the N M cold standby function the standby blade remains in standby with its power off until a failover occurs from a failed active blade system console A computer from which a user monitors and configures the CB 2500 ...

Страница 145: ...s Array virtual media An image file that contains the data recorded on media such as a CD or DVD By converting the installation media for the OS and other software to virtual media you can make the software available for installation on a server blade VLAN Virtual Local Area Network W Web console A console that runs in a Web browser You can use the Web console to view hardware information for a se...

Страница 146: ...WoL Wake on LAN A B C D E F G H I J K L M N O P Q R S T U V W X Y Z Glossary 8 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 147: ...ing 3 13 Brocade 10Gb DCB switch module setting user account 3 9 C CB 2500 general procedure for setup 1 14 initial setup 3 1 CD media 5 14 CD DVD image creation tool 5 14 changing network address management module 3 7 server blade 3 7 switch module 3 7 changing password 3 6 cleaning C 1 CLI console 1 9 1 12 complete memory dump B 3 connecting by using LAN cable 3 3 connecting by using RS232C cabl...

Страница 148: ...ch 2 6 G general procedure for setup 1 14 global tabs 1 12 global task 1 12 H half wide blade 2 9 appearance 2 10 connector 2 10 ID tag 2 10 switch 2 10 hardware conditions 1 9 HDD slot 2 6 2 10 Hitachi Compute Systems Manager 1 7 host idle loop 1 7 hot swapping 1 3 I I O board module 1 2 1 6 appearance 2 16 ATN LED 2 17 ATN switch 2 17 indicator 2 17 LID LED 2 17 PWR LED 2 17 ID tag 2 3 2 6 2 10 ...

Страница 149: ... cable 3 4 connectors 2 19 indicators 2 19 logging in 3 5 MM ALM LED 2 19 MM HB LED 2 20 MM MSR LED 2 19 MM PWR LED 2 20 MM SHDN switch 2 20 serial port 2 19 standby management module 1 5 switches 2 19 system ALM LED 2 19 system LID LED 2 19 system PWR LED 2 19 system WRN LED 2 19 USB port 2 19 MEM LED 2 9 memory dump 5 2 memory scrubbing 1 4 MGMT port 2 20 MIS LED 2 9 MM ALM LED 2 19 MM HB LED 2 ...

Страница 150: ...D 2 8 EXP LED 2 12 forcibly powering off 4 3 full wide blade 2 5 half wide blade 2 9 HDD slot 2 6 2 10 image signal 5 3 iSCSI boot environment 3 14 issuing an NMI 6 6 KVM 2 7 2 11 LID LED 2 8 2 12 LNK SMP LED 2 8 LP LED 2 9 LP mode 3 11 MEM LED 2 9 memory scrubbing 1 4 MIS LED 2 9 NMI LED 2 9 NMI reset switch 2 11 onboard CNA 1 4 PCI expansion blade 2 12 PCI Express adapter 1 6 power capping 1 7 p...

Страница 151: ...ng to management LAN 3 10 hardware conditions 1 9 prerequisites 1 9 setting the network information 3 5 software conditions 1 11 system fan module 1 6 system LID LED 2 19 system PWR LED 2 19 system unit cleaning C 1 connecting to management LAN 3 10 connecting to power supply 3 2 system WRN LED 2 19 T taking memory dump Linux B 5 VMware B 5 Windows B 2 TEMP LED 2 9 troubleshooting 6 2 U UEFI 3 12 ...

Страница 152: ...Index 6 Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 153: ...Hitachi Compute Blade 2500 Series Getting Started Guide ...

Страница 154: ...Street Santa Clara California 95050 2639 U S A www hds com Regional Contact Information Americas 1 408 970 1000 info hds com Europe Middle East and Africa 44 0 1753 618000 info emea hds com Asia Pacific 852 3189 7900 hds marketing apac hds com MK 99CB2500003 12 ...

Отзывы: