NEC DX2000 Скачать руководство пользователя страница 88 | Manualshive

Страница: 88 / 132

background image

1. Server Module BIOS

Scalable Modular Server DX2000 Maintenance Guide

88

Chapter 2 Useful Features

For details about the options, see the table below.

Option Parameter

Description

Power Management Policy

Performance Preferred
[Energy Preferred]
Custom

Set Power Management Policy.
Set this item to [Custom] to select the following menu
items.

Processor C3 Report

[Disabled]
Enabled

Enable or disable the function to report the availability of

the processor C3 state to OS.

Processor C6 Report

Disabled
[Enabled]

Enable or disable the function to report the availability of
the processor C6 state to OS.

Package C-State Limit

C0/C1
C2
C6(non Retention)
C6(Retention)
[No Limit]

Specify the upper limit for Package C State of processor.

EIST Disabled

[Enabled]

Enable or disable the Enhanced Intel SpeedStep
Technology feature. This option is displayed only when
the installed processor supports this feature.

Energy Performance

Performance
[Balanced Performance]
Balanced Power
Power

Set a ratio to decide whether a priority is given to the
performance or the power saving in the operation of the
processor.
This item is displayed when you set “EIST” to [Enabled].

Turbo Boost

Disabled
[Enabled]

Enable or disable the Turbo Boost Technology feature.
This option is displayed only when the installed
processor supports this feature.

Energy Efficient Turbo

Disabled
[Enabled]

Enable or disable Energy Efficient Turbo. This item is
displayed when you set. ”EIST” to [Enabled] and ”Turbo
Boost” to [Enabled].

P-State Coordination

[HW_ALL]
SW_ALL
SW_ANY

Specify the P-STATE Coordination type.
This parameter is displayed when EIST is set to
“Enabled”.

Workload Sensitivity

[Balanced]
I/O sensitive

Set [I/O sensitive] when you want to put a higher priority
to the system performance for the I/O workload.

Uncore Frequency Scaling

Disabled
[Enabled]

Enable or disable the function to adjust the operating
frequency of Uncore.

Uncore Frequency
Coordination

[Disabled]
Enabled

Enable or disable the function to adjust the operating
frequency of Uncore between processors. This item is
displayed when you set ”Uncore Frequency Scaling” to
[Enabled].

[ ]: Factory setting

«
...
86
87
88
89
90
...
»

Содержание DX2000

Страница 1: ...Second Edition April 2016 NEC Corporation 2016 NEC Scalable Modular Server DX2000 Maintenance Guide Chapter 1 Maintenance Chapter 2 Useful Features Chapter 3 Appendix ...

Страница 2: ...onfigurations and summary of DX2000 Utility Disk Chapter 4 NW Switch Modules Support function of internal NW switch modules setup and initial settings for network Chapter 5 Replacing Modules How to replace internal modules Chapter 6 Appendix Specifications and block diagram of this server Maintenance Guide Chapter 1 Maintenance Server maintenance troubleshooting management tool and management util...

Страница 3: ...om BMC and CSC modules 15 2 3 Checking STATUS LED 15 2 4 Backup 15 2 5 Cleaning 16 2 5 1 Cleaning the server 16 2 5 2 Cleaning the AC cable 17 3 User Support 18 3 1 Maintenance Services 18 3 2 Before Asking for Repair 18 4 Troubleshooting 19 4 1 From Power On to When the NW Switch Module Starts Operating 19 4 2 Upon Using the DX2000 Utility Disc 20 4 3 Upon Installing an OS 20 4 4 When OS Is Start...

Страница 4: ...69 7 1 1 Install 70 7 1 2 Starting the management utility 70 7 1 3 Command Table 70 7 1 4 Search search 71 7 1 5 Implemented module display slotlist 73 7 1 6 IP MAC address display niclist 74 7 1 7 FRU GUID display frulist 75 7 1 8 UID LED Light On uidon 76 7 1 9 UID LED Light Off uidoff 76 7 1 10 Creating MAC address list savelist 77 7 1 11 Quit quit 79 7 1 12 Chapter 2 Useful Features 80 1 Serve...

Страница 5: ...serious malfunctions could occur Note Indicates items that must be confirmed when handling the server or operating software Tips Indicates information that is helpful to keep in mind when using this server Hard disk drives Unless otherwise stated hard disk drive described in this document refers to both of the following Hard disk drives HDD Solid state drive SSD POST POST described in this documen...

Страница 6: ...on of the United States Adobe the Adobe logo and Acrobat are trademarks of Adobe Systems Incorporated PCI Express is a trademark of Peripheral Component Interconnect Special Interest Group All other product brand or trade names used in this publication are the trademarks or registered trademarks of their respective trademark owners ...

Страница 7: ...ut specific prior written permission THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS AS IS AND ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT INDIRECT INCIDENTAL SPECIAL EXEMPLARY OR CONSEQUE...

Страница 8: ... IS AND ANY EXPRESSED OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED IN NO EVENT SHALL THE OpenSSL PROJECT OR ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT INDIRECT INCIDENTAL SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES INCLUDING BUT NOT LIMITED TO PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DA...

Страница 9: ...ER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE Tools Module Name Ver Source FOSS License FOSS Licensor AES 29 04 09 http gladman plushost co uk oldsite AES aes src 29 04 09 zip BSD style Copyright c 1998 2008 Brian Gladman Worcester UK inttypes h N A http msinttypes googl...

Страница 10: ...t without permission from NEC Corporation 4 Every effort has been made to ensure the completeness of this document However if you have any concerns or discover errors or omissions please contact your retailer 5 Regardless of article 4 NEC Corporation assumes no responsibility for effects resulting from operations 6 The sample values used in this document are not the actual values Keep this documen...

Страница 11: ... Maintenance Describes what you must confirm for daily use how to manage files and how to clean the server 3 User Support Describes various services on this product 4 Troubleshooting Describes how to identify the causes of problems and actions to take to address them Refer to this section when you suspect a failure 5 Resetting and turning off the power forcibly Describes how to reset the server Re...

Страница 12: ...is the user s responsibility to erase the data Important NEC assumes no liability for data leakage should the product be transferred to a third party without erasing the data Data seems to be erased when you empty Recycle Bin of Windows or execute the format command of the operating system However the actual data remains on the hard disk drive Data not erased completely might be restored by specia...

Страница 13: ...d require replacement For stable operation of the server NEC recommends you regularly replace these components Contact your sales representative for information on replacement or the lifetime of components The replacement service of SSDs that reached their end of life will be charged 1 3 Regarding the Transportation of This Server This server and or some of the associated optional devices uses lit...

Страница 14: ...ty 20 to 80 No dew condensation is permitted Tips Make backup copies of important data stored in the hard disk drives 1 Shut down all the server modules installed in this server 2 Unplug the power cord of the server from the power outlet 3 Confirm that the POWER LED is off 4 Disconnect all the cables from the server Pull out the server from the rack slowly withthree or more people For the procedur...

Страница 15: ...ormality such as a hardware error occurs For how to configure alert features refer to Chapter 1 7 3 1 SNMP trap setting command For a list of alerts refer to Chapter 1 4 Troubleshooting Platform event trap PET events are SNMP trap events compliant with the Intelligent Platform Management Interface IPMI The user needs to prepare a program for receiving traps 2 3 Checking STATUS LED Check the STATUS...

Страница 16: ...ow if stains remain on the surfaces Important Do not use volatile solvents such as thinner and benzene to clean the server Those solvents could damage or tarnish the material The power outlet cables connectors on the rear panel of the server and the inside of the server must be kept dry 1 Shut down all the server modules installed in this server 2 Unplug the power cord of the server from a power o...

Страница 17: ...ables connectors on the rear panel of the server and the inside of the server must be kept dry 1 Shut down all the server modules installed in this server 2 Unplug the power cord of the server from a power outlet 3 Confirm that the POWER LED is off 4 Wipe off dust from power plug of AC cable with a dry cloth 5 Soak a soft cloth in neutral detergent that is diluted with cold or lukewarm water and s...

Страница 18: ... occurred follow the steps below 1 Check if the power cord and cables to other products are properly connected 2 Refer to Chapter 1 4 Troubleshooting If you find a symptom similar to your problem take the action as instructed 3 Confirm that the required software has been properly installed 4 Scan for viruses using a commercial Antivirus Software If the problem persists after taking the measures ab...

Страница 19: ...onnected power outlet is on Is the POWER LED on the front panel of this server lighting green When the power cord is connected and the AC power is supplied initialization of the BMC and CSC modules of the server modules and the NW switch module starts During initialization the POWER LED lights up in amber When initialization is complete the POWER switch and POWER LED light green If the POWER LED l...

Страница 20: ...llation screen Add the following option to the OS installation configuration file used for booting the server by using PXE console ttyS2 115200n8 4 4 When OS Is Started Unable to start OS Are server modules installed firmly Install the modules properly Check that server modules on which OS is installed properly mounted Is the OS installed correctly Check that the OS has been installed by using the...

Страница 21: ... that comes with 10G LAN module and change the setting value to ON You have to specify it in the startup script file to change the flow control Example Changing the flow control setting by using the ethtool command To change the setting to on ethtool A ens1f0 rx on tx on To confirm the result ethtool a ens1f0 4 6 When Power Off The power is not turned off Did you issue an IPMI command to power off...

Страница 22: ...ower forcibly 5 1 Reset 5 1 1 Resetting the server module If the server module halts before starting the OS reset the server module remotely via EXPRESSSCOPE Engine 3 or ipmi This clears all the data being processed in the memory and restarts the server For EXPRESSSCOPE Engine 3 Select System Reset to reset it Via ipmi Command example ipmitool I lanplus U user name P password H IP address of the B...

Страница 23: ...ailable for about 60 seconds after the BMC is reset 5 1 3 Resetting the NW switch module The Reset switch in the NW switch module allows you to reset only the NW switch module The network connection is not available for about 90 seconds after the reset 5 1 4 Resetting the CSC module The CSC Reset switch in the CSC module allows you to reset only the CSC module Hold it down for at least 4 seconds t...

Страница 24: ...ion when an OS command does not shut down the server the POWER Switch does not turn off the server or software reset does not work Note If the remote power on function is used cycle the power once to load the OS after the power has been forcibly turned off and then turn off the power again by shutting down the OS DX2000 Hold down the Power switch on the front panel at least eight seconds to forcib...

Страница 25: ... and start TeDoLi with the server module that conduct diagnosis using RemoteKVM function to use it Function You can diagnose if the standalone server module is properly operating 6 1 System Diagnostic Management Tool Use the system diagnostic management tool to maintain this sever System diagnostic management tool can diagnose CSC modules and multiple server modules of this server simultaneously S...

Страница 26: ...stic management tool to an existing network and start it it may have an impact on the connected existing network Command support function The system diagnostic management tool is equipped with the tab complimentary function and the history function to help user input the command Tab complimentary function This functions by pressing the tab twice quickly Available commands are all listed if the com...

Страница 27: ...tool To start the system diagnostic management tool by the management terminal see Chapter 1 6 1 2 Starting by the management terminal 1 Preparation The following preparation is required to execute the system diagnostic management tool 1 Console terminal A console terminal is required to display screens of the Deploy server module A console terminal is conditional on having at least one LAN port a...

Страница 28: ...ic management tool When you have connected a console terminal to this server start the tool in the following procedure 1 Connect AC cable to this server 2 Open the top panel of this server and press the POWER button of the Deploy server module 3 Boot the DX2000 Utility Disk from the CD DVD ROM drive connected to the front side USB 4 When you have started the system diagnostic management tool conne...

Страница 29: ...and displays the following message on the Management Tool window When 10 seconds have passed without any entry the process moves on and operates with the default User Password Default User Mainte Password Mainte To start the tool with other User Password than the default User Password press the ENTER key within10 seconds Enter User Password accessible to CSC modules and server modules If you chang...

Страница 30: ... check the DHCP setting for each module The following prompt is displayed when CSC modules and server modules in the server have been searched 9 Enter the command list in the Management Tool window and check if server modules are recognized 10 If you find all server modules unrecognized enter the command reload to start searching again If you change User ID Password press enter key in 10 seconds U...

Страница 31: ...erver to start the system diagnostic management tool in this server 1 Preparation The following preparation is required to execute the system diagnostic management tool Do not connect the LAN cable see 5 until you are instructed by the starting procedure 1 Management terminal The terminal supports Linux and needs to be equipped with the following devices 1 LAN port Intel Broadcom and Realtek suppo...

Страница 32: ...ssword accessible to CSC modules and server modules 4 The following message is displayed in the Management Tool window 5 On the top of the screen select Application System Tools Terminal and start a new window 6 Execute the following command in the window started above 3 7 Connect the management terminal to this sever with the LAN cable 8 On completion of searching CSC modules and server modules i...

Страница 33: ...erver modules 3 Enter list again after a few minutes If Type of CPU Board is TeDoLi and Status is Run CSC modules server modules have been diagnosed MNG PC list Module List Power Type Status CSC ON TP CPU Board1 ON MNG PC CPU Board2 OFF CPU Board3 OFF CPU Board43 OFF CPU Board44 OFF CSC 1 CPU 43 MNG PC PASS 0 PASS 0 FAIL 0 FAIL 0 MNG PC MNG PC start tedoli all MNG PC MNG PC list Module List Power ...

Страница 34: ...tart of execution Enter list again and check that Status of server modules is PASS TP Diagnosis program of CSC modules is infinitely executed out of synchronization with TeDoLi Enter stop 0 to stop it 4 If you find Fail in Status in step 3 specify the module with list server module number and check the detail Example When you find the server module 23 to be Fail enter list 23 MNG PC list Module Li...

Страница 35: ...g example the log is saved in the log2015 folder of the USB memory 1 The log file is saved in the tar gz format Extract it to check the content MNG PC list 23 Please wait a moment CPU Board 023 List Num Device List Type Prev Status 1 MEMORY MEMORY normal end 2 CPU CPU normal end 3 SATA SATA 4 HDD Slot0 TOSHIBA THNSNJ256G8NU HDD INT fail 5 NIC NIC 6 Port1 eth0 LOM PORT 7 Port2 eth1 LOM PORT 8 SENSO...

Страница 36: ...d MNG PC shutdown f all 8 Enter list at the management tool prompt MNG PC and check that all server modules are turned OFF 9 On the top of the screen select System Shut Down and press the Shut Down button in the displayed dialog box to shut down the system When you restart the system note that the system diagnostic management tool may not find server modules 10 Pull out the AC cable MNG PC shutdow...

Страница 37: ... Description start Starts diagnosis stop Stops diagnosis shutdown Shuts down the server module list Displays the status of modules log Displays the diagnosis log or saves it in a USB memory sol Displays the server module screen on the system diagnostic management tool smart Displays the s m a r t information of server modules reload Reloads the server module information ver Displays the version of...

Страница 38: ...server module to start the tool all CSC modules of this server and all server modules are specified x Specify the server module number Set 0 to specify a CSC module You can specify multiple server modules Example 1 6 23 36 Server modules 1 6 to 23 and 36 have been specified Screen image Description number MNG PC start tedoli all ENTER MNG PC list ENTER Module List Power Type Status CSC ON TP Run C...

Страница 39: ...his server and all server modules are specified x Specify the server module number Set 0 to specify a CSC module You can specify multiple server modules Example 1 6 23 36 Server modules 1 6 to 23 and 36 have been specified Screen image Description number MNG PC stop all ENTER MNG PC list ENTER Module List Power Type Status CSC ON TP PASS CPU Board1 ON MNG PC CPU Board2 ON TeDoLi PASS CPU Board3 ON...

Страница 40: ...r module forcibly from BMC arg2 cpu board Specify the server module to be shut down all Specify all server modules of this server x Specify the server module number You can specify multiple server modules Example 1 6 23 36 Server modules 1 6 to 23 and 36 are specified Screen image Description number MNG PC shutdown all ENTER MNG PC list ENTER Module List Power Type Status CSC ON TP PASS CPU Board1...

Страница 41: ... of this server and all server modules x Specify the server module number Set 0 to specify a CSC module You can specify multiple server modules Example 1 6 23 36 Server modules 1 6 to 23 and 36 are specified Screen image Description number MNG PC list ENTER Module List Power Type Status CSC ON TP Run CPU Board1 ON MNG PC CPU Board2 ON TeDoLi Run CPU Board3 ON TeDoLi Run CPU Board43 ON TeDoLi Run C...

Страница 42: ... Indicates the server module number loaded position CSC Indicates CSC modules CPU Board Indicates server modules Power ON Indicates DC ON status OFF Indicates DC OFF status Type Indicates the tool type blank Indicates that the tool is not started TP Indicates the diagnosis module of CSC MNG PC Indicates the system diagnostic management tool TeDoLi Indicates the diagnostic tool TeDoLi Status Indica...

Страница 43: ...odules whose diagnostic result was abnormal The display includes ABRT WARN MNG PC Indicates that the system diagnostic management tool is operating on the Deploy server module This is not displayed when the tool is operating on a management terminal c The time of the server module is displayed Time Indicates the time of the server module d Statuses of CSC modules and server module devices are disp...

Страница 44: ...er and all server modules x Specify the server module number Set 0 to specify a CSC module You can specify multiple server modules Example 1 6 23 36 Server modules 1 6 23 and 36 have been specified Screen image Description number MNG PC log save all ENTER Select the device to save log 1 USB Flash Disk Enter selection 1 ENTER a Enter directory name tedoli ENTER b Please wait a moment Now saving All...

Страница 45: ...ce Guide 45 Chapter 1 Maintenance Description Description number Description a Specify the target to collect log b Enter a directory name to create a directory to save the data Enter a space default to create no directory c A message to indicate successful log saving ...

Страница 46: ...the server module to be displayed all Specify all server modules of this server x Specify the server module number You can specify multiple server modules Example 1 6 23 36 Server modules 1 6 to 23 and 36 are specified Screen image Description number MNG PC sol 2 ENTER a MNG PC Description Description number Description a Open a window on the system diagnostic management tool to display the consol...

Страница 47: ...4 linux 2 6 32 358 14 1 el6 x86_64 local build Copyright C 2002 12 by Bruce Allen http smartmontools sourceforge net START OF INFORMATION SECTION Device Model SAMSUNG MZNLN128HCGR 00000 Serial Number S1ZXNXAG603879 LU WWN Device Id 5 002538 d401eb1a7 Firmware Version EMT2100Q User Capacity 128 035 676 160 bytes 128 GB Sector Size 512 bytes logical physical Device is Not in smartctl database for de...

Страница 48: ...NG PC reload ENTER Please wait Searching MNG PC ver Function Display the version of the system diagnostic management tool Screen image Descripti on number MNG PC ver ENTER Release Version MM20 00 11 Build Date Dec 11 2015 12 03 53 Rootfs Version 16 01 00 a Description Description number Description a The version of the system diagnostic management tool and the build date are displayed ...

Страница 49: ... TeDoLi 1 Console terminal A console terminal is required to display screens of server modules to execute TeDoLi A console terminal is conditional on having at least one LAN port and connecting to Web console with BMC of the server module to execute TeDoLi by Internet Explorer Connect LAN to the dedicated LAN connector of NW switch module 1 of this server The IP address of BMC is required to conne...

Страница 50: ... When you have connected a console terminal to this server start the tool in the following procedure 1 Insert the DX2000 Utility Disk into the console terminal 2 Specify the IP address on the console terminal and connect BMC of the server module to execute TeDoLi 3 Login Default User Administrator Password Administrator ...

Страница 51: ...le Modular Server DX2000 Maintenance Guide 51 Chapter 1 Maintenance 4 Use the RemoteKVM function to display the server module screen to execute TeDoLi 5 Select Enabled on the menu to enable the remote device of the RemoteKVM function ...

Страница 52: ...6 System Diagnostic Tool Scalable Modular Server DX2000 Maintenance Guide 52 Chapter 1 Maintenance 6 Select Remote Media of the RemoteKVM function 7 Press the Browse button of CD DVD of the remote media ...

Страница 53: ...stic Tool Scalable Modular Server DX2000 Maintenance Guide 53 Chapter 1 Maintenance 8 Select the tedoli iso file in the diagnosis folder on the DX2000 Utility Disk 9 Connect the ISO image of TeDoLi by the remote media ...

Страница 54: ...save the log to the remote media connect USB Mem to save the log before ISO image of TeDoLi is started Note that you can select the drive if you mount the media such as USB memory drives to save the log on the console terminal machine before RemoteKVM is started 11 Close the setting window of the remote media ...

Страница 55: ...ance Guide 55 Chapter 1 Maintenance 12 Press the power button to turn on the power of the server module 13 Wait for several minutes until TeDoLi starts When TeDoLi starts the following message appears in the RemoteKVM window W e l c o m e t o T e D o L I SXX XX XX ...

Страница 56: ...heck that the device configuration is correct and that there is no unrecognized device Device confoguration information screen System diagnosis 6 2 4 Decide whether or not to undertake system diagnostic evaluation Select Execute to execute system diagnosis Then move on to step 1 Select Cancel to ignore system diagnosis Then move on to Chapter 1 6 2 5 Saving log System diagnostic execution screen P...

Страница 57: ...xecution Time Limit 8 minutes ElapsedTime 000 00 29 MEMORY Run CPU Run HDD0 Run SENSOR Run SENSOR Run ElapsedTime 000 01 29 MEMORY Run CPU Run HDD0 Run SENSOR Run SENSOR Run 1 Set the system diagnosis time Enter time to change the displayed default value System diagnosis time setting screen 2 System diagnosis starts System diagnosis status is displayed in every one minute System diagnosis status s...

Страница 58: ...y if you find FAIL System diagnosis result check screen Completed all the tests Please confirm the execution results Num Device List Simple Status round 1 MEMORY MEMORY PASS 213 2 CPU CPU PASS 43 3 SATA 4 HDD Slot0 SAMSUNG MZNLN256HCHP HDD0 PASS 4086 5 NIC 6 Port1 eth0 NIC Port 7 Port2 eth1 NIC Port 8 Slot2 9 Port1 eth2 Slot2 Po 10 Port2 eth3 Slot2 Po 11 SENSOR TEMPERATURE SENSOR T PASS 45 12 SENS...

Страница 59: ... collect the result of the system diagnosis and check the life of SAS SSD Select a destination device to save the log in a removable disk If you save the log in the remote media connected at the startup the device name is shown as Remote USB MEM Select an output console to save the log using the terminal log function Destination to save log selection screen 1 When you select save xxx You can speci...

Страница 60: ...Attempt to retry the test 2 Reboot 3 Shutdown Enter selection Shutdown 3 2 When you select display serial Start saving the log according to the direction after having set the automatic log collection function of the displayed terminal software Screen to specify terminal output Move on to Chapter 1 6 2 6 Quitting system diagnosis after having saved the log Quitting system diagnosis 6 2 6 Quit the s...

Страница 61: ...EVICE INFORMATION SAS SSD device information in the file Use the search key SSD Life Level An example of displaying the device information of SAS SSD is shown below The meaning of the life information level is shown in the table below Level Life Meaning 5 100 51 Safe 4 50 21 Live 3 20 11 Coming to the end 2 10 6 Replacement required 1 5 0 Dead Host 0 Channel 0 SCSI ID 0 Lun 0 Vendor Name ATA Model...

Страница 62: ...sole terminal by connecting the console terminal and a UART port on the front side In this case you need a USB UART Bridge driver Install it on the console terminal from the DX2000 Utility Disk in advance This section describes the procedure to confirm connection using Tera Term on Windows OS Prepare remote logon client software to fit your environment Driver storage path of DX2000 Utility Disk US...

Страница 63: ...ps Data 8 bit Parity none Stop 1 bit Flow control none 2 To connect Deploy server modules select Silicon Labs Quad CP210x USB to UART Bridge Interface 0 COMxx 3 When you start Deploy server modules the POST progress status is displayed on the Tera Term screen as shown below Press the F4 key during POST and select ROM Utility Use a keyboard connected to Deploy server modules ...

Страница 64: ... 4 ROM Utility starts and displays the Keyboard selection screen at the beginning Select a keyboard type 5 Change the BMC network setting from the Server Configuration Utility Menu Select the menu as follows Server Configuration Utility Menu EXPRESSSCOPE Engine 3 Configuration Network IPv4 Property ...

Страница 65: ...e setting 7 After the setting is reflected confirm the connection to BMC 2 Redisplaying the Management Tool window If you have closed the Management Tool window by mistake you can redisplay it in the following steps 1 On the system diagnostic management tool select Applications System tools and Terminal on the top of the screen and start a local terminal 2 The following command redisplays the Mana...

Страница 66: ...below 4 Network testing with the diagnostic tool This section describes how to test a network with the system diagnostic management tool Network test is disabled by default Change the following configuration file 1 On the system diagnostic management tool select Applications System tools and Terminal on the top of the screen to start a terminal 2 Open the configuration file by using the following ...

Страница 67: ...ostic test tool performs data communication to LAN 1 and LAN 2 of the server module Therefore you need to connect NW switch modules with the cable and enable communication between LAN 1 and LAN 2 of the server module 5 Conduct a diagnosis test Set Flow Control to ON of all QSFP ports for NW switch module setting to conduct a network test in multiple server modules at a time See Configuration guide...

Страница 68: ...You can check the information of modules implemented on this server You can turn on off the UID LED on each module installed on this server 7 1 Server Management Utility Use the Server Management Utility to set manage and maintain this server We recommend that you permanently set a PC on which the Server Management Utility can be executed described as server management terminal here Execute the Se...

Страница 69: ...perates on CentOS6 4 x86_64 Install the ipmitool Version 1 6 0 or later on a server management terminal Allocate an IP address that can communicate with CSC server modules to the IP address of a server management terminal and confirm that the server management terminal can communicate with this server The following figure is an example of connection when using Server Management Utility 1 Server Ma...

Страница 70: ...ment utility 7 1 3 After installing the management utility execute the mng_util command at the installation directory and start the management utility When executing the mng_util command specify the username and password that has the permission to access CSC modules and server modules If the username and password are omitted default settings username Mainge password Mainte is used Command Table 7 ...

Страница 71: ...esses of the LAN for the modules that were found are grouped by chassis Format search IP address Low IP address High search 192 168 0 01 192 168 0 254 Chassis serial 123456789012 Board ManagementLAN MAC IP DataLAN1 MAC DataLAN2 MAC CSC 94 DE 80 AB 01 91 192 168 0 01 LAN SW1 94 DE 80 AB 02 A8 192 168 0 02 LAN SW2 94 DE 80 AB 02 A9 192 168 0 03 CPU Board1 94 DE 80 AB 00 34 192 168 0 11 94 DE 80 AB 0...

Страница 72: ...ses are contained in the search range there is a possibility that the target CSC modules or server modules may not be found In such a case change the settings in the server management terminal to set the sufficient value for the maximum number of ARP cache entries for the server management terminal search OWN Chassis serial 123456789012 Board ManagementLAN MAC IP DataLAN1 MAC DataLAN2 MAC CSC 94 D...

Страница 73: ...CSC modules found by search commands IP address IP address of the CSC module Information displayed by implemented modules Chassis serial Serial number of the chassis on which the CSC module specified by IP address is mounted Slot Slot number in the chassis Module Type Implemented module type N Code A part of FRU data of the server module slotlist I 192 168 0 13 Chassis serial 123456789012 Slot Mod...

Страница 74: ... of the management LAN IP IP address of the management LAN DataLAN1 MAC MAC address of NW switch module 1 in each server module DataLAN2 MAC MAC address of NW switch module 2 in each server module niclist I all C all Chassis serial 123456789012 Board ManagementLAN MAC IP DataLAN1 MAC DataLAN2 MAC CSC 94 DE 80 AB 01 91 192 168 0 1 LAN SW1 94 DE 80 AB 02 A8 192 168 0 2 LAN SW2 94 DE 80 AB 02 A9 192 ...

Страница 75: ...ision of BMC of the CSC module and each server module System GUID GUID included in FRU of the CSC module and each server module FRU data managed by each module for subsequent lines after that beginning with CSC frulist I 192 168 0 11 C all Chassis serial 123456789012 Board Firmware Revision System GUID CSC 1 8 0006ce2c 0f6a e311 0180 94de80aafdee CPU Board1 1 8 001280c9 ca2e e011 0180 94de80ab02d8...

Страница 76: ...ress IP address of the CSC module C module number all CSC module all server modules number CSC module 0 server modules 1 44 UID LED Light Off uidoff 7 1 10 UID LED of the CSC module and server modules turn off Format uidoff I CSC module C module number options I CSC module all All CSC modules found by search commands IP address IP address of the CSC module C module number all CSC module all server...

Страница 77: ...t destination file path name Output file sample savelist I all f tmp aaa csv Chassis serial Board ManagementLAN MAC ManagementLAN IP DataLAN1 MAC DataLAN2 MAC 123456789012 CSC 94 DE 80 AB 01 91 192 168 0 01 123456789012 LAN SW1 94 DE 80 AB 02 A8 192 168 0 02 123456789012 LAN SW2 94 DE 80 AB 02 A9 192 168 0 03 123456789012 CPU Board1 94 DE 80 AB 00 34 192 168 0 11 94 DE 80 AB 03 D2 94 DE 80 AB 03 D...

Страница 78: ...e BB Slot number 00 CSC and NW switch module 01 44 server module C Identification number of LAN See below for CSC and NW switch module 1 CSC module 2 NW switch module 1 3 NW switch module 2 See below for Server module 1 BMC of the server module 2 LAN of NW switch module 1 in each server module 3 LAN of NW switch module 2 in each server module savelist I all f tmp aaa conf dhcpconf host client01001...

Страница 79: ...7 Server Management Utility Scalable Modular Server DX2000 Maintenance Guide 79 Chapter 1 Maintenance Quit quit 7 1 12 Quit the management utility Format quit quit ...

Страница 80: ...Scalable Modular Server DX2000 Useful Features This chapter describes useful features for using the server Refer to this chapter according to your purpose and need 1 Server Module BIOS Describes how to configure the server module BIOS settings and parameters ...

Страница 81: ... 1 1 Starting SETUP Refer to Chapter 3 Setup in DX2000 User s Guide and turn on the server module and then proceed with POST Wait until the following message appears at the bottom left of the screen Press F2 SETUP If you press F2 SETUP will start after POST and the Main menu appears You can also start SETUP by pressing F2 while expanding option ROM 1 2 Parameter Descriptions The SETUP utility has ...

Страница 82: ...nistrator is displayed UEFI Spec Version Information only UEFI version supported by this server Memory Information Total Memory Information only The total capacity of installed memory is displayed System Date WWW MM DD YYYY Set the system date System Time HH MM SS Set the system time Factory setting Tips Be sure to confirm that the date and time are correctly configured Check the system clock mont...

Страница 83: ...the cursor to Advanced the Advanced menu appears For the menus that show a to their left select a menu and press the Enter key to display its submenu When Boot Mode is UEFI When Boot Mode is Legacy Note You cannot select the Network Stack Configuration submenu and the UEFI Driver Configuration submenu when Boot Mode is Legacy ...

Страница 84: ...r 2 Useful Features 1 Processor Configuration submenu From the Advanced menu select Processor Configuration and then press the Enter key to display the menu screen shown below For the menu that has on the left move the cursor to it and then press the Enter key to show its submenus ...

Страница 85: ...cute Disable Bit Disabled Enabled Enable or disable Execute Disable Bit feature This option is displayed only when the installed processor supports this feature VT x Disabled Enabled Enable or disable Intel Virtualization Technology feature to virtualize processor Active Processor Cores 1 X Specify the number of cores to enable in each processor package The number of cores that can be specified de...

Страница 86: ...ffective cores in Processor 1 Max Cores Information Only The maximum number of cores in Processor 1 L2 Cache RAM Information Only The secondary cache size of Processor 1 L3 Cache RAM Information Only The tertiary cache size of Processor 1 Microcode Revision Information Only The revision of the microcode applied to Processor 1 is displayed EMT64 Information Only Supported is displayed when the proc...

Страница 87: ...1 Server Module BIOS Scalable Modular Server DX2000 Maintenance Guide 87 Chapter 2 Useful Features b Processor Power Management submenu ...

Страница 88: ...ced Power Power Set a ratio to decide whether a priority is given to the performance or the power saving in the operation of the processor This item is displayed when you set EIST to Enabled Turbo Boost Disabled Enabled Enable or disable the Turbo Boost Technology feature This option is displayed only when the installed processor supports this feature Energy Efficient Turbo Disabled Enabled Enable...

Страница 89: ...g POST When it is Enabled and an error is detected during memory test the corresponding memory resource is reduced Memory Retest No Yes Set Yes to clear the error information in the memory and reconfigure all memories at next POST This parameter automatically returns to No after memories are reconfigured Memory Frequency Limit Auto 1600 MHz 1866 MH Specify the upper limit of operating frequency of...

Страница 90: ...ed memory is displayed Current Memory Speed Information Only The current memory operating frequency is displayed DIMM1 4 Status Information Only The status of each DIMM is indicated as follows Value Normal The value indicates the amount of memory Value Error In trouble the memory is effective The value indicates the amount of memory Disabled In trouble The memory is reduced Not Present Not impleme...

Страница 91: ...ink Speed Settings AER Support Disabled Enabled Enable or disable the function to control PCIe Advanced Error Reporting from OS Onboard LAN VGA is precluded from this function Above 4G Decoding Disabled Enabled Enable or disable the function to map memory in the address space exceeding 64GB for 64bit PCIe devices PCIe Completion Timeout 50 µsec to 10 msec 16 msec to 55 msec 65 msec to 210 msec 260...

Страница 92: ...bled when there is no valid VGA controller except those onboard and Serial Redirection is disabled VGA Priority Offboard Onboard Set a priority of the option ROM expansion of the VGA controller LAN1 2 Option ROM Scan Disabled Enabled Enable or disable the option ROM expansion of the onboard LAN This item is displayed according to the number of LAN ports Disabled is assumed except LAN1 PCI1 Slot Op...

Страница 93: ...rameter Description Target Link Speed Limit Auto 2 5GT s 5 0GT s Set the upper limit of the Link speed of onboard PCI devices and PCI devices connected to PCI slots PCI1 Slot Link Speed Limit 2 5GT s 5 0GT s 8 0GT s Set the upper limit of the Link speed of PCI slots in the range of the Link speed set by Target Link Speed Limit Factory setting ...

Страница 94: ...enu screen as shown below Option Parameter Description VT d Disabled Enabled Enable or disable Intel Virtualization Technology for Directed I O I O virtualization support function This is displayed only when a processor supporting this function is mounted I OAT Disabled Enabled Enable or disable Intel I O Acceleration Technology Wake On LAN Disabled Enabled Enable or disable the feature that remot...

Страница 95: ...om the Advanced menu select Storage Configuration and then press the Enter key to display the menu screen shown below For the menu that has on the left move the cursor to it and then press the Enter key to show its submenus Option Parameter Description SATA Information SATA Mode Information only SATA Mode is displayed Factory setting ...

Страница 96: ...X2000 Maintenance Guide 96 Chapter 2 Useful Features a SATA Information submenu Option Parameter Description SATA Port0 Information Only Each device connected to each port is displayed Not Present is displayed for those unconnected Factory setting ...

Страница 97: ...nced menu and press Enter key The following screen is displayed Option Parameter Description Device Reset Timeout 10 sec 20 sec 30 sec 40 sec Set a timeout time to issue the Start Unit command to a USB device Controller Timeout 1 sec 5 sec 10 sec 20 sec Set a timeout time to issue Control Bulk and Interrupt Transfer commands to the USB controller Factory setting ...

Страница 98: ...r DX2000 Maintenance Guide 98 Chapter 2 Useful Features 7 Serial Port Configuration submenu Select Serial Port Configuration in the Advanced menu and press Enter key The following screen is displayed See the table in the next page for each item ...

Страница 99: ...edirection function of the serial port Set Serial Port B to use direct connection using a terminal such as ESMPRO ServerManager Set the screen size of a terminal to 80x25 The following connection set items are displayed Terminal Type VT100 VT UTF8 PC ANSI Select a type of the terminal Baud Rate 9600 19200 57600 115200 Set the baud rate Data Bits 7 8 Set a bit width of the data Parity None Even Odd...

Страница 100: ...e Advanced menu and press Enter key The following screen is displayed Option Parameter Description Network Stack Disabled Enabled Enable or disable UEFI network stack Enable this function to display following menu items PXE Configuration Network Mode Disabled IPv4 IPv6 Set PXE network mode Media Detect Count 1 50 Set a count to try detecting media to connect PXE factory setting ...

Страница 101: ... Chapter 2 Useful Features 9 UEFI Driver Configuration submenu Option Parameter Description UEFI Driver Name This item is displayed when the onboard LAN controller or the UEFI driver of each PCI device is loaded This submenu varies depending on the UEFI driver Driver Health ...

Страница 102: ...res a Driver Health submenu Option Parameter Description UEFI Driver Name Information only The status of UEFI Driver Health is displayed This item is displayed when the onboard LAN controller or the UEFI driver of each PCI device is loaded and when the UEFI driver is compatible with Driver Health ...

Страница 103: ...the menu that has on the left move the cursor to it and then press the Enter key to show its submenus Then configure the settings Select Administrator Password or User Password and then press the Enter key to display the screen where you can register change the password Tips Set Administrator Password first before setting User Password Do not set any password before installing OS ...

Страница 104: ...ccess all SETUP menus Password can be set only when SETUP is started by administrative right If no password is set SETUP starts with administrative right User Password 3 to 20 alphanumeric characters When the Enter key is pressed the password entry screen to set user right is displayed With this password access to SETUP menus is limited The user password can be set when SETUP is started by adminis...

Страница 105: ...rver DX2000 Maintenance Guide 105 Chapter 2 Useful Features Server 1 2 4 If you move the cursor to Server the Server menu screen appears For the menu that has on the left move the cursor to it and then press the Enter key to show its submenus ...

Страница 106: ...bled 5 60 minutes Enabler or disable the boot monitoring function and set the timeout time Install ESMPRO ServerAgentService to use this function Set this function to Disabled if ESMPRO ServerAgentService is not installed Boot Monitoring Policy Retry 3 times Always Reset Set the operation to automatically reset this server when a timeout occurs during boot monitoring Set Retry 3times to try starti...

Страница 107: ...number of the system System Serial Number Information Only Serial number of the system Chassis Part Number Information Only Part number of the chassis Chassis Serial Number Information Only Serial number of the chassis Onboard LAN1 MAC Information Only MAC address of the standard LAN1 Onboard LAN2 MAC Information Only MAC address of the standard LAN2 Management LAN MAC Information Only MAC address...

Страница 108: ...with random value for the waiting time until the DC power is turned on Set Last State or Power On to AC LINK to select this item Delay Time 50 600 Set a waiting time in seconds The minimum configurable time may be changed from the factory setting Set Manually Setting to Power On Delay Time Mode to select this item The table below shows AC LINK setting and the DC power operation when the power is s...

Страница 109: ...ad of the logo When BIOS Redirection Port is enabled Unavailable is displayed You cannot change the setting automatic operation in Disabled Bootup NumLock State On Off Enable or disable NumLock of the keyboard Setup Prompt Timeout 0 65535 Set F2 key input waiting time to start SETUP from 0 to 65535 seconds Boot Mode Legacy UEFI Set a boot mode Set UEF1 to this item for the following OS ー Red Hat E...

Страница 110: ...ems display the priority of boot devices If all of the Boot Options are set to Disabled the SETUP starts running upon completion of POST When Boot Mode is changed devices are displayed after rebooting Boot Option 2 Boot Option 3 Boot Option 4 CD DVD Priorities Set a starting priority for each device type Removable Priorities HDD Priorities Network Priorities Factory setting ...

Страница 111: ...y device of the next device type For example if there are two priorities in each device type Boot Option and priorities of device types are as follows Boot Option and priorities of device types 1 to 8 1 Boot Option 1 CD DVD CD DVD Priorities Boot Option 1 CD DVD device 1 1 CD DVD Priorities Boot Option 2 CD DVD device 2 2 2 Boot Option 2 Removable Removable Priorities Boot Option 1 Removable devic...

Страница 112: ...lows AMI Remote CD DVD devices are registered as the highest priority devices in CD DVD Priorities Execute Load Setup Defaults in Save Exit menu to register AMI Remote CD DVD devices as the highest priority devices in CD DVD Priorities 1 See EXPRESSSCOPE Engine 3 User s Guide for the detail of the remote management extended license Tips To use remote media connect remote KVM enable the remote devi...

Страница 113: ...tartup of SETUP utility is retained After the SETUP utility closes the system automatically reboots c Save Changes and Power Off The utility closes with all the changes saved in NVRAM Then the server automatically turns off the power d Discard Changes and Power Off The utility closes without saving changes in NVRAM BIOS setting at starting the utility is inherited After closing the server automati...

Страница 114: ... Appendix 1 POST Error Message List of error messages detected by the self diagnostic function POST 2 BMC Error Messages List of error messages detected by BMC 3 Error Messages of CSC Modules List of error messages detected by CSC modules 4 Revision Record Revison history of this document ...

Страница 115: ...nance personnel please take note of the content of the display The list only contains messages for the server For details about error messages of optional devices and the actions to take refer to the instructions that come with each product System Monitoring Check Passed ERROR AE81 DIMM1 with error is enabled AE01 DIMM1 has been disabled AE02 DIMM2 has been disabled Press F2 to setup Message indic...

Страница 116: ...you have replaced the lithium battery 8007 SETUP Menu Password cleared by Jumper SETUP password is cleared by Jumper Start SETUP and set each item again If the problem persists contact a maintenance service company 8020 BIOS update error BIOS update failed Update BIOS again If you find the same error continually detected contact a maintenance service company 8800 DXE_NB_ERROR An error was detected...

Страница 117: ...mory error was detected in DIMM4 As all memories are disabled it was forcibly enabled B000 Expansion ROM not initialized Failed to expand option ROM Disable option ROM expansion for the option board on which OS is not started Start SETUP and disable PCI Configuration PCI Device Controller and Option ROM Settings PCIxx Slot Option ROM in the Advanced menu xx PCI slot number B001 Expansion ROM not i...

Страница 118: ...C10F IPMB signal lines do not respond Failure of Satellite Management Controller was detected C110 BMC FRU device failure FRU storing device information corrupted C111 BMC SDR Repository failure Failure was detected in SROM that stores the SDR C112 BMC SEL device failure BMC SEL device failure C113 BMC RAM test error BMC RAM error was detected C114 BMC Fatal hardware error A hardware error was det...

Страница 119: ...update mode C300 Out of band setup configuration failure Access to the BMC Mezzanine card failed Remove the server module wait for 30 seconds implement it again and start it If you find the same error continually detected contact a maintenance service company C302 Hardware configuration error The hardware configuration of this server is incorrect Contact a maintenance service company C501 ME is in...

Страница 120: ...BIOS Rev YYYY Displayed during POST XX indicates the POST code in execution YYYY indicates the version of the system BIOS POST Completed Successfully POST is successfully completed POST ERROR XXXX Error XXXX was detected during POST Check the error message displayed on LCD and take an action No Available Memory in System No available memory is implemented Check if memory is correctly implemented E...

Страница 121: ... is not an error Wait until BIOS has been recovered BIOS Updater Running Updating BIOS This is not an error Wait until BIOS has been updated PCI Bus System Error 1 A system error of the PCI bus was detected Contact a maintenance service company PCI Bus Parity Error 1 A parity error of the PCI bus was detected CPUx_DIMMxCorrectab le Error Recoverable errors of the memory were detected frequently ...

Страница 122: ...sages You can check error messages on the virtual LCD displayed in the Web browser of EXPRESSSCOPE engine 3 See EXPRESSSCOPE Engine 3 User s guide for the virtual LCD BMC is used for the server and the chassis The table below shows error messages meaning and actions BMC message displayed on the bottom of LCD ...

Страница 123: ...ure1 Lower Critical Baseboard Temperature1 Upper Critical Ambient Temperature Lower Non Critical Ambient Temperature Upper Non Critical Ambient Temperature Lower Critical Ambient Temperature Upper Critical DIMM Area Temperature Lower Non Critical DIMM Area Temperature Upper Non Critical DIMM Area Temperature Lower Critical DIMM Area Temperature Upper Critical Processor Thermal Control Upper Non Cr...

Страница 124: ...process IPMI Watchdog timer timeout Power off Timeout of the watchdog timer was detected Node Manager Firmware Image execution Failed Node Manager failed Pull out the power cord wait at least 30 seconds and start it again If the problem persists contact a maintenance service company Node Manager Firmware Flash Erase Error Node Manager Firmware Flash Corrupted Node Manager Internal Error Node Manag...

Страница 125: ... VBAT Lower Critical VBAT Upper Critical Baseboard Temperature Lower Non Critical Temperature error detected The fan may be faulty or clogged Contact a maintenance service company Baseboard Temperature Upper Non Critical Baseboard Temperature Lower Critical Baseboard Temperature Upper Critical Baseboard Temperature1 Lower Non Critical Baseboard Temperature1 Upper Non Critical Baseboard Temperature...

Страница 126: ...tected The power supply unit failed Check if the power cord is connected If the problem persists contact a maintenance service company Power Supply1 Failure detected Power Supply2 Failure detected Power Supply AC Lost Power Supply1 AC Lost Power Supply2 AC Lost Power Supply Predictive Failure detected Power Supply1 Predictive Failure detected Power Supply2 Predictive Failure detected Sensor Failur...

Страница 127: ...wer Non Critical The rotating speed of FAN2 is lower than the threshold System FAN1 Lower Non Critical The rotating speed of FAN1 is lower than the threshold Power Supply2 Failure detected PSU2 failed Replace the PSU Contact a maintenance service company Power Supply1 Failure detected PSU1 failed Power Supply Failure detected PSU3 or PSU DC failed Power Supply2 Temperature Lower Non Critical The t...

Страница 128: ...re and find no problem replace the PSU Contact a maintenance service company Power Supply1 Temperature Lower Critical The temperature of PSU1 is lower than the threshold Power Supply Temperature Lower Critical The temperature of PSU3 is lower than the threshold Power Supply2 Temperature Upper Critical The temperature of PSU2 is higher than the threshold Check the environment temperature and the du...

Страница 129: ... the threshold Replace the chassis Contact a maintenance service company BaseBoard Voltage Upper Critical The sensor on the baseboard detected the voltage over the threshold Processor2 Voltage Lower Critical The sensor of the CPU of on LAN SW 2 detected the voltage below the threshold Replace the LAN SW Contact a maintenance service company Processor2 Voltage Upper Critical The sensor of the CPU o...

Страница 130: ...ed to Chapter 1 Notes are added to Section 6 1 System Diagnostic Management Tool in Chapter 1 Descriptions are added to step 8 of 2 Starting the system diagnostic management tool of Section 6 1 1 Starting in the server in Chapter 1 Important notes are added to step 3 and 7 of Section 6 1 3 Operating the system diagnostic management tool in Chapter 1 Step 10 is added to Section 6 2 2 Starting TeDoL...

Страница 131: ... MEMO ...

Страница 132: ...2000 Maintenance Guide April 2016 NEC Corporation 7 1 Shiba 5 Chome Minato Ku Tokyo 108 8001 Japan NEC Corporation 2016 The contents of this manual may not be copied or altered without the prior written permission of NEC Corporation ...

Отзывы:

Нет отзывов

Похожие инструкции для DX2000

Бренд: Gateway Страницы: 138

Бренд: Supero Страницы: 112

SuperServer 5039MD18-H8TNR

Бренд: Supermicro Страницы: 134

SUPERSERVER 5018D-FN8T

Бренд: Supermicro Страницы: 124

Бренд: Yawarra Страницы: 4

Бренд: Sun Microsystems Страницы: 12

LaserJet 4200/n

Бренд: HP Страницы: 28

Jetdirect 2700w

Бренд: HP Страницы: 43

Jetdirect 2900nw

Бренд: HP Страницы: 20

LaserJet 4200/n

Бренд: HP Страницы: 36

wp110 - 802.11b Wireless Print Server

Бренд: HP Страницы: 35

JetDirect Ew2500

Бренд: HP Страницы: 2

Бренд: HP Страницы: 18

Бренд: HP Страницы: 25

Jetdirect J7974E

Бренд: HP Страницы: 186

Бренд: HP Страницы: 223

Бренд: HP Страницы: 228

Jetdirect J7949e

Бренд: HP Страницы: 232

Бренды по названию

0 1 2 3 4 5 6 7 8 9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Популярные бренды

Загрузить еще бренды