Intel BX80571E7500 - Core 2 Duo 2.93 GHz Processor Скачать руководство пользователя страница 38 | Manualshive

Страница: 38 / 146

background image

I

NTEL

® X

EON

® P

ROCESSOR

7500 S

ERIES

U

NCORE

P

ROGRAMMING

G

UIDE

UNCORE PERFORMANCE MONITORING

2-26

SINKS_S2C

• Title: S2C Sinks
• Category: Ring - WIR
• Event Code: 0x07, Max. Inc/Cyc: 3,
• Definition: Number of messages sunk from the ring at the C-Box that were sent by the S-Box.
• NOTE: Each sink represents the transfer of 32 bytes, or 2 sinks per cache line.

SINKS_S2P_BL

• Title: S2P Sinks
• Category:  Ring - WIRa
• Event Code: 0x08, Max. Inc/Cyc: 1,
• Definition:  Number BL ring messages sunk by the processor that were sent from the S-Box. This cov-

ers BL only, because that is the only kind of message the S-Box can send to a processor.

• NOTE: Each sink represents the transfer of 32 bytes, or 2 sinks per cache line.

SNP_HITS

• Title: Snoop Hits in LLC
• Category: Local - CC
• Event Code: 0x28, Max. Inc/Cyc: 1,
• Definition: Number of Intel QPI snoops that hit in the LLC according to state of LLC when the hit

occurred. GotoS: LLC Data or Code Read Snoop Hit ‘x’ state in remote cache. GotoI: LLC Data Read
for Ownership Snoop Hit ‘x’ state in remote cache.

Extension

umask

[15:8]

Description

---

b00000000

(*nothing will be counted*)

AD

b00000001

AD (Intel QPI snoop request of LLC)

AK

b00000010

AK (Intel QPI completions sent to LLC)

BL

b00000100

BL (Data Fills sent to the LLC in response to RD requests)

Extension

umask

[15:8]

Description

---

b00000000

(*nothing will be counted*)

REMOTE_RD_HITM

bxxxxxxx1

Intel QPI SnpData or SnpCode hit M line in LLC

REMOTE_RD_HITE

bxxxxxx1x

Intel QPI SnpData or SnpCode hit E line in LLC

REMOTE_RD_HITS

bxxxxx1xx

Intel QPI SnpData or SnpCode hit S line in LLC

REMOTE_RD_HITF

bxxxx1xxx

Intel QPI SnpData or SnpCode hit F line in LLC

REMOTE_RFO_HITM

bxxx1xxxx

Intel QPI SnpInvOwn or SnpInvItoE hit M line in LLC

REMOTE_RFO_HITE

bxx1xxxxx

Intel QPI SnpInvOwn or SnpInvItoE hit E line in LLC

REMOTE_RFO_HITS

bx1xxxxxx

Intel QPI SnpInvOwn or SnpInvItoE hit S line in LLC

REMOTE_RFO_HITF

b1xxxxxxx

Intel QPI SnpInvOwn or SnpInvItoE hit F line in LLC

REMOTE_HITM

bxxx1xxx1

Intel QPI Snoops that hit M line in LLC

REMOTE_HITE

bxx1xxx1x

Intel QPI Snoops that hit E line in LLC

REMOTE_HITS

bx1xxx1xx

Intel QPI Snoops that hit S line in LLC

REMOTE_HITF

b1xxx1xxx

Intel QPI Snoops that hit F line in LLC

REMOTE_ANY

b11111111

Intel QPI Snoops that hit in LLC (any line state)

«
...
36
37
38
39
40
...
»

Содержание BX80571E7500 - Core 2 Duo 2.93 GHz Processor

Страница 1: ...Intel Xeon Processor 7500 Series Uncore Programming Guide Reference Number 323535 001 March 2010 ...

Страница 2: ...ogy and an HT Technology enabled chipset BIOS and operating system Performance will vary depending on the specific hardware and software you use For more information see http www intel com technology hyperthread index htm including details on which processors support HT Technology Intel Virtualization Technology requires a computer system with an enabled Intel processor BIOS virtual machine monito...

Страница 3: ...N state Counter Control Pairs 2 14 2 3 4 C BOX Performance Monitoring Events 2 16 2 3 4 1 An Overview 2 16 2 3 4 2 Acronyms frequently used in C Box Events 2 16 2 3 4 3 The Queues 2 17 2 3 4 4 Detecting Performance Problems in the C Box Pipeline 2 17 2 3 5 C Box Events Ordered By Code 2 17 2 3 6 C Box Performance Monitor Event List 2 18 2 4 B BOX PERFORMANCE MONITORING 2 29 2 4 1 Overview of the B...

Страница 4: ...7 2 6 6 R Box Performance Monitor Event List 2 88 2 7 M BOX PERFORMANCE MONITORING 2 95 2 7 1 Overview of the M Box 2 95 2 7 2 Functional Overview 2 95 2 7 2 1 Intel 7500 Scalable Memory Buffer 2 96 2 7 3 M Box Performance Monitoring Overview 2 96 2 7 3 1 Choosing An Event To Monitor Example using subcontrol registers 2 96 2 7 3 2 M Box PMU Overflow Freeze and Unfreeze 2 97 2 7 4 M BOX Performance...

Страница 5: ...N PROCESSOR 7500 SERIES UNCORE PROGRAMMING GUIDE FIGURES FIGURES Figure 1 1 Intel Xeon Processor 7500 Series Block Diagram 1 1 Figure 2 1 R Box Block Diagram 2 72 Figure 2 2 Memory Controller Block Diagram 2 95 ...

Страница 6: ...INTEL XEON PROCESSOR 7500 SERIES UNCORE PROGRAMMING GUIDE FIGURES This page intentionally left blank ...

Страница 7: ...26 S_MSR_PMON_SUMMARY Register Fields 2 47 Table 2 27 S_CSR_PMON_GLOBAL_CTL Register Fields 2 47 Table 2 28 S_MSR_PMON_GLOBAL_STATUS Register Fields 2 47 Table 2 29 S_MSR_PMON_OVF_CTRL Register Fields 2 48 Table 2 30 S_CSR_PMON_CTL 3 0 Register Field Definitions 2 48 Table 2 31 S_CSR_PMON_CTR 3 0 Register Field Definitions 2 49 Table 2 32 S_MSR_MM_CFG Register Field Definitions 2 49 Table 2 33 S_M...

Страница 8: ..._ZDP_CTL_FVC Register Field Definitions 2 108 Table 2 81 M_MSR_PMU_ZDP_CTL_FVC evnt 4 1 Encodings 2 108 Table 2 82 M_MSR_PMU_ZDP_CTL_FVC RESP Encodings 2 109 Table 2 83 M_MSR_PMU_ZDP_CTL_FVC BCMD Encodings 2 109 Table 2 84 Performance Monitor Events for M Box Events 2 110 Table 2 85 Unit Masks for CYCLES_DSP_FILL 2 111 Table 2 86 Unit Masks for CYCLES_PGT_STATE 2 112 Table 2 87 Unit Masks for CYCL...

Страница 9: ...outlined in Section 2 1 Global Performance Monitoring Control All processor uncore performance monitoring features can be accessed through RDMSR WRMSR instruc tions executed at ring 0 Since the uncore performance monitors represent socket wide resources that are not context switched by the OS it is highly recommended that only one piece of software per socket attempt to program and extract informa...

Страница 10: ... Control R Box L 0xE1F 0xE10 Counter Config Registers 7 0 0xE0F 0xE0C QLX SubConfig Registers for Ports 3 0 0xE0B 0xE04 IPERF 0 SubConfig Registers 0xE02 0xE00 Global Control Status Ovf Control C Box Counters C Box 7 0xDFB 0xDF0 Counter Config Registers 0xDE2 0xDE0 Global Control Status Ovf Control C Box 3 0xDDB 0xDD0 Counter Config Registers 0xDC2 0xDC0 Global Control Status Ovf Control C Box 5 0...

Страница 11: ... 0xCA4 Subconfig Registers FVC PLD PGT THR MAP ISS DSP Timestamp Register 0xCA2 0xCA0 Global Control Status Ovf Control S Box Counters S Box 1 0xE5A 0xE58 Match Mask Registers 0xCD7 0xCD0 Counter Config Registers 0xCC3 0xCC0 Global Control Status Ovf Control S Box 0 0xE4A 0xE48 Match Mask Registers 0xC57 0xC50 Counter Config Registers 0xC43 0xC40 Global Control Status Ovf Control B Box Counters B ...

Страница 12: ...INTEL XEON PROCESSOR 7500 SERIES UNCORE PROGRAMMING GUIDE INTRODUCTION 1 4 Section 2 8 W Box Performance Monitoring Section 2 9 Packet Matching Reference ...

Страница 13: ...all counting 2 1 1 2 PMI on Counter Overflow The uncore may also be configured to upon detection of a performance counter overflow send a PMI signal to the core executing the monitoring software To do so the pmi_en in the individual counter s control register must be set to 1 and U_MSR_PMON_GLOBAL_CTL pmi_core_sel must be set to point to the core the monitoring software is executing on Note PMI is...

Страница 14: ...l Software can either poll the counters whenever it chooses or wait to be notified that a counter has overflowed by receiving a PMI a Polling before reading it is recommended that software freeze and disable the counters by clearing U_MSR_PMON_GLOBAL_CTL en_all b Frozen counters If software set up the counters to freeze on overflow and send notification when it happens the next question is Who cau...

Страница 15: ...5 Global Performance Monitors Table 2 1 Global Performance Monitoring Control MSRs 2 1 5 1 Global PMON Global Control Status Registers The following registers represent state governing all PMUs in the uncore both to exert global control and collect box level information U_MSR_PMON_GLOBAL_CTL contains bits that can reset rst_all and freeze enable en_all all the uncore counters The en_all bit must b...

Страница 16: ...rflowing counter 00000000 No PMI sent 00000001 Send PMI to core 0 10000000 Send PMI to core 7 11000100 Send PMI to core 2 6 7 etc en 0 0 Enable U Box PMON counters Field Bits HW Reset Val Description cond 31 0 Condition Change pmi 30 0 PMI Received from box with overflowing counter ig 31 4 0 Read zero writes ignored ov_s0 3 0 Set if overflow is detected from a S Box 0 PMON register ov_s1 2 0 Set i...

Страница 17: ... do if an overflow is detected edge_detect Rather than accumulating the raw count each cycle the register can capture transitions from no event to an event incoming Table 2 6 U_MSR_PMON_EVT_SEL Register Field Definitions MSR Name Access MSR Address Size bits Description U_MSR_PMON_CTR RW_RW 0x0C11 64 U Box PMON Counter U_MSR_PMON_EV_SEL RW_RO 0x0C10 32 U Box PMON Event Select Field Bits HW Reset V...

Страница 18: ...tored in the U Box are summarized in the following section Tracks NcMsgS packets generated by the U Box as they arbitrate to be broadcast They are prioritized as follows Special Cycle StopReq1 StartReq2 Lock Unlock Remote Interrupts Local Interrupts Errors detected and distinguished between recoverable corrected uncorrected and fatal Number of times cores were sent IPIs or were Woken up Requests t...

Страница 19: ...Number of cycles the Remote IPI packet buffer contained a valid entry BUF_VALID_SPC_CYCLES Title SpcCyc Buffer Valid Category U Box Events Event Code 0x004 Max Inc Cyc 1 Definition Number of uncore cycles the Special Cycle packet buffer contains a valid entry Special Cycles are NcMsgS packets generated by the U Box and broadcast to internal cores to cover such things as Shutdown Invd_Ack and WbInv...

Страница 20: ... Max Inc Cyc 1 Definition Number of recoverable errors U2R_REQUESTS Title Number U2R Requests Category U Box Events Event Code 0x050 Max Inc Cyc 1 Definition Number U Box to Ring Requests U2B_REQUEST_CYCLES Title U2B Active Request Cycles Category U Box Events Event Code 0x051 Max Inc Cyc 1 Definition Number U to B Box Active Request Cycles UNCORRECTED_ERR Title Uncorrected Error Category U Box Ev...

Страница 21: ...eding to communicate with the other C Boxes in that same socket Each C Box is uniquely associated with a single S Box All messages which a given C Box sends out to the system memory or Intel QPI pass through the S Box that is physically closest to that C Box 2 3 2 C Box Performance Monitoring Overview Each of the C Boxes in the Intel Xeon Processor 7500 Series supports event monitoring through six...

Страница 22: ...AL_OVF_CT L WO_R O 0xDE2 32 C Box 7 PMON Global Overflow Control CB7_CR_C_MSR_PMON_GLOBAL_STATUS RW_R W 0xDE1 32 C Box 7 PMON Global Status CB7_CR_C_MSR_PMON_GLOBAL_CTL RW_RO 0xDE0 32 C Box 7 PMON Global Control CB3_CR_C_MSR_PMON_CTR_5 RW_R W 0xDDB 64 C Box 3 PMON Counter 5 CB3_CR_C_MSR_PMON_EVT_SEL_5 RW_RO 0xDDA 64 C Box 3 PMON Event Select 5 CB3_CR_C_MSR_PMON_CTR_4 RW_R W 0xDD9 64 C Box 3 PMON C...

Страница 23: ..._GLOBAL_CTL RW_RO 0xDA0 32 C Box 5 PMON Global Control CB1_CR_C_MSR_PMON_CTR_5 RW_R W 0xD9B 64 C Box 1 PMON Counter 5 CB1_CR_C_MSR_PMON_EVT_SEL_5 RW_RO 0xD9A 64 C Box 1 PMON Event Select 5 CB1_CR_C_MSR_PMON_CTR_4 RW_R W 0xD99 64 C Box 1 PMON Counter 4 CB1_CR_C_MSR_PMON_EVT_SEL_4 RW_RO 0xD98 64 C Box 1 PMON Event Select 4 CB1_CR_C_MSR_PMON_CTR_3 RW_R W 0xD97 64 C Box 1 PMON Counter 3 CB1_CR_C_MSR_P...

Страница 24: ...N_EVT_SEL_5 RW_RO 0xD5A 64 C Box 2 PMON Event Select 5 CB2_CR_C_MSR_PMON_CTR_4 RW_R W 0xD59 64 C Box 2 PMON Counter 4 CB2_CR_C_MSR_PMON_EVT_SEL_4 RW_RO 0xD58 64 C Box 2 PMON Event Select 4 CB2_CR_C_MSR_PMON_CTR_3 RW_R W 0xD57 64 C Box 2 PMON Counter 3 CB2_CR_C_MSR_PMON_EVT_SEL_3 RW_RO 0xD56 64 C Box 2 PMON Event Select 3 CB2_CR_C_MSR_PMON_CTR_2 RW_R W 0xD55 64 C Box 2 PMON Counter 2 CB2_CR_C_MSR_P...

Страница 25: ...MON_GLOBAL_STATUS RW_R W 0xD21 32 C Box 4 PMON Global Status CB4_CR_C_MSR_PMON_GLOBAL_CTL RW_RO 0xD20 32 C Box 4 PMON Global Control CB0_CR_C_MSR_PMON_CTR_5 RW_R W 0xD1B 64 C Box 0 PMON Counter 5 CB0_CR_C_MSR_PMON_EVT_SEL_5 RW_RO 0xD1A 64 C Box 0 PMON Event Select 5 CB0_CR_C_MSR_PMON_CTR_4 RW_R W 0xD19 64 C Box 0 PMON Counter 4 CB0_CR_C_MSR_PMON_EVT_SEL_4 RW_RO 0xD18 64 C Box 0 PMON Event Select 4...

Страница 26: ...ol bits include pmi_en governs what to do if an overflow is detected threshold since C Box counters can increment by a value greater than 1 a threshold can be applied If the threshold is set to a non zero value that value is compared against the incoming count for that event in each cycle If the incoming count is the threshold value then the event count captured in the data register will be increm...

Страница 27: ...Read zero writes ignored rsv 62 61 0 Reserved Must write to 0 else behavior is undefined ig 60 50 0 Read zero writes ignored threshold 31 24 0 Threshold used in counter comparison invert 23 0 When 0 the comparison that will be done is threshold event When set to 1 the comparison that is inverted e g threshold event en 22 0 Local Counter Enable When set the associated counter is locally enabled NOT...

Страница 28: ... counters for each C Box instance For any event to get an aggregate count of that event for the entire LLC the counts across the C Box instances must be added together The counts can be averaged across the C Box instances to get a view of the typical count of an event from the perspective of the individual C Boxes Individual per C Box deviations from the average can be used to identify hot spottin...

Страница 29: ...ses if the message is in the IRQ IPQ then the C Box hasn t acknowledged it yet and the request hasn t yet entered the LLC s coherence domain It deallocates from the IRQ IPQ at the moment that the C Box does acknowledge it In optimal performance scenarios where there are minimal conflicts between transactions and loads are low enough to keep latencies relatively near to idle IRQ and IPQ occupancies...

Страница 30: ..._WINS_AD 0x0E 1 Ingress S Box Non S Box Bypass Wins MAF_ACK 0x10 1 MAF Acknowledges LLC_MISSES 0x14 1 LLC Misses LLC_HITS 0x15 1 LLC Hits LLC_S_FILLS 0x16 1 LLC lines filled from S Box LLC_VICTIMS 0x17 1 LLC lines victimized Queue Occupancy Events OCCUPANCY_IRQ 0x18 24 IRQ Occupancy TRANS_IRQ 0x19 1 IRQ Transactions dealloc OCCUPANCY_IPQ 0x1A 8 IPQ Occupancy TRANS_IPQ 0x1B 1 IPQ Transactions OCCUP...

Страница 31: ...e direction that points toward the nearest S Box AK_NSB b00001000 AK ring in the direction that points away from the nearest S Box AK_ALL b00001100 AK ring in either direction BL_SB b00010000 BL ring in the direction that points toward the nearest S Box BL_NSB b00100000 BL ring in the direction that points away from the nearest S Box BL_ALL b00110000 BL ring in either direction IV b01000000 IV rin...

Страница 32: ...NCES_P2C_AD Title P2C AD Bounces Category Ring WIR Event Code 0x01 Max Inc Cyc Definition Core request to LLC bounces on AD ring Extension umask 15 8 Description b00000000 nothing will be counted SB b000000x1 Direction that points toward the nearest S Box NSB b0000001x Direction that points away from the nearest S Box ALL b00000011 Either direction Extension umask 15 8 Description b00000000 nothin...

Страница 33: ...S Title LLC Hits Category Local LLC Event Code 0x15 Max Inc Cyc 1 Definition Last Level Cache Hits NOTE LRU hints are included in count Extension umask 15 8 Description b00000000 nothing will be counted AD_BYP0 b00000001 0 cycle AD egress bypass AD_BYP1 b00000010 1 cycle AD egress bypass AK_BYP0 b00000100 0 cycle AK egress bypass AK_BYP1 b00001000 1 cycle AK egress bypass BL_BYP0 b00010000 0 cycle...

Страница 34: ...0xx1 Shared request requires S line to be upgraded due to RFO F b00000x1x Forward request requires F line to be upgraded due to RFO I b000001xx Invalid address not found ALL b00000111 All misses Extension umask 15 8 Description b00000000 nothing will be counted M b0000xxx1 Filled to LLC in Modified remote socket forwarded M data without writing back to memory controller E b0000xx1x Filled to LLC i...

Страница 35: ... QPI snoop was delayed because it conflicted with an existing MAF transaction that had an Ack Conflict pending IDX_BLOCK bxxx1xxxx An incoming local core RD that missed the LLC was delayed because a victim way could not be immediately chosen PA_BLOCK bxx1xxxxx If this count is very high it likely means that software is frequently issuing requests to the same physical address from disparate threads...

Страница 36: ...e ring was delayed due to ring back pressure VIQ_FULL bxxxxx1xx An incoming local processor RD request that missed the LLC was delayed because the LLC victim buffer was full NO_TRACKER_CREDITS bxxxx1xxx An incoming local processor RD or WR request was delayed because it required a Home tracker credit for example LLC RD Miss and no credit was available NO_S_FIFO_CREDITS bxxx1xxxx Some incoming mess...

Страница 37: ... NOTE Each sink represents the transfer of 32 bytes or 2 sinks per cache line SINKS_P2C Title P2C Sinks Category Ring WIR Event Code 0x05 Max Inc Cyc 3 Definition Number of messages sunk from the ring at the C Box that were sent by one of the local processors NOTE Each sink represents the transfer of 32 bytes or 2 sinks per cache line Extension umask 15 8 Description b00000000 nothing will be coun...

Страница 38: ...sk 15 8 Description b00000000 nothing will be counted AD b00000001 AD Intel QPI snoop request of LLC AK b00000010 AK Intel QPI completions sent to LLC BL b00000100 BL Data Fills sent to the LLC in response to RD requests Extension umask 15 8 Description b00000000 nothing will be counted REMOTE_RD_HITM bxxxxxxx1 Intel QPI SnpData or SnpCode hit M line in LLC REMOTE_RD_HITE bxxxxxx1x Intel QPI SnpDa...

Страница 39: ... I this subevent will capture those snoops REMOTE_RFO b0000001x Remote RFO Goto I State Intel QPI snoops SnpInvOwn or SnpInvItoE to LLC that caused an invalidate of a cache line REMOTE_ANY b00000011 Intel QPI snoops to LLC that hit in the cache line Extension umask 15 8 Description b00000000 nothing will be counted P2C_AD_SB b00000001 Processor to C Box AD Egress that injects in the direction towa...

Страница 40: ...hat passed through the Snoop Response FIFO The RSPF is a buffer that sits between each processor and the ring that buffers the processor s snoop responses in the event that there is back pressure due to ring congestion TRANS_RWRF Title RWRF Transactions Category Queue Occupancy Event Code 0x21 Max Inc Cyc 1 Definition Number of requests that passed through the Read Write Request FIFO The RWRF is a...

Страница 41: ...of these four counters is dedicated to observe a specific set of events as specified in its control register BBx_CR_B_MSR_PERF_CTL 3 0 The B Box counters will increment by a maximum of 1 per cycle For information on how to setup a monitoring session refer to Section 2 1 Global Performance Monitoring Control 2 4 2 1 B Box PMU On Overflow and the Consequences PMI Freeze If an overflow is detected fr...

Страница 42: ...R_PERF_CNT2_REG RW_RW 0x0C75 64 B Box 1 PMON Counter 2 BB1_CR_B_MSR_PERF_CTL2_REG RW_RW 0x0C74 64 B Box 1 PMON Event Select 2 BB1_CR_B_MSR_PERF_CNT1_REG RW_RW 0x0C73 64 B Box 1 PMON Counter 1 BB1_CR_B_MSR_PERF_CTL1_REG RW_RW 0x0C72 64 B Box 1 PMON Event Select 1 BB1_CR_B_MSR_PERF_CNT0_REG RW_RW 0x0C71 64 B Box 1 PMON Counter 0 BB1_CR_B_MSR_PERF_CTL0_REG RW_RW 0x0C70 64 B Box 1 PMON Event Select 0 ...

Страница 43: ...to 1 to enable counting Additional control bits include pmi_en governs what to do if an overflow is detected NOTE In the B Box each control register can only select from a specific set of events see Table 2 24 Performance Monitor Events for B Box Events for the mapping Field Bits HW Reset Val Description ctr_en 3 0 0 Must be set to enable each B Box counter bit 0 to enable ctr0 etc NOTE U Box enab...

Страница 44: ...ting each B Box provides a MATCH MASK register pair that allow a user to filter packet traffic incoming and outgoing according to the packet Opcode Message Class and Physical Address Various events can be programmed to enable a B Box performance counter i e OPCODE_ADDR_IN_MATCH for counter 0 to capture the filter match as an event The fields are laid out as follows Note Refer to Table 2 103 Intel ...

Страница 45: ...everal smaller queues COHQ0 1 Coherence Queues 256 entry read request is pushed onto COHQ when it is in the ready state e g a read request that has received all of its snoop responses and is waiting for the M Box ACK All snoop responses received or RspFwd or RspWb will result in pushing that transaction onto the COHQ NDROQ NDR Output Queue 256 entry request is pushed when an NDR message has to be ...

Страница 46: ...e latency valid cnt 32 IMT inserts The 256 entry TF Tracker File holds all transactions that arrive in the B Box from the time they arrive until they are completed and leave the B Box Transactions could stay in this structure much longer than they are needed IMT is the critical resource each transaction needs before being sent to the M Box memory controller TF average occupancy valid cnt 256 cycle...

Страница 47: ...s Counter 1 Events MSG_IN_MATCH 0x01 1 Message In Match MSG_OUT_MATCH 0x02 1 Message Out Match OPCODE_IN_MATCH 0x03 1 Opcode In Match OPCODE_OUT_MATCH 0x04 1 Opcode Out Match MSG_OPCODE_IN_MATCH 0x05 1 Message Opcode In Match MSG_OPCODE_OUT_MATCH 0x06 1 Message Opcode Out Match IMT_INSERTS_ALL 0x07 1 IMT All Inserts DRSQ_INSERTS 0x09 1 DRSQ Inserts IMT_INSERTS_IOH 0x0A 1 IMT IOH Inserts IMT_INSERT...

Страница 48: ...ch at B Box Input Use B_MSR_MATCH MASK_REG CONFLICTS Title Conflicts Category Miscellaneous Event Code 0x17 Max Inc Cyc 1 PERF_CTL 3 Definition Number of conflicts COHQ_BYPASS Title COHQ Bypass Category ARB Queues Event Code 0x0E Max Inc Cyc 1 PERF_CTL 3 Definition Coherence Queue Bypasses IMT_NE_CYCLES 0x07 1 IMT Non Empty Cycles Counter 3 Events EARLY_ACK 0x02 1 Early ACK IMT_PREALLOC 0x06 1 IMT...

Страница 49: ... Queue Depth is 256 DEMAND_FETCH Title Demand Fetches Category Miscellaneous Event Code 0x0F Max Inc Cyc 1 PERF_CTL 3 Definition Counts number of times a memory access was issued after CohQ pop i e IMT prefetch was not used DRSQ_INSERTS Title DRSQ Inserts Category ARB Queues Event Code 0x09 Max Inc Cyc 1 PERF_CTL 1 Definition DRSQ Inserts DRSQ_OCCUPANCY Title DRSQ Occupancy Category ARB Queues Eve...

Страница 50: ...ered IMT insert events If the conflict rate CONFLICTS IMT_INSERTS_ALL 100 is 5 it is not recommended that this event along with IMT_VALID_OCCUPANCY be used to derive average IMT latency or latency for specific flavors of inserts IMT_INSERTS_IOH Title IMT IOH Inserts Category In Flight Memory Table Event Code 0x0A Max Inc Cyc 1 PERF_CTL 1 Definition In Flight Memory Table inserts of IOH requests e ...

Страница 51: ... inserts of Non IOH InvItoE requests e g all non IOH triggered InvItoE memory transactions targeting this B Box as their home node and processed by this B Box NOTE Conflicts and AckConflicts are considered IMT insert events If the conflict rate CONFLICTS IMT_INSERTS_ALL 100 is 5 it is not recommended that this event along with IMT_VALID_OCCUPANCY be used to derive average IMT latency or latency fo...

Страница 52: ...ID_OCCUPANCY be used to derive average IMT latency or latency for specific flavors of inserts IMT_NE_CYCLES Title IMT Non Empty Cycles Category In Flight Memory Table Event Code 0x07 Max Inc Cyc 1 PERF_CTL 2 Definition In Flight Memory Table Non Empty Cycles IMT_PREALLOC Title IMT Prealloc Category In Flight Memory Table Event Code 0x06 Max Inc Cyc 1 PERF_CTL 3 Definition In Flight Memory Table in...

Страница 53: ...e In Match Category Mask Match Event Code 0x05 Max Inc Cyc 1 PERF_CTL 1 Definition Message Class and Opcode Match at B Box Input Use B_MSR_MATCH MASK_REG MSG_OPCODE_OUT_MATCH Title Message Opcode Out Match Category Mask Match Event Code 0x06 Max Inc Cyc 1 PERF_CTL 1 Definition Message Class and Opcode Match at B Box Output Use B_MSR_MATCH MASK_REG MSG_OUT_MATCH Title Message Out Match Category Mas...

Страница 54: ...tegory S Box Interface Event Code 0x14 Max Inc Cyc 1 PERF_CTL 3 Definition Number of times S Box VN0 credit was not available when needed SNPOQ_INSERTS Title SNPOQ Inserts Category ARB Queues Event Code 0x12 Max Inc Cyc 1 PERF_CTL 1 Definition SNP Output Queue Inserts Queue Depth is 256 SNPOQ_OCCUPANCY Title SNPOQ Occupancy Category ARB Queues Event Code 0x12 Max Inc Cyc 1 PERF_CTL 0 Definition SN...

Страница 55: ...inition Tracker File occupancy for IOH InvItoE requests Accumulates lifetimes of IOH triggered InvItoE memory transactions that have arrived in this B Box TF starts tracking transactions before they are sent to the M Box NOTE This event captures overflows from a subcounter tracking all requests Multiply by 256 to determine the correct count TF_IOH_NON_INVITOE_RD Title TF Occupancy IOH Non InvItoE ...

Страница 56: ...de 0x05 Max Inc Cyc 1 PERF_CTL 0 Definition Tracker File occupancy for write requests Accumulates lifetimes of write memory transactions that have arrived in this B Box TF starts tracking transactions before they are sent to the M Box NOTE This event captures overflows from a subcounter tracking all requests Multiply by 256 to determine the correct count ...

Страница 57: ...the overflow bit is set at the box level S_MSR_PMON_GLOBAL_STATUS ov and forwarded up the chain to the U Box where it will be stored in U_MSR_PMON_GLOBAL_STATUS ov_s0 Each S Box collects overflow bits for all boxes on it s side of the chip Refer to Table 2 26 S_MSR_PMON_SUMMARY Register Fields to determine how these bits are accumulated before they are forwarded to the U Box HW can be also configu...

Страница 58: ..._S_MSR_PMON_SUMMARY RO_WO 0x0CC3 32 S Box 1 PMON Global Summary SR1_CR_S_MSR_PMON_OVF_CTL WO_RO 0x0CC2 32 S Box 1 PMON Global Overflow Control SR1_CR_S_MSR_PMON_GLOBAL_STATUS RW_RW 0x0CC1 32 S Box 1 PMON Global Overflow Status SR1_CR_S_MSR_PMON_GLOBAL_CTL RW_RO 0x0CC0 32 S Box 1 PMON Global Control SR0_CR_S_MSR_PMON_CTR3 RW_RW 0x0C57 64 S Box 0 PMON Counter 3 SR0_CR_S_MSR_PMON_CTL3 RW_RO 0x0C56 64...

Страница 59: ...ble 2 28 S_MSR_PMON_GLOBAL_STATUS Register Fields Field Bits HW Reset Val Description ig 63 20 Read zero writes ignored ov_r 19 0 Overflow in R Box In S Box0 indicates overflow from Left R Box In S Box1 indicates overflow from Right R Box ov_s 18 0 Overflow in S Box ig 17 Read zero writes ignored ov_mb 16 0 Overflow in M or B Box ig 15 3 Read zero writes ignored ov_c_l 2 0 Overflow in left C Boxes...

Страница 60: ...vent incoming reset_occ_cnt Reset 7b occupancy counter associated with this counter Table 2 30 S_CSR_PMON_CTL 3 0 Register Field Definitions Field Bits HW Reset Val Description clr_ov 3 0 0 Writing 1 to bit in filed causes corresponding bit in Overflow PerfMon Counter field in S_CSR_PMON_GLOBAL_STATUS register to be cleared to 0 Field Bits HW Reset Val Description ig 63 0 Read zero writes ignored ...

Страница 61: ...ters for Mask Match Facility In addition to generic event counting each S Box provides a MATCH MASK register pair that allows a user to filter outgoing packet traffic system bound according to the packet Opcode Message Class Response HNID and Physical Address Program the selected S Box counter to capture TO_R_PROG_EV to capture the filter match as an event To use the match mask facility a Set MM_C...

Страница 62: ...n request NOTE Match will be ignored if field set to all 0s opc 58 48 0 Match on Opcode see Table 2 34 S_MSR_MATCH opc Opcode Match by Message Class NOTE Match will be ignored if field set to all 0s mc 47 43 0 Match on Message Class b1xxxx NCB bx1xxx NCS bxx1xx NDR bxxx1x HOM1 bxxxx1 HOM0 NOTE Match will be ignored if field set to all 0s addr 42 5 0 Match on PA address bits 43 6 hnid 4 0 0 Match o...

Страница 63: ...general purpose counters is a 7b queue occupancy counter which supports the various queue occupancy events found in Section 2 5 5 S Box Events Ordered By Code Each System Bound and Ring Bound data storage structure within the S Box queue FIFO buffer has an associated tally counter which can be used to provide input into one of the S Box performance counters The data structure the tally counter is ...

Страница 64: ...m Bound NDR Message Queue TO_R_NDR_MSGQ_OCCUPANCY 16 1 NDR Packet to System System Bound DRS Message Queue TO_R_DRS_MSGQ_OCCUPANCY 16 4 DRS Packet to System System Bound NCB Message Queue TO_R_NCB_MSGQ_OCCUPANCY 16 4 NCB Packet to System System Bound NCS Message Queue TO_R_NCS_MSGQ_OCCUPANCY 16 4 NCS Packet to System Ring Bound Message Queue TO_RING_MSGQ_OCCUPANCY SNP 31 1 Packets from System SNP ...

Страница 65: ...d HOM Message Queue Occupancy TO_R_SNP_MSGQ_CYCLES_FULL 0x08 1 Cycles System Bound SNP Message Queue Full TO_R_SNP_MSGQ_CYCLES_NE 0x09 1 Cycles System Bound SNP Message Queue Not Empty TO_R_SNP_MSGQ_OCCUPANCY 0x0A 32 System Bound SNP Message Queue Occupancy TO_R_NDR_MSGQ_CYCLES_FULL 0x0B 1 Cycles System Bound NDR Message Queue Full TO_R_NDR_MSGQ_CYCLES_NE 0x0C 1 Cycles System Bound NDR Message Que...

Страница 66: ...LOSSES 0x42 1 Egress ARB Losses EGRESS_STARVED 0x43 1 Egress Cycles in Starvation RBOX_HOM_BYPASS 0x50 1 R Box HOM Bypass RBOX_SNP_BYPASS 0x51 1 R Box SNP Bypass S2B_HOM_BYPASS 0x52 1 S Box to B Box HOM Bypass B2S_DRS_BYPASS 0x53 1 B Box to S Box DRS Bypass BBOX_HOM_BYPASS 0x54 1 B Box HOM Bypass PKTS_SENT_HOM 0x60 1 HOM Packets Sent to System PKTS_SENT_SNP 0x62 1 SNP Packets Sent to System PKTS_S...

Страница 67: ...ox BBOX_CREDIT_RETURNS Title B Box Credit Returns Category System Bound Transmission Event Code 0x6B Max Inc Cyc 1 Definition Number credit return idle flits sent to the B Box BBOX_HOM_BYPASS Title B Box HOM Bypass Category System Bound Enhancement Event Code 0x54 Max Inc Cyc 1 Definition B Box HOM Bypass optimization utilized NO_CREDIT_HOM 0x80 1 HOM Credit Unavailable NO_CREDIT_SNP 0x81 1 SNP Cr...

Страница 68: ...g multiple subevents in this category will result in the counter being increased by the number of selected subevents that occur in a given cycle Because only one of the even odd FIFOs can arbitrate to send onto the ring in each cycle the event for the even odd FIFOs in each direction are exclusive The bypass event for each direction is the sum of the bypass events of the even odd FIFOs Extension u...

Страница 69: ...starvation NOTE Enabling multiple subevents in this category will result in the counter being increased by the number of selected subevents that occur in a given cycle Because only one of the even odd FIFOs can arbitrate to send onto the ring in each cycle the event for the even odd FIFOs in each direction are exclusive The bypass event for each direction is the sum of the bypass events of the eve...

Страница 70: ...eline Bypass optimization where the line is sent early was utilized NO_CREDIT_AD Title AD Ring Credit Unavailable Category Ring Bound Credits Event Code 0x87 Max Inc Cyc 1 Definition Number of times the S Box has a pending SNP NCS or NCB message to send and there is no credit for the target egress FIFO NO_CREDIT_AK Title AK Ring Credit Unavailable Category Ring Bound Credits Event Code 0x88 Max In...

Страница 71: ... Inc Cyc 1 Definition Number of times the S Box has a pending non coherent bypass message to send and there is no NCB or VNA credit available NO_CREDIT_NCS Title NCS Credit Unavailable Category System Bound Credits Event Code 0x83 Max Inc Cyc 1 Definition Number of times the S Box has a pending non coherent standard message to send and there is no NCS or VNA credit available NO_CREDIT_NDR Title ND...

Страница 72: ...ages which are 9 flits Multiply this event by 9 to derive flit traffic from the R Box due to DRS messages PKTS_RCVD_NCB Title NCB Packets Received from System Category Ring Bound Transmission Event Code 0x75 Max Inc Cyc 1 Definition Number of non coherent bypass packets the S Box has received from the system NOTE The only ring bound NCB message types are NcMsgB StartReq2 VLW IntLogical IntPhysical...

Страница 73: ... 0x60 Max Inc Cyc 1 Definition Number of home packets the S Box has transmitted to the R Box or B Box If both R Box and B Box are selected counts the total number of home packets sent to both boxes PKTS_SENT_NCB Title NCB Packets Sent to System Category System Bound Transmission Event Code 0x68 Max Inc Cyc 11 Definition Number of NCB packets the S Box has transmitted to the system NOTE If multiple...

Страница 74: ...ition Number of SNP packets the S Box has transmitted to the system This event only counts the first snoop that is spawned from a home request When S Box broadcast is enabled this event does not count the additional snoop packets that are spawned RBOX_CREDIT_CARRIERS Title R Box Credit Carrying Flits Category Ring Bound Transmission Event Code 0x76 Max Inc Cyc 1 Definition Number credit carrying i...

Страница 75: ...egory System Bound Enhancement Event Code 0x52 Max Inc Cyc 1 Definition Number of cycles the S Box to B Box HOM channel bypass optimization was utilized Includes cycles used to transmit message flits and credit carrying idle credit flits TO_RING_B2S_MSGQ_CYCLES_FULL Title Ring Bound B2S Message Queue Full Category Ring Bound Queue Event Code 0x2B Max Inc Cyc 1 Definition Number of cycles in which ...

Страница 76: ...ge Queue Full Category Ring Bound Queue Event Code 0x21 Max Inc Cyc 1 Definition Number of cycles in which the header buffer containing NCB messages on their way to the Ring is full TO_RING_NCB_MSGQ_CYCLES_NE Title Cycles Ring Bound NCB Message Queue Not Empty Category Ring Bound Queue Event Code 0x24 Max Inc Cyc 1 Definition Number of cycles in which the header buffer containing NCB messages on t...

Страница 77: ...g is full TO_RING_R2S_MSGQ_CYCLES_NE Title Cycles Ring Bound R2S Message Queue Not Empty Category Ring Bound Queue Event Code 0x2C Max Inc Cyc 1 Definition Number of cycles in which the header buffer containing R to S Box messages on their way to the Ring has one or more entries allocated TO_RING_R2S_MSGQ_OCCUPANCY Title Ring Bound R2S Message Queue Occupancy Category Ring Bound Queue Event Code 0...

Страница 78: ...nt is asserted when any of the selected C Box DRS header buffers are not empty TO_R_DRS_MSGQ_OCCUPANCY Title System Bound DRS Message Queue Occupancy Category System Bound Queue Event Code 0x10 Max Inc Cyc 16 Definition Number of entries in the header buffer for the selected C Box containing DRS messages heading to a System Agent through the R Box When more than one C Box is selected the queue occ...

Страница 79: ...R Box has one or more entries allocated If both R Box and B Box sub events are selected this event is asserted when the total number of entries in the R Box and B Box Home header buffers is equal to 64 TO_R_B_HOM_MSGQ_OCCUPANCY Title System Bound HOM Message Queue Occupancy Category System Bound Queue Event Code 0x07 Max Inc Cyc 64 Definition Number of entries in the header buffer containing HOM m...

Страница 80: ...ent is asserted when any of the selected C Box DRS header buffers are not empty TO_R_NCB_MSGQ_OCCUPANCY Title System Bound NCB Message Queue Occupancy Category System Bound Queue Event Code 0x13 Max Inc Cyc 8 Definition Number of entries in the header buffer for the selected C Box containing NCB messages heading to a System Agent through the R Box When more than one C Box is selected the queue occ...

Страница 81: ...event is asserted when any of the selected C Box NCS header buffers are not empty TO_R_NCS_MSGQ_OCCUPANCY Title System Bound NCS Message Queue Occupancy Category System Bound Queue Event Code 0x16 Max Inc Cyc 2 Definition Number of entries in the header buffer for the selected C Box containing NCS messages heading to a System Agent through the R Box When more than one C Box is selected the queue o...

Страница 82: ...nt Code 0x0D Max Inc Cyc 16 Definition Number of entries in the header buffer containing NDR messages heading to a System Agent through the R Box TO_R_PROG_EV Title System Bound Programmable Event Category System Bound Queue Event Code 0x00 Max Inc Cyc 1 Definition Programmable Event heading to a System Agent through the R Box Match Mask on criteria set in S_MSR_MATCH MASK registers Refer to Secti...

Страница 83: ... is full TO_R_SNP_MSGQ_CYCLES_NE Title Cycles System Bound SNP Message Queue Not Empty Category System Bound Queue Event Code 0x09 Max Inc Cyc 1 Definition Number of cycles in which the header buffer containing SNP messages heading to a Sys tem Agent through the R Box has one or more entries allocated TO_R_SNP_MSGQ_OCCUPANCY Title System Bound SNP Message Queue Occupancy Category System Bound Queu...

Страница 84: ...storing incoming packets from the B and S Boxes as well as off chip requests using the Intel QuickPath Interconnect protocol Data from each packet header is consolidated and sent to the R Box arbiter R Box input ports have two structures of important to performance monitoring Entry overflow table EOT and Entry Table ET R Box PMU supports performance monitoring in these two structures 2 6 1 2 R Box...

Страница 85: ... the right side Since the R Box consists of 12 almost identical ports R Box perfmon events consist of an identical set of events for each port The R Box perfmon usage model allows monitoring of multiple ports at the same time R Box PMUs do not provide any global performance monitoring events However unlike many other uncore boxes event programming in the R Box is hierarchical It is necessary to pr...

Страница 86: ... the overflow field responsible for the freeze must be cleared by setting the corresponding bit in R_MSR_PMON_GLOBAL_OVF_CTL clr_ov Assuming all the counters have been locally enabled en bit in data registers meant to monitor events and the overflow bit s has been cleared the R Box is prepared for a new sample interval Once the global controls have been re enabled Section 2 1 4 Enabling a New Samp...

Страница 87: ...RW_NA 0x0E79 64 R Box Port 2 Match 2 R_MSR_PORT2_XBR_SET2_MM_CFG RW_NA 0x0E78 64 R Box Port 2 Mask Match Config 2 R_MSR_PORT2_XBR_SET1_MASK RW_NA 0x0E6A 64 R Box Port 2 Mask 1 R_MSR_PORT2_XBR_SET1_MATCH RW_NA 0x0E69 64 R Box Port 2 Match 1 R_MSR_PORT2_XBR_SET1_MM_CFG RW_NA 0x0E68 64 R Box Port 2 Mask Match Config 1 R_MSR_PORT1_XBR_SET2_MASK RW_NA 0x0E76 64 R Box Port 1 Mask 2 R_MSR_PORT1_XBR_SET2_...

Страница 88: ..._PORT5_IPERF_CFG1 RW_NA 0x0E29 32 R Box Port 5 RIX Perf Event Cfg 1 R_MSR_PORT4_IPERF_CFG1 RW_NA 0x0E28 32 R Box Port 4 RIX Perf Event Cfg 1 R_MSR_PORT3_IPERF_CFG1 RW_NA 0x0E27 32 R Box Port 3 RIX Perf Event Cfg 1 R_MSR_PORT2_IPERF_CFG1 RW_NA 0x0E26 32 R Box Port 2 RIX Perf Event Cfg 1 R_MSR_PORT1_IPERF_CFG1 RW_NA 0x0E25 32 R Box Port 1 RIX Perf Event Cfg 1 R_MSR_PORT0_IPERF_CFG1 RW_NA 0x0E24 32 R...

Страница 89: ...Box Port 3 RIX Perf Event Cfg 0 R_MSR_PORT2_IPERF_CFG0 RW_NA 0x0E06 32 R Box Port 2 RIX Perf Event Cfg 0 R_MSR_PORT1_IPERF_CFG0 RW_NA 0x0E05 32 R Box Port 1 RIX Perf Event Cfg 0 R_MSR_PORT0_IPERF_CFG0 RW_NA 0x0E04 32 R Box Port 0 RIX Perf Event Cfg 0 R_MSR_PMON_OVF_CTL_15_8 RW1C_ WO 0x0E02 32 R Box PMON Overflow Ctrl for ctrs 7 0 R_MSR_PMON_GLOBAL_STATUS_7_0 RO_WO 0x0E01 32 R Box PMON Global Statu...

Страница 90: ...F_CTL_ 15_8 7_0 Register Fields 2 6 3 3 R Box PMON state Counter Control Pairs Filters The following table defines the layout of the R Box performance monitor control registers The main task of these configuration registers is to select the subcontrol register that selects the event to be monitored by the respective data counter Setting the ev_sel fields performs the subcontrol register selection ...

Страница 91: ...eption is sent to the U Box ev_sel 5 1 0 Event Select For the R Box this means choosing which sub register contains the actual event select Each control register can redirect the event select to one of 3 sets of registers QLX RIX or Mask Match registers It can further select from one of two subselect fields either in the same or different registers And finally each control can listen to events occ...

Страница 92: ...RT5_QLX1 0x09 Select Event Configured in R_CSR_PORT5_QLX_EVENT_CFG 1 PORT5_XBAR_MM1 0x0A Set1 Port5 XBAR Mask Match PORT5_XBAR_MM2 0x0B Set2 Port5 XBAR Mask Match PORT6_IPERF0 0x0C Select Event Configured in R_CSR_PORT6_IPERF0 PORT6_IPERF1 0x0D Select Event Configured in R_CSR_PORT6_IPERF1 PORT6_QLX0 0x0E Select Event Configured in R_CSR_PORT6_QLX_EVENT_CFG 0 PORT6_QLX1 0x0F Select Event Configure...

Страница 93: ...T0_QLX_EVENT_CFG 0 PORT0_QLX1 0x03 Select Event Configured in R_CSR_PORT0_QLX_EVENT_CFG 1 PORT0_XBAR_MM1 0x04 Set1 Port0 XBAR Mask Match PORT0_XBAR_MM2 0x05 Set2 Port0 XBAR Mask Match PORT1_IPERF0 0x06 Select Event Configured in R_CSR_PORT1_IPERF0 PORT1_IPERF1 0x07 Select Event Configured in R_CSR_PORT1_IPERF1 PORT1_QLX0 0x08 Select Event Configured in R_CSR_PORT1_QLX_EVENT_CFG 0 PORT1_QLX1 0x09 S...

Страница 94: ... this Output Port RCVD_SPEC_FLT 25 0x0 Special Flit Received RCVD_ERR_FLT 24 0x0 Flit Received which caused CRC Error ig 23 22 0x0 Read zero writes ignored MC_ROLL_ALLOC 21 0x0 Used with MC field If set every individual allocation of selected MC into EOT is reported If 0 a rolling count is reported count whenever count overflows 7b count val of 128 for selected MC s allocation into EOT MC 20 17 0x...

Страница 95: ..._PC 5 0 0x0 New Packet Class Bit Mask Bit mask to select which packet types to count Anded with New Packet VN Select b1XXXXX Snoop bX1XXXX Home bXX1XXX Non Data Response bXXX1XX Data Response bXXXX1X Non Coherent Standard bXXXXX1 Non Coherent Bypass Table 2 49 R_MSR_PORT 7 0 _QLX_CFG Register Fields Sheet 1 of 2 Field Bits HW Reset Val Description ig 31 16 Read zero writes ignored ev1_sub 15 0x0 P...

Страница 96: ...lobal Arb Bid 0011 Queue Arb Fail 0100 Local Arb Fail 0101 Global Arb Fail 0110 Queue Arb Home Order Kill 0111 Local Arb Home Order Kill 1000 Global Arb Home Order Kill 1001 Target Available 1010 Starvation Detected 1011 1111 Reserved ev0_sub 7 0x0 Performance Event 0Sub Class Select 0 VN0 1 VN1 ev0_cls 6 4 0x0 Performance Event 0 Class Select 000 HOM 001 SNP 010 NDR 011 NCS 100 DRS 101 NCB 110 VN...

Страница 97: ...2 1 _MM_CFG Registers Field Bits HW Reset Val Description dis 63 0x0 Disable Set to 0 to enable use by PMUs ig 62 22 0x0 Read zero writes ignored mm_trig_en 21 0x0 Match Mask trigger enable Set to 1 to enable mask match trigger ig_flt_cnt 20 0x0 Ignore flit count Set to ignore match_flt_cnt field match_flt_cnt 19 16 0x0 Match flit count Set number of flit count in a packet on which to trigger a ma...

Страница 98: ...ved Must write to 0 else behavior is undefined Table 2 52 R_MSR_PORT 7 0 _XBR_SET 2 1 _MASK Registers Field Bits HW Reset Val Description 63 52 0x0 Reserved Must write to 0 else behavior is undefined RDS 51 48 0x0 Response Data State for certain DRS messages 47 36 0x0 Reserved Must write to 0 else behavior is undefined RNID 35 31 0x0 Remote Node ID 30 18 0x0 Reserved Must write to 0 else behavior ...

Страница 99: ...ll compute the number of DRS writeback and non snoop write messages DRS DataC_M 0x1C00 Match 51 48 0x8 0x1FE0 Mask 51 48 0xF Data Response message of a cache line in M state that is response to a core request The DRS DataC_M messages are only sent to S Boxes DRS WblData 0x1C80 0x1FE0 Data Response message for Write Back data where cacheline is set to the I state DRS WbSData 0x1CA0 0x1FE0 Data Resp...

Страница 100: ...UE_READ_WIN 8 0x1 1 Input Queue Read Win ALLOC_TO_ARB 15 9 0xX 1 Transactions allocated to Arb EOT_NE_CYCLES 16 0x1 1 Cycles EOT Not Empty EOT_OCCUPANCY 21 0x0 1 EOT Occupancy EOT_INSERTS 21 0x1 1 Number of Inserts into EOT FLITS_RECV_ERR 24 0x1 1 Error Flits Received FLITS_RECV_SPEC 25 0x1 1 Special Flits Received OUTPUTQ_NE 26 0x1 1 Output Queue Not Empty OUTPUTQ_OVFL 27 0x1 1 Output Queue Overf...

Страница 101: ...or ALLOC_TO_ARB Extension IPERF Bit Values 15 9 Description b0000000 nothing will be counted NCB bxxxxxx1 Non Coherent Bypass Messages NCS bxxxxx1x Non Coherent Standard Messages DRS_VN01 bxxxx1xx Data Response VN0 VN1 Messages NDR bxxx1xxx Non Data Response Messages SNP bxx1xxxx Snoop Messages HOM_VN0 bx1xxxxx Home VN0 Messages HOM_VN1 b1xxxxxx Home VN1 Messages ALL b1111111 All Messages Table 2 ...

Страница 102: ...Inc Cyc 1 Definition Counts all special flits received FLITS_SENT Title Flits Sent Category RIX Bit s Value 31 0x1 Max Inc Cyc 1 Definition Counts all flits Output queue receives active beat GLOBAL_ARB_BID Title Global ARB Bids Category QLX Bit s Value 3 0 0x2 Max Inc Cyc 1 Definition Count global arbitration bids from the port Table 2 57 Unit Masks for EOT_ROLL_DEPTH_ACC Extension IPERF Bit Value...

Страница 103: ... Failed Local ARB Bids Category QLX Bit s Value 3 0 0x4 Max Inc Cyc 1 Definition Number of clocks of non zero Local ARB Bids that were rejected Table 2 58 Unit Masks for GLOBAL_ARB_BID_FAIL Extension QLX_EVENT _CFG Bit Values 7 4 Description VN0 HOM b0000 VN0 Home Messages VN0 SNP b0001 VN0 Snoop Messages VN0 NDR b0010 VN0 Non Data Response Messages VN0 NCS b0011 VN0 Non Coherent Standard Messages...

Страница 104: ...ages VN0 HOM b001xxxxx VN0 Home Messages VN0 ALL b00111111 VN0 All Messages VN1 NCB b01xxxxx1 VN1 Non Coherent Bypass Messages VN1 NCS b01xxxx1x VN1 Non Coherent Standard Messages VN1 DRS b01xxx1xx VN1 Data Response Messages VN1 NDR b01xx1xxx VN1 Non Data Response Messages VN1 SNP b01x1xxxx VN1 Snoop Messages VN1 HOM b011xxxxx VN1 Home Messages VN1 ALL b01111111 VN1 All Messages VNA NCB b10xxxxx1 ...

Страница 105: ...RB_BID_FAIL Title Failed Queue ARB Bids Category QLX Bit s Value 3 0 0x3 Max Inc Cyc 1 Definition Number of Queue ARB bids for input port that were rejected Table 2 60 Unit Masks for QUE_ARB_BID Extension QLX_EVENT _CFG Bit Values 6 4 Description HOM b000 Home Messages SNP b001 Snoop Messages NDR b010 Non Data Response Messages NCS b011 Non Coherent Standard Messages DRS b100 Data Response Message...

Страница 106: ... was available at output port Table 2 62 Unit Masks for TARGET_AVAILABLE Extension QLX_EVENT _CFG Bit Values 7 4 Description VN0 HOM b0000 VN0 Home Messages VN0 SNP b0001 VN0 Snoop Messages VN0 NDR b0010 VN0 Non Data Response Messages VN0 NCS b0011 VN0 Non Coherent Standard Messages VN0 DRS b0100 VN0 Data Response Messages VN0 NCB b1001 VN0 Non Coherent Bypass Messages VN0 VSM b0110 VN0 VNA small ...

Страница 107: ...ween the home node controller B Box and the Intel Scalable Memory Interconnect and basically translates read and write commands into specific memory commands and schedules them with respect to memory timing The other main function of the memory controller is advanced ECC support There are two memory controllers per socket each controlling two Intel SMI channels in lockstep Because of the data path...

Страница 108: ...MMs must be the same type except that all DIMMs attached to an Intel 7500 Scalable Memory Buffer must run with a common frequency and core timings Host lockstep requirements may impose additional requirements on DIMMs on separate Intel SMI channels DDR buses may contain different number of DIMMs zero through two Host lockstep requirements may impose additional requirements on DIMMs on separate Int...

Страница 109: ...GLOBAL_STATUS ov and forwarded up the chain towards the U Box If a M Box0 counter overflows a notification is sent and stored in S Box0 S_MSR_PMON_SUMMARY ov_mb which in turn sends the overflow notification up to the U Box U_MSR_PMON_GLOBAL_STATUS ov_s0 Refer to Table 2 26 S_MSR_PMON_SUMMARY Register Fields to determine how each M Box s overflow bit is accumulated in the attached S Box HW can be a...

Страница 110: ...MU_CNT_CTL_2 RW_RW 0x0CF4 64 M Box 1 PMON Control 2 MB1_CR_M_MSR_PMU_CNT_1 RW_RW 0x0CF3 64 M Box 1 PMON Counter 1 MB1_CR_M_MSR_PMU_CNT_CTL_1 RW_RW 0x0CF2 64 M Box 1 PMON Control 1 MB1_CR_M_MSR_PMU_CNT_0 RW_RW 0x0CF1 64 M Box 1 PMON Counter 0 MB1_CR_M_MSR_PMU_CNT_CTL_0 RW_RW 0x0CF0 64 M Box 1 PMON Control 0 MB1_CR_M_MSR_PMU_ZDP_CTL_FVC RW_RW 0x0CEB 32 M Box 1 PMON SubControl for FVC events MB1_CR_M...

Страница 111: ...SR_PMU_CNT_CTL_2 RW_RW 0x0CB4 64 M Box 0 PMON Control 2 MB0_CR_M_MSR_PMU_CNT_1 RW_RW 0x0CB3 64 M Box 0 PMON Counter 1 MB0_CR_M_MSR_PMU_CNT_CTL_1 RW_RW 0x0CB2 64 M Box 0 PMON Control 1 MB0_CR_M_MSR_PMU_CNT_0 RW_RW 0x0CB1 64 M Box 0 PMON Counter 0 MB0_CR_M_MSR_PMU_CNT_CTL_0 RW_RW 0x0CB0 64 M Box 0 PMON Control 0 MB0_CR_M_MSR_PMU_ZDP_CTL_FVC RW_RW 0x0CAB 32 M Box 0 PMON SubControl for FVC events MB0_...

Страница 112: ... event select Many of the events selected may be broken into components through use of companion subcontrol registers See Section 2 7 7 M Box Performance Monitor Event List for more details The en bit must be set to 1 to enable counting Additional control bits include pmi_en governs what to do if an overflow is detected Field Bits HW Reset Val Description ctr_en 5 0 0 Must be set to enable each MB...

Страница 113: ...d Bits HW Reset Val Description ig 63 0 Read zero writes ignored rsv 62 61 0 Reserved Must write to 0 else behavior is undefined ig 60 25 0 Read zero writes ignored rsv 24 22 0 Reserved Must write to 0 else behavior is undefined set_flag_sel 21 19 0 Selects the set condition for enable flag Secondary event select See Table 2 84 Performance Monitor Events for M Box Events for events elected by this...

Страница 114: ...egisters govern events coming from subunits within the M Box which can be roughly categorized as follows MAP Memory Mapper receives read and write commands addresses from the B Box and translates the received addresses physical into DRAM addresses rank bank row and column The commands and translated addresses are sent to the PLD In parallel the broken DRAM addresses are also sent to the PGT PLD Pa...

Страница 115: ...o the issue logic Once the ISS returns the subcommand choice the oldest DSP entry containing that subcommand kind for a particular DIMM is allowed to execute During subcommand execution the DSP sends the original B Box transaction s FVID that was stored in the DSP entry to the PLD After subcommand execution the DSP s queue entry state is updated to the next required subcommand kind based on the or...

Страница 116: ...tency for this FVID is counted fvid 5 0 0 FVID Fill Victim Index of transaction for which scheduler latency is to be counted Only fully completed transactions are counted Field Bits Access HW Reset Val Reset Type ig 31 10 Reads 0 writes ignored sched_mode_pld_trig 9 7 RW 0 Selects the scheduling mode for which the number of DRAM commands is counted in MA_PLD Here for implementation reasons Uses sa...

Страница 117: ... MAP PMU Event 3 1 Count each time a trigger set up in MAP fires 0 Count every valid response from MBOX This includes responses to B Box as well as response to patrol scrub FSM anycmd 8 0 Count all B Box commands to M Box Event is counted by PGT Event0 cmd 7 5 0 B Box command to count Event is counted by PGT Event0 NOTE anycmd MUST be 0 opn2cls_cnt 4 0 0 Selects FVID Fill Victim Index for which ov...

Страница 118: ...gnored empty_or_clspg 11 0 Selects between counting number of page auto close commands or counting empty page hits 0 Auto Page Close Commands 1 Empty Pages Hits by new access to page table trig_sel 10 9 0 Selects PGT event to be counted 00 Time which PGT stays in open closed page policy 01 Number of PGT Trigger 1 matches 10 Number of PGT Trigger 2 matches trans_cmd_cnt 8 7 0 Selects translated com...

Страница 119: ...with address match as specified by M_CSR_INJ_ERR_ADDR_1 M_CSR_INJ_ERR_CTL_1 match_ and M_CSR_INJ_ERR_CTL_1 inj_err_ fields control the match condition dram_cmd 12 8 0 The DRAM command type to be counted 11110 ZQCAL_SCMD 11101 RCR_SCMD 11100 WCR_SCMD 11000 NOWPE_SCMD 10111 SFT_RST_SCMD 10110 IBD_SCMD 10101 CKEL_SCMD 10100 CKEH_SCMD 10011 POLL_SCMD 10010 SYNC_SCMD 10001 PRE_SCMD 10000 TRKL_SCMD 0111...

Страница 120: ... match on See Table 2 82 M_MSR_PMU_ZDP_CTL_FVC RESP Encodings fvid 4 0 0 FVID to match on Name Value Description smi_nb_trig 0b111 Select Intel SMI Northbound debug event bits from the Intel SMI status frames as returned from the Intel 7500 Scalable Memory Buffers OR PBOX init error see pbox_init_err field These bits are denoted NBDE in the Intel SMI spec status frame description An OR of all the ...

Страница 121: ... Memory commands ras cas pre prefesh preall etc Page hits and page misses Name Value Description spr_uncor_resp 0b111 Uncorrectable response for command to misbehaving DIMM during sparing Reserved 0b110 spr_ack_resp 0b101 Positive acknowledgment for command to misbehaving DIMM during sparing No error was detected for the transaction spec_ack_resp 0b100 Speculative early positive acknowledgment for...

Страница 122: ...HR 1 DIMM Dn Thermal Trip Points Crossed REFRESH 0x06 1 Refresh Commands REFRESH_CONFLICT 0x07 1 Refresh Conflict SCHED_MODE_CHANGES 0x08 1 Scheduling Mode Changes ISS_EV 0x09 ISS 1 ISS Related Events DRAM_CMD 0x0a PLD ISS 1 DRAM Commands PLD_RETRY_EV 0x0b PLD 1 PLD Related Retry Events MA_PAR_ERR 0x0c 1 MA Parity Error FVC_EV0 0x0d FVC 1 FVC Event 0 FVC_EV1 0x0e FVC 1 FVC Event 1 FVC_EV2 0x0f FVC...

Страница 123: ...LES_MFULL Title M Box Full Cycles Category Cycle Events Event Code 0x01 Max Inc Cyc 1 Definition Number of cycles spent in the mfull state Also known as the badly starved state Symbol Name CNT_CTLx 21 19 CNT_CTLx 7 1 Max Inc Cyc Max Inc Cyc Description Events selected by set_flag_sel CYCLES_DSP_FILL 0x0 DSP 1 Time in DSP_FILL State CYCLES_SCHED_MODE 0x1 ISS 1 Time in SCHED_MODE State CYCLES_RETRYQ...

Страница 124: ...01 7 0x1 Max Inc Cyc 1 Definition Counts cycles spent in scheduling mode specified in M_CSR_PMU_ISS sched_mode regis ter DRAM_CMD Title DRAM Commands Category DRAM Commands Event Code 0x0a Max Inc Cyc 1 Definition Count PLD Related DRAM Events NOTE In order to measure a non filtered version of the subevents it is necessary to make sure the PLD Dep bits 13 7 0 are also set to 0 Table 2 86 Unit Mask...

Страница 125: ... 12 8 0x4 0 0x1 9 7 0x0 Count CAS Write no auto precharge open page mode DRAM commands during static trade off scheduling mode CAS_WR_OPN RDPRIO 12 8 0x4 0 0x1 9 7 0x1 Count CAS Write no auto precharge open page mode DRAM commands during static read priority scheduling mode CAS_WR_OPN WRPRIO 12 8 0x4 0 0x1 9 7 0x2 Count CAS Write no auto precharge open page mode DRAM commands during static write p...

Страница 126: ...RAM commands EXSR 12 8 0xB Count Exit Self Refresh DRAM commands NOP 12 8 0xC Count NOP DRAM commands TRKL 12 8 0x10 Count Write Trickle DRAM commands PRE 12 8 0x11 Count PRE DRAM commands SYNC 12 8 0x12 Count SYNC DRAM commands CKE_HI 12 8 0x14 Count CKE High DRAM commands CKE_LO 12 8 0x15 Count CKE Low DRAM commands SOFT_RST 12 8 0x17 Count Soft Reset DRAM commands WR_CFG 12 8 0x1C Count Write C...

Страница 127: ...m B box e g reads from memory BBOX_CMDS WRITES 0x5 0x1 Write commands from B box to M box e g writes to memory BBOX_RSP ACK 0x6 0x0 Counts positive acknowledgements No error was detected BBOX_RSP RETRY 0x6 0x1 Count Retry Responses Possibly a correctable error Retries are generated until it is decided that the error was either correctable or uncorrectable BBOX_RSP COR 0x6 0x2 Counts corrected for ...

Страница 128: ...correctable or uncorrectable BBOX_RSP COR 0x6 0x2 Counts corrected for example after error trials or just by a retry BBOX_RSP UNCOR 0x6 0x3 Count Uncorrectable Responses BBOX_RSP SPEC_ACK 0x6 0x4 Speculative positive acknowledgement for optimized read flow No error was detected for the transaction BBOX_RSP SPR_ACK 0x6 0x5 Count positive acknowledgements for command to misbehaving DIMM during spari...

Страница 129: ...ion BBOX_RSP SPR_ACK 0x6 0x5 Count positive acknowledgements for command to misbehaving DIMM during sparing No error was detected for the transaction 0x6 0x6 nothing will be counted BBOX_RSP SPR_UNCOR 0x6 0x7 Counts Uncorrectable responses to B Box as a result of commands issued to misbehaving DIMM during sparing SMI_NB_TRIG 0x7 Select Intel SMI Northbound debug event bits from Intel SMI status fr...

Страница 130: ...or optimized read flow No error was detected for the transaction BBOX_RSP SPR_ACK 0x6 0x5 Count positive acknowledgements for command to misbehaving DIMM during sparing No error was detected for the transaction 0x6 0x6 nothing will be counted BBOX_RSP SPR_UNCOR 0x6 0x7 Counts Uncorrectable responses to B Box as a result of commands issued to misbehaving DIMM during sparing SMI_NB_TRIG 0x7 Select I...

Страница 131: ...nsactions accessing a bank with no open page The page was previously closed either because it has never been opened was closed via a CASpre closed explicitly by the idle page closing mechanism or closed by a PREALL in order to do a refresh This is command that requires a RAS CAS to complete PAGE_HIT Title Page Table Hit Category Page Table Related Event Code 0x14 Max Inc Cyc 1 Definition Number of...

Страница 132: ...de 0x0b Max Inc Cyc 1 Definition Count PLD Related Retry Events REFRESH Title Refresh Commands Category DRAM Commands Event Code 0x06 Max Inc Cyc 1 Definition Advance counter when a refresh command is detected REFRESH_CONFLICT Title Refresh Conflict Category DRAM Commands Event Code 0x07 Max Inc Cyc 1 Definition Number of refresh conflicts detected A refresh conflict is a conflict between a read w...

Страница 133: ...x Inc Cyc 1 Definition Counts when a specified thermal trip point is crossed in the down direction Extension THR Bits 10 9 3 Description ALL GT_MID_RISE 0x3 0x1 Advance the counter when the above mid temp thermal trip point rising is crossed in the down direction for any DIMM ALL GT_MID_FALL 0x2 0x1 Advance the counter when the above mid temp thermal trip point falling is crossed in the down direc...

Страница 134: ...DIMM ALL GT_MID_FALL 0x2 0x1 Advance the counter when the above mid temp thermal trip point falling is crossed in the up direction for any DIMM ALL GT_LO 0x1 0x1 Advance the counter when the above low temp but below mid temp thermal trip point is crossed in the up direction for any DIMM ALL LT_LO 0x0 0x1 Advance the counter when the below low temp thermal trip point is crossed in the up direction ...

Страница 135: ...NCORE PROGRAMMING GUIDE UNCORE PERFORMANCE MONITORING 2 123 TT_CMD_CONFLICT Title Thermal Throttling Command Conflicts Category Thermal Throttle Event Code 0x19 Max Inc Cyc 1 Definition Count command conflicts due to thermal throttling ...

Страница 136: ... 2 8 2 1 W Box PMU Overflow Freeze and Unfreeze If an overflow is detected from a W Box performance counter the overflow bit is set at the box level W_MSR_PMON_GLOBAL_STATUS ov ov_fixed and forwarded to the U Box where it is also captured U_MSR_PMON_GLOBAL_STATUS ov_w HW can be also configured by setting the corresponding pmi_en to 1 to send a PMI to the U Box when an overflow is detected The U Bo...

Страница 137: ...s Size bits Description W_MSR_PMON_FIXED_CTR_CTL RW_RW 0x395 64 W Box PMON Fixed Counter Control W_MSR_PMON_FIXED_CTR RW_RW 0x394 64 W Box PMON Fixed Counter W_MSR_PMON_CTR_3 RW_RW 0xC97 64 W Box PMON Counter 3 W_MSR_PMON_EVT_SEL_3 RW_RW 0xC96 64 W Box PMON Control 3 W_MSR_PMON_CTR_2 RW_RW 0xC95 64 W Box PMON Counter 2 W_MSR_PMON_EVT_SEL_2 RW_RW 0xC94 64 W Box PMON Control 2 W_MSR_PMON_CTR_1 RW_RW...

Страница 138: ... incoming count is the threshold value then the event count captured in the data register will be incremented by 1 Not present in fixed counter invert Changes the threshold test condition to Not present in fixed counter edge_detect Rather than accumulating the raw count each cycle for events that can increment by 1 per cycle the register can capture transitions from no event to an event incoming N...

Страница 139: ...ption ig 63 0 Read zero writes ignored rsv 62 61 0 Reserved Must write to 0 else behavior is undefined ig 60 51 0 Read zero writes ignored rsv 50 0 Reserved Must write to 0 else behavior is undefined ig 49 32 0 Read zero writes ignored thresh 31 24 0 Threshold used for counter comparison invert 23 0 Invert threshold comparison When 0 the comparison will be thresh event When 1 the comparison will b...

Страница 140: ... events across the uncore Beyond that the W Box provides a smattering of events that indicating when and under what circumstances the W Box throttled the chip due to power constraints 2 8 5 W Box Events Ordered By Code Table 2 102 summarizes the directly measured W Box events 2 8 6 W Box Performance Monitor Event List This section enumerates Intel Xeon Processor 7500 Series uncore performance moni...

Страница 141: ...e 0x03 Max Inc Cyc 1 Definition Selected core is in C0 and is throttling due to FORCEPR assertion C_THROTTLE_TMP Title Core Throttled due to Temp Category W Box Events Event Code 0x00 Max Inc Cyc 1 Definition Temperature of the selected core is at or above the throttle temperature PROCHOT Title Prochot Category W Box Events Event Code 0x02 Max Inc Cyc 1 Definition Package is asserting the PROCHOT ...

Страница 142: ...ter packet traffic according to certain fields A couple common fields the Message Class Opcode fields have been summarized in the following tables Table 2 103 Intel QuickPath Interconnect Packet Message Classes Code Name Definition b0000 HOM0 Home Requests b0001 HOM1 Home Responses b0010 NDR Non Data Responses b0011 SNP Snoops b0100 NCS Non Coherent Standard b1100 NCB Non Coherent Bypass b1110 DRS...

Страница 143: ...CnfltOwn WbEData 0111 NonSnpWr NonSnpWrData 1000 InvItoE RspFwd SnpInvItoE WbIDataPtl 1001 AckCnfltWbI RspFwdI 1010 RspFwdS WbEDataPtl 1011 RspFwdIWb NonSnpWrdataPtl 1100 WbMtoI RspFwdSWb 1101 WbMtoE RspIWb 1110 WbMtoS RspSWb 1111 AckCnflt PrefetchHint Opc NDR NCB NCS 0000 Gnt_Cmp NcWr NcRd 0001 Gnt_FrcAckCnflt WcWr IntAck 0010 0011 FERR 0100 CmpD NcRdPtl 0101 AbortTO NcCfgRd 0110 0111 NcIORd 1000...

Страница 144: ... Series supports getting data in E F or I state DataC_ FEIMS _FrcAckC nflt 0001 DRS Data Response in FEIMS state Force Acknowledge NOTE Set RDS field to specify which state is to be measured Intel Xeon Processor 7500 Series supports getting data in E F or I state DataNc 0011 DRS Uo Non Coherent Data DebugData 1111 NCB Debug Data EvctCln 0110 HOM0 Clean cache line eviction notification to home agen...

Страница 145: ...d The choice between F or S and E is determined by whether or not per caching agent has cache line in S state RdInvOwn 0100 HOM0 Read Invalidate Own requests a cache line in M or E state M or E is determined by whether requester is forwarded an M copy by a peer caching agent or sent an E copy by home agent RspCnflt 0100 HOM1 Peer is left with line in I or S state and the peer has a conflicting out...

Страница 146: ...ushing any M state data to home WbEData 0110 DRS Writeback data downgrade to E state WbEDataPtl 1010 DRS Partial byte masked writeback data downgrade to E state WbIData 0100 DRS Writeback data downgrade to I state WbIDataPtl 1000 DRS Partial byte masked writeback data downgrade to I state WbMtoI 1100 HOM0 Write a cache line in M state back to memory and transition its state to I WbMtoE 1101 HOM0 W...

Отзывы:

Нет отзывов

Похожие инструкции для BX80571E7500 - Core 2 Duo 2.93 GHz Processor

Бренд: D-Link Страницы: 38

Web Smart DES-1210-28P

Бренд: D-Link Страницы: 6

xStack Storage DSN-4000 Series

Бренд: D-Link Страницы: 23

Express EtherNetwork DI-604

Бренд: D-Link Страницы: 7

xStack Storage DSN-4000 Series

Бренд: D-Link Страницы: 97

Бренд: NEC Страницы: 10

Бренд: NEC Страницы: 8

Бренд: Salus Страницы: 16

Бренд: Commell Страницы: 58

Бренд: SeaLevel Страницы: 17

Scorpio WD2500BPVT

Бренд: Western Digital Страницы: 3

Scorpio WD2500BPVT

Бренд: Western Digital Страницы: 4

Бренд: Caen Страницы: 14

4 Channel Input Buffer Board SDAIBB

Бренд: B&B Electronics Страницы: 22

Бренд: OPTO 22 Страницы: 28

Бренд: 3Com Страницы: 53

Developer's Board

Бренд: Sony Ericsson Страницы: 28

CORE 2 DUO E4000 - 3-2008

Бренд: Intel Страницы: 122

Бренды по названию

0 1 2 3 4 5 6 7 8 9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Популярные бренды

Загрузить еще бренды