manualshive.com logo in svg

IBM z13s, Technical Manual

Looking for a comprehensive Technical Manual for your IBM z13s? Look no further! Download the user manual for free from our website and get all the detailed information you need to optimize your usage of this cutting-edge product. Experience seamless performance with the IBM z13s, available for download at manualshive.com.

Share
Download
background image

 Contents 

v

3.4.12  Instruction fetching, decoding, and grouping  . . . . . . . . . . . . . . . . . . . . . . . . . . .  99
3.4.13  Extended Translation Facility  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  99
3.4.14  Instruction set extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  100
3.4.15  Transactional Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  100
3.4.16  Runtime Instrumentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  100

3.5  Processor unit functions  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  100

3.5.1  Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  100
3.5.2  Central processors  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  102
3.5.3  Integrated Facility for Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  103
3.5.4  Internal Coupling Facility. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  103
3.5.5  IBM z Integrated Information Processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  105
3.5.6  System assist processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  109
3.5.7  Reserved processors  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  110
3.5.8  Integrated firmware processor  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  110
3.5.9  Processor unit assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  110
3.5.10  Sparing rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  111
3.5.11  Increased flexibility with z/VM mode partitions . . . . . . . . . . . . . . . . . . . . . . . . .  112

3.6  Memory design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  112

3.6.1  Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  112
3.6.2  Main storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  115
3.6.3  Expanded storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  115
3.6.4  Hardware system area (HSA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  116

3.7  Logical partitioning  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

3.7.1  Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  117
3.7.2  Storage operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  124
3.7.3  Reserved storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  127
3.7.4  Logical partition storage granularity  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  128
3.7.5  LPAR dynamic storage reconfiguration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  129

3.8  Intelligent Resource Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  129
3.9  Clustering technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  131

3.9.1  Coupling Facility Control Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  132
3.9.2  Coupling Thin Interrupts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  133
3.9.3  Dynamic CF dispatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  133
3.9.4  CFCC and Flash Express use. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  135

Chapter 4.  Central processor complex I/O system structure . . . . . . . . . . . . . . . . . . .  137
4.1  Introduction to the InfiniBand and PCIe for I/O infrastructure . . . . . . . . . . . . . . . . . . .  138

4.1.1  InfiniBand I/O infrastructure  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  138
4.1.2  PCIe I/O infrastructure  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  138
4.1.3  InfiniBand specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  139
4.1.4  PCIe Generation 3  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  139

4.2  I/O system overview  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  140

4.2.1  Characteristics  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  140
4.2.2  Summary of supported I/O features  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  141

4.3  I/O drawer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  142
4.4  PCIe I/O drawer  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  143
4.5  PCIe I/O drawer and I/O drawer offerings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  146
4.6  Fanouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  147

4.6.1  PCIe Generation 3 fanout (FC 0173)  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  149
4.6.2  HCA2-C fanout (FC 0162). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  149
4.6.3  Integrated Coupling Adapter (FC 0172)  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  149
4.6.4  HCA3-O (12x IFB) fanout (FC 0171). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  150
4.6.5  HCA3-O LR (1x IFB) fanout (FC 0170) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  151

Summary of Contents for z13s

Page 1: ...partnership with IBM Academy of Technology Front cover IBM z13s Technical Guide Octavian Lascu Barbara Sannerud Cecilia A De Leon Edzard Hoogerbrug Ewerson Palacio Franco Pinto Jin J Yang John P Troy Martin Soellig ...

Page 2: ......

Page 3: ...International Technical Support Organization IBM z13s Technical Guide June 2016 SG24 8294 00 ...

Page 4: ...S Government Users Restricted Rights Use duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp First Edition June 2016 This edition applies to IBM z13s servers Note Before using this information and the product it supports read the information in Notices on page xix ...

Page 5: ... connectivity PCIe and InfiniBand 16 1 3 6 I O subsystem 16 1 3 7 Parallel Sysplex Coupling and Server Time Protocol connectivity 20 1 3 8 Special purpose features 23 1 3 9 Reliability availability and serviceability 26 1 4 Hardware Management Consoles and Support Elements 27 1 5 IBM z BladeCenter Extension zBX Model 004 27 1 5 1 Blades 28 1 5 2 IBM WebSphere DataPower Integration Appliance XI50 f...

Page 6: ...y identifier and MSU values 73 2 7 5 Capacity BackUp 74 2 7 6 On Off Capacity on Demand and CPs 76 2 8 Power and cooling 77 2 8 1 Power considerations 77 2 8 2 High voltage DC power 78 2 8 3 Internal Battery Feature 78 2 8 4 Power capping 78 2 8 5 Power Estimation tool 78 2 8 6 Cooling requirements 79 2 9 Summary of z13s structure 79 Chapter 3 Central processor complex system design 81 3 1 Overvie...

Page 7: ...e operations 124 3 7 3 Reserved storage 127 3 7 4 Logical partition storage granularity 128 3 7 5 LPAR dynamic storage reconfiguration 129 3 8 Intelligent Resource Director 129 3 9 Clustering technology 131 3 9 1 Coupling Facility Control Code 132 3 9 2 Coupling Thin Interrupts 133 3 9 3 Dynamic CF dispatching 133 3 9 4 CFCC and Flash Express use 135 Chapter 4 Central processor complex I O system ...

Page 8: ...channel subsystem 187 5 1 Channel subsystem 188 5 1 1 Multiple logical channel subsystems 189 5 1 2 Multiple subchannel sets 190 5 1 3 Channel path spanning 193 5 2 I O configuration management 196 5 3 Channel subsystem summary 196 Chapter 6 Cryptography 199 6 1 Cryptography in IBM z13 and z13s servers 200 6 2 Some fundamentals on cryptography 200 6 2 1 Modern cryptography 200 6 2 2 Kerckhoffs pri...

Page 9: ...AR memory upgrade 252 7 3 13 LPAR physical capacity limit enforcement 252 7 3 14 Capacity Provisioning Manager 253 7 3 15 Dynamic PU add 253 7 3 16 HiperDispatch 254 7 3 17 The 63 75 K subchannels 255 7 3 18 Multiple Subchannel Sets 256 7 3 19 Three subchannel sets 256 7 3 20 IPL from an alternative subchannel set 257 7 3 21 Modified Indirect Data Address Word facility 257 7 3 22 HiperSockets Comp...

Page 10: ...n for z VM 278 7 3 61 QDIO interface isolation for z OS 278 7 3 62 QDIO optimized latency mode 279 7 3 63 Large send for IPv6 packets 279 7 3 64 OSA Express5S and OSA Express4S checksum offload 280 7 3 65 Checksum offload for IPv4and IPv6 packets when in QDIO mode 280 7 3 66 Adapter interruptions for QDIO 280 7 3 67 OSA Dynamic LAN idle 281 7 3 68 OSA Layer 3 virtual MAC for z OS environments 281 ...

Page 11: ...307 7 16 4 Advanced Entry Workload License Charge 308 7 16 5 System z New Application License Charges 308 7 16 6 Midrange workload license charges 308 7 16 7 Parallel Sysplex License Charges 309 7 16 8 z Systems International Program License Agreement 309 7 16 9 zBX licensed software 310 7 17 References 310 Chapter 8 System upgrades 311 8 1 Upgrade types 312 8 1 1 Overview of upgrade types 312 8 1...

Page 12: ...63 BladeCenter components 363 zBX firmware 364 9 6 1 zBX RAS and the IBM z Unified Resource Manager 364 9 6 2 zBX Model 004 2458 004 364 9 7 Considerations for PowerHA in zBX environment 366 9 8 IBM z Advanced Workload Analysis Reporter 367 9 9 RAS capability for Flash Express 368 Chapter 10 Environmental requirements 369 10 1 IBM z13s power and cooling 370 10 1 1 Power and I O cabling 370 10 1 2 ...

Page 13: ...and Enhanced IBM Service Support System 404 11 4 3 HMC and SE remote operations 404 11 5 HMC and SE key capabilities 405 11 5 1 Central processor complex management 405 11 5 2 Logical partition management 406 11 5 3 Operating system communication 407 11 5 4 HMC and SE microcode 409 11 5 5 Monitoring 411 11 5 6 Capacity on demand support 415 11 5 7 Features on Demand support 416 11 5 8 Server Time ...

Page 14: ... interface 464 B 3 5 IBM zAware complements your existing tools 464 B 4 IBM zAware prerequisites 464 B 4 1 IBM zAware features and ordering 464 12 7 3 Feature on Demand FoD 465 B 4 2 IBM zAware operating requirements 467 B 5 Configuring and using IBM zAware virtual appliance 468 Appendix C Channel options 471 C 1 Channel options supported on z13s servers 472 C 2 Maximum unrepeated distance for FIC...

Page 15: ...ement Console 515 F 2 4 Open source virtualization 515 F 2 5 What comes with KVM for IBM z Systems 516 F 3 Managing the KVM for IBM z Systems environment 518 F 3 1 Hypervisor Performance Manager 520 F 4 Using IBM Cloud Manager with OpenStack 520 Appendix G Native Peripheral Component Interconnect Express 521 G 1 Design of native PCIe I O adapter management 522 G 1 1 Native PCIe adapter 522 G 1 2 I...

Page 16: ... zEnterprise Data Compression Express 551 J 1 Overview 552 J 2 zEDC Express 552 J 3 Software support 553 J 3 1 z VM V6R3 support with PTFs 554 J 3 2 IBM SDK 7 for z OS Java support 554 J 3 3 IBM z Systems Batch Network Analyzer 554 Related publications 555 IBM Redbooks 555 Other publications 555 Online resources 555 Help from IBM 555 ...

Page 17: ...0 two CPC drawer Locations of the PCIe and IFB fanouts 66 2 25 Redundant I O interconnect for I O drawer 68 2 26 Redundant I O interconnect for PCIe I O drawer 69 2 27 z13s upgrade paths 71 3 1 z13s cache levels and memory hierarchy 84 3 2 z13s cache topology 85 3 3 z13s and zBC12 cache level comparison 86 3 4 z13s CPC drawer communication topology 87 3 5 Point to point topology z13s two CPC drawe...

Page 18: ...usage 300 8 1 The provisioning architecture 320 8 2 Example of temporary upgrade activation sequence 321 8 3 Memory sizes and upgrades for the N10 325 8 4 Memory sizes and upgrades for the single drawer N20 326 8 5 Memory sizes and upgrades for the two drawer N20 326 8 6 Feature on Demand window for zBX Blades features HWMs 327 8 7 Permanent upgrade order example 329 8 8 CIU eligible order activat...

Page 19: ...BX components 422 11 30 Cryptographic Configuration window 426 11 31 Enabling Dynamic Partition Manager 427 11 32 HMC welcome window CPC in DPM Mode 428 11 33 Unified Resource Manager functions and suites 429 11 34 Ensemble example with primary and alternate HMCs 435 12 1 z13s to zBC12 z114 z10 BC and z9 BC performance comparison 438 12 2 Memory hierarchy on the z13s one CPC drawer system two node...

Page 20: ... window 506 E 5 Traditional HMC Welcome window when no CPCs are running in DPM mode 507 E 6 User Options when the HMC presents the DPM welcome window 507 E 7 DPM wizard welcome window options 509 F 1 KVM running in z Systems LPARs 514 F 2 Open source virtualization KVM for IBM z Systems 516 F 3 KVM for IBM z Systems management interface 518 F 4 KVM management by using the libvirt API layers 519 G ...

Page 21: ...IBM websites are provided for convenience only and do not in any manner serve as an endorsement of those websites The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk IBM may use or distribute any of the information you provide in any way it believes appropriate without incurring any obligation to you The performance data ...

Page 22: ...ctor Active Energy Manager IBM z Systems IBM z13 IBM IMS Language Environment Lotus MVS NetView OMEGAMON Parallel Sysplex Passport Advantage Power Systems POWER6 POWER7 PowerHA PowerPC PowerVM PR SM Processor Resource Systems Manager RACF Redbooks Redpaper Redpapers Redbooks logo Resource Link Resource Measurement Facility RMF System Storage System z10 System z9 System z SystemMirror Tivoli VIA VT...

Page 23: ...iOS Android Place a Sponsorship Promotion in an IBM Redbooks publication featuring your business or solution with a link to your web site Qualified IBM Business Partners may place a full page promotion in the most popular Redbooks publications Imagine the power of being seen by users who download millions of Redbooks publications each year Promote your business in an IBM Redbooks publication ibm co...

Page 24: ...THIS PAGE INTENTIONALLY LEFT BLANK ...

Page 25: ... 4 TB of client memory The IBM z13s Model N20 is estimated to provide up to 100 more total system capacity than the IBM zEnterprise BC12 Model H13 This book provides information about the IBM z13s server and its functions features and associated software support Greater detail is offered in areas relevant to technical planning It is intended for systems engineers consultants planners and anyone wh...

Page 26: ...r for the last generations of the IBM high end servers Ewerson leads the Mainframe Specialty Services Area MF SSA which is part of GTS Delivery Technology and Engineering DT E He is a member of the IBM Academy of Technology Franco Pinto is a Client Technical Specialist in IBM Switzerland He has 20 years of experience in the mainframe and z OS fields His areas of expertise include z Systems technic...

Page 27: ...ing edge technologies Your efforts will help to increase product acceptance and customer satisfaction as you expand your network of technical contacts and relationships Residencies run from two to six weeks in length and you can participate either in person or as a remote resident working from your home base Find out more about the residency program browse the residency index and apply online at i...

Page 28: ...us on Facebook http www facebook com IBMRedbooks Follow us on Twitter http twitter com ibmredbooks Look for us on LinkedIn http www linkedin com groups home gid 2130806 Explore new Redbooks publications residencies and workshops with the IBM Redbooks weekly newsletter https www redbooks ibm com Redbooks nsf subscribe OpenForm Stay current on recent Redbooks publications with RSS Feeds http www red...

Page 29: ...omers with the agility and insight they need to succeed in today s marketplace The IT infrastructure must always be on to serve customers in a 24 x 7 global business have flawless data integrity and be capable of thwarting cyber security threats It needs to be resilient to support the introduction of new workloads and handle spikes in processing while meeting the most demanding service levels It m...

Page 30: ...on z13s servers sets the groundwork for digital business by providing support for demanding workloads such as operational analytics and cloud along with traditional mission critical applications z OS V2 2 continues to support the IBM z Integrated Information Processor zIIP 3 that can take advantage of the simultaneous multithreading SMT feature implemented in the IBM z Systems processor unit PU fo...

Page 31: ...ture zACI Dynamic Partition Manager DPM 1 2 1 Processor and memory IBM continues its technology leadership at a consumable entry point with the z13s servers z13s servers are built using the IBM modular multi drawer design that supports 1 or 2 CPC drawers per CPC Each CPC drawer contains eight core SCMs with either 6 or 7 cores enabled for use The SCMs host the redesigned complementary metal oxide ...

Page 32: ...ation The second drawer for an N20 is added when more than 2 TB of memory or more I O features are needed IBM z13s Model Capacity Identifier A01 is estimated to provide 60 more total system capacity than the zBC12 Model capacity Identifier A01 with the same amount of memory and power requirements With up to 4 TB of main storage and SMT the performance of the z13s processors provide considerable im...

Page 33: ...anually or the z OS Capacity Provisioning Manager can automatically start the activation when Workload Manager WLM policy thresholds are reached Tokens are available that can be purchased for On Off CoD either before or after workload execution pre or post paid LPAR group absolute capping IBM PR SM and the Hardware Management tool have been enhanced to limit the amount of physical processor capaci...

Page 34: ...nd short wavelength SX FICON Express8S 10 KM long wavelength LX and short wavelength SX FICON Express8 10 KM LX and SX carry forward only Networking connectivity Open Systems Adapter OSA OSA Express5S 10 GbE LR and SR OSA Express5S GbE LX and SX OSA Express5S 1000BASE T Ethernet OSA Express4S 10 GbE LR and SR carry forward only OSA Express4S GbE LX and SX carry forward only OSA Express4S 1000BASE ...

Page 35: ...e that uses RDMA over Converged Ethernet and is designed to provide fast memory to memory communications between two z Systems CPCs It is transparent to applications Use of the 10GbE RoCE Express feature helps reduce CPU consumption for applications that use the TCP IP stack such as IBM WebSphere Application Server accessing a DB2 database on z OS It can also help reduce network latency with memor...

Page 36: ...sly existing compression technology on z Systems zEDC compression provides these advantages among others QSAM BSAM for better disk utilization and batch elapsed time improvements SMF for increased availability and online storage reduction DFSMSdss for better disk and tape utilization for backup data Java for high throughput standard compression through java util zip Encryption Facility for z OS fo...

Page 37: ... guests that use Linux on z Systems on IFLs or CPs and manage IBM z VSE z TPF and z OS guests on CPs Run designated z OS workloads such as parts of IBM DB2 Distributed Relational Database Architecture DRDA processing and XML on zIIPs Coupling Facility mode logical partition Parallel Sysplex is the clustering technology used with z13s servers To use this technology a special LIC is used This code i...

Page 38: ... in LPARs Both provide network access for TCP IP socket applications that use the Linux Fast Path However no TCP IP stack is required on the z VSE system and no Linux on z Systems needs to be installed IBM zAware IBM zAware is a feature that was introduced with the zEC12 that provides the next generation of system monitoring IBM zAware is designed to offer a near real time continuous learning diag...

Page 39: ...without the need to reproduce the context of an event With a properly configured z13s server further reduction of outages can be attained through improved nondisruptive replace repair and upgrade functions for memory drawers and I O adapters In addition z13s servers have extended nondisruptive capability to download and install Licensed Internal Code LIC updates Enhancements include removing pre p...

Page 40: ... purchase can assume any of the following characterizations Central processor CP for general purpose use Integrated Facility for Linux IFL for the use of Linux on z Systems z Systems Integrated Information Processor zIIP7 One CP must be installed with or before the installation of any zIIPs ICF is used by the CFCC Extra SAPs are used by the channel subsystem A PU that is not characterized cannot b...

Page 41: ... but all z13s upgrade paths are disruptive zBC12 upgrade to z13s server When a BC12 is upgraded to a z13s server the zBC12 must be at least at Driver level 15 If a zBX is involved Driver 15 must be at Bundle 22 or later The following processes are not supported Reverting to an earlier level within the z13 or z13s models Upgrades from IBM System z10 or earlier systems Attachment of a zBX Model 002 ...

Page 42: ... Processor features The processor chip runs at 4 3 GHz Depending on the model either 13 PUs N10 or 26 PUs N20 are available Each core on the PU chip includes an enhanced dedicated coprocessor for data compression and cryptographic functions which are known as the Central Processor Assist for Cryptographic Function CPACF 8 Hardware data compression can play a significant role in improving performan...

Page 43: ... has a set of instructions that allows defining groups of instructions that are run atomically that is either all the results are committed or none are The facility provides for faster and more scalable multi threaded execution and is known as hardware transactional memory Out of order execution As with its predecessor zBC12 a z13s server has an enhanced superscalar microprocessor with OOO executi...

Page 44: ... its associated I O features by using redundant I O interconnect to the I O drawer Up to four HCA fanouts are available per CPC drawer 1 3 5 I O connectivity PCIe and InfiniBand z13s servers offer various improved I O features and uses technologies such as PCIe and InfiniBand This section briefly reviews the most relevant I O capabilities z13s servers take advantage of PCIe Generation 3 to impleme...

Page 45: ...the traditional PCIe features The device drivers for these adapters are available in the operating system The diagnostic tests for the adapter layer functions of the native PCIe features are managed by LIC that is designated as a resource group partition which runs on the IFP For availability two resource groups are present and share the IFP During the ordering process of the native PCIe features ...

Page 46: ...for programs that process data sets that use striping and compression such as DB2 VSAM partitioned data set extended PDSE hierarchical file system HFS and z OS file system zFS It does so by reducing channel director and control unit processor usage Enhanced problem determination analysis and manageability of the storage area network SAN by providing registration information to the fabric name serv...

Page 47: ...4S GbE Short Wave OSA Express4S 1000BASE T Ethernet OSA Express features provide important benefits for TCP IP traffic which are reduced latency and improved throughput for standard and jumbo frames Performance enhancements are the result of the data router function being present in all OSA Express features For functions that were previously performed in firmware the OSA Express5S and OSA Express4...

Page 48: ...ore information see Appendix D Shared Memory Communications on page 475 Shared Memory Communications Direct Memory Access In addition to supporting SMC R z13s support is a new protocol called SMC D over ISM Unlike SMC R SMC D has no dependency on the RoCE Express feature SMC D provides low latency communications within a CPC by using shared memory SMC R by using RDMA can vastly improve the transac...

Page 49: ...IFB type has twice the number of links per fanout card as compared to type HCA2 O LR 1xIFB that was used in the previous z Systems generations Internal Coupling Channels ICs Operate at memory speed CFCC Level 21 CFCC level 21 is delivered on z13s servers with driver level 27 CFCC Level 21 introduces the following enhancements Support for up to 20 ICF processors per z Systems CPC The maximum number...

Page 50: ...e STP code on the z13 z13s zEC12 zBC12 z196 and z114 servers With this function the z13 z13s zEC12 zBC12 z196 and z114 servers can be configured to use an NTP server as an external time source ETS This implementation answers the need for a single time source across the heterogeneous platforms in the enterprise An NTP server becomes the single time source for the z13 z13s zEC12 zBC12 z196 and z114 ...

Page 51: ... cryptographic solution that is implemented in z Systems has received the highest standardized security certification FIPS 140 2 Level 49 In addition to the integrated cryptographic features the cryptographic features Crypto Express5S the only crypto card that is supported on z13s servers allows adding or moving crypto coprocessors to LPARs without pre planning z13s servers implement PKCS 11 one o...

Page 52: ...tographic coprocessor mode introduced the PKCS 11 secure key function An accelerator for public key and private key cryptographic operations is used with Secure Sockets Layer Transport Layer Security SSL TLS acceleration The Crypto Express5S is designed to meet these cryptographic standards among others FIPS 140 2 Level 4 ANSI 9 97 Payment Card Industry PCI HSM Deutsche Kreditwirtschaft DK FIPS 14...

Page 53: ...ess card implements an internal NAND Flash SSD and has a capacity of 1 4 TB of usable storage Cards are installed in pairs which provide mirrored data to ensure a high level of availability and redundancy A maximum of four pairs of cards four features can be installed on a z13s server for a maximum capacity of 5 6 TB of storage The Flash Express feature recently enhanced is designed to allow each ...

Page 54: ...press on page 551 1 3 9 Reliability availability and serviceability The z13 and z13s RAS strategy employs a building block approach which is developed to meet the client s stringent requirements for achieving continuous reliable operation Those building blocks are error prevention error detection recovery problem determination service structure change management measurement and analysis The initia...

Page 55: ...d the zBX Model 004 machines Also when the z Systems and a zBX Model 004 are members of an ensemble the HMC10 has a global ensemble management scope which is compared to the SEs on the zBX Model 004 and on the CPCs which have local node management responsibility When tasks are performed on the HMC the commands are sent to one or more CPC SEs or zBX Model 004 SEs which then issue commands to their ...

Page 56: ...failover for automated multi site recovery These capabilities facilitate the management of planned and unplanned outages 1 5 1 Blades Two types of blades can be installed and operated in the IBM z BladeCenter Extension zBX Optimizer Blades IBM WebSphere DataPower Integration Appliance XI50 for zBX blades IBM Blades A selected subset of IBM POWER7 blades A selected subset of IBM BladeCenter HX5 bla...

Page 57: ...se System Network management Networks Creates and manages virtual networks including access control which allows virtual servers to be connected Workload Awareness and platform performance management Performance Manages CPU resource across virtual servers that are hosted in the same hypervisor instance to achieve workload performance policy objectives The Unified Resource Manager provides energy m...

Page 58: ...nded that these be on separate features for primary and backup SE connectivity 1 8 Operating systems and software IBM z13s servers are supported by a large set of software including independent software vendor ISV applications This section lists only the supported operating systems Use of various features might require the latest releases For more information see Chapter 7 Software support on page...

Page 59: ...er with support for IBM WebSphere software IBM z13s servers provide full support for SOA web services Java Platform Enterprise Edition Linux and Open Standards The z13s server is intended to be a platform of choice for the integration of the newest generations of applications with existing applications and data z Systems software is also designed to take advantage of the many enhancements on z13 a...

Page 60: ...e but are not limited to the following z OS V2 2 supports zIIP processors in SMT mode to help improve throughput for zIIP workloads This support is also available for z OS V2 1 with PTFs z OS V2 2 supports up to 141 processors per LPAR or up to 128 physical processors per LPAR in SMT mode z OS V2 2 also supports up to 4 TB of real memory per LPAR This support is also available on z OS V2 1 with PT...

Page 61: ...ic Support for z OS V1R13 z OS V2R2 This web deliverable is available for download from http www ibm com systems z os zos tools downloads More support for the z13 processor family is planned with the ICSF Cryptographic Support for z OS V1R13 z OS V2R2 web deliverable in PTFs in the first quarter of 2016 Support for the new TKE 8 1 workstation 1 8 2 IBM compilers The following IBM compilers for z S...

Page 62: ...13s servers Because specifying the architecture level of 11 results in a generated application that uses instructions that are available only on the z13s or z13 servers the application will not run on earlier versions of the hardware If the application must run on z13s servers and on older hardware specify the architecture level corresponding to the oldest hardware on which the application needs t...

Page 63: ...ain the z13s hardware building blocks and how these components interconnect from a physical point of view This information can be useful for planning purposes and can help to define configurations that fit your requirements This chapter provides information about the following topics Frame and drawers Processor drawer concept Single chip modules Memory Reliability availability and serviceability C...

Page 64: ...A frame The external LAN interface connections are now directly connected to the SEs at the rear of the system Optional Internal Battery Features IBFs that provide the function of a local uninterrupted power source The IBF further enhances the robustness of the power design increasing power line disturbance immunity It provides battery power to preserve processor data in a loss of power on both po...

Page 65: ...troller SCM SC SCM and the associated memory slots The N10 model is always the single CPC drawer Model N20 CPC drawer contains two nodes four PU SCMs and two SC SCMs The N20 model can be a one or two CPC drawers system Memory dual inline memory modules DIMMs point of load regulators and fanout cards for internal and external connectivity A newly designed swing tray that is in front of the I O draw...

Page 66: ...wo CPC Drawers I O Drawer Configurations 2 1 2 PCIe I O drawer and I O drawer features Each CPC drawer has PCIe Generation3 fanout slots and InfiniBand fanout slots to support two types of I O infrastructure for data transfer PCIe I O infrastructure with a bandwidth of 16 GBps InfiniBand I O infrastructure with a bandwidth of 6 GBps N10 N20 1 BPA CPC 1 6 13 21 25 29 31 39 41 42 N10 N20 1 BPA CPC P...

Page 67: ...llation as a Secure IBM Common Cryptographic Architecture CCA coprocessor as a Secure IBM Enterprise Public Key Cryptography Standards PKCS 11 EP11 coprocessor or as an accelerator Flash Express Each Flash Express feature occupies two slots in the PCIe I O Drawer but does not have a CHPID type Logical partitions LPARs in all channel subsystems CSSs have access to the features 10 GbE Remote Direct ...

Page 68: ... L4 cache Four PCIe fanout slots Two InfiniBand fanout slots One memory controller per PU chip Five DDR3 DIMM slots per memory controller Two Flexible Support Processors FSPs Node 1 resources SCMs memory and fanouts are not installed in the Model N10 Two node drawer contains Figure 2 5 on page 41 Four PU chips up to 26 active PUs Two SC chips 960 MB L4 cache Eight PCIe fanout slots Four InfiniBand...

Page 69: ...2 6 shows the front view of the CPC drawer with fanout slots and FSP slots for both a Model N20 and a Model N10 Figure 2 6 CPC drawer front view 20 Memory DIMM Slots InfiniBand fanoutslots x4 NODE 0 PCIefanout slots x4 Front Rear 4x PU SCMs air cooled 2x SC SCMs air cooled PCIefanout slots x4 NODE 1 FSP FSP ...

Page 70: ...the same node The S bus provides interconnects between SC chips in the same drawer The A bus provides interconnects between SC chips in different drawers through SMP cables Processor support interfaces PSIs are used to communicate with FSP cards for system control 1 The drawing shows eight cores per PU chip This is by design The PU chip is a FRU shared with the z13 When installed in a z13s server ...

Page 71: ...ut The SEs provide the Simple Network Time Protocol SNTP client function When Server Time Protocol STP is used the time of an STP only Coordinated Timing Network CTN can be synchronized to the time that is provided by an NTP server This configuration allows time of day TOD synchronization in a heterogeneous platform environment The accuracy of an STP only CTN is improved by using an NTP server wit...

Page 72: ...ements are managed through the flexible service processors FSPs An FSP is based on the IBM PowerPC microprocessor technology Each FSP card has two ports that connect to two internal Ethernet LANs through system control hubs SCH1 and SCH2 The FSPs communicate with the SEs and provide a subsystem interface SSI for controlling components Tip STP is available as FC 1021 It is implemented in the Licens...

Page 73: ...plexed n 1 and each element has at least one FSP Two internal Ethernet LANs are used by two SEs for redundancy Crossover capability between the LANs is available so that both SEs can operate on both LANs The HMCs and SEs are connected directly to one or two Ethernet Customer LANs One or more HMCs can be used Note The maximum number of drawers CEC and I O is four for z13s servers The diagram in Fig...

Page 74: ...rom the rear of the frame During a blower failure the adjacent blower increases speed to satisfy cooling requirements until the failed blower is replaced Figure 2 11 Redundant DCAs and blowers for CPC drawers 2 3 Single chip modules The SCM is a multi layer metal substrate module that holds either one PU chip or a SC chip Its size is 678 76 mm2 28 4 mm x 23 9 mm Each node of a CPC drawer has three...

Page 75: ...nectivity between the CPC drawers is accomplished through SMP connectors and cables One inter drawer connection per node of the CPC drawer is used for the two drawer Model N20 configuration This configuration allows a multi drawer system to be displayed as a symmetric multiprocessor SMP system Each node has three SCMs Two PU SCMs and one SC SCM 2 3 2 Processor unit PU chip The z13s PU chip install...

Page 76: ...dual address sliced and dual store pipe support an integrated on chip coherency manager cache and cross bar switch The L3 directory filters queries from the local L4 Both L3 slices can deliver up to 16 GBps bandwidth to each core simultaneously The L3 cache interconnects the eight cores GX I O buses PCIe I O buses and memory controllers MCs with SC chips The MC function controls access to memory T...

Page 77: ... and handling of instruction resource dispatch This unit is also central to performance measurement through a function called instrumentation Instruction fetch and branch IFB prediction and instruction cache and merge ICM These two subunits IFB and ICM contain the instruction cache branch prediction logic instruction fetching controls and buffers The relative size of these subunits is the result o...

Page 78: ...on about PU characterization see 3 5 Processor unit functions on page 100 The maximum number of characterized PUs depends on the z13s model Some PUs are characterized by the system as standard system assist processors SAPs to run the I O processing By default there are at least two spare PUs per Model N20 system that are used to assume the function of a failed PU The remaining installed PUs can be...

Page 79: ...dle 125 simultaneous cache transactions per chip The L3 caches on PU chips communicate with the L4 caches through the attached SC chip by using unidirectional buses L3 is divided into two logical slices Each slice is 32 MB and consists of two 16 MB banks L3 is 16 way set associative Each bank has 4 K sets and the cache line size is 256 bytes The bus clock ratio 2 1 between the L4 cache and the PU ...

Page 80: ...ctive cores has 64 MB and is designed as a store in cache Cache levels L2 and L3 are implemented on the PU chip to reduce the latency between the processor and the L4 large shared cache which is on the two SC chips Each SC chip has 480 MB which is shared by PU chips on the node The S bus provide the inter node interface between the two L4 caches SC chips in each node The L4 cache uses a store in d...

Page 81: ...installed memory is 160 GB per CPC drawer The minimum initial amount of customer memory that can be ordered is 64 GB for all z13 models The maximum customer memory size is based on the physical installed memory minus the RAIM 20 of physical and the hardware system area HSA memory which has a fixed amount of 40 GB Table 2 5 shows the memory granularity that is based on the installed customer memory...

Page 82: ...drawer Model N20 can have different but not mixed DIMM sizes in each drawer Note The required granularity for all main storage fields of an LPAR for which an origin has been specified for example initial main storage amount reserved main storage amount and main storage origin is fixed at 2 GB This configuration helps to simplify customer management of absolute storage regarding 2 GB large page sup...

Page 83: ...component can be detected and corrected dynamically This system simplifies the memory subsystem design while maintaining a fully fault tolerant RAIM design The RAIM design provides the following layers of memory recovery ECC with 90B 64B Reed Solomon code DRAM failure with marking technology in which two DRAMs can be marked and no half sparing is needed A call for replacement occurs on the third D...

Page 84: ...ve memory slots DIMM changes require a disruptive IML on z13s Model N10 memory configurations The memory in Model N10 can be configured in the following manner Ten DIMM slots per N10 drawer supported by two memory controllers on PU chips with five slots each Any remaining unpopulated DIMM banks MD06 MD10 and MD11 MD15 must be plugged with memory DIMM airflows Drawer memory configuration based on D...

Page 85: ... unpopulated DIMM banks MD06 MD10 and MD21 MD25 must be plugged with memory DIMM airflows Drawer memory configurations based on DIMM sizes of 16 32 64 128 supporting up to 2008 GB customer available memory in the drawer HSA size 40 GB is fixed and is not taken from customer purchased memory Customer Memory Total Physical Increment Node 0 DIMM location size GB GB GB GB MD16 MD20 MD21 MD25 Dial Max ...

Page 86: ... by node for new build systems Table 2 7 Model N20 single CPC drawer physically installed memory Memory DIMMS MD16 MD20 Front NODE 0 NODE 1 Rear Memory DIMMS MD21 MD25 Memory DIMMS MD06 MD10 Memory DIMMS MD11 MD15 PU PU PU PU Cust Mem Total Phys Increm ent Node 1 DIMM location size GB Node 0 DIMM location size GB GB GB GB MD06 MD10 MD11 MD15 MD16 MD20 MD21 MD25 Dial Max 64 160 8 16 16 88 72 16 16 ...

Page 87: ... GB customer available memory per system HSA size 40 GB is fixed and is not taken from customer purchased memory 120 320 32 16 16 16 16 216 152 16 16 16 16 184 16 16 16 16 216 16 16 16 16 248 640 32 32 32 32 472 280 32 32 32 32 312 32 32 32 32 344 32 32 32 32 408 64 32 32 32 32 472 32 32 32 32 536 1280 64 64 64 64 984 600 64 64 64 64 728 128 64 64 64 64 856 64 64 64 64 984 64 64 64 64 1112 2560 12...

Page 88: ...ysically installed memory Memory DIMMS MD16 MD20 Front NODE 0 NODE 1 Rear Memory DIMMS MD21 MD25 Memory DIMMS MD06 MD10 Memory DIMMS MD11 MD15 PU PU PU PU Drawer 0 Drawer 1 Model N20 Cust Mem Physi cal Increm ent Drawer 1 Drawer 0 Second drawer GB GB GB Node 1DIMM loc size Node 0 D IMM loc size Node 1 DIMM loc size Node 0 DIMM loc size Dial Max MD06 MD10 MD11 MD15 MD16 MD20 MD21 MD25 MD06 MD10 MD1...

Page 89: ...4 16 16 728 128 64 64 64 64 16 16 856 64 64 64 64 16 16 984 64 64 64 64 16 16 1112 2720 128 128 128 128 16 16 2008 1240 128 128 128 128 16 16 1368 128 128 128 128 16 16 1496 128 128 128 128 16 16 1624 128 128 128 128 16 16 1752 128 128 128 128 16 16 1880 128 128 128 128 16 16 2008 128 128 128 128 16 16 Cust Mem Physi cal Increm ent Drawer 1 Drawer 0 Second drawer GB GB GB Node 1DIMM loc size Node ...

Page 90: ... 16 2264 2520 3200 128 128 128 128 32 32 32 32 2520 2776 3840 128 128 128 128 64 64 64 64 3032 3032 128 128 128 128 64 64 64 64 3288 5120 128 128 128 128 128 128 128 128 4056 3544 128 128 128 128 128 128 128 128 3800 128 128 128 128 128 128 128 128 4056 128 128 128 128 128 128 128 128 Cust Mem Physi cal Increm ent Drawer 1 Drawer 0 Second drawer GB GB GB Node 1DIMM loc size Node 0 D IMM loc size N...

Page 91: ...enabled using Licensed Internal Code Configuration Control LICCC by upgrading the DIMM cards by adding new DIMM cards or by adding a CPC drawer For a model upgrade that results in the addition of a CPC drawer the minimum memory increment is added to the system Each CPC drawer has a minimum physical memory size of 160 GB During a model upgrade adding a CPC drawer is a disruptive operation Adding ph...

Page 92: ...RAS features of z Systems servers A planned outage because of the addition of extra processor capacity A planned outage because of the addition of extra I O cards An unplanned outage because of a failure of a power supply An unplanned outage because of a memory failure The z Systems hardware has decades of intense engineering behind it which has resulted in a robust and reliable platform The hardw...

Page 93: ...hip connectors and use SER hardened latches throughout the design z13s servers have N 2 point of load POL power conversion This redundancy protects the processor from the loss of voltage because of POL failures z13s servers have N 2 redundancy on the environmental sensors ambient temperature relative humidity air density2 and corrosion Enhanced bus structure using integrated time domain reflectome...

Page 94: ...undant I O interconnect on page 67 Five types of fanouts are available PCIe Generation 3 fanout card This copper fanout provides connectivity to the PCIe switch cards in the PCIe I O drawer Host Channel Adapter HCA2 C This copper fanout provides connectivity to the IFB MP cards in the I O drawer Integrated Coupling Adapter ICA SR This adapter provides coupling connectivity between z13s and z13 z13...

Page 95: ...ins two CPC drawers two PCIe I O drawers and PCIe ICA Coupling adapters All PCIe Generation 3 fanouts are plugged in CPC Drawer 1 and all PCIe ICA coupling adapters are plugged in CPC Drawer 0 top alternating between the internal node affinity A fanout can be repaired concurrently with the use of redundant I O interconnect 2 6 1 Redundant I O interconnect Redundancy is provided for both InfiniBand...

Page 96: ... HCA2 C fanout of a CPC drawer connects to the IFB MP B card and services domain B in an I O drawer If the connection from the CPC drawer IFB MP A to the I O drawer is removed connectivity to domain B is maintained The I O is guided to domain B through the interconnect between IFB MP A and IFB MP B Note Both IFB MP cards must be installed in the I O Drawer to maintain the interconnect across I O d...

Page 97: ...o interconnects to each pair must be driven from two different PCIe fanouts Normally each PCIe interconnect in a pair supports the eight features in its domain In backup operation mode one PCIe interconnect supports all 16 features in the domain pair 2 7 Model configurations When a z13s order is configured PUs are characterized according to their intended usage They can be ordered as any of the fo...

Page 98: ... six for IFLs and 20 for ICFs The maximum number of zIIPs is always up to twice the number of PUs that are characterized as CPs Not all PUs on a model must be characterized The following items are present in z13s servers but they are not part of the PUs that clients purchase and require no characterization An SAP to be used by the channel subsystem The number of predefined SAPs depends on the z13s...

Page 99: ... upgrade to Model N20 is required The upgrade involves extra hardware to add the second node in the first drawer and might force the addition of a second drawer The upgrade from N10 to N20 is disruptive Supported upgrade paths see Figure 2 27 on page 71 Upgrade to z13s processors from earlier processors From both the z114 and zBC12 servers Upgrade from z13s servers Model N10 to N20 Model N20 to z1...

Page 100: ...nformation STSI instruction returns a model capacity identifier MCI The MCI determines the number and speed of characterized CPs Characterization of a PU as an IFL an ICF or a zIIP is not reflected in the output of the STSI instruction because characterization has no effect on software charging For more information about STSI output see Processor identification on page 351 Important Upgrades from ...

Page 101: ...8 E06 81 F06 93 G01 21 H01 24 I01 27 J01 30 K01 34 L01 38 G02 40 H02 45 I02 51 J02 57 K02 64 L02 71 G03 58 H03 65 I03 74 J03 82 K03 92 L03 103 G04 75 H04 84 I04 95 J04 206 K04 119 L04 133 G05 92 H05 103 I05 115 J05 128 K05 144 L05 161 G06 107 H06 120 I06 135 J06 150 K06 168 L06 188 M01 42 N01 48 O01 53 P01 60 Q01 67 R01 75 M02 80 N02 90 O02 100 P02 112 Q02 125 R02 140 M03 115 N03 129 O03 144 P03 1...

Page 102: ...Ls and zIIPs always run at full capacity and also when running as CBU to replace lost capacity for disaster recovery When you order CBU specify the maximum number of CPs ICFs IFLs zIIPs and SAPs to be activated for disaster recovery If disaster strikes you decide how many of each of the contracted CBUs of each type must be activated The CBU rights are registered in one or more records on the serve...

Page 103: ...cord and related configuration features On Demand Capacity Selections NEW00001 CBU CP 1 Years 5 Tests 0 Expiration 09 10 2013 Resulting feature numbers in configuration 6817 Total CBU Years Ordered 5 6818 CBU Records Ordered 1 6820 Single CBU CP Year 5 In Example 2 2 a second CBU record is added to the same configuration for two CP CBUs two IFL CBUs two zAAP CBUs and two zIIP CBUs with five additi...

Page 104: ...ized PUs Relative to granular capacity On Off CoD for CPs is treated similarly to the way CBU is handled On Off CoD and granular capacity When temporary capacity requested by On Off CoD for CPs matches the model capacity identifier range of the permanent CP feature the total number of active CP equals the sum of the number of permanent CPs plus the number of temporary CPs ordered For example when ...

Page 105: ...n is nearly the same as a zBC12 However with a maximum performance ratio of 1 58 1 it has a much higher exploitation on the same footprint 2 8 1 Power considerations The z Systems CPCs operate with two redundant power supplies Each power supply has an individual power cord for the z13s server For redundancy the servers have two power feeds Power cords attach either 50 60 hertz Hz alternating curre...

Page 106: ... 36 for a pictorial view of the location of this feature This optional IBF provides the function of a local uninterrupted power source UPS The IBF further enhances the robustness of the power design increasing power line disturbance immunity It provides battery power to preserve processor data during a loss of power on all four AC feeds from the utility company The IBF can hold power briefly durin...

Page 107: ...s all aspects of the z13s structure Table 2 14 System structure summary Power consumption The exact power consumption for your system will vary The object of the tool is to produce an estimation of the power requirements to aid you in planning for your system installation Actual power consumption after installation can be confirmed on the HMC System Activity Display Description Model N10 Model N20...

Page 108: ...nnect Bus InfiniBand cable 6 GBps 6 GBps 6 GBps IInterconnect Bus PCIe cable 16 GBps 16 GBps 16 GBps Number of Support Elements 2 2 2 External AC power 1 phase 3 phase 1 phase 3 phase 1 phase 3 phase Optional external DC 520 V 380 V DC 520 V 380 V DC 520 V 380 V DC Internal Battery Feature Optional Optional Optional Description Model N10 Model N20 1 Model N20 2 ...

Page 109: ...it This information can be used to understand the functions that make the z13s server a system that accommodates a broad mix of workloads for large enterprises This chapter includes the following sections Overview Design highlights CPC drawer design Processor unit design Processor unit functions Memory design Logical partitioning Intelligent Resource Director Clustering technology 3 ...

Page 110: ...ultaneous multithreading SMT single instruction multiple data SIMD and flexible configuration options It is the next implementation of z Systems to address the ever changing IT environment The CPC drawer packaging is derived from the z13 The PU single chip modules SCMs are air cooled rather than water cooled like the z13 because of lower power and cooling requirements for the z13s SCMs z13s server...

Page 111: ...fer the high capacity and scalability that are required by the most demanding applications both from the single system and clustered systems points of view Offer the capability of concurrent upgrades for processors memory and I O connectivity avoiding system outages in planned situations Implement a system with high availability and reliability These goals are achieved with the redundancy of criti...

Page 112: ...rmance z13s servers use CMOS 14S0 Silicon On Insulator SOI 22 nm chip technology with advanced low latency pipeline design creating high speed yet power efficient circuit designs The PU SCM has a dense packaging but due to lower cycle frequency can be air cooled 3 3 1 Cache levels and memory structure The z13s memory subsystem focuses on keeping data closer to the PU core With the current processo...

Page 113: ...PC drawer using 20 DIMMs total of 5 per feature A z13s Model N20 with two CPC drawers can have up to 4 TB of addressable main storage Considerations Cache sizes are being limited by ever diminishing cycle times because they must respond quickly without creating bottlenecks Access to large caches costs more cycles Instruction and data cache L1 sizes must be limited because larger distances must be ...

Page 114: ...d data as close to the processors as possible You can do so by directing as much work of a particular LPAR workload to the processors in the same CPC drawer as the L4 cache This configuration is achieved by having the IBM Processor Resource Systems Manager PR SM scheduler and the z OS Workload Manager WLM and dispatcher work together Have them keep as much work as possible within the boundaries of...

Page 115: ...ove processors CPs zIIPs IFLs ICFs SAPs and spares to a different chip node and drawer to improve the reuse of shared caches by processors of the same partition It can use dynamic memory relocation DMR to move a running partition s memory to different physical memory to improve the affinity and reduce the distance between the memory of a partition and the processors of the partition For more infor...

Page 116: ... improved capacity and performance Although the cycle time of the z13s processor has only slightly decreased compared to zBC12 2 4 the processor performance was increased much more through improved processor design such as redesigned pipeline and out of order execution branch prediction time of access to high speed buffers caches redesign and the relative nest intensity RNI For more information ab...

Page 117: ...grated Facility for Linux IFL and IBM z Systems Integrated Information Processor zIIP speciality engines on z13s servers and it requires operating system support An operating system with SMT support can be configured to dispatch work to a thread on a zIIP for eligible workloads in z OS or an IFL for z VM core in single thread or SMT mode so that HiperDispatch cache optimization can be considered F...

Page 118: ...functionally transparent to middleware and applications and no changes are required to run them in an SMT enabled partition 3 4 2 Single instruction multiple data The z13s superscalar processor has 32 vector registers and an instruction set architecture that includes a subset of 139 new instructions known as SIMD added to improve the efficiency of complex mathematical models and vector processing ...

Page 119: ...ons have a non destructive operand encoding that allows the addition of the register vector A and register vector B and stores the result in the register vector A A A B Figure 3 7 shows a schematic representation of a SIMD instruction with 16 byte size elements in each vector operand Figure 3 7 Schematic representation of add SIMD instruction with 16 elements in each vector The vector register fil...

Page 120: ...ahead of the stalled instruction Reordering storage accesses Instructions that access storage can stall because they are waiting on results that are needed to compute the storage address In an in order core later instructions are stalled In an out of order core later storage accessing instructions that can compute their storage address are allowed to run Hiding storage access latency Many instruct...

Page 121: ...z13s PU core logical diagram Memory address generation and memory accesses can occur out of program order This capability can provide a greater use of the z13s superscalar core and can improve system performance L2 Dir VBQ Decode Dispatch1 Clump of instruction up to 6 Rename Group of micro ops up to 6 Issue Queue Issue Queue Fixed Pt Load Store2 Fixed Pt Bin Hex Floating Pt VBU VBU GPR FPR VBQ Ins...

Page 122: ... This technology results in a shorter run time Branch prediction If the branch prediction logic of the microprocessor makes the wrong prediction removing all instructions in the parallel pipelines might be necessary The wrong branch prediction is expensive in a high frequency processor design Therefore the branch prediction techniques that are used are important to prevent as many wrong branches a...

Page 123: ...ossible Many challenges exist in creating an efficient superscalar processor The superscalar design of the PU has made significant strides in avoiding address generation interlock AGI situations Instructions that require information from memory locations can suffer multi cycle delays while getting the needed memory content Because high frequency processors wait faster spend processor cycles more q...

Page 124: ... chip Its implementation meets business application requirements for better performance precision and function Base 10 arithmetic is used for most business and financial computation Floating point computation that is used for work that is typically done in decimal arithmetic involves frequent data conversions and approximation to represent decimal numbers This process has made floating point arith...

Page 125: ...ing point packed conversion instruction Converting from packed can now force the input packed value to positive instead of requiring a separate OI OILL or load positive instruction Converting to packed can now force a positive zero result instead of requiring ZAP instruction Software support Decimal floating point is supported in the following programming languages and products Release 4 and later...

Page 126: ... The following techniques are used Branch history table BHT Branch target buffer BTB Pattern history table PHT BTB data compression The success rate of branch prediction contributes significantly to the superscalar aspects of z13s servers because the architecture rules prescribe that for successful parallel execution of an instruction stream the correctly predicted result of the branch is essentia...

Page 127: ...far ahead of instruction decoding and execution as possible because of the relatively large instruction buffers available In the microprocessor smaller instruction buffers are used The operation code is fetched from the I cache and put in instruction buffers that hold prefetched data that is awaiting decoding Instruction decoding The processor can decode up to six instructions per cycle The result...

Page 128: ...ted The execution is optimistic The instructions are run but previous state values are saved in a transactional memory If the transaction succeeds the saved values are discarded Otherwise they are used to restore the original values The Transaction Execution Facility provides instructions including declaring the beginning and end of a transaction and canceling the transaction TX is expected to pro...

Page 129: ...led CPC drawer node have no available remaining PUs an upgrade results in a model upgrade upgrade Model N10 to N20 same drawer If the upgrade is for more than 2048 GB of memory or additional I O features then installation of an extra CPC drawer is necessary However there is a limit of two CPC drawers CPC drawer installation is disruptive and takes more time than a simple LIC upgrade For more infor...

Page 130: ... PUs can be characterized as CPs depending on the configuration z13s servers can be initialized either in LPAR mode or in Elastic PR SM mode For more information see Appendix E IBM Dynamic Partition Manager on page 501 CPs are defined as either dedicated or shared Reserved CPs can be defined to an LPAR to allow for nondisruptive image upgrades If the operating system in the LPAR supports the logic...

Page 131: ...chased but not activated is registered as an unassigned IFL FC 1928 When the system is later upgraded with an additional IFL the system recognizes that an IFL already was purchased and is present 3 5 4 Internal Coupling Facility An Internal Coupling Facility ICF is a PU that is used to run the CFCC for Parallel Sysplex environments Within the capacity of the sum of all unassigned PUs in two CPC dr...

Page 132: ... processed in a more timely manner After the image is dispatched existing poll for work logic in both CFCC and z OS can be used largely as is to locate and process the work The new interrupt speeds up the redispatching of the partition LPAR presents these Coupling Thin Interrupts to the guest partition so CFCC and z OS both require interrupt handler support that can deal with them CFCC also change...

Page 133: ...upling Thin Interrupts and the new DYNDISP specification It allows more environments with multiple CF images to coexist in a server and to share CF engines with reasonable performance For more information see 3 9 3 Dynamic CF dispatching on page 133 3 5 5 IBM z Integrated Information Processor A zIIP3 reduces the standard processor CP capacity requirements for z OS Java XML system services applica...

Page 134: ...ol to the JVM it passes control back to the dispatcher The dispatcher then reassigns the JVM code execution to a CP If no zIIP is available all busy the z OS dispatcher allows the Java task to run on a standard CP This process depends on the option that is used in the OPT statement in the IEAOPTxx member of SYS1 PARMLIB Figure 3 14 Logical flow of Java code execution on a zIIP Standard Processor z...

Page 135: ...CP even if the zIIPs are overcommitted and CPs are unused The exception is that zIIP eligible work can sometimes run on a CP to resolve resource conflicts Therefore zIIP eligible work does not affect the CP utilization that is used for reporting through the subcapacity reporting tool SCRT no matter how busy the zIIPs are If zIIPs are defined to the LPAR but are not online the zIIP eligible work un...

Page 136: ...ended Remote Copy XRC uses the zIIP as well Most z OS Data Facility Storage Management Subsystem DFSMS system data mover SDM processing that is associated with z OS Global Mirror can run on the zIIP This configuration requires z OS V1R10 or later releases The first IBM user of z OS XML system services is DB2 V9 For DB2 V9 before the z OS XML System Services enhancement z OS XML System Services non...

Page 137: ...ations for all LPARs All models have standard SAPs configured The number of standard SAPs depends on the z13s model as shown in Table 3 2 Table 3 2 SAPs per model SAP configuration A standard SAP configuration provides a well balanced system for most environments However some application environments have high I O rates which are typically various Transaction Processing Facility TPF environments I...

Page 138: ...d Do not define more active and reserved processors than the operating system for the LPAR can support For more information about logical processors and reserved processors and their definitions see 3 7 Logical partitioning on page 116 3 5 8 Integrated firmware processor An IFP is allocated from the pool of PUs This is available for the whole system Unlike other characterized PUs the IFP is standa...

Page 139: ... prevent them from sharing L3 and L4 caches with z OS processors This initial PU assignment which is done at POR can be dynamically rearranged by an LPAR by swapping an active core with a core in a different PU chip in a different CPC drawer or node to improve system performance For more information see LPAR dynamic PU reassignment on page 123 When an extra CPC drawer is added after a POR processo...

Page 140: ...PAR z VM can perform these tasks Manage guests that use Linux on z Systems on IFLs and those that use z VSE or z OS on CPs Run designated z OS workloads such as Java and XML execution and parts of DB2 DRDA processing on zIIPs If the only operating system to run under z VM is Linux define a Linux only LPAR 3 6 Memory design This section describes various considerations of the z13s memory design 3 6...

Page 141: ...her to address the priority ranks that have been created When partitions are activated PR SM tries to find a home assignment CPC drawer home assignment node and home assignment chip for the logical processors that are defined to them The PR SM goal is to allocate all the partition logical processors and memory to a single CPC drawer the home drawer for that partition If all logical processors can ...

Page 142: ... segment table which is needed to find the page table and the page table which is needed to find the entry for the particular page in question are not in cache This case involves two complete real memory access delays plus the address translation delay The duration of a processor cycle is much shorter than the duration of a memory cycle so a TLB miss is relatively costly It is preferable to have a...

Page 143: ...mmunication with and control of optional expanded storage Error checking and correction Main storage can be accessed by all processors but cannot be shared between LPARs Any system image LPAR must have a main storage size defined This defined main storage is allocated exclusively to the LPAR during partition activation 3 6 3 Expanded storage Expanded storage can optionally be defined on z13s serve...

Page 144: ...nfiguration dependent control blocks On z13s servers the HSA has a fixed size of 40 GB and is not part of the purchased memory that customers order and is installed by IBM The fixed size of the HSA eliminates planning for future expansion of the HSA because the hardware configuration definition HCD input output configuration program IOCP always reserves space for the following items Three channel ...

Page 145: ...s optimization was done by using a round robin algorithm with a number of increments per book to match the number of memory controllers MCs per book This memory allocation design is driven by performance results with minimized variability for most workloads With z13 and z13s servers memory allocation has changed from the model that was used for the z9 Partition memory is now allocated in a per CPC...

Page 146: ... independently from the other LPARs An LPAR can be added removed activated or deactivated at any time Changing the number of LPARs is not disruptive and does not require POR Certain facilities might not be available to all operating systems because the facilities might have software corequisites Each LPAR has the same resources as a real CPC Processors Called logical processors they can be defined...

Page 147: ...capacity of the system In support of WLC the user can specify a defined capacity in MSU per hour The defined capacity sets the capacity of an individual LPAR when soft capping is selected The defined capacity value is specified on the Options tab in the Customize Image Profiles window WLM keeps a 4 hour rolling average of the processor usage of the LPAR When the 4 hour average processor consumptio...

Page 148: ...ations including their operating modes and processor types operating systems and addressing modes Only the currently supported versions of operating systems are considered Table 3 4 z13s modes of operation The 64 bit z Architecture mode has no special operating mode because the architecture mode is not an attribute of the definable images operating mode The 64 bit operating systems are in 31 bit m...

Page 149: ...LPAR that is defined as one of these types Dedicated or shared CPs Dedicated or shared ICFs Linux only mode to run these systems A Linux on z Systems operating system on either of these types Dedicated or shared IFLs Dedicated or shared CPs A z VM operating system on either of these types Dedicated or shared IFLs Dedicated or shared CPs z VM mode to run z VM on dedicated or shared CPs or IFLs plus...

Page 150: ...ncurrently added to a logical partition dynamically by using the Support Element Logical Processor Add function under the CPC Operational Customization task This SE function allows the initial and reserved processor values to be dynamically changed The operating system must support LPAR mode PU type Operating systems PUs usage ESA 390 CPs z Architecture operating systems ESA 390 operating systems ...

Page 151: ...can include spare PUs The goals are to pack the LPAR on fewer CPC drawers and also on fewer PU chips based on the z13s CPC drawers topology The effect of this process is evident in dedicated and shared LPARs that use HiperDispatch LPAR dynamic PU reassignment is transparent to operating systems LPAR group capacity limit The group capacity limit feature allows the definition of a group of LPARs on ...

Page 152: ...nment that coordinates the physical system resources shared by the partitions The partitions resources include processors memory network storage crypto and accelerators DPM provides a new mode of operation for z Systems that provides these capabilities Facilitates defining configuring and operating PR SM LPARs in a similar way to how someone performs these tasks on another platform Lays the founda...

Page 153: ...ng systems Table 3 6 shows the z13 storage allocation and usage possibilities depending on the image mode Table 3 6 Main storage definition and usage possibilities ESA 390 mode In ESA 390 mode storage addressing can be 31 or 64 bits depending on the operating system architecture and the operating system configuration An ESA 390 mode image is always initiated in 31 bit addressing mode During its in...

Page 154: ...GB can be configured as expanded storage In addition this mode allows the use of either 24 bit or 31 bit addressing under program control Because an ESA 390 mode image can be defined with up to 128 GB of main storage the main storage above 2 GB is not used Instead it remains configured to this image ESA 390 TPF mode In ESA 390 TPF mode storage addressing follows the ESA 390 architecture mode The z...

Page 155: ...3s definition limit for LPARs is 4 TB of storage Currently the IBM zAware Version 2 feature which is available on z13 and z13s servers runs in a zACI LPAR For more information see Appendix B IBM z Systems Advanced Workload Analysis Reporter IBM zAware on page 453 3 7 3 Reserved storage Reserved storage can be optionally defined to an LPAR allowing a nondisruptive image memory upgrade for this part...

Page 156: ... partition main storage granularity z13s servers The granularity applies across all main storage that is defined both initial and reserved For example for an LPAR with an initial storage amount of 30 GB and a reserved storage amount of 48 GB the main storage granularity of both initial and reserved main storage is 512 MB Expanded storage granularity is fixed at 512 MB LPAR storage granularity info...

Page 157: ...em running on an LPAR releases a storage increment 3 8 Intelligent Resource Director IRD is a z13 z13s and z Systems capability that is used only by z OS IRD is a function that optimizes processor and channel resource utilization across LPARs within a single z Systems server This feature extends the concept of goal oriented resource management It does so by grouping system images that are on the s...

Page 158: ...ad HiperDispatch also adjusts the number of logical processors The goal is to map the logical processor to as few physical processors as possible This configuration uses the processor resources more efficiently by trying to stay within the local cache structure Doing so makes efficient use of the advantages of the high frequency microprocessors and improves throughput and response times Dynamic ch...

Page 159: ... Parallel Sysplex technology is an enabling technology allowing highly reliable redundant and robust z Systems technology to achieve near continuous availability A Parallel Sysplex makes up one or more z OS operating system images that are coupled through one or more Coupling Facilities The images can be combined together to form clusters A correctly configured Parallel Sysplex cluster maximizes a...

Page 160: ... competitive price for performance scalable growth and continuous availability 3 9 1 Coupling Facility Control Code The LPAR running the CFCC can be on z13 z13s zEC12 zBC12 z196 or z114 systems For more information about CFCC requirements for supported systems see 7 8 Coupling facility and CFCC considerations on page 292 CFCC Level 21 CFCC level 21 is delivered on z13s servers with driver level 27...

Page 161: ...ronments that share Coupling Facility engines Although dedicated engines are preferable to obtain the best Coupling Facility performance Coupling Thin Interrupts can help facilitate the use of a shared pool of engines helping to lower hardware acquisition costs The interrupt causes a shared logical processor coupling facility partition that has not already been dispatched to be dispatched by PR SM...

Page 162: ...ew thin interrupt now gets the shared processor that is dispatched when a command or duplexing signal is presented to the shared engine This function saves processor cycles and is an excellent option to be used by a production backup CF or a testing environment CF This function is activated by the CFCC command DYNDISP ON The CPs can run z OS operating system images and CF images For software charg...

Page 163: ...es resilience while providing cost effective standby capacity to help manage the potential overflow of IBM WebSphere MQ shared queues Structures can now be allocated with a combination of real memory and SCM that is provided by the Flash Express feature For more information about Flash Express and CF Flash use see Appendix H Flash Express on page 529 ...

Page 164: ...136 IBM z13s Technical Guide ...

Page 165: ...ailable on z13s servers This chapter includes the following sections Introduction to the InfiniBand and PCIe for I O infrastructure I O system overview I O drawer PCIe I O drawer PCIe I O drawer and I O drawer offerings Fanouts I O features cards Connectivity Parallel Sysplex connectivity Cryptographic functions Integrated firmware processor Flash Express 10GbE RoCE Express zEDC Express 4 ...

Page 166: ...s PCIe I O infrastructure consists of the following components PCIe fanouts that support 16 GBps I O bus interconnection for CPC drawer connectivity to the PCIe I O drawers The 7U 32 slot and 4 domain PCIe I O drawer for PCIe I O features The z13s PCIe I O infrastructure provides these benefits Increased bandwidth from the CPC drawer to the I O domain in the PCIe I O drawer through a 16 GBps bus T...

Page 167: ...details and the standard for InfiniBand see the InfiniBand website at http www infinibandta org 4 1 4 PCIe Generation 3 The z13 and z13s servers are the first generation of z Systems servers to support the PCIe Generation 3 Gen3 protocol PCIe Generation 3 uses 128b 130b encoding for data transmission This encoding reduces the encoding effort by approximately 1 54 when compared to the PCIe Generati...

Page 168: ...rvers use InfiniBand as the internal interconnect protocol to drive I O drawers and CPC to CPC connections InfiniBand supports I O bus infrastructure data rates up to 6 GBps Connectivity options z13s servers can be connected to an extensive range of interfaces such as FICON FCP for SAN connectivity 10 Gigabit Ethernet Gigabit Ethernet and 1000BASE T Ethernet for LAN connectivity For CPC to CPC con...

Page 169: ...gged in an I O drawer or PCIe I O drawer supports concurrent card replacement during a repair action 4 2 2 Summary of supported I O features The following I O features are supported Up to 128 FICON Express16S channels Up to 128 FICON Express8S channels Up to 32 FICON Express8 channels Up to 96 OSA Express5S ports Up to 96 OSA Express4S ports Up to 8 ICA SR links with up to 16 coupling links at 8 G...

Page 170: ...tribute power to the I O drawer The locations of the DCAs I O feature cards and IFB MP cards in the I O drawer are shown in Figure 4 1 Figure 4 1 I O drawer The I O structure in a z13s CPC is illustrated in Figure 4 2 on page 143 An IFB cable connects the HCA fanout card in the CPC drawer to an IFB MP card in the I O drawer The passive connection between two IFB MP cards is a design of redundant I...

Page 171: ... PCIe I O drawer The PCIe I O drawers are attached to the CPC drawer through a PCIe bus and use PCIe as the infrastructure bus within the drawer The PCIe I O bus infrastructure data rate is up to 16 GBps PCIe switch application specific integrated circuits ASICs are used to fan out the host bus from the CPC drawers to the individual I O features Up to 64 channels 32 PCIe I O features are supported...

Page 172: ...tions of the DCAs AMDs PCIe switch cards and I O feature cards in the PCIe I O drawer are shown in Figure 4 3 Figure 4 3 PCIe I O drawer 7U 7U 311mm Rear Front Domain 3 Domain 1 Domain 0 Domain 2 AMD 16 GBps PCIe switch card 560mm max DCA ...

Page 173: ...th for each other through the passive connection in the PCIe I O drawer backplane During a PCIe fanout card or cable failure all 16 I O cards in the two domains can be driven through a single PCIe switch card As a RII design a switch card in the front is connected to a switch card in the rear through the PCIe I O drawer board Switch cards in same PCIe I O drawer are connected to PCIe fanouts acros...

Page 174: ...fferings A maximum of two PCIe I O drawers can be installed that support up to 64 PCIe I O features Only PCIe I O features can be ordered for a new system and the total number of PCIe I O Note The limitation of up to two native PCIe features Flash Express zEDC Express and 10GbE RoCE Express per I O domain is eliminated on z13 and z13s servers Domain I O slot in domain 0 01 02 03 04 06 07 08 and 09...

Page 175: ...card is always installed in pairs to support I O domain redundant connectivity The InfiniBand HCA2 C fanout card is available for carry forward of FICON Express8 features that are installed in I O drawers The PCIe and InfiniBand fanouts are installed in the front of each CPC drawer A two node CPC drawer has eight PCIe Gen3 fanout slots and four InfiniBand fanout slots The PCIe fanout slots are nam...

Page 176: ...ense wavelength division multiplexing DWDM equipment distance to z13 z13s zEC12 zBC12 z196 and z114 servers The PCIe Gen3 fanout card comes with one port the HCA3 O LR 1xIFB fanout comes with four ports and other fanouts come with two ports Figure 4 6 illustrates the following PCIe and IFB connections from the CPC drawer PCIe Gen3 for connectivity to PCIe I O drawer not represented here see Figure...

Page 177: ...apter FC 0172 The IBM ICA SR introduced on the z13 and z13s platform is a two port fanout that is used for short distance coupling connectivity with a new coupling channel type the CS5 The ICA SR uses PCIe Gen3 technology with x16 lanes that are bifurcated into x8 lanes for coupling No performance degradation is expected compared to the Coupling over the InfiniBand 12x IFB3 protocol The ICA SR is ...

Page 178: ...and z114 CPCs Up to 8 HCA3 O 12x IFB fanouts are supported and provide up to 16 ports for coupling links The fiber optic cables are industry standard OM3 2000 MHz km 50 µm multimode optical cables with MPO connectors The maximum cable length is 150 meters 492 ft There are 12 pairs of fibers 12 fibers for transmitting and 12 fibers for receiving The HCA3 O 12xIFB fanout supports a link data rate of...

Page 179: ... protocol is used When you use the 12x IFB3 protocol synchronous service times are up to 40 faster than when you use the 12x IFB protocol Unlike the HCA3 O 12x InfiniBand links the ICA SR cannot define more than four CHPIDS per port 4 6 5 HCA3 O LR 1x IFB fanout FC 0170 The HCA3 O LR fanout for 1x InfiniBand provides an optical interface that is used for coupling links The four ports on the fanout...

Page 180: ...d forward Table 4 3 shows for the z13s Models N10 and N20 where the limitation of HCA3 O and HCA3 O LR fanouts that are caused by carry forward of FICON Express8 features might require attention during migration For migration considerations see 4 9 2 Migration considerations on page 178 Table 4 3 CPC drawer InfiniBand fanout availability limitation with carry forward FICON Express8 features Adapte...

Page 181: ...e first drawer location A21A slot LG03 and one fanout is installed in the second drawer location A25A slot LG11 The assigned AID for the fanout in the first drawer is 1B The AID that is assigned to the fanout in the second drawer is 15 Example 4 1 AID assignment in PCHID REPORT CHPIDSTART 12345678 PCHID REPORT Oct 31 2014 Machine 2965 N20 SNXXXXXXX Source Cage Slot F C PCHID Ports or AID Comment A...

Page 182: ...mum distance Link data ratea a The link data rates do not represent the actual performance of the link The actual performance is dependent upon many factors that include latency through the adapters cable lengths and the type of workload HCA2 C 0162 Connect to I O drawer Copper N A 3 5 m 11 48 ft 6 GBps PCIe fanout 0173 Connect to PCIe I O drawer Copper N A 4 m 13 1 ft 16 GBps HCA3 O 12xIFB 0171 C...

Page 183: ...A Express5S 10 GbE LR 0415 Y Y OSA Express5S 10 GbE SR 0416 Y Y OSA Express5S GbE LX 0413 Y Y OSA Express5S GbE SX 0414 Y Y OSA Express5S 1000BASE T Ethernet 0417 Y Y OSA Express4S 10 GbE LR 0406 N Y OSA Express4S 10 GbE SR 0407 N Y OSA Express4S GbE LX 0404 N Y OSA Express4S GbE SX 0405 N Y OSA Express4S 1000BASE T Ethernet 0408 N Y Integrated Coupling Adapter ICA SR 0172 Y N A HCA3 O 12xIFB 0171...

Page 184: ...page 153 Example 4 2 PCHID REPORT CHPIDSTART 12345678 PCHID REPORT Nov 17 2015 Machine 2965 N20 SN1 Source Drwr Slot F C PCHID Ports or AID Comment A25 LG07 A25A LG07 0171 AID 00 A21 LG07 A21A LG07 0170 AID 04 A21 LG09 A21A LG09 0171 AID 06 A21 LG04 A21A LG04 0172 AID 1C A21 LG03 J01 A13B 01 0403 100 Mem paired 12C Mem A21 LG03 J01 A13B 03 0420 108 RG1 A21 LG03 J01 A13B 06 0413 110 D1D2 A21 LG03 J...

Page 185: ... km 6 2 miles is installed in PCIe I O drawer 1 location A13B slot 07 PCHID 114 is assigned to port D1 and 115 assigned to port D2 One feature code 0411 10GbE RoCE Express is installed in PCIe I O drawer 1 location A13B slot 18 attached to resource group RG1 with PCHID 138 assigned and shared by port D0 and D1 Another 10GbE RoCE Express is installed in PCIe I O drawer 1 location A13B slot 37 attac...

Page 186: ... OSM OSN or OSX OSA Express4S features that are defined as OSC OSD OSE OSM OSN or OSX I O feature Number of Maximum number of PCHID CHPID definition Portsper card Port increments Ports I O slots FICON Express16S LX SX 2 2 128 64 Yes FC FCP FICON Express8S LX SX 2 2 128 64 Yes FC FCP FICON Express8 LX SX 4 4 32 8 Yes FC FCP OSA Express5S 10 GbE LR SR 1 1 48 48 Yes OSD OSX OSA Express5S GbE LX SX 2 ...

Page 187: ...he end to end enterprise The Enterprise Fiber Cabling Services use a proven modular cabling system the fiber transport system FTS which includes trunk cables zone cabinets and panels for servers directors and storage devices FTS supports Fiber Quick Connect FQC a fiber harness that is integrated in the frame of a z13s server for quick connection The FQC is offered as a feature on z13s servers for ...

Page 188: ...X LC Duplex 50 62 5 µm MM 0415 OSA Express5S 10 GbE LR LC Duplex 9 µm SM 0416 OSA Express5S 10 GbE SR LC Duplex 50 62 5 µm MM 0413 OSA Express5S GbE LX LC Duplex 9 µm SM 0414 OSA Express5S GbE SX LC Duplex 50 62 5 µm MM 0417 OSA Express5S 1000BASE T RJ 45 Category 5 UTPc 0404 OSA Express4S GbE LX LC Duplex 9 µm SM 0405 OSA Express4S GbE SX LC Duplex 50 62 5 µm MM 0406 OSA Express4S 10 GbE LR LC Du...

Page 189: ...N channels CHPID type FC or FCP can be shared among LPARs and can be defined as spanned All ports on a FICON feature must be of the same type either LX or short wavelength SX The features are connected to a FICON capable control unit either point to point or switched point to point through a Fibre Channel switch FICON Express16S The FICON Express16S feature installs exclusively in the PCIe I O dra...

Page 190: ...ink speed is auto negotiated point to point and is transparent to users and applications The following types of FICON Express8 optical transceivers are supported FICON Express8 10 km LX feature FC 3325 with four ports per feature supporting LC Duplex connectors FICON Express8 SX feature FC 3326 with four ports per feature supporting LC Duplex connectors Each port of FICON Express8 10 km LX feature...

Page 191: ...le Brocade s Exchange Based Routing EBR and Cisco s Open Exchange ID Routing OxID Check with the switch provider for their support statement A static SAN routing policy normally assigns the ISL routes according to the incoming port and its destination domain port based routing or the source and destination ports pairing device based routing Port based routing PBR assigns the ISL routes statically ...

Page 192: ...isplay command For Linux on z Systems z VM z VSE and KVM LPARs using FCP channels this diagnostic data is available in a new window in the SAN Explorer tool N_Port ID Virtualization enhancement N_Port ID Virtualization NPIV allows multiple system images in LPARs or z VM guests to use a single FCP channel as though each were the sole user of the channel This feature first introduced with z9 EC can ...

Page 193: ...e on a z13s server All FICON features use LC Duplex connectors Table 4 9 z13 channel feature support Channel feature Feature codes Bit rate Cable type Maximum unrepeated distancea MHz km FICON Express16S 10KM LX 0418 4 8 or 16 Gbps SM 9 µm 10 km FICON Express16S SX 0419 16 Gbps MM 50 µm 35 m 500 100 m 2000 125 m 4700 8 Gbps MM 62 5 µm MM 50 µm 21 m 200 50 m 500 150 m 2000 190 m 4700 4 Gbps MM 62 5...

Page 194: ...he port supports CHPID types OSD and OSX When FICON Express8 SX 3326 8 Gbps MM 62 5 µm MM 50 µm 21 m 200 50 m 500 150 m 2000 190 m 4700 4 Gbps MM 62 5 µm MM 50 µm 70 m 200 150 m 500 380 m 2000 400 m 4700 2 Gbps MM 62 5 µm MM 50 µm 150 m 200 300 m 500 500 m 2000 N A 4700 a Minimum fiber bandwidths in MHz km for multimode fiber optic links are included in parentheses where applicable Channel feature...

Page 195: ...Ethernet LAN or Ethernet switch that is capable of 10 Gbps The port can be defined as a spanned channel and can be shared among LPARs within and across logical channel subsystems The OSA Express5S 10 GbE SR feature supports the use of an industry standard small form factor LC Duplex connector Ensure that the attaching or downstream device has an SR transceiver The sending and receiving transceiver...

Page 196: ...er or switch If you allow the LAN speed and duplex mode to default to auto negotiation the OSA Express port and the attached router or switch auto negotiate the LAN speed and duplex mode settings between them They then connect at the highest common performance speed and duplex mode of interoperation If the attached Ethernet router or switch does not support auto negotiation the OSA Express port ex...

Page 197: ...ivers must be the same LR to LR The OSA Express4S 10 GbE LR feature does not support auto negotiation to any other speed and runs in full duplex mode only OSA Express4S 10 GbE LR supports 64B 66B encoding However the GbE supports 8B 10 encoding making auto negotiation to any other speed impossible A 9 µm single mode fiber optic cable that terminates with an LC Duplex connector is required for conn...

Page 198: ... and receiving transceivers must be the same LX to LX A 9 µm single mode fiber optic cable that terminates with an LC Duplex connector is required for connecting each port on this feature to the selected device If multimode fiber optic cables are being reused a pair of Mode Conditioning Patch cables is required one for each end of the link OSA Express4S Gigabit Ethernet SX FC 0405 The OSA Express4...

Page 199: ...does not match the speed and duplex mode of the signal on the cable the OSA Express port does not connect 4 8 5 OSA Express for ensemble connectivity The following OSA Express features are used to connect z13s servers to IBM z BladeCenter Extension zBX Model 004 and other ensemble nodes OSA Express5S 10 Gigabit Ethernet GbE Long Reach LR FC 0415 OSA Express5S 10 Gigabit Ethernet GbE Short Reach SR...

Page 200: ...rtual LAN attachments just like the IBM zEnterprise zEC12 IBM zEnterprise BC12 IBM zEnterprise 196 and IBM zEnterprise 114 servers Previous servers provided 16 attachments HiperSockets can be customized to accommodate varying traffic sizes Because HiperSockets does not use an external network it can free up system and network resources This advantage can help eliminate attachment costs and improve...

Page 201: ...Sockets network and an external Ethernet network It also can be used to connect to the HiperSockets Layer 2 networks of different servers HiperSockets Layer 2 in the z13s server is supported by Linux on z Systems and by z VM for Linux guest use z13s z13 and the other zEnterprise CPCs zEC12 zBC12 z196 and z114 servers support the HiperSockets Completion Queue function that is designed to allow Hipe...

Page 202: ...er CPC and each VCHID supports up to 255 VFs with a total maximum of 8K VFs For more information about the SMC D and ISM see Appendix D Shared Memory Communications on page 475 HiperSockets can have network latency and CPU reduction benefits and performance improvement by using the SMC D over ISM 4 9 Parallel Sysplex connectivity Coupling links are required in a Parallel Sysplex configuration to p...

Page 203: ...r IC connections require two CHPIDs to be defined which can be defined only in peer mode A maximum of 32 IC CHPIDs 16 connections can be defined Table 4 12 shows the coupling link options Table 4 12 Coupling link options Note ISC 3 HCA2 O and HCA2 O LR are not supported on z13s servers The zEC12 and zBC12 are the last servers that support ISC 3 HCA2 O and HCA2 O LR coupling links by carry forward ...

Page 204: ... LR 1xIFB on zEC12 and later when both sides of the connection use 32 subchannels Otherwise change the default value from 7 to 32 subchannels on each CIB definition In a Parallel Sysplex configuration z OS and coupling facility images can run on the same or separate servers There must be at least one CF that is connected to all z OS images although there can be other CFs that are connected only to...

Page 205: ...0171 For more information see 4 6 4 HCA3 O 12x IFB fanout FC 0171 on page 150 HCA3 O LR fanout for 1x InfiniBand FC 0170 For more information see 4 6 5 HCA3 O LR 1x IFB fanout FC 0170 on page 151 ICA SR fanout FC0172 For more information see 4 6 3 Integrated Coupling Adapter FC 0172 on page 149 Extended distance support For extended distance support see the System z End to End Extended Distance Gu...

Page 206: ...previous generations of z Systems servers in an existing Parallel Sysplex to z13s servers in that same Parallel Sysplex requires proper planning for coupling connectivity The change in the supported type of coupling link adapters and the number of available fanout slots of the z13s CPC drawer as compared to the number of available fanout slots of the previous generation z Systems z114 and zBC12 ne...

Page 207: ... as stand alone coupling facility with all four fanouts that are used for IFB connectivity In a 1 1 link migration scenario this server cannot be migrated to a z13s N10 because the z13s N10 cannot accommodate more than two InfiniBand fanouts Furthermore if FICON Express8 features are carried forward the total number of InfiniBand fanout are reduced by two For more information see 4 6 6 Fanout cons...

Page 208: ...hey also are the last z Systems that can be part of a Mixed CTN Consider Long Distance Coupling requirements first HCA3 O 1x is the only long distance coupling link that is available on z13s servers Keep slots free for HCA3 O 1x LR where possible ICA SR or HCA3 O 12x should be used for short distance coupling requirements ISC 3 Migration z13s servers do not support ISC 3 Evaluate current ISC 3 usa...

Page 209: ...nfiniBand However after the second z13s server is migrated coupling connectivity can be moved to ICA SR links Upgrade CPCs with fewer coupling constraints first Consider upgrading CPCs to have sufficient IFB links in the target z13s configuration first for example multi CPC drawer CPCs Test the new ICA SR link on the least constrained CPCs for example z OS Host CPCs that have only the lowest numbe...

Page 210: ...plex Timer A z13s server cannot be added into a Mixed CTN and can participate only in an STP only CTN STP recovery enhancement All coupling host channel adapters ICA SR HCA3 O 12xIFB and HCA3 O LR 1xIFB are designed to send a reliable and unambiguous going away signal This signal indicates that the server on which the ICA SR or HCA3 is running is about to enter a failed check stopped state The goi...

Page 211: ...he ETS is maintained ETSs with PPS output are available from various vendors that offer network timing solutions 4 10 Cryptographic functions Cryptographic functions are provided by the CP Assist for Cryptographic Function CPACF and the PCI Express cryptographic adapters z13s servers support the Crypto Express5S feature 4 10 1 CPACF functions FC 3863 FC 3863 is required to enable CPACF functions F...

Page 212: ... be ordered as new FC 403 or carry forward FC 402 They are plugged into PCIe I O drawers in pairs for availability Like the Crypto Express5S cards each card has a PCHID and no HCD IOCP definition is required Flash Express subchannels are predefined and are allocated from the 256 subchannels that are reserved in subchannel set 0 Flash Express cards are internal to the CPC and are accessible by usin...

Page 213: ...t use a CHPID It is defined by using the input output configuration program IOCP FUNCTION statement or in the HCD For zEC12 and zBC12 each feature can only be dedicated to an LPAR and z OS can support only one of the two ports In z13s servers both two ports are supported by z OS and can be shared by up to 31 partitions LPARs The 10GbE RoCE Express feature uses an SR laser as the optical transceive...

Page 214: ...and Appendix G Native Peripheral Component Interconnect Express on page 521 4 14 zEDC Express zEDC Express is an optional feature FC 0420 that is available on z13s z13 zEC12 and zBC12 servers It is designed to provide hardware based acceleration for data compression and decompression The feature installs exclusively on the PCIe I O drawer One to eight features can be installed on the system There ...

Page 215: ...he concepts of the IBM z13s channel subsystem including multiple channel subsystems and multiple subchannel sets It also describes the technology terminology and implementation aspects of the channel subsystem This chapter includes the following sections Channel subsystem I O configuration management Channel subsystem summary 5 ...

Page 216: ...e and I O connectivity In support of the larger I O capability the CSS structure has been scaled up correspondingly by introducing the multiple logical channel subsystem LCSS with z990 and multiple subchannel sets MSS with z9 z13s servers are designed to support up to three logical channel subsystems LCSS and three subchannel sets Figure 5 1 summarizes the multiple channel subsystems and multiple ...

Page 217: ...has extended z Systems total number of I O connections to support a balanced configuration for the growth of processor and I O capabilities A one digit number ID starting from 0 CSSID is assigned to an LCSS and a one digit hexadecimal ID MIF ID starting from 0 is assigned to an LPAR within the LCSS Subchannels A subchannel provides the logical appearance of a device to the program and contains the...

Page 218: ...ies that are necessary to operate and control an I O device It adapts the characteristics of each device so that it can respond to the standard form of control that is provided by the CSS A control unit can be housed separately or can be physically and logically integrated with the I O device the channel subsystem or within the z Systems server itself I O devices An I O device provides external st...

Page 219: ...figuration for naming an I O device The device number must be unique within a subchannel set of a channel subsystem It is assigned to the corresponding subchannel by channel subsystem while an I O configuration is activated Thus a subchannel in a subchannel set of a channel subsystem has a device number together with a subchannel number for designating an I O operation The device number provide a ...

Page 220: ...AR or program temporary fix PTF installed Alias devices of the parallel access volumes PAV Secondary devices of GDPS Metro Mirror Copy Service formerly PPRC FlashCopy SOURCE and TARGET devices with PTF OA46900 DB2 data backup volumes with PTFOA24142 Using an extra subchannel set for these special devices helps to relieve the numbers of devices in the subchannel set 0 which improves growth capabili...

Page 221: ...e LCSSs at the same time That CHPID is then spanned across those LCSSs Using spanned channels paths decreases the number of channels needed in an installation of z Systems servers D IOS CONFIG ALL IOS506I 11 32 19 I O CONFIG DATA 340 ACTIVE IODF DATA SET SYS6 IODF39 CONFIGURATION ID L06RMVS1 EDT ID 01 TOKEN PROCESSOR DATE TIME DESCRIPTION SOURCE SCZP303 14 10 31 08 51 47 SYS6 IODF39 ACTIVE CSS 0 S...

Page 222: ...nal links HiperSockets and IC links and for certain types of external links External links that are supported on z13s servers include FICON Express16S FICON Express8S FICON Express8 channels OSA Express5S OSA Express4S and Coupling Links The definition of LPAR name MIF image ID and LPAR ID are used by the channel subsystem to identify an LPAR to identify I O functions from different LPARs of multi...

Page 223: ... a z13s server has these maximum number of LPARs LCSS0 and LCSS1 support 15 LPARs each and MIF image ID range from 1 to F within each LCSS LCSS2 supports 10 LPARs and MIF image ID range from 1 to A LPAR ID The LPAR ID is defined by user in an image activation profile for each LPAR It is a 2 digit heximal number from 00 to 7F The LPAR ID must be unique across the server Though it is arbitrary defin...

Page 224: ...2827DEVICE 2818DEVICE and 2817DEVICE Preventive Service Planning PSP buckets before implementation Hardware Configuration Manager HCM HCM is a priced optional feature that supplies a graphical interface for HCD It is installed on a PC and allows you to manage both the physical and the logical aspects of a mainframe s hardware configuration CHPID Mapping Tool CMT The CMT helps to map CHPIDs onto PC...

Page 225: ...Chapter 5 Central processor complex channel subsystem 197 Maximum number of subchannels per CSS 191 74 K SS0 65280 SS1 SS2 65535 Maximum number of CHPIDs per CSS 256 ...

Page 226: ...198 IBM z13s Technical Guide ...

Page 227: ... to the principles of cryptography It then looks at the implementation of cartography in the hardware and software architecture of z Systems followed by a more detailed description of the features the IBM z13s servers are offering The chapter concludes with a summary of the cryptographic features and the software required This chapter includes the following sections Cryptography in IBM z13 and z13...

Page 228: ...ayment Card Industry PCI hardware security module HSM criteria and the criteria for Deutsche Kreditwirtschaft DK evaluation The cryptographic functions include the full range of cryptographic operations that are necessary for e business e commerce and financial institution applications User Defined Extensions UDX allow you to add custom cryptographic functions to the functions that z13s servers of...

Page 229: ...main elements The algorithm is the mathematical or logical formula that is applied to the key and the clear text to deliver a ciphered result or to take a ciphered text and deliver the original clear text The key ensures that the result of the encrypting data transformation by the algorithm is only the same when the same key is used The decryption of a ciphered message only results in the original...

Page 230: ...hat is transferred from the application in clear to the cryptographic function The key value is stored in the clear at least briefly somewhere in unprotected memory areas so under certain circumstances the key can be exposed by someone accessing this memory area This risk must be considered when using clear keys However there are many applications where this risk can be accepted For example the tr...

Page 231: ...thms are the already mentioned DES TDES and AES Asymmetric algorithms use two distinct but related keys the public key and the private key As the names imply the private key must be kept secret while the public key is shown to everyone However with asymmetric cryptography it is not important who sees or knows the public key Whatever is done with one key can only be undone by the other key For inst...

Page 232: ...cryptographic hardware that is supported on IBM z13s servers and Table 6 1 on page 205 lists the corresponding feature codes and describes the purpose of these hardware features All these features are described in more detail later in this chapter Figure 6 2 Cryptographic hardware supported on IBM z13s servers Attached to every processor unit PU on a single chip module SCM in a central processor c...

Page 233: ... and security administration It is optional for running a Crypto Express5S card in CCA mode and required for running it in EP11 mode The TKE workstation has one Ethernet port and supports connectivity to an Ethernet local area network LAN operating at 10 100 or 1000 Mbps Up to 10 features per z13s server can be ordered 0097 Trusted Key Entry TKE rack mounted workstation The rack mounted version of...

Page 234: ...ress5S card depending on performance or requested cryptographic function ICSF is also the means by which the secure Crypto Express5S features are loaded with master key values allowing the hardware features to be used by applications The cryptographic hardware installed in the z13s server determines the cryptographic features and services available to the applications The users of the cryptographi...

Page 235: ... network VPN and data storing applications that do not require FIPS 140 2 Level 4 security CPACF is designed to facilitate the privacy of cryptographic key material when used for data encryption through key wrapping implementation It ensures that key material is not visible to applications or operating systems during encryption operations For more information see 6 4 2 CPACF protected key on page ...

Page 236: ...data depending on the function that is being used These tools might benefit from the throughput improvements DB2 IMS encryption tool DB2 built in encryption z OS Communication Server IPsec IKE AT TLS z OS System SSL z OS Network Authentication Service Kerberos DFDSS Volume encryption z OS Java SDK z OS Encryption Facility Linux on z Systems Kernel openSSL openCryptoki and GSKIT The z13s hardware i...

Page 237: ...is done synchronously on the CPACF Protected keys blend the security of Crypto Express5S coprocessors and the performance characteristics of the CPACF This process allows it to run closer to the speed of clear keys An enhancement to CPACF facilitates the continued privacy of cryptographic key material when used for data encryption In Crypto Express5S coprocessors a secure key is encrypted under a ...

Page 238: ...re keys are considered not eligible to be used as protected keys The process that is described in Figure 6 5 considers a secure key as being the source of a protected key The source key in this case already is stored in the ICSF cryptographic key data set CKDS as a secure key which has been encrypted under the master key This secure key is sent to Crypto Express5S to be deciphered and sent to the ...

Page 239: ...nistration of the Crypto Express5S when it is configured in EP11 mode Accelerator CEX5A for acceleration of public key and private key cryptographic operations that are used with SSL TLS processing See 6 5 4 Crypto Express5S as an accelerator on page 218 These modes can be configured by using the SE and the PCIe adapter must be configured offline to change the mode The Crypto Express5S feature has...

Page 240: ... task from the SE which reflects the cryptographic definitions in the image profile for the partition With this function adding and removing the cryptographic feature on a running operating system can be done dynamically The same usage domain index can be defined more than once across multiple LPARs However the PCIe cryptographic adapter number that is coupled with the usage domain index that is s...

Page 241: ...amir and Leonard Adleman It is a widely used asymmetric public key algorithm which means that the encryption key is public while the decryption key is kept secret It is based on the difficulty of factoring the product of two large prime numbers The number describes the length of the keys ECDSA 192 224 256 384 521 Prime NIST Elliptic Curve Cryptography ECC is a family of asymmetric cryptographic al...

Page 242: ...dit card transaction processing Visa Data Secure Platform DSP Point to Point Encryption P2PE Europay MasterCard and Visa EMV card transaction processing Card personalization Other financial transaction processing Integrated role based access control system User Defined Extensions support UDX allows the user to add customized operations to a cryptographic coprocessor User Defined Extensions to the ...

Page 243: ...omains in an AP As such the Crypto Express cards are enhanced to handle 256 domains and the z Systems firmware provides up to 40 on z13s and 85 on z13 domains to customers to match the current LPAR maximum Customers have the flexibility of mapping individual LPARs to unique crypto domains or continuing to share crypto domains across LPARs These are the requirements to support 40 or 85 domains Hard...

Page 244: ...he internal systems of banks and their servers CCA is designed to support the functions that are essential to those parts of the German banking industry that are governed by DK requirements The functions include key management support for new AES key types AES key derivation support and several DK specific PIN and administrative functions This support includes PIN method APIs PIN administration AP...

Page 245: ...rmware zEC12 zBC12 z196 or z114 servers and Crypto Express3 with CCA V4 4 firmware Software requirements z OS V2 2 z OS V2 1 or z OS V1 13 with the Cryptographic Support for z OS V1R13 z OS V2R1 web deliverable FMID HCR77A1 with PTFs z OS V2 1 FMID HCR77A0 with PTFs z OS V1 13 with the Cryptographic Support for z OS V1R13 z OS V2R1 web deliverable FMID HCR77A1 with PTFs z OS V1 12 or z OS V1 13 wi...

Page 246: ...able to the EP11 When defined in EP11 mode the TKE workstation is required to manage the Crypto Express5S feature 6 5 4 Crypto Express5S as an accelerator A Crypto Express5S card running in accelerator mode supports only RSA clear key and SSL Acceleration A request is processed fully in hardware The Crypto Express accelerator is a coprocessor that is reconfigured by the installation process so tha...

Page 247: ...rtition The installation of the CP Assist for Cryptographic Functions CPACF DES TDES enablement Feature Code 3863 is required to use the Crypto Express5S feature Each Crypto Express5S feature contains one PCI X adapter The adapter can be in the following configurations IBM Enterprise Common Cryptographic Architecture CCA Coprocessor CEX5C IBM Enterprise Public Key Cryptography Standards 11 PKCS Co...

Page 248: ...gical partition if it is being set up as the TCP IP host for the TKE If you are setting up the host TCP IP in this logical partition to communicate with the TKE the partition is used as a path to other domains Master Keys Indicate all the control domains that you want to access including this partition s own control domain from this partition Control and Usage Domain Identifies the cryptographic c...

Page 249: ...om the SE Select the CPCs and click View LPAR Cryptographic Controls in the CPC Operational Customization window The resulting window displays the definition of Usage and Control domain indexes and PCI Cryptographic candidate and online lists as shown in Figure 6 7 The information is provided only for active logical partitions Figure 6 7 SE View LPAR Cryptographic Controls Operational changes can ...

Page 250: ...BC z9 EC z9 BC z990 and z890 servers 6 6 1 Logical partition TKE host and TKE target If one or more LPARs are configured to use Crypto Express5S coprocessors the TKE workstation can be used to manage DES AES ECC and PKA master keys This management can be done for all cryptographic domains of each Crypto Express coprocessor feature that is assigned to the LPARs defined to the TKE workstation Each L...

Page 251: ...domain and push it to a set of domains This feature is valuable for deploying new domains Coordinated Master Key roll Ability to start Coordinated Master Key roll from the TKE Three new wizard like features Create new TKE zone Create new Migration Zone and Configure Host Roles and Authorities Operational Key Option This feature allows the client to decide whether operational key commands are limit...

Page 252: ...ng a MES upgrade to a TKE 8 1 installation the following steps must be performed Upgrade from TKE 8 0 to TKE 8 1 1 Save Upgrade Data on TKE 8 0 to USB memory to save client data 2 Upgrade firmware to TKE 8 1 3 Perform a Frame Roll install to apply Save Upgrade Data client data to the TKE 8 1 system 4 Run the TKE Workstation Setup wizard Upgrade from TKE 7 3 to TKE 8 1 1 Save Upgrade Data on TKE 7 ...

Page 253: ...rypto Express4S running in EP11 mode requires smart cards to hold administrator certificates and master key material The smart card must be P N 74Y0551 d A Crypto Express4S running in EP11 mode requires smart cards to hold administrator certificates and master key material The smart card must be P N 74Y0551 Attention The TKE is unaware of the CPC type where the host crypto module is installed That...

Page 254: ...res system master keys to be loaded X X System master key storage X X Retained key storage X Tamper resistant hardware packaging X X Xc Designed for FIPS 140 2 Level 4 certification X X X Supports Linux applications that perform SSL handshakes X RSA functions X X X High performance SHA 1 and SHA2 X X X Clear key DES or triple DES X AES for 128 bit 192 bit and 256 bit keys X X X Pseudorandom number...

Page 255: ...BrainPool X ECDH 192 224 256 384 521 Prime NIST X ECDH 160 192 224 256 320 384 512 BrainPool X PNG Prime Number Generator X a To make the addition of the Crypto Express features nondisruptive the logical partition must be predefined with the appropriate PCI Express cryptographic adapter number This number must be selected in its candidate list in the partition image profile b This feature is not r...

Page 256: ...228 IBM z13s Technical Guide ...

Page 257: ...eric software support for IBM z BladeCenter Extension zBX Model 004 Support of z13s functions depends on the operating system its version and release This chapter covers the following topics Operating systems summary Support by operating system Support by function Cryptographic support GDPS Virtual Appliance z OS migration considerations IBM z Advanced Workload Analysis Reporter zAware Coupling fa...

Page 258: ... IBM Software Support Services Service Extension for z OS V1 12 fee based corrective service can be obtained up to September 2017 No Yes Service is required See the following box which is titled Features z VM V6R2b b z VM V6R2 with PTF provides compatibility support CEX5S with enhanced crypto domain support No Yesc c z VM supports both 31 bit and 64 bit mode guests z VSE V5R1 No Yes z TPF V1R1 Yes...

Page 259: ...ided on z OS V1R12 Exploitation support is provided only on z OS V2R1 and later For a list of supported functions and their minimum required levels see Table 7 3 on page 234 7 2 2 z VM At general availability z VM V6R2 and V6R3 provide compatibility support with limited use of new z13s functions For a list of supported functions and their minimum required support levels see Table 7 3 on page 234 F...

Page 260: ...pport Table 7 2 shows the service levels of SUSE and Red Hat releases supported at the time of writing Table 7 2 Current Linux on z Systems distributions Statements of Direction Removal of support for Expanded Storage XSTORE z VM V6 3 is the last z VM release that supports XSTORE for either host or guest use The IBM z13 and z13s servers will be the last z Systems servers to support XSTORE Stabiliz...

Page 261: ...ms KVM for IBM z is an open virtualization alternative for z Systems built on Linux and KVM KVM for IBM z delivers a Linux familiar administrator experience that can enable simplified virtualization management and operation See Table F 1 on page 513 for a list of supported features 7 2 7 z13s function support summary The following tables summarize the z13s functions and their minimum required oper...

Page 262: ... GB large page support Y Y Yf N N N Out of order execution Y Y Y Y Y Y Hardware decimal floating pointg Y Y Y Y Y Y 40 LPARs Y Y Y N Y Y CPU measurement facility Y Y Y Y Y Y Enhanced flexibility for Capacity on Demand CoD Y Y Y Y Y Y HiperDispatch Y Y Y Y Y N Three logical channel subsystems LCSSs Y Y Y N N N Three subchannel sets per LCSS Y Y Y N Y Y Simultaneous MultiThreading SMT Y Y N N Yh N S...

Page 263: ...ect Memory Access RDMA over Converged Ethernet RoCE 10GbE RoCE Express Y Y Y Y Y N Shared RoCE environment Y Y N N Y N Shared Memory Communications Direct Memory Access SMC D SMC D over ISM Internal Shared Memory Y N N N Yjh N FICON Fibre Connection and FCP Fibre Channel Protocol FICON Express 8S channel path identifier CHPID type FC when using z13s FICON or channel to channel CTC Y Y Y Y Y Y FICO...

Page 264: ...ICON Express 16S CHPID type FCP support of hardware data router Y N N N Y N T10 DIF support by the FICON Express16S features when defined as CHPID type FCP N N N N Y Y Health Check for FICON Dynamic routing Y Y Y Y N N OSA Open Systems Adapter OSA Express5S 10 Gigabit Ethernet Long Reach LR and Short Reach SR CHPID type OSD Y Y Y Y Y Y OSA Express5S 10 Gigabit Ethernet LR and SR CHPID type OSX Y Y...

Page 265: ...hernet LX and SX CHPID type OSD one port per CHPID Y Y Y Y Y Y OSA Express4S 1000BASE T CHPID type OSC one or two ports per CHPID Y Y Y Y Y Y OSA Express4S 1000BASE T CHPID type OSD two ports per CHPID Y Y Y Y Y Y OSA Express4S 1000BASE T CHPID type OSD one port per CHPID Y Y Y Y Y Y OSA Express4S 1000BASE T CHPID type OSE one or two ports per CHPID Y Y Y Y Y Y OSA Express4S 1000BASE T CHPID type ...

Page 266: ... channel reporting Y Y Y Y N N a PTFs might be required for toleration support or use of z13s functions and features b Hardware limit of 6 CP and 2zIIP 1CP ratio c With SMT 40 threads d zAware V2 0 Linux Guest support e 4 TB of real storage is supported with PTF for APAR OA47439 f PTF support required and with RSM enabled web delivery g Packed decimal conversion support h PTF support required i On...

Page 267: ... MB page support N N N N Y 2 GB Large Page Support Out of order execution Y Y Y Y Y 40 logical partitions Y Y Y Y Y HiperDispatch N N N N N Three logical channel subsystems LCSSs N N N N Y Three subchannel set per LCSS Y Y Y N Y Simultaneous multithreading SMT N N N N Y Single instruction multiple data SIMD N N N N N Multi vSwitch Link Aggregation N N N N N Cryptography CP Assist for Cryptographic...

Page 268: ... over Converged Ethernet RoCE 10GbE RoCE Express N N N N Nk Shared Memory Communications Direct SMC D SMC D over Internal Shared Memory ISM N N N N N FICON and FCP FICON Express8S support of zHPF single track CHPID type FC N N N N Y FICON Express8 support of zHPF multitrack CHPID type FC N N N N Y High Performance FICON for z Systems zHPF N N N N Yl GRS FICON CTC toleration N_Port ID Virtualizatio...

Page 269: ...type FCP support of hardware data router N N N N Y T10 DIF support by the FICON Express16S features when defined as CHPID type FCP N N N N Y OSA Large send for IPv6 packets Inbound workload queuing for Enterprise Extender N N N N N Checksum offload for IPV6 packets N N N N N Checksum offload for LPAR to LPAR traffic N N N N N OSA Express5S 10 Gigabit Ethernet LR and SR CHPID type OSD Y Y Y Yo Y OS...

Page 270: ... SX CHPID type OSD two ports per CHPID Y Y Y Yt Y OSA Express4S Gigabit Ethernet LX and SX CHPID type OSD one port per CHPID Y Y Y Y Y OSA Express4S 1000BASE T CHPID type OSC one or two ports per CHPID Y Y Y N OSA Express4S 1000BASE T CHPID type OSD two ports per CHPID Y Y Y Yg Y OSA Express4S 1000BASE T CHPID type OSD one port per CHPID Y Y Y Y Y OSA Express4S 1000BASE T CHPID type OSE one or two...

Page 271: ...itions therefore must be migrated to support the HiperSockets definitions CHPID type IQD VCHID specifies the virtual channel identification number associated with the channel path Valid range is 7E0 7FF VCHID is not valid on z Systems before z13s servers j Applicable to guest operating systems k IBM is working with its Linux distribution partners to include support in future Linux on z Systems dis...

Page 272: ...nd the IBM zAware firmware will now run in this partition There are no hardware dependencies zACI will be delivered as part of the base code on each z13s and z13 driver level 27 server The IBM zAware virtual appliance can pinpoint deviations in z OS normal system behavior It also improves real time event diagnostic tests by monitoring the z OS operations log Operating system Maximum number of PUs ...

Page 273: ...management for users who intend to run KVM for IBM z Systems as hypervisor or Linux on z Systems running in a partition LPAR The new mode DPM provides simplified consumable and enhanced partition lifecycle and dynamic I O management capabilities by using the Hardware Management Console HMC For more information see Appendix E IBM Dynamic Partition Manager on page 501 7 3 2 zIIP support zIIPs do not...

Page 274: ...ion The IBM zEnterprise EC12 introduced an architectural feature called Transactional Execution TX This capability is known in academia and industry as hardware transactional memory Transactional execution is also implemented in z13 z13s and zBC12 servers This feature enables software to indicate to the hardware the beginning and end of a group of instructions that need to be treated in an atomic ...

Page 275: ... that cannot afford to page The z OS design for Flash Express storage does not completely remove the virtual storage constraints that are created by a paging spike in the system However some scalability relief is expected because of faster paging I O with Flash Express storage Flash Express storage is allocated to an LPAR similarly to main memory The initial and maximum amount of Flash Express Sto...

Page 276: ...e area ESQA per CPU and RSM stack is increased Also a new memory pool for pageable large pages is added For web deliverable code on z OS see the z OS downloads website http www ibm com systems z os zos downloads The support also is delivered in z OS V2R1 included with the base product or later Linux on z Systems also offers the support for Flash Express as an SCM device This support is useful for ...

Page 277: ...port can be achieved by any of the following ways Data class level Two new values zEDC Required ZR and zEDC Preferred ZP can be set with the COMPACTION option in the data class System Level Two new values zEDC Required ZEDC_R and zEDC Preferred ZEDC_P can be specified with the COMPRESS parameter found in the IGDSMSXX member of the SYS1 PARMLIB data sets Data class takes precedence over system leve...

Page 278: ... 1 MB large pages Table 7 10 Minimum support requirements for 1 MB large page Operating system Support requirements z OS z OS V2R1 with supporting PTFs SPE for IBM VTAM TCP IP and IOS The IOS PTF is a minor change to allow greater than 256 xFF PFIDs for RoCE z VM z VM V6 3 with supporting PTFs The z VM V6 3 SPE for base PCIe support is required When running z OS as a guest z OS APAR OA43256 is req...

Page 279: ...mal floating point and z OS XL C C considerations on page 291 Table 7 12 Minimum support requirements for decimal floating point 7 3 10 Up to 40 LPARs This feature first made available in z13s allows the system to be configured with up to 40 LPARs Because channel subsystems can be shared by up to 15 LPARs you must configure three channel subsystems to reach the 40 LPARs limit Table 7 13 lists the ...

Page 280: ...nondisruptively if supported by the guest operating system Releasing memory from z VM is not supported Releasing memory from the z VM guest depends on the guest s operating system support Linux on z Systems also supports both acquiring and releasing memory nondisruptively This feature is enabled for SUSE Linux Enterprise Server 11 and RHEL 6 Dynamic LPAR memory upgrade is not supported for zACI mo...

Page 281: ... supporting manual operation with guidelines can help ensure that sufficient processing power is available with the least possible delay Support requirements are listed in Table 7 16 Table 7 16 Minimum support requirements for capacity provisioning 7 3 15 Dynamic PU add Planning of an LPAR configuration allows defining reserved PUs that can be brought online when extra capacity is needed Operating...

Page 282: ...regardless of the HIPERDISPATCH specification If more logical processors are added to an LPAR that has 64 or fewer logical processors and the additional logical processors raise the number of logical processors to more than 64 the LPAR automatically operates in HiperDispatch Management Mode regardless of the HIPERDISPATCH YES NO specification That is even if the LPAR has the HIPERDISPATCH NO speci...

Page 283: ...uses the HiperDispatch facility for improved processor efficiency by better use of the processor cache to take advantage of the cache rich processor node and drawer design of the z13s system The supported processor limit has been increased to 64 whereas with SMT it remains at 32 supporting up to 64 threads running simultaneously The operating system support requirements for HiperDispatch are liste...

Page 284: ...st support for the MSS facility to allow using an alternative subchannel set for a Peer to Peer Remote Copy PPRC secondary volumes 7 3 19 Three subchannel sets With z13s a third subchannel set SS2 is introduced Together with the second subchannel set SS1 SS2 can be used for disk alias devices of both primary and secondary devices and as Metro Mirror secondary devices These subchannel sets help fac...

Page 285: ...e minimum support requirements for MIDAW Table 7 22 Minimum support requirements for MIDAW 7 3 22 HiperSockets Completion Queue The HiperSockets Completion Queue function is implemented on z13 z13s zEC12 zBC12 z196 and z114 servers The HiperSockets Completion Queue function is designed to allow HiperSockets to transfer data synchronously if possible and asynchronously if necessary Therefore it com...

Page 286: ...he IEDN through OSA Express for zBX OSX adapters It can then bridge to another central processor complex CPC through OSD adapters This configuration extends the reach of the HiperSockets network outside of the CPC to the entire ensemble and hosts that are external to the CPC The system is displayed as a single Layer 2 network Table 7 25 lists the minimum support requirements for HiperSockets Virtu...

Page 287: ...7 lists the minimum support requirements for HiperSockets IPv6 CHPID type IQD Table 7 27 Minimum support requirements for HiperSockets IPv6 CHPID type IQD 7 3 27 HiperSockets Layer 2 support For flexible and efficient data transfer for IP and non IP workloads the HiperSockets internal networks on z13s can support two transport modes These modes are Layer 2 Link Layer and the current Layer 3 Networ...

Page 288: ...igher performance helping to prepare for an end to end 16 Gbps infrastructure to meet the increased bandwidth demands of your applications The new features for the multimode and single mode fiber optic cabling environments have reduced latency for large read write operations and increased bandwidth compared to the FICON Express8S features Table 7 29 lists the minimum support requirements for FICON...

Page 289: ...ss8S Table 7 30 Minimum support requirements for FICON Express8S Support of SCSI devices CHPID type FCP N A V6R2b V5R1 N A SUSE Linux Enterprise Server 12 SUSE Linux Enterprise Server 11 Red Hat RHEL 7 Red Hat RHEL 6 Support of hardware data router CHPID type FCP N A V6R3 N A N A SUSE Linux Enterprise Server 12 SUSE Linux Enterprise Server 11 SP3 Red Hat RHEL 7 Red Hat RHEL 6 4 Support of T10 DIF ...

Page 290: ... Red Hat RHEL 6 4 Support of T10 DIF CHPID type FCP N A V6R2b N A N A SUSE Linux Enterprise Server 12 SUSE Linux Enterprise Server 11 SP3 Red Hat RHEL 7 Red Hat RHEL 6 4 a PTFs are required to support GRS FICON CTC toleration b PTFs are required Operating system z OS z VM z VSE z TPF Linux on z Systems Important The z13 and z13s servers are the last z Systems servers to support FICON Express 8 cha...

Page 291: ...posed configuration All added or changed logical control units and devices are added into the proposed configuration They are assigned proposed control unit and device numbers and channel paths that are based on the defined policy zDAC uses channel path chosen algorithms to minimize single points of failure The zDAC proposed configurations are created as work I O definition files IODF that can be ...

Page 292: ...pported the transfer of small blocks of fixed size data 4 K from a single track This capability has been extended first to 64 KB and then to multitrack operations The 64 KB data transfer limit on multitrack operations was removed by z196 This improvement allows the channel to fully use the bandwidth of FICON channels resulting in higher throughputs and lower response times The multitrack operation...

Page 293: ...he I O service time for these write operations at extended distances This enhancement to zHPF removes a key inhibitor for clients adopting zHPF over extended distances especially when using the IBM HyperSwap capability of z OS Table 7 33 lists the minimum support requirements for zHPF Table 7 33 Minimum support requirements for zHPF For more information about FICON channel performance see the perf...

Page 294: ...l for persistent IU pacing Extended distance FICON is transparent to operating systems and applies to all FICON Express16S FICON Express8S and FICON Express8 features that carry native FICON traffic CHPID type FC To use this enhancement the control unit must support the new IU pacing protocol IBM System Storage DS8000 series supports extended distance FICON for IBM z Systems environments The chann...

Page 295: ... performance technical papers on the z Systems I O connectivity website at http www ibm com systems z hardware connectivity fcp_performance html 7 3 40 N_Port ID Virtualization N_Port ID Virtualization NPIV allows multiple system images in LPARs or z VM guests to use a single FCP channel as though each were the sole user of the channel This feature first introduced with z9 EC can be used with earl...

Page 296: ...t Ethernet LX and SX The OSA Express5S Gigabit Ethernet feature is installed exclusively in the PCIe I O drawer Each feature has one PCIe adapter and two ports The two ports share a channel path identifier CHPID type OSD exclusively Each port supports attachment to a 1 Gbps Ethernet local area network LAN The ports can be defined as a spanned channel and can be shared among LPARs and across logica...

Page 297: ...anagement network on page 274 Each adapter can be configured in the following modes QDIO mode with CHPID types OSD and OSN Non QDIO mode with CHPID type OSE Local 3270 emulation mode including OSA ICC with CHPID type OSC OSA ICC OSC Channel supports Secure Sockets Layer on z13s and z13 Driver 27 servers Designed to improve security of console operations Up to 48 secure sessions per CHPID the overa...

Page 298: ... to better avoid redundancy Operating system Support requirements using two ports per CHPID Support requirements using one port per CHPID z OS OSC OSD OSE and OSNb z OS V1R12a OSC OSD OSE OSM and OSNb z OS V1R12a z VM OSC OSDa OSE and OSNb z VM V6R2 a PTFs are required b Although CHPID type OSN does not use any ports because all communication is LPAR to LPAR it is listed here for completeness OSC ...

Page 299: ...rnet feature Table 7 42 lists the minimum support requirements for OSA Express4S Gigabit Ethernet LX and SX Table 7 42 Minimum support requirements for OSA Express4S Gigabit Ethernet LX and SX Operating system Support requirements z OS OSD z OS V1R12a OSX z OS V1R12a a PTFs are required z VM OSD z VM V6R2 OSX z VM V6R2a and V6R3 for dynamic I O only z VSE OSD z VSE V5R1 OSX z VSE V5R1 z TPF OSD z ...

Page 300: ...ctions is unchanged Ensemble management with CHPID type OSM Operating system support is required to recognize and use the second port on the OSA Express4S 1000BASE T feature Table 7 43 lists the minimum support requirements for OSA Express4S 1000BASE T Table 7 43 Minimum support requirements for OSA Express4S 1000BASE T Ethernet Linux on z Systems OSD SUSE Linux Enterprise Server 12 SUSE Linux Ent...

Page 301: ...ent network INMN use these features OSA Express5S 1000BASE T Ethernet FC 0417 OSA Express4S 1000BASE T Ethernet FC 0408 For the IEDN use these features CHPID type OSX OSA Express5S 10 Gigabit Ethernet GbE Long Reach LR FC 0415 OSA Express5S 10 Gigabit Ethernet GbE Short Reach SR FC 0416 OSA Express4S 10 Gigabit Ethernet GbE Long Reach LR FC 0406 OSA Express4S 10 Gigabit Ethernet GbE Short Reach SR...

Page 302: ...etwork that connects all elements of an ensemble It is access controlled by using integrated virtual LAN VLAN provisioning No client managed switches or routers are required The IEDN is managed by the primary HMC that controls the ensemble This configuration helps reduce the need for firewalls and encryption and simplifies network configuration and management It also provides full redundancy for h...

Page 303: ...d has multiple LPAR support both as shared or spanned channels With the OSA ICC function 3270 emulation for console session connections is integrated in the z13s through a port on the OSA Express5S 1000BASE T or OSA Express4S 1000BASE T features This function eliminates the requirement for external console controllers such as 2074 or 3174 helping to reduce cost and complexity Each port can support...

Page 304: ...s OSA to differentiate workloads off the wire It then assigns work to a specific input queue per device to z OS The support is also available with OSA Express5S and OSA Express4S CHPID types OSD and OSX are supported Each input queue is a unique type of workload and has unique service and processing requirements The IWQ function allows z OS to preassign the appropriate processing resources for eac...

Page 305: ...equirements for IWQ 7 3 56 Inbound workload queuing for Enterprise Extender IWQ for the OSA Express features is enhanced to differentiate and separate inbound Enterprise Extender traffic to a dedicated input queue IWQ for Enterprise Extender is exclusive to OSA Express5S and OSA Express4S CHPID types OSD and OSX and the z OS operating system This limitation applies to z13 z13s zEC12 zBC12 z196 and...

Page 306: ...alent The minimum support requirements are listed in Table 7 49 Table 7 49 Minimum support requirements for Multi VSwitch Link Aggregation 7 3 60 QDIO data connection isolation for z VM The QDIO data connection isolation function provides a higher level of security when sharing an OSA connection in z VM environments that use VSWITCH The VSWITCH is a virtual network device that provides switching b...

Page 307: ...s does not require a Signal Adapter SIGA instruction to determine whether more data is available 7 3 63 Large send for IPv6 packets Large send for IPv6 packets improves performance by offloading outbound TCP segmentation processing from the host to an OSA Express5S and OSA Express4S feature by employing a more efficient memory transfer into OSA Express5S and OSA Express4S Large send support for IP...

Page 308: ... is owned by another IP stack that is sharing the OSA Express OSA Express sends the IP packet directly to the other IP stack The packet does not have to be placed out on the LAN which is termed LPAR to LPAR traffic Checksum offload is enhanced to support the LPAR to LPAR traffic which was not originally available Checksum offload is supported by the GbE features which include FC 0404 FC 0405 FC 04...

Page 309: ... and OSA Express4S features on z13s when in QDIO mode CHPID type OSD It is used by z OS V1R12 and higher releases 7 3 68 OSA Layer 3 virtual MAC for z OS environments To help simplify the infrastructure and facilitate load balancing when an LPAR is sharing an OSA MAC address with another LPAR each operating system instance can have its own unique logical or virtual MAC VMAC address All IP addresse...

Page 310: ...finiBand IFB and ICA using PCIe Gen3 technology can potentially provide high speed interconnection at short distances longer distance fiber optic interconnection and interconnection between partitions on the same system without external cabling For more information see 4 9 Parallel Sysplex connectivity on page 174 Integrated Coupling Adapter Support for PCIe Gen3 fanout also known as Integrated Co...

Page 311: ...nd does not support the use of InfiniBand or ICA coupling links by guests Table 7 56 lists the minimum support requirements for dynamic I O support for InfiniBand CHPIDs Table 7 56 Minimum support requirements for dynamic I O support for InfiniBand CHPIDs 7 3 74 Simultaneous multithreading SMT is the hardware capability to process up to two simultaneous threads in a single core sharing the resourc...

Page 312: ...fit immediately without requiring application software of IP topology changes Similar to SMC R this protocol utilizes shared memory architectural concepts eliminating TCP IP processing in the data path yet preserving TCP IP Qualities of Service for connection management purposes Operating system Support requirements z OS z OS V2R1 with APARs z VM z VM V6R3 Statement of Direction IBM is working wit...

Page 313: ...he following encryption types The Advanced Encryption Standard AES symmetric encryption The Data Encryption Standard DES symmetric encryption The Secure Hash Algorithm SHA hashing For more information see 6 4 CP Assist for Cryptographic Functions on page 207 Table 7 60 lists the support requirements for CPACF on z13s Table 7 60 Support requirements for CPACF Operating system Support requirements z...

Page 314: ...y ICSF is a base component of z OS It is designed to transparently use the available cryptographic functions whether CPACF or Crypto Express to balance the workload and help address the bandwidth requirements of the applications Despite being a z OS base component ICSF functions are generally made available through web deliverable support a few months after a new z OS release Therefore new functio...

Page 315: ... 4 HMAC Support ANSI X9 8 Pin ANSI X9 24 CBC Key Wrapping CKDS constraint relief PCI Audit All callable services AMODE 64 PKA RSA OAEP with SHA 256 algorithma HCR7790 V1R13 V1R12 V1R11 Cryptographic Support for z OS V1R11 V1R13 Expanded key support for AES algorithm Enhanced ANSI TR 31 PIN block decimalization table protection Elliptic Curve Diffie Hellman ECDH algorithm RSA in the Modulus Exponen...

Page 316: ...cryption VFPE services DK AES PIN and AES MAC Generate and Verify Services Support for exploitation of counter mode CTR for AES based encryption on z196 and later processors Enhanced random number generation exploiting CPACF Deterministic Random Number Generate DRNG instruction with ability to disable RNG Cache Services and support for key archiving and key material validity Enhancement to the ICS...

Page 317: ... through APAR OA42011 All systems in a sysplex that share a PKDS TKDS must be at HCR77A1 to use the PKDS TKDS Coordinated Administration support For more information see Migration from z OS V1R13 and z OS V1R12 to z OS V2R1 GA32 0889 7 5 GDPS Virtual Appliance The GDPS Virtual Appliance solution implements GDPS PPRC Multiplatform Resilience for z Systems also known as xDR xDR coordinates near cont...

Page 318: ...tivity is available only when other systems also support the same type of coupling When you plan to use the InfiniBand coupling or ICA coupling links technology consult the Coupling Facility Configuration Options white paper which is available at the following website http www ibm com systems z advantages pso whitepaper html 7 6 4 Large page support The large page support function must not be enab...

Page 319: ...itecture level ARCH 11 uses instructions that are available on the z13s TUNE This option allows optimization of the application for a specific system architecture within the constraints that are imposed by the ARCHITECTURE option The TUNE level must not be lower than the setting in the ARCHITECTURE option For more information about the ARCHITECTURE and TUNE compiler options see the z OS V1R13 0 XL...

Page 320: ...he following tasks in z OS 1 For each z OS that is to be monitored through the IBM zAware client configure a network connection in the TCP IP profile If necessary update the firewall settings 2 Verify that each z OS system meets the sysplex configuration and OPERLOG requirements for monitored clients of the IBM zAware virtual appliance 3 Configure the z OS system logger to send data to the IBM zAw...

Page 321: ...intains robust failure recovery capability through the redundancy of duplexing Reduces z OS CF and link utilization overhead costs associated with synchronous duplexing of lock structures Requires CFCC level 21 and service level 02 16 and z OS V2 2 SPE with PTFs for APAR OA47796 Large memory support Improve availability scalability for larger CF cache structures and data sharing performance with l...

Page 322: ...ble at the following web page http www ibm com systems support z cfsizer altsize html Before the migration install the compatibility coexistence PTFs A planned outage is required when you upgrade the CF or CF LPAR to CFCC Level 21 7 8 2 Flash Express exploitation by CFCC CFCC Level 19 is the minimum level that supports Flash Express Initial CF Flash Express exploitation is targeted for WebSphere M...

Page 323: ... as SMT and is available only in zIIP and IFL cores For more information about SMT see 3 4 1 Simultaneous multithreading on page 89 The z OS and the z VM have SMT support if the support is enabled by PTFs z OS 2 2 supports the operation of zIIP processors in SMT mode The following APARs must be applied to z OS V2R1 to use SMT5 OA43366 BCP OA43622 WLM OA44439 XCF The use of SMT on z OS V2R1 and lat...

Page 324: ...g the SET OPT xx setting in the IEAOPTxx parmlib Changing the MT mode for all cores can take some time to complete The activation of SMT mode also requires that the HMC Customize Delete Activation Profiles task Do not end the time slice if a partition enters a wait state must not be selected This is the recommended default setting PROCVIEW CORE requires DISPLAY M CORE and CONFIG CORE to display th...

Page 325: ... SMT controls each core and is responsible for maximizing their throughput and meeting workload goals with the smallest number of cores In z OS HiperDispatch cache optimization should be considered when you must choose the two threads to be dispatched in the same processor HiperDispatch attempts to dispatch guest virtual CPU on the same logical processor on which they have run previously PR SM att...

Page 326: ... through XL C C ARCH 11 and TUNE 11 or SIMD usage by MASS and ATLAS libraries requires the z13s support for z OS V2R1 XL C C web deliverable The followings compilers have built in functions for SMID IBM Java XL C C Enterprise COBOL Enterprise PL I Code must be developed to take advantage of the SIMD functions and applications with SIMD instructions abend if they run on a lower hardware level syste...

Page 327: ...rtual environment 7 The existing IDAW design allows the first IDAW in a list to point to any address within a page Subsequent IDAWs in the same list must point to the first byte in a page Also IDAWs except the first and last IDAW in a list must deal with complete 2 K or 4 K units of data Figure 7 3 shows a single CCW that controls the transfer of data that spans non contiguous 4 K frames in main s...

Page 328: ...ves and link speed increases data chaining techniques become less efficient because of switch fabrics control unit processing and exchanges and other issues The MIDAW facility is a method of gathering and scattering data from and into discontinuous storage locations during an I O operation The modified IDAW MIDAW format is shown in Figure 7 4 It is 16 bytes long and is aligned on a quadword Figure...

Page 329: ...suffix is used to improve data reliability and facilitates other functions that are described in the following paragraphs Therefore for example if the DCB BLKSIZE or VSAM CI size is equal to 8192 the actual block on storage consists of 8224 bytes The control unit itself does not distinguish between suffixes and user data The suffix is transparent to the access method and database In addition to re...

Page 330: ... the MIDAW facility can provide significant performance benefits when used with applications that use EF data sets such as DB2 or long chains of small blocks For more information about FICON and MIDAW see the following resources The I O Connectivity website contains material about FICON channel performance http www ibm com systems z connectivity DS8000 Performance Monitoring and Tuning SG24 7146 7...

Page 331: ...reated manually or exported from the HCD HCM The WWPN tool on z13s CHPID type FCP requires the following levels z OS V1R12 and later z VM V6R2 and later The WWPN tool is applicable to all FICON channels that are defined as CHPID type FCP for communication with SCSI devices on z13s It is available for download at the Resource Link at the following website http www ibm com servers resourcelink 7 14 ...

Page 332: ...m support for zBX Model 003 HX5 blades 7 15 2 IBM WebSphere DataPower Integration Appliance XI50 for zEnterprise The IBM WebSphere DataPower Integration Appliance XI50 for zEnterprise DataPower XI50z is a special purpose double wide blade The DataPower XI50z is a multifunctional appliance that provides these features Offers multiple levels of XML optimization Streamlines and secures valuable servi...

Page 333: ...are licensing are available from IBM depending on the software product Monthly license charge MLC International Program License Agreement IPLA MLC pricing metrics have a recurring charge that applies each month In addition to the permission to use the product the charge includes access to IBM product support during the support period MLC metrics in turn include various offerings IPLA metrics have ...

Page 334: ...arges Software products that are licensed under flat charges are not eligible for subcapacity pricing There is a single charge for each CPC on the z13s Subcapacity license charges For eligible programs subcapacity licensing allows software charges that are based on the measured utilization by logical partitions instead of the total number of MSUs of the CPC Subcapacity licensing removes the depend...

Page 335: ...or Sysplexes when not all of the nodes are z13 z13s zEC12 zBC12 z196 or z114 servers zNALC PSLC 7 16 3 Advanced Workload License Charges Advanced Workload License Charges AWLC was introduced with the IBM zEnterprise 196 They use the measuring and reporting mechanisms and the existing MSU tiers from VWLCs although the prices for each tier were lowered AWLC can be implemented in full capacity or sub...

Page 336: ...P z OS with zNALC provides a strategic pricing model that is available on the full range of z Systems servers for simplified application planning and deployment zNALC allows for aggregation across a qualified Parallel Sysplex which can provide a lower cost for incremental growth across new workloads that span a Parallel Sysplex For more information see the zNALC website http www ibm com systems z ...

Page 337: ...E V12 and IMS Tools Application development tools Certain WebSphere for z OS products Linux middleware products z VM V5 and V6 Generally three pricing metrics apply to IPLA products for z13s and z Systems Value unit VU VU pricing applies to the IPLA products that run on z OS Value Unit pricing is typically based on the number of MSUs and allows for a lower cost of incremental growth Examples of el...

Page 338: ...ource Manager The IBM z Unified Resource Manager is available through z13 z13s zEC12 and zBC12 hardware features either ordered with the system or ordered later No separate software licensing is required 7 17 References For current planning information see the support website for each of the following operating systems z OS http www ibm com systems support z zos z VM http www ibm com systems suppo...

Page 339: ...gives the customer an unprecedented flexibility and a finer control over cost and value Given today s business environment the growth capabilities of z13s servers provide these benefits among others Enabling exploitation of new business opportunities Supporting the growth of dynamic smart environments Managing the risk of volatile high growth and high volume applications Supporting 24x365 applicat...

Page 340: ...U facility supports LICCC upgrades only Temporary upgrades All temporary upgrades are LICCC based The billable capacity offering is On Off CoD The two replacement capacity offerings that are available are Capacity BackUp CBU and Capacity for Planned Events CPE For more information see 8 1 2 Terminology related to CoD for z13s systems on page 313 Concurrent and nondisruptive upgrades Depending on t...

Page 341: ... the use of spare capacity in a CPC to replace capacity from another CPC within an enterprise for a limited time Typically CBU is used when another CPC of the enterprise has failed or is unavailable because of a disaster event The CPC using CBU replaces the missing CPC s capacity CPE Used when temporary replacement capacity is needed for a short term event CPE activates processor capacity temporar...

Page 342: ...here xx indicates the number of active CPs Note that xx can have a range of 01 06 Model permanent capacity identifier MPCI Keeps information about capacity settings that were active before any temporary capacity was activated Model temporary capacity identifier MTCI Reflects the permanent capacity with billable capacity only without replacement capacity If no billable temporary capacity is active ...

Page 343: ...e Link IBM Resource Link is a technical support website that is included in the comprehensive set of tools and resources available from the IBM Systems technical support site http www ibm com servers resourcelink Secondary approval An option selected by the customer that a second approver control each CoD order When a secondary approval is required the request is sent for approval or cancellation ...

Page 344: ...es to a backup server during an unforeseen loss of server capacity because of an emergency CPE This offering enables you to replace model capacity or specialty engines due to a relocation of workload during system migrations or a data center move CBU or CPE temporary upgrades can be ordered by using the CIU application through Resource Link or by calling your IBM marketing representative Temporary...

Page 345: ... currently installed to a higher MCI In addition a capacity increase can be delivered by increasing the CP capacity setting and at the same time decreasing the number of active CPs z13s servers support the concurrent addition of processors to a running logical partition LPAR As a result you can have a flexible infrastructure in which you can add capacity without pre planning The OS running on the ...

Page 346: ...s zIIPs and SAPs Unused PUs if they are available on the installed CPC drawers or if the MCI for the CPs can be increased Memory when unused capacity is available on the installed memory cards The plan ahead memory option is available for customers to gain better control over future memory upgrades See 2 4 4 Memory upgrades on page 63 for more details Concurrent hardware installation upgrades MES ...

Page 347: ...requisites The CIU facility supports LICCC upgrades only It does not support I O upgrades All additional capacity that is required for an upgrade must be previously installed An additional CPC drawer or I O cards cannot be installed as part of an order placed through the CIU facility The sum of CPs unassigned CPs ICFs zIIPs IFLs and unassigned IFLs cannot exceed the PU count of the installed drawe...

Page 348: ...ly add processors CPs ICFs zIIPs IFLs and SAPs and memory or change the MCI up to the limits of the installed CPC drawers on an existing z13s CPC Temporary upgrades The z13s base model describes permanent and dormant capacity Figure 8 1 using the capacity marker and the number of PU features installed on the CPC Up to eight temporary offerings or records can be present Each offering has its own po...

Page 349: ...iguration is the only possible activation In addition this approach enables you to update and replenish temporary upgrades even in situations where the upgrades are already active Likewise depending on the configuration permanent upgrades can be performed while temporary upgrades are active Figure 8 2 shows examples of activation sequences of multiple temporary upgrades Figure 8 2 Example of tempo...

Page 350: ...rd CBU contract provides for five 10 day tests and one 90 day disaster activation over a five year period Contact your IBM representative for details CPE is a concurrent and temporary activation of extra CPs ICFs zIIPs IFLs and SAPs an increase of the MCI or both The CPE offering is used to replace temporary lost capacity within a customer s enterprise for planned downtime events such as during da...

Page 351: ...ades on page 63 Installing an additional CPC drawer and using LICCC to activate extra memory capacity on the installed CPC drawer MES upgrades for I O are done by the following method Installing additional I O cards and supporting infrastructure if required in the I O drawers or PCIe I O drawers that are already installed or installing extra PCIe I O drawers to hold the new cards MES upgrades for ...

Page 352: ...re memory upgrades See 2 4 5 Preplanned memory on page 63 for details about plan ahead memory features The N10 model has as a minimum ten 16 GB dual inline memory modules DIMMs resulting in 160 GB of installed memory The minimum customer addressable storage is 64 GB If you require more than that amount a non concurrent upgrade can install up to 984 GB of memory for customer use by changing the exi...

Page 353: ... that will be usable by the customer FC 1899 is used to activate previously installed preplanned memory and can activate all the preinstalled memory or subsets of it For each additional 16 GB of memory to be activated one FC 1899 must be added and one FC 1996 must be removed FC 1993 specifies memory to be installed but not used Order one feature for each 8 GB that will be usable by the customer FC...

Page 354: ...40 Client 216 120 152 184 216 32 128 RAIM 4 x 32GB 384 for HSA 40 Client 344 248 280 312 344 32 128 RAIM 4 x 32GB 512 for HSA 40 Client 472 408 472 64 256 RAIM 4 x 64GB 640 for HSA 40 Client 600 536 600 64 256 RAIM 4 x 64GB 1024 for HSA 40 Client 984 728 856 984 128 512 RAIM 4 x 128GB 2048 for HSA 40 Client 2008 1112 1240 1368 1496 1624 1752 1880 2008 128 N20 two drawers Physical Memory RAIM GB Ad...

Page 355: ...a Single Object Operation SOO to the servers SE Figure 8 6 shows the window for the FoD Blades feature values shown under the Perform a Model Conversion Feature on Demand Manage function Figure 8 6 Feature on Demand window for zBX Blades features HWMs Only one FoD LICCC record is installed or staged at any time in the system and its contents can be viewed under the Manage window as shown in Figure...

Page 356: ...been withdrawn from marketing WDfM Except for DataPower XI50z Blades supported x and p blades might still be available from distributors or on the used market Empty slot entitlements for x and p blades if already existing can be populated New entitlements for blades were WDfM as of December 2015 Consider upgrading zBX to a Model 004 until withdrawn WDfM date not announced 8 4 Permanent upgrade thr...

Page 357: ...nk initiates an order 1 Sign on to Resource Link 2 Select Customer Initiated Upgrade from the main Resource Link page The customer and server details that are associated with the user ID are listed 3 Select the server that will receive the upgrade The current configuration PU allocation and memory is shown for the selected server 4 Select Order Permanent Upgrade Resource Link limits the options to...

Page 358: ...provides the interface that enables you to order a concurrent upgrade for a server You can create cancel view the order and view the history of orders that were placed through this interface Configuration rules enforce that only valid configurations are generated within the limits of the individual server Warning messages are issued if you select invalid upgrade options The process enables only on...

Page 359: ...de of the web page Resource Link retrieves and stores relevant data that is associated with the processor configuration such as the number of CPs and installed memory cards It enables you to select only those upgrade options that are deemed valid by the order process It supports upgrades only within the bounds of the currently installed hardware 8 4 2 Retrieval and activation After an order is pla...

Page 360: ...ess as shown in Figure 8 10 Figure 8 10 IBM z13s Perform Model Conversion window 2 The window provides several possible options If you select the Retrieve and apply option you are prompted to enter the order activation number to start the permanent upgrade as shown in Figure 8 11 Figure 8 11 Customer Initiated Upgrade Order Activation Number window 8 5 On Off Capacity on Demand On Off CoD enables ...

Page 361: ...lling window for each capacity type An activation period is one or more complete billing windows It represents the time from the first activation of resources in a record until the end of the billing window in which the last resource in a record is deactivated At the end of each billing window the tokens are decremented by the highest usage of each resource during the billing window If any resourc...

Page 362: ...ontains the resource descriptions MSUs number of speciality engines and tokens that describe the total capacity that can be consumed For CP capacity the token contains MSU days For speciality engines the token contains speciality engine days When resources on a prepaid offering are activated they must have enough capacity tokens to support the activation for an entire billing window which is 24 ho...

Page 363: ...turn the z13s server to its original capacity If the capacity is increased multiple times within a 24 hour period the charges apply to the highest amount of capacity active in the period If additional capacity is added from an already active record that contains capacity tokens a check is made to ensure that the resource in question has enough capacity to be active for an entire billing window 24 ...

Page 364: ...urations are generated based on the configuration of the individual z13s server After completing the prerequisites orders for the On Off CoD can be placed The order process is to use the CIU facility on Resource Link You can order temporary capacity for CPs ICFs zIIPs IFLs or SAPs Memory and channels are not supported on On Off CoD The amount of capacity is based on the amount of owned capacity fo...

Page 365: ...n Off CoD test automatically stops at the end of the 24 hour period In addition you can perform administrative testing During this test no additional capacity is added to the z13s server but you can test all of the procedures and automation for the management of the On Off CoD facility Figure 8 13 is an example of an On Off CoD order on the Resource Link web page Figure 8 13 On Off CoD order examp...

Page 366: ... 8 5 5 Termination A customer is contractually obliged to terminate the On Off CoD right to use feature when a transfer in asset ownership occurs A customer can also choose to terminate the On Off CoD right to use feature without transferring ownership Application of FC 9898 terminates the right to use the On Off CoD This feature cannot be ordered if a temporary session is already active Similarly...

Page 367: ...acity is allocated to LPARs and actually consumed Results from the STSI instruction reflect the current permanent and temporary CPs See Store system information instruction on page 351 for more details 8 5 6 IBM z OS capacity provisioning The z13s provisioning capability combined with CPM functions in z OS provides a flexible automated process to control the activation of On Off Capacity on Demand...

Page 368: ...ompletely consumed during an activation driven by the CPM the corresponding On Off CoD record is deactivated prematurely by the system This deactivation occurs even if the CPM has activated this record or parts of it Warning messages are issued when capacity tokens are getting close to being fully consumed The messages start being generated five days before a capacity token is fully consumed The f...

Page 369: ... the CPD but participating members must all be part of the same CPD The CPCC is the user interface component Administrators work through this interface to define domain configurations and provisioning policies but it is not needed during production The CPCC is installed on a Microsoft Windows workstation CPM operates in four modes enabling various levels of automation Manual mode Use this command ...

Page 370: ...ervice class periods must exceed the activation threshold for a specified duration before the work is considered to be suffering Deactivation threshold The PI of the service class periods must fall below the deactivation threshold for a specified duration before the work is considered to no longer be suffering Included service classes are eligible service class periods Excluded service classes are...

Page 371: ...tion in question might be running through several service class periods where the first period might be the important one The application might be defined as importance level 2 or 3 but it might depend on other work running with importance level 1 Therefore determining which workloads to control and which service class periods to specify is important 8 6 Capacity for Planned Event CPE is offered w...

Page 372: ...igured CP is maintained for the contracted configuration The processors that can be activated by CPE come from the available unassigned PUs on any installed CPC drawer CPE features can be added to an existing z13s server non disruptively A one time fee is applied for each individual CPE event depending on the contracted configuration and its resulting feature codes Only one CPE contract can be ord...

Page 373: ...U IFL year FC 6823 25 CBU IFL year FC 6824 Single CBU ICF year FC 6825 25 CBU ICF year FC 6826 Single CBU zAAP year FC 6827 25 CBU zAAP year FC 6828 Single CBU zIIP year FC 6829 25 CBU zIIP year FC 6830 Single CBU SAP year FC 6831 25 CBU SAP year FC 6832 CBU replenishment The CBU entitlement record FC 6818 contains an expiration date that is established at the time of order and depends upon the qu...

Page 374: ...tures if the capacity setting of the CPs is changed CBU can add CPs through LICCC only and the z13s server must have the correct number of CPC drawers installed to support the required upgrade CBU can change the MCI to a higher value than the base setting but it does not change the z13s model The CBU feature cannot decrease the capacity setting A CBU contract must be in place before the special co...

Page 375: ...ged or the number of logical processors can be concurrently increased by configuring reserved processors online The OS must have the capability to concurrently configure more processors online If necessary additional LPARs can be created to use the newly added capacity CBU deactivation To deactivate the CBU the additional resources must be released from the LPARs by the OSs In certain cases releas...

Page 376: ...02 with three CBU features Tip The CBU test activation is done the same way as the real activation by using the same SE Perform a Model Conversion window and selecting the Temporary upgrades option The HMC windows have been changed to avoid real CBU activations by setting the test activation as the default option Z01 Z02 Z03 Z04 Z05 Z06 Y01 Y02 Y03 Y04 Y05 Y06 X01 X02 X03 X04 X05 X06 W01 W02 W03 W...

Page 377: ...eptable Although Parallel Sysplex clustering technology is the best continuous availability solution for z OS environments nondisruptive upgrades within a single server can avoid system outages and are suitable to additional OS environments z13s servers support concurrent upgrades meaning that dynamically adding more capacity to the CPC is possible If OS images running on the upgraded CPC do not r...

Page 378: ...ons are supported by certain OSs z OS and z VM enabling nondisruptive I O upgrades However having dynamic I O reconfiguration on a stand alone CF server is not possible because there is no OS with this capability running on this server 8 8 4 Cryptographic adapters Crypto Express5S features can be added concurrently if all of the required infrastructure is in the configuration 8 8 5 Special feature...

Page 379: ... fully supports the concurrent upgrade functions and is the preferred way to request processor information The store CPU ID STIDP instruction STIDP is provided for purposes of compatibility with a previous version Store system information instruction The STSI instruction returns the MCI for the permanent configuration and the MCI for any temporary capacity This process is key to the functioning of...

Page 380: ...sts the possible output returned to the issuing program for an OS running as a guest under z VM Table 8 4 IBM z VM guest STIDP output for z13s servers Planning for nondisruptive upgrades Online permanent upgrades On Off CoD CBU and CPE can be used to concurrently upgrade a z13s server However certain situations require a disruptive task to enable the new capacity that was recently added to the CPC...

Page 381: ... and be able to configure processors online The total number of defined and reserved CPs cannot exceed the number of CPs supported by the OS IBM z OSV1R11 z OS V1R12 and z OS V1R13 with program temporary fixes PTFs supports up to 100 processors IBM z OS V2R1 also supports up to 141 way in non SMT mode and up to 128 way in SMT mode IBM z VM supports up to 64 processors Configure reserved storage to...

Page 382: ...ny sequence and any combination The offerings can be controlled manually through command interfaces on the HMC or programmatically through a number of APIs so that IBM applications ISV programs or customer written applications can control the use of the offerings Resource consumption and therefore financial exposure can be controlled by using capacity tokens in On Off CoD offering records The CPM ...

Page 383: ...tem to help reduce power consumption and meet cooling requirements The design of the memory on z13s servers is based on the fully redundant memory infrastructure redundant array of independent memory RAIM RAIM was first introduced with the z196 The z Systems servers are the only systems in the industry that offer this memory design RAS also provides digitally signed delivery and transmission of LI...

Page 384: ...rvers provides the ability to plan for nondisruptive memory upgrades by having the system pre plugged with dual inline memory modules DIMMs based on a target configuration Pre plugged memory is enabled when you place an order through LICCC Enhanced Driver Maintenance EDM One of the greatest contributors to downtime during planned outages is LIC driver updates performed in support of new features a...

Page 385: ...external network connections are redundant throughout all of the zBX racks TORs and Blade Centers Balanced Power Plan Ahead FC 3003 The Balanced Power Plan Ahead feature enables you to order the maximum number of Bulk Power Regulators BPRs on any server configuration This feature helps to ensure that your configuration will be in a balanced power environment if you intend to add books and I O draw...

Page 386: ...he configuration It is possible to perform the following tasks dynamically Add a logical partition LPAR up to 40 Add a logical channel subsystem LCSS up to 3 Add a subchannel set up to 3 Add a logical CP to an LPAR Add a cryptographic coprocessor Remove a cryptographic coprocessor Enable I O connections Swap processor types Add memory by using LICCC Add a physical processor In addition by addressi...

Page 387: ...onnection FICON feature enhancements have been implemented to avoid any unwanted side effects that have been detected on previous servers Server Time Protocol STP recovery enhancement When HCA3 O 12xIFB or HCA3 O Long Reach LR 1xIFB or PCIe based ICA SR coupling links are used an unambiguous going away signal is sent when the server on which the HCA3 is running is about to enter a failed check sto...

Page 388: ...maintain redundancy and provide assistance while concurrently replacing a part Through the implementation of the VLAN to the redundant internal Ethernet support network the VLAN makes the support network itself easier to handle and more flexible PCIe I O drawer The PCIe I O drawer is available for z13s servers It can be installed concurrently as can all PCIe I O drawer supported features 9 4 Enhan...

Page 389: ... upgrades Upgrades might require a system level or a functional element scheduled outage to activate the new LIC The following circumstances require a scheduled outage Specific complex code changes might dictate a disruptive driver upgrade You are alerted in advance so that you can plan for the following changes Design data or hardware initialization data fixes CFCC release level change OSA channe...

Page 390: ...e The alternate acts as the backup Once per day information is mirrored The SE servers have N 1 redundant power supplies For more information see 11 2 3 New Support Elements on page 393 Hardware Management Console HMC in an ensemble The serviceability function for the components of an ensemble is delivered through the traditional HMC SE constructs as for earlier z Systems servers From a serviceabi...

Page 391: ...ration and monitoring functions by using its connectivity to its internal SEs Apart from a zBX configuration with one chassis that are installed the zBX is configured to provide N 1 components All the components are designed to be replaced concurrently In addition zBX configuration upgrades can be performed concurrently The zBX has two ToR switches These switches provide N 1 connectivity for the p...

Page 392: ...Unified Resource Manager powers up the blade verifies that there is an entitlement for the blade and verifies that the blade can participate in an ensemble If these conditions are not met the Unified Resource Manager powers down the blade The blade is populated with the necessary microcode and firmware The appropriate hypervisor is loaded onto the blade The management scope is deployed according t...

Page 393: ...endent of the z Systems servers for management and control functions The zBX Model 004 internal SEs are used for firmware updates and other related service activities Therefore the distance restriction of 23 meters from the CPC to the zBX that existed with the previous models was removed Not only hardware and firmware provide RAS capabilities The operating system can also contribute to improving R...

Page 394: ...nage and monitor network resources service IP addresses Automate application control start stop restart move The virtual servers that are defined and managed in zBX use only virtual I O resources PowerHA can manage both physical and virtual I O resources virtual storage and virtual network interface cards PowerHA can be configured to perform automated service recovery for the applications that run...

Page 395: ...abnormal events It represents a first in a new generation of smart monitoring products with pattern based message analysis IBM zAware runs as a firmware virtual appliance in a z13s LPAR It is an integrated set of analytic applications that creates a model of normal system behavior that is based on prior system data It uses pattern recognition techniques to identify unexpected messages in current d...

Page 396: ...can be accessed only by the host by using a unique protocol In each Flash Express card data is stored in four solid state drives SSDs in a RAID configuration If an SSD fails the data is reconstructed dynamically The cards in a pair mirror each other over a pair of SAS cables in a RAID 10 configuration If either card fails the data is available on the other card Card replacement is concurrent and d...

Page 397: ...ottom of the frame or off the top of the frame and the option to have a high voltage DC power supply directly into the z13s server instead of the usual AC power supply For comprehensive physical planning information see z13s Installation Manual for Physical Planning GC28 6953 This chapter includes the following sections IBM z13s power and cooling IBM z13s physical specifications IBM zBX environmen...

Page 398: ...es as shown in Figure 10 1 Figure 10 1 IBM z13s cabling options Top Exit I O Like the zBC12 z13s servers support the Top Exit I O Cabling feature FC 7920 This feature routes all coupling links and all I O cables including a 1000BASE T Ethernet cable from I O drawers or PCIe I O drawers through two frame extensions out the top of the frame Top exit I O and top exit power Bottom cover Top exit I O a...

Page 399: ...on The system operates with two redundant power supplies Each of the power supplies has its individual power cords For redundancy the server must have two power feeds Each power feed is one power cord Power cords attach to either 3 phase 50 60 hertz Hz 250 or 450 volt V alternating current AC power or 380 to 520 V direct current DC power Depending on the configuration single phase power cords can ...

Page 400: ...re listed in Table 10 2 Table 10 2 IBM hold up times I O drawer PCIe I O drawer N10 FC 1043 N20 one CPC drawer FC 1044 N20 two CPC drawers FC 1045 kW a a Environmental conditions Warm room 40oC and altitude of less than 1500 feet with system blowers speed increased for the appropriate environmental zone 0 0 2 8b b Indicates that single phase input power is available There is no balanced power opti...

Page 401: ...vide Unit Emergency Power Off UEPO switch protection from accidental tripping The option consists of a clear cover that can be installed into the UEPO switch opening The cover itself and the additional hardware that is required to install the option are included in the z13s Shipping Group material Determine whether you want this optional UEPO cover installed 10 1 6 Cooling requirements z13s server...

Page 402: ...ding to equipment failure unless the computer room environment is adequately maintained to prevent it That is where operating your data center with the goal of reaching recommended specifications instead of just the required numbers will pay off for you Figure 10 3 depicts the environmental limits for previous and current generation of z Systems servers Figure 10 3 Recommended environmental condit...

Page 403: ...ons such as those generated by an earthquake 10 3 IBM zBX environmental components The following sections provide information about the environmental components in summary for zBX For a full description of the environmental components for the zBX see zBX Model 004 Installation Manual for Physical Planning GC27 2630 z13s single frame Maximum Model N10 Model N20 1 drawer Model N20 2 drawers W O IBFs...

Page 404: ...e zBX has its own power supplies and cords which are independent of the z13s server power Depending on the configuration of the zBX up to 16 customer supplied power feeds can be required A fully configured four rack zBX has 16 Power Distribution Units PDUs The zBX has these power specifications 50 60 Hz AC power Voltage 240 V Both single phase and three phase wiring PDUs and power cords The follow...

Page 405: ... consumption depends on the zBX configuration in terms of the number of BladeCenters and blades installed Input power in kilovolt ampere kVA is equal to the outgoing power in kilowatt kW Heat output expressed in kBTU per hour is derived by multiplying the table entries by a factor of 3 4 For 3 phase installations phase balancing is accomplished with the power cable connectors between the BladeCent...

Page 406: ... systems before it enters the room An easy to mount rear door design attaches to customer supplied water by using industry standard fittings and couplings Up to 50 000 BTUs or approximately 15 kW of heat can be removed from the air exiting the back of a zBX rack Number of blades Maximum utility power kW Heat output kBTU hour 7 7 3 24 82 14 12 1 41 14 28 21 7 73 78 42 31 3 106 42 56 40 9 139 06 70 ...

Page 407: ...ure enables you to route I O and power cabling through the top of the zBX rack The feature as shown in Figure 10 5 on page 380 will add 177 mm 7 in to the height and 9 75 kg 21 5 lbs to the weight of the zBX rack after it is installed It can be ordered with a new zBX but can also added later You need one feature per installed rack zBX rack Building chilled water Real Door Heat eXchanger Cold air l...

Page 408: ...t This section provides information about the elements of energy management in areas of tooling to help you understand the requirement for power and cooling monitoring and trending and reducing power consumption The hardware components in the z Systems CPC and the optional zBX are monitored and managed by the Energy Management component in the SE and HMC The GUI of the SE and the HMC provide views...

Page 409: ...s resourcelink The tool provides an estimate of the anticipated power consumption of a machine model given its configuration You enter the machine model memory size number of I O drawers and quantity of each type of I O feature card The tool outputs an estimate of the power requirements for that configuration If you have a registered machine in Resource Link you can access the power estimation too...

Page 410: ...wer and cooling requirements See 10 4 1 Power estimation tool on page 381 10 4 3 System Activity Display and Monitors Dashboard The System Activity Display presents you with the current power usage among other information as shown in Figure 10 7 Figure 10 7 Power usage on the System Activity Display Label Power Power watts Time Trending weeks months Power typically allocated to a server Over Alloc...

Page 411: ...lculate energy costs and savings view properties and settings for resources and view active energy related events Active Energy Manager does not directly connect to the z Systems servers but it attaches through a LAN connection to the HMC See 11 3 HMC and SE connectivity on page 398 Active Energy Manager discovers the HMC managing the server by using a discovery profile that specifies the HMC s IP...

Page 412: ...ite The z Unified Resource Manager offers multiple energy management tasks as part of the automate suite These tasks enable you to change system behaviors for optimized energy usage and energy saving Power cap Group power cap Various options are presented depending on the scope that is selected in the z Unified Resource Manager graphical user interface GUI Set Power Cap The power cap function can ...

Page 413: ...ing message that informs you that the z13s server went above the set cap level Being under the limit of the cap level is equal to the maximum potential power value see 10 4 2 Query maximum potential power on page 381 More information about energy management with zManager is available in IBM zEnterprise Unified Resource Manager SG24 7921 ...

Page 414: ...386 IBM z13s Technical Guide ...

Page 415: ...ne or more Support Elements SEs which then issue commands to their central processor complexes CPCs or IBM z BladeCenter Extension zBX This chapter addresses the HMC and SE in general and adds appropriate information for HMCs that manage ensembles with the IBM z Unified Resource Manager This chapter includes the following sections Introduction to the HMC and SE HMC and SE enhancements and changes ...

Page 416: ...he z Systems CPC The SE then issues those commands to their CPC and or zBX if any One HMC can control up to 100 SEs and one SE can be controlled by up to 32 HMCs Some functions are available only on the SE With Single Object Operation SOO these functions can be used from the HMC For more information see Single object operating on page 405 With Driver 27 Version 2 13 1 the IBM Dynamic Partition Man...

Page 417: ...n page 412 HMC Data Replication If you are using HMC Data Replication and you have an HMC at Driver 27 this HMC can only do Data Replication with another HMC at Driver 27 or Driver 22 Data Replication with driver levels older than Driver 22 are not supported on an HMC at Driver 27 If IBM z Unified Resource Manager is used data replication is not available Mirroring between the primary and alternat...

Page 418: ...ore data Hard disk drive HDD restore Support multi partitions for LPAR weights Ability to change LPAR weight related information at the same time New option for CPUMF Diagnostic Sampling Collection Can only be selected if Basic sampling is selected as shown in Figure 11 1 Figure 11 1 Diagnostic sampling authorization control Note When a UEFI admin password is set it must be available for the SSR t...

Page 419: ... Firmware updates are saved locally and installed during a scheduled MCL apply session Firmware updates are installed from the HMC and zBX SEs by using the same process and controls as used for z Systems servers zBX hardware related and firmware related failures are reported to IBM and the IBM support structure is engaged by using the HMC RSF This is the same process that is used for reporting z S...

Page 420: ...BM 1U standard tray containing a monitor and a keyboard The system unit and tray must be mounted in the rack in two adjacent 1U locations in the ergonomic zone between 21U and 26U in a standard 19 rack The customer must provide the rack Three C13 power receptacles are required Two for the system unit and one for the display and keyboard as shown in Figure 11 3 Figure 11 3 Rack mounted HMC HMC Syst...

Page 421: ...rt description of the new backup options that are available for HMC Version 2 13 1 Backup of primary SEs or HMCs to an FTP server With Driver 22 or later you can perform a backup of primary SEs or HMCs to an FTP server The FTP server must be supplied by the customer You can enable a secure FTP connection to your server Support Elements x2 Monitor Keyboardfor SEs System Control Hubs Front View Rear...

Page 422: ...ilable for backups The default that is shipped with the system is an 8 GB UFD SE and HMC backup files have been getting larger with newer z Systems The following features require more storage space for a backup IBM zEnterprise Unified Resource Manager zBX IBM zAware Several previous z Systems SE backups Backup of primary SEs The backup for the primary SE of a z13s or zBX Model 004 server can be ma...

Page 423: ...ata for different CPC machine types Figure 11 7 Backup Critical Data destinations of SEs For more information see the HMC and SE Version 2 13 1 console help system or go to the IBM Knowledge Center at the following link http www ibm com support knowledgecenter After you get to the IBM Knowledge Center click z Systems and then click z13s System Type UFD Media FTP Server z13 z13s No Yes zBX 004 No Y...

Page 424: ...duled Operation for SEs backup 11 2 5 SE driver support with the HMC driver The driver of the HMC and SE is equivalent to a specific HMC and SE version as shown in these examples Driver 27 is equivalent to Version 2 13 1 Driver 22 is equivalent to Version 2 13 0 Driver 86 is equivalent to Version 2 11 0 Driver 79 is equivalent to Version 2 10 2 An HMC with Version 2 13 1 can support different z Sy...

Page 425: ...91 are similar except the depth for FC 0092 is in round numbers that is 95 mm 3 75 in more FC 0094 M T 7914 FC 0094 is a rack mounted HMC It contains 16 GB of memory FC 0096 M T 2461 FC 0096 is the new rack mounted HMC It contains also 16 GB of memory z Systems family name Machine type SE driver SE version Ensemble node potential z13s 2965 27 2 13 1 Yesa a A CPC in DPM mode cannot be a member of a...

Page 426: ... to the J03 or Statements of Direction Removal of support for Classic Style User Interface on the Hardware Management Console and Support Element The IBM z13 and z13s servers will be the last z Systems servers to support Classic Style User Interface In the future user interface enhancements will be focused on the Tree Style User Interface Removal of support for the Hardware Management Console Comm...

Page 427: ...to the SEs Figure 11 10 shows the connectivity between HMCs and the SEs Figure 11 10 HMC and SE connectivity The LAN ports for the SEs installed in the CPC are shown in Figure 11 11 Figure 11 11 SE Physical connection Various methods are available for setting up the network It is your responsibility to plan and design the HMC and SE connectivity Select the method based on your connectivity and sec...

Page 428: ...ment applications such as IBM System Director Active Energy Manager AEM Security Configuration of network components such as routers or firewall rules is beyond the scope of this book Whenever the networks are interconnected security exposures can exist For more information about HMC security see z Systems Integrating the Hardware Management Console s Broadband Remote Support Facility into your En...

Page 429: ... Hardware prerequisite changes The following HMC changes are important for z13s servers IBM does not provide Ethernet switches with the system RSF is broadband only Ethernet switches Ethernet switches for HMC and SE connectivity are customer provided Existing supported switches can still be used Ethernet switches hubs typically have these characteristics Sixteen auto negotiation ports 100 1000 Mbp...

Page 430: ...gurations Statically assigned IPv4 or statically assigned IPv6 addresses Dynamic Host Configuration Protocol DHCP assigned IPv4 or DHCP assigned IPv6 addressees Auto configured IPv6 Link local is assigned to every network interface Router advertised which is broadcast from the router can be combined with a Media Access Control MAC address to create a unique address Privacy extensions can be enable...

Page 431: ...rovided Problem reporting and repair data Microcode Change Level MCL delivery Hardware inventory data which is also known as vital product data VPD On demand enablement 11 4 1 Security characteristics The following security characteristics are in effect RSF requests always are initiated from the HMC to IBM An inbound connection is never initiated from the IBM Service Support System All data that i...

Page 432: ...ollowing host names and IP addresses are used and your network infrastructure must allow the HMC to have access to them Host names www 945 ibm com on port 443 esupport ibm com on port 443 IP addresses IPv4 IPv6 or both can be used IPv4 129 42 26 224 443 129 42 34 224 443 129 42 42 224 443 129 42 50 224 443 129 42 56 189 443 enhanced 129 42 58 189 443 enhanced 129 42 60 189 443 enhanced 129 42 54 1...

Page 433: ...you get to the IBM Knowledge Center click z Systems and then click z13s Single object operating It is not necessary to be physically close to a SE to use it The HMC can be used to access the SE remotely by using the SOO task The interface that is displayed is the same as the one on the SE For more information see the HMC and SE Version 2 13 1 console help system or go to the IBM Knowledge Center a...

Page 434: ... are set to LPARs at the scheduled time Channel paths can be dynamically configured on and off as needed for each partition from an HMC The Change LPAR Controls task for z13s servers can export the Change LPAR Controls table data to a comma separated value csv formatted file This support is available to a user when connected to the HMC remotely by a web browser Partition capping values can be sche...

Page 435: ...specified as an absolute number of processors to cap the group s activity to The value is specified to hundredths of a processor for example 4 56 processors worth of capacity The value is not tied to the LICCC Any value from 0 01 to 255 00 can be specified This makes the profiles more portable and means you will not have issues in the future when profiles are migrated to new machines Although the ...

Page 436: ...icate Management task Manage Trusted Signing Certificates is used to add trusted signing certificates For example if the certificate that is associated with the 3270 server on the IBM host is signed and issued by a corporate certificate it must be imported as shown in Figure 11 14 Figure 11 14 Manage Trusted Signing Certificates If the connection between the console and the IBM host can be trusted...

Page 437: ...duled fix application session or to schedule one earlier if the risk assessment warrants Microcode terms The microcode has these characteristics The driver contains EC streams Each EC stream covers the code for a specific component of z13s servers It has a specific name and an ascending number The EC stream name and a specific number are one MCL MCLs from the same EC stream must be installed in se...

Page 438: ...410 IBM z13s Technical Guide Figure 11 17 shows how the driver bundle EC stream MCL and MCFs interact with each other Figure 11 17 Microcode terms and interaction ...

Page 439: ...group on the same date You can install an MCL to a specific target bundle level The System Information window is enhanced to show a summary bundle level for the activated level as shown in Figure 11 18 Figure 11 18 System Information Bundle level OSC Concurrent Patch Concurrent patch for OSC channels is now supported 11 5 5 Monitoring This section addresses monitoring considerations ...

Page 440: ...shboard task supersedes the System Activity Display SAD In the z13s server the Monitors Dashboard task in the Monitor task group provides a tree based view of resources Multiple graphical views exist for displaying data including history charts The Open Activity task which is known as SAD monitors processor and channel usage It produces data that includes power monitoring information power consump...

Page 441: ...sk Figure 11 20 Monitors Dashboard task Starting with Driver 27 you can display the activity for an LPAR by processor type as shown in Figure 11 21 Figure 11 21 Display the activity for an LPAR by processor type The Monitors Dashboard is enhanced to show SMT usage as shown in Figure 11 22 Figure 11 22 Display the SMT usage ...

Page 442: ...o number Adjunct Processor Number for this PCHID is also shown in the table It provides information about the usage rate on a system wide basis not per LPAR as shown in Figure 11 23 Figure 11 23 Monitors Dashboard Crypto function integration For Flash Express a new window is added as shown in Figure 11 24 Figure 11 24 Monitors Dashboard Flash Express function integration ...

Page 443: ...ity on demand CoD upgrades are performed by using the SE Perform a Model Conversion task Use the task to retrieve and activate a permanent upgrade and to retrieve install activate and deactivate a temporary upgrade The task shows a list of all installed or staged LIC configuration code LICCC records to help you manage them It also shows a history of recorded activities The HMC for z13s servers has...

Page 444: ...Ps Integrated Facility for Linux IFL and IBM z Integrated Information Processors zIIPs and zBX HWMs providing entitlement controls for each blade type It is also used as LICCC support for the following features IBM zAware Enablement max connections Base proprietary service Expiration date New features Yet to be announced or developed 11 5 8 Server Time Protocol support With the STP functions the r...

Page 445: ...n the Customize User Controls window that allows the enablement of the View System Sysplex Time option This option provides view only access to the STP windows for each individual user role The Initialize Time window shows the current values of leap second offset time zone and date and time if the CTN time was set before Set Date and Time is modified so that Use External Time Source ETS is the fir...

Page 446: ... Figure 11 26 the NTP option is the recommended option if an NTP server is available If an NTP server is not available for this HMC any defined CPC SE can be selected after you select Selected CPCs Figure 11 26 Customize Console Date and Time ...

Page 447: ...rms its SE and HMC 2 The PTS SE recognizes the check stop pending condition and the PTS SE STP code is called 3 The PTS SE sends an ECAR request through the HMC to the BTS SE 4 The Backup Time Server BTS SE communicates with the BTS to start the takeover ECAR support is faster than the original CAR support for these reasons The console path goes from a 2 way path to a one way path There is almost ...

Page 448: ...ients The HMC can act as an NTP server With this support the z13s server can get time from the HMC without accessing a LAN other than the HMC SE network When the HMC is used as an NTP server it can be configured to get the NTP source from the Internet For this type of configuration a LAN that is separate from the HMC SE LAN can be used HMC NTP broadband authentication support HMC NTP authenticatio...

Page 449: ... keep this key secret Messages encrypted with a secret key can be decrypted only with the same secret key Symmetric key authentication supports network address translation NAT Symmetric key autokey NTP V4 authentication This autokey uses public key cryptography as described in RFC 5906 which was made available in NTP Version 4 You can generate keys for the HMC NTP by clicking Generate Local Host K...

Page 450: ...ion for zBX components 11 5 10 Security and user ID management This section addresses security and user ID management considerations HMC and SE security audit improvements With the Audit and Log Management task audit reports can be generated viewed saved and offloaded The Customize Scheduled Operations task allows you to schedule audit report generation saving and offloading The Monitor System Eve...

Page 451: ...y user IDs and view only access for HMC and SE With HMC and SE user ID support users can be created that have view only access to selected tasks Support for view only user IDs is available for the following purposes Hardware messages Operating system messages Customize or delete activation profiles Advanced facilities Configure on and off HMC and SE secure FTP support You can use a secure FTP conn...

Page 452: ...rted to an UFD for later viewing The following five views are available PCHID Control Unit View shows PCHIDs channel subsystems CSS channel path identifiers CHPIDs and their control units PCHID Partition View shows PCHIDS CSS CHPIDs and the partitions in which they exist Control Unit View shows the control units their PCHIDs and their link addresses in each CSS Link Load View shows the Link addres...

Page 453: ... Standards PKCS 11 EP11 coprocessor or an accelerator When EP11 mode is selected unique Enterprise PKCS 11 firmware is loaded into the cryptographic coprocessor It is separate from the CCA firmware that is loaded when a CCA coprocessor is selected CCA firmware and PKCS 11 firmware cannot coexist at the same time in a card The Trusted Key Entry TKE Workstation with smart card reader feature is requ...

Page 454: ...fying the digital signature It helps ensure that no malware can be installed on z Systems products during firmware updates It enables the z13s Central Processor Assist for Cryptographic Function CPACF functions to comply with Federal Information Processing Standard FIPS 140 2 Level 1 for Cryptographic Licensed Internal Code LIC changes The enhancement follows the z Systems focus on security for th...

Page 455: ...nts 427 Setting up is a disruptive action The selection of DPM mode of operation is done by using a function called Enable Dynamic Partition Manager under the SE CPC Configuration menu as shown in Figure 11 31 Figure 11 31 Enabling Dynamic Partition Manager ...

Page 456: ...a zBX Model 004 The ensemble provides an integrated way to manage virtual server resources and the workloads that can be deployed on those resources The zEnterprise is a workload optimized technology system that delivers a multiple platform integrated hardware system This system spans z Systems System p and System x blade server technologies Management of the ensemble is provided by the IBM zEnter...

Page 457: ... management Provides tasks for creating an ensemble and controlling membership of the ensemble Storage management Provides a common user interface for allocation and deallocation of physical and virtual storage resources for an ensemble Virtual server management Provides lifecycle management to create delete activate deactivate and modify the definitions of virtual servers Virtual network manageme...

Page 458: ...zBX EAM enhancements EAM availability enhancements are based on Workload Resource Group WRG definitions A WRG is a grouping mechanism and management view of the virtual servers that support a business application The availability definitions are created at the HMC and include these functions Creation of element groups an element is a virtual server that is associated to a specific workload Element...

Page 459: ... management resource inventory provisioning monitoring automation workload based optimization and so on The Web Services API consists of two major components that are accessed by customer applications through Internet Protocol network connections with the HMC For more information about the API and the Unified Resource Manager see z Systems Hardware Management Console Web Services API Version 2 13 ...

Page 460: ...ent tasks on CPCs that are not part of the ensemble These tasks include Load Activate and so on The ensemble specific managed objects include the following objects Ensemble Members Blades BladeCenters Hypervisors Storage resources Virtual servers Workloads When another HMC accesses an ensemble node s CPC the HMC can perform the same tasks as though the CPC were not a part of an ensemble A few of t...

Page 461: ...an ensemble which can consist of a maximum of eight CPCs and up to eight zBX Model 004 systems The HMC that ran the Create Ensemble wizard becomes the primary HMC An alternate HMC is elected and paired with the primary The Primary HMC Version 2 13 1 or later and Alternate HMC Version 2 13 1 or later are displayed on the HMC banner When the ensemble is deleted the titles change back to the default ...

Page 462: ...this configuration all management actions are possible to any of the reachable systems while logged in to a session on any of the HMCs subject to access control With the Unified Resource Manager this paradigm has changed One ensemble is managed by one primary and alternate HMC pair Multiple ensembles require an equal number of primary and alternate HMC pairs to manage them If a z13s server a zEnte...

Page 463: ...omponents of the ensemble by using the System Control Hub SCH in the z13s server the BPH in the zEC12 and the INMN switch in the zBX Model 004 The OSA Express5S or OSA Express 4S 10 GbE ports CHPID type OSX in the z13s server are plugged with customer provided 10 GbE cables to the IEDN zBX switch These cables are either short reach SR or long reach LR depending on the OSA feature SCH SE BladeCente...

Page 464: ...436 IBM z13s Technical Guide ...

Page 465: ...ns for IBM z13s This chapter includes the following sections Performance information LSPR workload suite Fundamental components of workload capacity performance Relative nest intensity LSPR workload categories based on relative nest intensity Relating production workloads to LSPR workloads Workload performance variation 12 ...

Page 466: ...uration than a zBC12 6 way However variations on the observed performance increase depend on the workload type Consult the LSPR when you consider performance on the zBC12 The performance ratings across the individual LSPR workloads are likely to have a large spread More performance variation of individual logical partitions LPARs exists because the fluctuating resource requirements of other partit...

Page 467: ... and the previous generation processor families These ratios are based on measurements and projections that use standard IBM benchmarks in a controlled environment The throughput that any user experiences can vary depending on the amount of multiprogramming in the user s job stream the I O configuration and the workload processed Therefore no assurance can be given that an individual user can achi...

Page 468: ...ut of order execution Branch prediction As workloads are moved between microprocessors with various designs performance varies However when on a processor this component tends to be similar across all models of that processor 12 3 3 Memory hierarchy and memory nest The memory hierarchy of a processor generally refers to the caches data buses and memory arrays that stage the instructions and data t...

Page 469: ...aries because the average time to retrieve instructions and data from within the memory hierarchy varies Additionally when on a processor this component continues to vary This variation is because the location of a workload s instructions and data within the memory hierarchy is affected by many factors that include but are not limited to these factors Locality of reference I O rate Competition fro...

Page 470: ...e done to affect most of these factors An application type is whatever is necessary to do the job The data reference pattern and processor usage tend to be inherent to the nature of the application The LPAR configuration and application mix are mostly a function of what must be supported on a system The I O rate can be influenced somewhat through buffer pool tuning However one factor software conf...

Page 471: ...al factor in determining workload performance Other more traditional factors such as application type or I O rate have RNI tendencies However it is the net RNI of the workload that is the underlying factor in determining the workload s capacity performance The LSPR now runs various combinations of former workload primitives such as CICS DB2 IMS OSAM VSAM WebSphere COBOL and utilities to produce ca...

Page 472: ...PR provided performance ratios for individual workloads and for the default mixed workload This default workload was composed of equal amounts of four of the previous workloads OLTP T OLTP W WASDB and CB L Guidance in converting the previous LSPR categories to the new ones is given in Figure 12 5 The IBM Processor Capacity Reference for z Systems zPCR tool1 has been changed to support the new z OS...

Page 473: ...rom CPU MF the L1MP percentage of data and instruction references that miss the L1 cache a workload can be classified as LOW AVERAGE or HIGH RNI This classification and resulting hit are automated in the zPCR tool It is preferable to use zPCR for capacity sizing 12 7 Workload performance variation Performance variability from application to application is expected This variation is similar to that...

Page 474: ...ip Improved out of order OOO execution design Improved pipeline balance with up to six instructions that can be decoded per cycle and up to 10 instructions operations that can be initiated to run per clock cycle Simultaneous multithreading Single instruction multiple data SIMD unit and 139 new instructions for vector operations Enhanced branch prediction latency and instruction fetch throughput Im...

Page 475: ... pool GBP Performance improvements for coupling facility cache structures to avoid flooding the coupling facility cache with changed data and avoid excessive delays and backlogs for cast out processing Performance throughput enhancements for parallel cache castout processing by extending the number of record code check RCC cursors beyond 512 Coupling facility CF storage class and castout class con...

Page 476: ...448 IBM z13s Technical Guide ...

Page 477: ...ce Container Infrastructure zACI framework available on z13s and z13 Driver Level 27 servers It briefly describes the reason why IBM has created the framework and provides a description about how the zACI environment is intended to be used This appendix includes the following sections What is zACI Why use zACI IBM z Systems servers and zACI A ...

Page 478: ...olution that uses them The zACI framework reduces the amount of work a team must do to create an appliance and enforces a common set of behaviors for operations that all appliances adhere to The zACI framework provides a set of utilities to implement common functions that all appliances need such as first failure data capture FFDC network setup appliance configuration and so on The framework enabl...

Page 479: ...M zAware logical partition LPAR type is not supported on z13s or z13 at Driver level 27 servers The zACI LPAR type is used instead Statement of Direction IBM plans to make available the z VSE Network Appliance based on zACI in June 2016 All statements regarding IBM plans directions and intent are subject to change or withdrawal without notice Any reliance on these statements of general direction i...

Page 480: ...BM zAware LPARs will be automatically converted during Enhanced Driver Maintenance from Driver 22 to Driver 27 No reconfiguration of IBM zAware is required A new icon will identify the zACI LPARs in the HMC user interface UI as shown in Figure A 3 right Classic UI left tree style UI Figure A 3 zACI icon in the HMC interface ...

Page 481: ...toring capabilities IBM zAware helps you pinpoint and resolve potential problems quickly enough to minimize impacts to your business This appendix includes the following sections Troubleshooting in complex IT environments Introducing IBM zAware Understanding IBM zAware technology IBM zAware prerequisites Configuring and using IBM zAware virtual appliance For more information about IBM zAware see E...

Page 482: ...workloads provisioning balancing deploying disaster recovery DR testing and so on Using a combination of existing system management tools helps to diagnose problems However they cannot quickly identify messages that precede system problems and cannot detect every possible combination of change and failure When using these tools you might need to look through message logs to understand the underlyi...

Page 483: ...oup multiple systems data into a combined model by workload one for all web servers one for all databases and so on by solution for example one model for your cloud and by VM host Heat map display that provides a consolidated aggregated and higher level view with the ability to drill down to detail views Improved usability and GUI functional enhancements that address customer requirements Enhanced...

Page 484: ...vironment B 2 2 Value of IBM zAware Early detection and focused diagnosis can help improve the time that is needed to recover from complex z OS problems These problems can be cross sysplex across a set of z Systems servers and beyond central processing complex CPC boundaries IBM zAware is enhanced to identify unusual system behavior of Linux on z Systems images running natively or as a guest on z ...

Page 485: ... faster problem resolution B 2 3 IBM z OS Solutions to improve problem diagnostic procedures Table B 1 shows how IBM zAware can be combined with other monitoring and problem determination tools Table B 1 Using IBM zAware with other monitoring and problem determination tools Ineffective time spent in problem determination and trial and error More precise and early diagnosis can shorten impact time ...

Page 486: ... connected z OS and Linux on z Systems instances z OS RTDa Real time diagnostics of specific z OS system issues Yes Yes Rules based after an incident Linux on z Systems health checkerb Checks configurations Programmatic applies to IBM and ISV tools Yes Rules based to screen for conditions IBM zAware Pattern based message analysis Self learning Aids in diagnosing complex z OS problems including cro...

Page 487: ...1 requires zEC12 or zBC12 systems with a priced feature code Needs processor memory disk storage and network resources to be assigned to the LPAR that it runs Is updated like all other firmware with a separate engineering change stream Is loaded from the Support Element SE hard disk by activating an Image Profile configured with a mode of IBM zAware Employs out of band monitoring with minimal effe...

Page 488: ...e General tab of the image profile Figure B 4 HMC Image Profile for an IBM zAware LPAR Figure B 5 shows the tab of the image profile on the HMC The setup information is configured on the tab of the image profile This setup must be performed for the initial activation of the image profile Figure B 5 HMC Image Profile for an IBM zAware LPAR ...

Page 489: ...ng system IBM zAware analyzes massive amounts of OPERLOG messages including all z OS console messages and ISV and application generated messages to build sysplex and LPAR detailed views in the IBM zAware graphical user interface GUI Linux on z Systems images must be configured so that the syslog daemon sends data to IBM zAware IBM zAware can create model groups based on similar operational charact...

Page 490: ...ore to be marked as interesting or rare An unstable system requires a larger interval score to be marked as interesting or rare For each interval IBM zAware provides details of all of the unique and unusual message IDs within the interval This data includes how many how rare and how much the messages contributed to the intervals score anomaly score interval contribution score rarity score and appe...

Page 491: ...oads that run on z OS monitored clients This training period applies for all monitored clients Different training periods cannot be defined for each client B 3 2 Priming IBM zAware Instead of waiting for the IBM zAware server to collect data over the course of the training period you can prime the server You do so by transferring prior data for monitored clients and requesting that the server buil...

Page 492: ...t often includes a corrective action for the issue that is highlighted by the message B 3 5 IBM zAware complements your existing tools Compared to existing tools IBM zAware works with relatively little customization It does not depend on other solutions or manual coding of rules and is always enabled to watch your system The XML output that is created by IBM zAware is consumed by existing system m...

Page 493: ...ored There can be multiple IBM zAware host partitions on one z13s z13 zEC12 or zBC12 server but only one IBM zAware FC0011 feature no additional charge for multiple host partitions IBM zAware monitored client Represents the z OS partition that sends OPERLOG files for processing to an IBM zAware partition Multiple z OS partitions monitored clients can be on the server The clients can also include L...

Page 494: ...he z13s z13 zEC12 or zBC12 server that hosts the IBM zAware partition You do not need to order IBM zAware connections for client systems The number of IBM zAware connections to be ordered can be calculated by completing the following steps 1 Determine which systems have z OS images to be monitored by IBM zAware including the z13s z13 zEC12 or zBC12 server where the IBM zAware LPAR resides 2 Count ...

Page 495: ...etween a partial engine to two engines depending on the size of the configuration Memory Minimum 4 GB initial memory for the first six z OS clients 256 MB required for each additional z OS client above the first six z OS clients 256 MB required for each additional LINUX client Flash Express is not supported DASDs 500 GB persistent DASD storage Only extended count key data ECKD format is supported ...

Page 496: ...ation of the IBM zAware environment b Plan the LPAR characteristics of the zACI LAPR to host IBM zAware firmware c Plan the network connections that are required for the partition and each monitored client d Plan the security requirements for the IBM zAware server its monitored clients and users of the IBM zAware GUI e Plan for using the IBM zAware GUI 2 Phase 2 Configuring the zACI partition for ...

Page 497: ...figure the z OS system logger to send data to the IBM zAware virtual appliance server i Configure the Linux SYSLOG to send data to the IBM zAware virtual appliance server j Prime the IBM zAware server with prior data from monitored clients k Build a model of normal system behavior for each monitored client 1 The IBM zAware server uses these models for analysis l Optional Use the IBM zAware ignore ...

Page 498: ...470 IBM z13s Technical Guide ...

Page 499: ...scribes all channel attributes the required cable types the maximum unrepeated distance and the bit rate for z13s servers This appendix includes the following sections C 1 Channel options supported on z13s servers on page 472 C 2 Maximum unrepeated distance for FICON SX features on page 473 C ...

Page 500: ...X 0418 4 8 or 16 SM 9 µm 10 km 6 2 miles New build FICON Express16S SX 0419 4 8 or 16 OM2 OM3 See Table C 2 on page 473 New build FICON Express8S 10KM LX 0409 2 4 or 8 SM 9 µm 10 km 6 2 miles New build FICON Express8 10KM LX 3325 Carry forward FICON Express8S SX 0410 2 4 or 8 OM1 OM2 OM3 See Table C 2 on page 473 New build FICON Express8 SX 3326 Carry forward Open Systems Adapter OSA OSA Express5S...

Page 501: ... build a The link data rate does not represent the actual performance of the link The actual performance depends on many factors that include latency through the adapters cable lengths and the type of workload b Where applicable the minimum fiber bandwidth distance in MHz km for multi mode fiber optic links is included in parentheses Channel feature Feature codes Bit ratea in Gbps or stated Cable ...

Page 502: ...474 IBM z13s Technical Guide ...

Page 503: ...ty implemented on IBM z Systems servers as Shared Memory Communications Remote Direct Memory Access SMC R and the new Shared Memory Communications Direct Memory Access SMC D of the IBM z13 and z13s servers This appendix includes the following sections Shared Memory Communications overview Shared Memory Communication over RDMA Shared Memory Communications Direct Memory Access D ...

Page 504: ...ns with a goal to expand exploitation to more operating systems and possibly appliances and accelerators Single root I O virtualization SR IOV technology provides the capability to share the 10GbE RoCE Express adapter between logical partitions LPARs D 2 1 RDMA technology overview RoCE is part of the InfiniBand Architecture Specification that provides InfiniBand transport over Ethernet fabrics It ...

Page 505: ...A SMC R is a protocol that allows TCP socket applications to transparently use RDMA SMC R is a hybrid solution as shown in Figure D 2 on page 478 It uses an existing TCP connection to establish the SMC R connection A TCP option SMCR controls switching from TCP to out of band SMC R The SMC R information is exchanged within the TCP data stream Socket application data is exchanged through RDMA write ...

Page 506: ...t immediately D 2 3 Single Root I O Virtualization SR IOV is a technology that is designed to provide the capability to share the adapter between up to 31 LPARs SR IOV is also designed to provide isolation of virtual functions within the Peripheral Component Interconnect Express PCIe 10GbE RoCE Express adapter For example one LPAR cannot cause errors visible to other virtual functions or other LPA...

Page 507: ... for reference D 2 5 10GbE RoCE Express feature The 10GbE RoCE Express feature is designed to help reduce the consumption of CPU resources for applications that use the TCP IP stack such as WebSphere accessing a DB2 database Using the 10GbE RoCE Express feature also helps to reduce network latency with memory to memory transfers that use SMC R in z OS V2 1 or later It is transparent to application...

Page 508: ...erminated with an LC Duplex connector Both point to point connection with another 10GbE RoCE Express adapter and switched connection with an enterprise class 10 GbE switch are supported Figure D 4 10GbE RoCE Express Although SMC R can be used with direct RoCE Express to RoCE Express connectivity without any switch this type of direct physical connectivity forms a single physical point to point con...

Page 509: ... with an LC Duplex connector supports 300 m 984 25 ft OM2 50 micron multimode fiber optic cable rated at 500 MHz km terminated with an LC Duplex connector support 82 m 269 ft OM1 62 5 micron multimode fiber optic cable rated at 200 MHz km terminated with an LC Duplex connector support 33 m 108 2 ft D 2 6 10GbE RoCE Express configuration example Figure D 5 illustrates a sample configuration that al...

Page 510: ...Figure D 6 RNIC and OSD interaction The following notes refer to Figure D 6 The z OS system administrator only must configure and manage the OSD interface The Communications Server transparently splits and converges network traffic to and from the converged interface Only OSD connectivity must be configured With SMC R the RNIC interface is dynamically and transparently added and configured Note Th...

Page 511: ...n ID Virtual Function ID is defined when PCIe hardware is shared between LPARs Virtual Function ID has a decimal Virtual Function Identifier VF in the range 1 n where n is the maximum number of partitions the PCIe feature supports For example the RoCE feature supports up to 31 partitions and a zEDC Express feature supports up to 15 Physical network ID As one parameter for the FUNCTION statement th...

Page 512: ...me PCHID must have the same PNETID for each port 10GbE RoCE Express Functions for LPAR 08 are reconfigurable to LP 09 with access to two networks 10GbE RoCE Express Functions for LPAR 12 are reconfigurable to LP 06 with access to two networks Physical RoCE Express adapters on PCHID 100 and 12C are shared between LPARs 08 and 12 D 2 8 Software exploitation of SMC R SMC R can be implemented on the R...

Page 513: ...and zBC12 servers Dual port 10 GbE adapter Maximum of 16 RoCE Express features per CPC HCD and input output configuration data set IOCDS PCIe FID VF sharing and RoCE configuration with PNet ID Optional Standard 10 GbE switch CEE enabled switch is not required Required queued direct input output QDIO Mode OSA connectivity between z OS LPARs as shown in Figure D 5 on page 481 Adapter must be dedicat...

Page 514: ...D IOCDS configuration changes are required Existing z OS RoCE users might be required to make a TCP IP configuration change that is existing TCP IP profile PFIDs might be compatible with shared RoCE Changes are required for existing RoCE users for the following cases z OS users who use multiple TCP IP stacks and both stacks currently use the same RoCE feature single z OS image sharing a physical c...

Page 515: ...ow directly between the two servers without having to traverse the Sysplex Distributor node for every inbound packet which is the current model with TCP IP In the new model only connection establishment flows must go through the Sysplex Distributor node Sysplex Distributor before RoCE Figure D 9 shows a traditional Sysplex Distributor Figure D 9 Sysplex Distributor before RoCE The traditional Sysp...

Page 516: ...Add the PNet ID for the current OSD Define PFIDs for RoCE with the same PNet ID Specify the GLOBALCONFIG SMCR parameter TCP IP Profile Must specify at least one PCIe Function ID PFID A PFID represents a specific RNIC adapter A maximum of 16 PFID values can be coded Up to eight TCP IP stacks can share a RoCE PCHID RoCE feature in a specific LPAR each stack must define a unique FID value Start the I...

Page 517: ...agement purposes From an operations standpoint SMC D is similar to SMC R The objective is to provide consistent operations and management tasks for both SMC D and SMC R SMC D uses a new virtual PCI adapter called Internal Shared Memory ISM The ISM Interfaces are associated with IP interfaces for example HiperSockets or OSA ISM interfaces do not exist without an IP interface ISM interfaces are not ...

Page 518: ... over Internal Shared Memory ISM is a virtual channel similar to IQD for HiperSockets A virtual adapter is created in each OS Using the SMC protocol the memory is logically shared The network is firmware provided A new device is required to manage that virtual function SMC is based on a TCP IP connection and preserves the entire network infrastructure SMC D is a protocol that allows TCP socket app...

Page 519: ...he hybrid model switches to ISM SMC D dynamically The TCP connection remains active idle and is used to control the SMC D connection The hybrid model preserves the following critical operational and network management TCP IP features Minimal or zero IP topology changes Compatibility with TCP connection level load balancers OSA ISM TCP IP Interface Sockets Middleware Application z OS LPAR B SMC D O...

Page 520: ...irtualization Virtual Functions with high scalability It supports up to 32 ISM VCHIDs per CPC z13 or z13s servers each VCHID represents a unique internal shared memory network each with a unique Physical Network ID Each VCHID supports up to 255 VFs per VCHID the maximum is 8k VFs per z13 or z13s CPC which provide significant scalability Each ISM VCHID represents a unique and isolated internal netw...

Page 521: ... D 14 shows the basic concept of vPCI adapters Figure D 14 Concept of vPCI adapter implementation Note Basic z VM support is available Generic zPCI pass through support starting from z VM 6 3 The use of the zPCI architecture remains basically unchanged ...

Page 522: ... PNet X ISM ISM ISM ISM IP Interface A 1 IP Interface A 2 VCHID X ISM Network X becomes a logical extension of the external Layer 2 Network X Ethernet LAN X Each host has an IP network interface connection to a Layer 2 Network with a defined PNet ID X associated with IP subnet A PNet X could be an external LAN or an internal HS Network IQD VCHID with PNet X Both hosts have access to the same IP su...

Page 523: ... TCP IP stacks per z OS LPAR SMC D can use up to 8 FIDs or VFs one per TCP IP stack IBM supports up to 32 ISM PNet IDs per CEC Each TCP IP stack can have access to a PNet ID that consumes up to 32 FIDs one VF per PNet ID D 3 6 Hardware configuration definitions Complete these steps to create HCDs 1 Configure ISM vPCI Functions HCD HCM 2 Define PNet IDs OSA HiperSockets IQD and ISM in HCD HCM 3 Act...

Page 524: ...nd IPCS formatted dumps ISM Functions must be associated with another Channel IQD a single IQD HiperSockets channel OSD channels D 3 7 Sample IOCP FUNCTION statements Example D 2 shows IOCP FUNCTION statements that describe the configuration that defines ISM adapters shared between LPARs on the same CPC as shown in Figure D 17 Figure D 17 ISM adapters shared between LPARs Example D 2 IOCP FUNCTION...

Page 525: ... PART LPARn LPAR6 LPARn PNETID PNETA TYPE ISM FUNCTION FID 1022 VCHID 7E2 VF 1 PART LPAR1 LPAR1 LPAR2 PNETID PNETB TYPE ISM FUNCTION FID 1023 VCHID 7E2 VF 2 PART LPAR2 LPAR1 LPAR2 PNETID PNETB TYPE ISM FUNCTION FID 1024 VCHID 7E2 VF 3 PART LPARn LPAR1 LPARn PNETID PNETB TYPE ISM Note On the IOCDS statement the VCHID is defined as 7E1 In Figure D 17 on page 496 the ISM network PNET 1 is referenced ...

Page 526: ...ranslation architecture Only minor changes required for z VM guests An OS can be enabled for both SMC R and SMC D SMC D is used when both peers are within the same CPC and ISM PNet and VLAN After the ISM HCD configuration is complete SMC D can be enabled in z OS with a single TCP IP parameter GLOBALCONFIG SMCD ISM FIDs must be associated with an IP network The association is accomplished by matchi...

Page 527: ... are discovered automatically based on matching PNETID associated with the OSD or HiperSockets Figure D 19 shows an extract from z OS Communications Server IP Configuration Reference Figure D 19 SMCD parameter in GLOBALCONFIG Subsystem FMID APAR IOS HBB77A0 OA47913 CommServer SNA VTAM HVT6220 OA48411 CommServer IP HIP6220 PI45028 HCD HCS77A0 HCS7790 HCS7780 HCS7770 HCS7760 HCS7750 OA46010 IOCP HIO...

Page 528: ...al servers that can communicate over the same ISM VCHID is 255 Each ISM VCHID in IOCDS HCD represents a unique isolated internal network each having a unique Physical Network ID PNet IDs are configured in HCD IOCDS ISM VCHIDs support VLANs can be subdivided into VLANs ISM provides a GID internally generated to correspond with each ISM FID MACs VMACs MTU physical ports and Frame size are all N A IS...

Page 529: ...oduction to IBM Dynamic Partition Manager DPM on z Systems It provides a description about how Dynamic Partition Manager environment can be initialized and managed This appendix includes the following sections What is IBM Dynamic Partition Manager Why use DPM IBM z Systems servers and DPM Setting up the DPM environment E ...

Page 530: ... hypervisor for z Systems servers DPM uses the existing PR SM hypervisor infrastructure and on top of it provides an intelligent interface that allows customers to define use and operate the platform virtualization with no z Systems experience or skills E 2 Why use DPM DPM mode is targeted at customers distributed market with no specific z Systems skills or knowledge of z VM who want to implement ...

Page 531: ...e system It fully virtualizes the system resources and does not require additional software to run PR SM allows the defining and managing of subsets of the z Systems resources in logical partitions LPARs The LPAR definitions include a number of logical processing units LPUs memory and I O resources LPARs can be added modified activated or deactivated in z Systems platforms by using the traditional...

Page 532: ...p the DPM environment The DPM is in fact a z Systems mode of operation Enabling DPM is a disruptive action The selection of DPM mode of operation is done by using a function called Enable Dynamic Partition Manager under the CPC Configuration menu in the SE interface as shown in Figure E 2 The DPM mode of operation is normally set at machine installation time by the service support representative S...

Page 533: ... and when finished the DPM mode becomes active and operational Note During the installation process the IBM SSR connects the two OSA Express5s 1000BASE T cables to the SCHs provided ports Important A CPC in DPM mode cannot be part of an Ensemble managed by Unified Resource Manager The HMC used to enable the CPC in DPM mode must NOT be an Ensemble HMC either Primary or Backup Ensemble HMC All defin...

Page 534: ...vide more functions Figure E 4 DPM mode welcome window The HMC can monitor and control up to 32 z Systems CPCs The monitored and controlled CPCs need to be defined to the HMC by using the Object Definition task and adding the CPC object The welcome window shown in Figure E 4 only appears when at least one HMC defined CPC is active in DPM mode Otherwise the traditional HMC window is presented when ...

Page 535: ...process of defining partitions browse the tutorials or start creating the environment by defining and activating the partitions Figure E 6 shows the available expanded views for two of the three options presented to the user in the HMC welcome page when there is a least one CPC running in DPM mode Figure E 6 User Options when the HMC presents the DPM welcome window The Guides option provides Tutor...

Page 536: ...tion for partition that is being created 3 Processors The partition s processing resources are defined in this window 4 Memory This window is used to define partition s initial and maximum memory 5 Network This window is where users define the partition s network NICs resources 6 Storage Where storage connectivity Host Bus Adapters HBA are defined 7 Accelerators Partition resources such as zEDC ca...

Page 537: ...onal facility that is provided by the DPM is Monitor System This option allows users to monitor and manage their DPM environment in many ways and for various events Some of the monitoring and management capabilities are Partition overall performance shown in usage percentages including Processors Storage utilization Network adapters Storage adapters Cryptos Accelerators Power Consumption in KW Env...

Page 538: ... assignment of processors and memory configuration of I O adapters Network FCP Storage Crypto and Accelerators Manage the environment Modification of system resources without disrupting running workloads Monitor and troubleshoot the environment Source identification of system failures conditions states and events that might lead to workload degradation A CPC can be in either the DPM mode or standa...

Page 539: ... introduction to open fertilization with KVM for IBM z Systems and a description of how the environment can be managed This appendix includes the following sections Why KVM for IBM z Systems IBM z Systems servers and KVM Managing the KVM for IBM z Systems environment Using IBM Cloud Manager with OpenStack F ...

Page 540: ...rity for non z Systems users leading to greater operational efficiencies Enables x86 workload consolidation and deployment on z Systems servers using KVM Uses standard interfaces to enable single cross platform virtualization and help simplify systems management Integrates into existing cloud environments and enables cloud deployments by seamlessly working with OpenStack Runs your Linux workloads ...

Page 541: ...its virtual machines PR SM allows the defining and managing of subsets of the z Systems resources in logical partitions LPARs Each KVM for IBM z instance runs in a dedicated LPAR The LPAR definition includes a number of logical processing units LPUs memory and I O resources LPUs are defined and managed by PR SM and are perceived by KVM for IBM z as real CPUs PR SM is responsible for accepting requ...

Page 542: ...ures support the following protocols Native FICON An enhanced protocol over FC providing for communication with FICON devices such as disks tapes and printers Native FICON supports IBM extended count key data ECKD devices Fibre Channel Protocol FCP A standard protocol for communicating with disk and tape devices FCP supports Small Computer System Interface SCSI devices Linux on z Systems and KVM f...

Page 543: ...ll KVM for IBM z and to provide an interface to the IBM z Systems hardware for configuration management functions For details about the HMC see Introduction to the Hardware Management Console at http www ibm com support knowledgecenter HW11P_2 13 1 com ibm hwmca kc_hmc doc i ntroductiontotheconsole introduction html F 2 4 Open source virtualization KVM technology is a cross platform virtualization...

Page 544: ...e boot process In Linux a generic virtual block device is used instead of specific devices such as ECKD or SCSI devices The virtual block devices are handled by the virtio_blk device driver module This module must either be compiled into the Linux kernel or loaded automatically during the boot process For more information about KVM on IBM z Systems see http www ibm com systems z solutions virtuali...

Page 545: ... qemu org The installer is a series of windows to assist and guide the user through the installation process Each window has setting selections that can be made to customize the KVM installation For more information about KVM installation process see Getting Started with KVM for IBM z Systems SG24 8332 Nagios Remote Plugin Executor NRPE can be used with KVM for IBM z NRPE is an addon that allows y...

Page 546: ...sed to administer the hypervisor Furthermore KVM can be administered by using open source tools such as virt manager or OpenStack KVM for IBM z Systems can also be administered and managed by using IBM Cloud Manager with OpenStack Figure F 3 IBM Cloud Manager is created and maintained by IBM and built on OpenStack Figure F 3 KVM for IBM z Systems management interface KVM for IBM z Systems can be m...

Page 547: ...s the libvirtd daemon This component interacts directly with QEMU and the KVM kernel QEMU manages and monitors the KVM virtual machines by performing the following tasks Manage the I O between virtual machines and KVM Create virtual disks Change the state of a virtual machine Start a virtual machine Stop a virtual machine Suspend a virtual machine Resume a virtual machine Delete a virtual machine ...

Page 548: ... of compute storage and networking resources throughout a data center and is based on the Open Stack project http www openstack org IBM Cloud Manager with OpenStack is an advanced management solution that is created and maintained by IBM and built on OpenStack IBM Cloud Manager with OpenStack can be used to get started with a cloud environment and continue to scale with users and workloads providi...

Page 549: ... the design of native Peripheral Component Interconnect Express PCIe features management on z13s and includes the concepts of the integrated firmware processor IFP and resource groups RGs The following topics are included Design of native PCIe I O adapter management Native PCIe feature plugging rules Native PCIe feature definitions G ...

Page 550: ... cards the application specific integrated circuit ASIC chip on the adapter always downloads the device drivers and diagnostic tools from the Support Element SE and runs the management functions on the adapter In the new design there is no ASIC chip for management function on the native PCIe feature adapters For the RoCE and zEDC device drivers and diagnostic tools are now running on the IFP and u...

Page 551: ...supported on the z13s For multiple PCIe features of the same type each feature is always assigned to a different I O domain of the two domains implemented and if available installed in a different PCIe I O drawer to eliminate the possibility of a single point of failure Currently an I O domain of the PCIe I O drawer can support a total of two native PCIe feature adapters together with the existing...

Page 552: ...P manages error recovery and collects error data The error data is sent by the IFP to the SE which then provides a message on the SE and the Hardware Management Console HMC If an error requires maintenance a call home to the IBM Support system is started by the HMC Diagnostic and maintenance tasks Any maintenance action on a native PCIe feature is managed by the IFP including testing or replacing ...

Page 553: ... RoCE Express features Figure G 2 Sample output of AO data or PCHID report 2 8 16 1 3a 8 16 2 4a 8 16 2 a Maximum number of Flash Express per drawer is two Number of Flash Express features 2 adapters feature Maximum number of zEDC Express features Maximum number of RoCE Express features Minimum number of PCIe I O drawers required Notes Although Flash Express features are counted as native PCIe car...

Page 554: ... not have a channel path identifier CHPID assigned but they have a PCHID assigned according to the physical location in the PCIe I O drawer To define the native PCIe adapters in the HCD or HMC a new I O configuration program IOCP FUNCTION statement is introduced that includes several feature specific parameters The IOCP example in Figure G 3 on page 527 define zEDC Express and 10GbE RoCE Express f...

Page 555: ... which is 15 for the zEDC Express feature and 31 for the RoCE Express feature G 3 3 Physical network identifier The physical network identifier PNETID is required to set up the Shared Memory Communications over RDMA SMC R communication between two 10GbE RoCE Express features Each FUNCTION definition supports up to four PNETIDs zEDC Express Functions for LPAR LP14 Reconfigurable to LP01 FUNCTION FI...

Page 556: ...iguration Program User s Guide for ICP IOCP SB10 7163 Definition of10 GbE RoCE Express feature is required to pair up with an OSD CHPID definition by the parameter of PNETID The OSD CHPID definition statement is not listed in the example The PNETID is limited to 2 for a 10GbE RoCE Express definition statement on z13s ...

Page 557: ...e enterprise applications An SSD sometimes called a solid state disk or electronic disk is a data storage device that uses integrated circuit assemblies as memory to store data persistently SSD technology uses electronic interfaces compatible with traditional block I O hard disk drives SSDs do not employ any moving mechanical components This characteristic distinguishes them from traditional magne...

Page 558: ...ge hierarchy Flash Express is an optional PCIe card feature that is available on zEC12 zBC12 z13 and z13s servers Flash Express cards are supported in PCIe I O drawers and can be mixed with other PCIe I O cards such as Fibre Channel connection FICON Express16S Crypto Express5S and Open Systems Adapter OSA Express5S cards You can order a minimum of two features FC 0403 and a maximum of eight featur...

Page 559: ...f the installed drawers slots 1 and 14 before you use the rear slots 25 and 33 Format each pair of cards before you use them Figure H 3 shows a PCIe I O drawer that is fully populated with Flash Express cards Figure H 3 PCIe I O drawer that is fully populated with Flash Express cards D o m a i n 2 D o m a i n 0 D o m a i n 3 D o m a i n 1 Card Slot 19 Card Slot 18 Card Slot 17 Card Slot 16 PCI IN ...

Page 560: ... stand alone memory dumps In z OS Flash Express memory is accessed by using the new z Systems Extended Asynchronous Data Mover EADM architecture It is started with a Start subchannel instruction The Flash Express PCIe cards are shareable across logical partitions LPARs Flash Express memory can be assigned to z OS LPARs such as the main storage It is dedicated to each LPAR You can dynamically incre...

Page 561: ...press provides the following types of actions Flash status and control Displays the list of adapters that are installed in the system and their state Manage Flash allocation Displays the amount of flash memory on the system View Flash allocations Displays a table of flash information for one partition View Flash Displays information for one pair of flash adapters Physical Flash Express PCIe cards ...

Page 562: ...able With the introduction of Flash Express 1 MB large pages can be on Flash Express and pageable Table H 2 introduces for a few z OS data types that are supported by Flash Express the choice criteria for data placement on Flash Express or on disk Table H 2 Flash Express z OS supported data types Data type Data page placement Pageable link pack area PLPA At IPL NIP time PLPA pages are placed both ...

Page 563: ...nt information D M SCM DETAIL displays detailed increment level information The CONFIG ONLINE command is enhanced to allow bringing more SCMs online CF SCM amount ONLINE H 3 Security on Flash Express Data that is stored on Flash Express is encrypted by a strong encryption symmetric key that is in a file on the SE hard disk This key is also known as the Flash encryption key authentication key The f...

Page 564: ... an SE It ensures that the encrypted files are unique and all such smart cards are uniquely tied to their SEs All key generation encryption and decryption occur on the smart card Keys are never in the clear The truly sensitive key the flash encryption key authentication key is only in the file on the SE until it is served to the firmware management of the Flash Express adapter Figure H 6 shows the...

Page 565: ...can encrypt the data with the Flash encryption key authentication key The firmware management of the Flash Express adapter generates its own transport RSA asymmetric key pair This pair is used to wrap the Flash encryption key authentication key while in transit between the SE and the firmware code Figure H 7 on page 538 shows the following key serving topology 1 The firmware management of the Flas...

Page 566: ...stem has no knowledge of the event However when the SE is powered on notification is sent to the system administrator Primary Support Element failure during IML serving of the flash key If the primary SE fails during the serving of the key the alternate SE takes over as the primary and restarts the key serving operation Alternate Support Element failure during switchover from the primary If the al...

Page 567: ...d to be available until the key serving operation is complete If both SEs are down and the Flash Express goes down and comes back online before the SEs become available all data on the Flash Express is lost Reformatting is then necessary when the device is started If both Flash Express devices are still started get the primary SE back online as fast as possible with the Flash encryption key authen...

Page 568: ...540 IBM z13s Technical Guide ...

Page 569: ...x I GDPS Virtual Appliance This appendix discusses the Geographically Dispersed Parallel Sysplex GDPS Virtual Appliance This appendix includes the following sections GDPS overview Overview of GDPS Virtual Appliance GDPS Virtual Appliance recovery scenarios I ...

Page 570: ...lete description of GDPS solution and terminology see IBM GDPS Family of Products An Introduction to Concepts and Capabilities SG24 6374 It is typical to deploy IT environments that are only running Linux on z Systems The GDPS Virtual Appliance is a building block of high availability HA and disaster recovery DR solutions for those environments that do not have or require z OS skills The following...

Page 571: ...g system installation and maintenance Application software maintenance Hardware and software upgrades Unplanned outages can be caused by the following situations Non disaster events such as Application failure Operator errors human error Power outages Network failure Hardware and software failures Disaster events such as Natural disasters or other catastrophes that damage the production facilities...

Page 572: ...ery solution for Linux on z Systems customers that include these components An operating system image The application components An appliance management layer which makes the image self contained Application programming interfaces APIs and user interfaces UIs for customization administration and operation tailored for the appliance function GDPS Virtual Appliance is designed to improve both consum...

Page 573: ...rage devices DASDs one for appliance image initial Installation and two for the upgrade scenario One or two Open Systems Adapter OSA attachments dependent on the network setup and the wanted network resilience System z VM Version 5 Release 4 or later or z VM 6 2 or later Note that z VM 5 4 is not supported on z13s servers The disks being used by z VM and Linux to be mirrored must be ECKD disks A s...

Page 574: ...M IPL z VM guest The remote copy environment is managed by using HyperSwap function and data is kept available and consistent for operating systems and applications Disaster detection ensures successful and faster recovery by using automated processes A single point of control is implemented from the GDPS Virtual Appliance There is no need for involvement of all experts for example storage team ha...

Page 575: ...tartup of z VM systems includes any z VSE guests Use of HyperSwap to non disruptively swap z VM and its guests from the primary to secondary Peer to Peer Remote Copy PPRC devices for both planned disk subsystem maintenance and unplanned disk subsystem failure I 3 GDPS Virtual Appliance recovery scenarios This section presents the following recovery scenarios using GDPS Virtual Appliance Planned di...

Page 576: ...0 000 volume pairs and the systems remain active I 3 2 Unplanned disk outage An unplanned HyperSwap is started automatically by GDPS when triggered by events that indicate the failure of a primary disk device HyperSwap events can include the following events Hard failure triggers I O errors Boxed devices Control unit failures Loss of all channel paths Soft failures such as I O response time trigge...

Page 577: ...ty of the secondary data The GDPS cluster then resets Site 1 and Site 2 systems and updates all the IPL information to point to the secondary devices and IPLs all the production systems in the LPARs in Site 2 again The GDPS Appliance scripting capability is key to recovering the systems in the shortest possible time following a disaster All recovery operations are carried out without operator inte...

Page 578: ...550 IBM z13s Technical Guide ...

Page 579: ...M zEnterprise Data Compression Express This appendix describes the optional IBM zEnterprise Data Compression zEDC Express feature of the z13 and z13s zEC12 and IBM zBC12 servers The appendix includes the following sections Overview zEDC Express Software support J ...

Page 580: ...of storage resources This solution provides a lower cost of computing and also helps to optimize the cross platform exchange of data J 2 zEDC Express zEDC Express is an optional feature FC 0420 It is designed to provide hardware based acceleration for data compression and decompression The feature installs exclusively on the Peripheral Component Interconnect Express PCIe I O drawer Between one and...

Page 581: ...uisite PTFs The fixcat function is explained here http www ibm com systems z os zos features smpe fix category html The fix category named IBM Function zEDC that identifies the fixes that enable or use the zEDC function For more Information about Implementation and usage of the zEDC feature see Reduce Storage Occupancy and Increase Operations Efficiency with IBM zEnterprise Data Compression SG24 8...

Page 582: ...t is available to clients IBM Business Partners and IBM employees zBNA replaces the BWATOOL It is based on Microsoft Windows It provides graphical and text reports including Gantt charts and support for alternative processors zBNA can be used to analyze client provided System Management Facilities SMF records to identify jobs and data sets that are candidates for zEDC compression across a specifie...

Page 583: ...atrix REDP 5157 You can search for view download or order these documents and other Redbooks Redpapers Web Docs draft and additional materials at the following website ibm com redbooks Other publications The IBM z13s product publications are also relevant as further information sources z13s Installation Manual for Physical Planning GC28 6953 z Systems Input Output Configuration Program User s Guid...

Page 584: ...556 IBM z13s Technical Guide ...

Page 585: ...ISBN 0738441678 SG24 8294 00 1 0 spine 0 875 1 498 460 788 pages IBM z13s Technical Guide ...

Page 586: ......

Page 587: ......

Page 588: ...ibm com redbooks Printed in U S A Back cover ISBN 0738441678 SG24 8294 00 ...

Reviews:

No comments

Related manuals for z13s

IBM System x3100 M4 Problem Determination And Service Manual preview
System x3100 M4
Brand: IBM Pages: 332
IBM x3655 - System - 7985 User Manual preview
x3655 - System - 7985
Brand: IBM Pages: 118
HPE ProLiant DL180 Gen10 Product End-Of-Life Disassembly Instructions preview
ProLiant DL180 Gen10
Brand: HPE Pages: 10
Supero 6027R-E1R12N User Manual preview
6027R-E1R12N
Brand: Supero Pages: 112
US Robotics USR8700 User Manual preview
USR8700
Brand: US Robotics Pages: 156
3onedata NP318T-8DI Quick Installation Manual preview
NP318T-8DI
Brand: 3onedata Pages: 4
H3C UniServer R4900 G5 Operating System Installation Manual preview
UniServer R4900 G5
Brand: H3C Pages: 141
Cyclades Cyclades-PR4000 Installation Manual preview
Cyclades-PR4000
Brand: Cyclades Pages: 174
IEI Technology AVL-2000PLUS User Manual preview
AVL-2000PLUS
Brand: IEI Technology Pages: 34
ZLAN ZLAN5443A User Manual preview
ZLAN5443A
Brand: ZLAN Pages: 13
Hewlett Packard Enterprise HPE ProLiant DL580 Gen10 User Manual preview
HPE ProLiant DL580 Gen10
Brand: Hewlett Packard Enterprise Pages: 197
Supermicro A+ AS -2114GT-DNR User Manual preview
A+ AS -2114GT-DNR
Brand: Supermicro Pages: 126
Antrica ANT-35000A User Manual preview
ANT-35000A
Brand: Antrica Pages: 68
LevelOne FNS-1000 User Manual preview
FNS-1000
Brand: LevelOne Pages: 53
Planet Multi-Port Fast Ethernet Print Server FPS-3121 User Manual preview
Multi-Port Fast Ethernet Print Server FPS-3121
Brand: Planet Pages: 57
Transition Networks SDSTX3110-121-LRT-B Quick Start Manual preview
SDSTX3110-121-LRT-B
Brand: Transition Networks Pages: 2
Allnet ALL60200 User Manual preview
ALL60200
Brand: Allnet Pages: 91
RoviSys OPC90 User Manual preview
OPC90
Brand: RoviSys Pages: 146

Brands by name

Popular brands

Load more brands