background image

Design Criteria                                                                                                                                

                                                                                                                                                                                              

 

 
 
 

 

30 

 

JBOD versus RAID0 Disks 

While Cloudian HyperStore as a SDS solution works with JBODs or with RAID0 disks, it is recommended to use 
JBOD for the solution. The 12G SAS RAID controller in S3260 provides up to 4G of cache that can be used for 
writes.  

Memory Sizing 

Memory sizing is based on the number of objects stored on each storage server, which is related to the average 
file size and the data protection scheme. Standard designs call for 384GB for the S3260 M5 single node and 
192GB for the S3260 M5 dual node configurations. 

Network Considerations 

Cloudian Network requirements are standard Ethernet only.  Please refer to the Network layout in Figure 9. While 
Cloudian software can work on a single network interface, it is recommended to create different virtual interfaces 
in Cisco UCS and segregate them. A client-access network and private-cluster network are required for the 
operation. Cisco UCS S3260 has two physical ports of 40G each and the VIC allows you to create out many 
Virtual interfaces on each physical port. 

It is recommended to have a private-cluster network on one port and the client-access networks on another port. 
This provides 40Gb bandwidth for each of these networks. While the client-access network requirements are 
minimal, every storage node can take up to 40Gb of client bandwidth requirements. Also, by having the client and 
cluster VIC’s pinned to each fabric of the fabric interconnects, there is a minimal overhead of network traffic 
passing through the upstream switches for inter-node communication, if any. This unique feature of fabric 
interconnects and VIC’s makes the design highly flexible and scalable.  

Uplinks 

The uplinks from fabric interconnects to upstream switches like Nexus, carry the traffic in case of FI failures or 
reboots. A reboot for instance is needed during a firmware upgrade. While there is complete high availability built-
in the infrastructure, the performance may drop, depending on the uplink connectors from each FI to the Nexus 
vPC pool.  If you want ‘no’ or a ‘minimal drop’, increase the uplink connectors. 

Multi-Site Deployments 

Similar to Amazon S3, the Cloudian HyperStore system supports the implementation of multiple “service regions.” 
Setting up the Cloudian HyperStore system to use multiple service regions is optional.  

The main benefits of deploying multiple service regions are:  

 

Each region has its own independent Cloudian HyperStore geo-cluster for S3 object storage. 
Consequently, deploying multiple regions is another means of scaling-out your overall Cloudian HyperStore 
service offering (beyond using multiple nodes and multiple datacenters to scale out a single geo-cluster). In 
a multi-region deployment, different S3 datasets are stored in each region. Each region has its own token 
space and there is no data replication across regions.  

 

With a multi-region deployment, your service users can choose the service region in which their storage 
buckets will be created. Users may choose to store their S3 objects in the region that’s geographically 
closest to them; or they may choose one region rather than another for reasons of regulatory compliance 
or corporate policy.  

Summary of Contents for UCS S3260 M5

Page 1: ...1 Cisco UCS S3260 M5 Server with Cloudian HyperStore Object Storage Design Guide Last Updated January 24 2019 ...

Page 2: ...Ex the Cisco logo DCE and Welcome to the Human Network are trademarks Changing the Way We Work Live Play and Learn and Cisco Store are service marks and Access Registrar Aironet AsyncOS Bringing the Meeting To You Catalyst CCDA CCDP CCIE CCIP CCNA CCNP CCSP CCVP Cisco the Cisco Certified Internetwork Expert logo Cisco IOS Cisco Press Cisco Systems Cisco Systems Capital the Cisco Systems logo Cisco...

Page 3: ...erprise Linux 7 5 13 Cloudian HyperStore 15 Cloudian Object Storage 15 Cloudian HyperStore Design 16 Cloudian HyperStore Architecture 16 Cloudian Management Console 17 S3 Compatible 18 Integrated Billing Management and Monitoring 18 Cloudian Features 19 Cisco and Cloudian 20 Infinitely Scalable on Demand 20 Security 20 Data Protection 20 Effortless Data Movement 20 Solution Design 21 Deployment Ar...

Page 4: ...ication versus Erasure Coding 27 Replication 27 Erasure Coding 27 Supported Erasure Coding Configurations 28 Flash Storage 29 JBOD versus RAID0 Disks 30 Memory Sizing 30 Network Considerations 30 Uplinks 30 Multi Site Deployments 30 Expansion of the Cluster 31 Deployment Guide 33 Summary 34 About the Authors 35 Acknowledgements 35 ...

Page 5: ...tinuous evolution of data there has been lack of enterprise manageability for large capacity storage servers Cisco UCS with Cloudian provide the solution to solve the challenges as it increased demand to have Cloudian HyperStore validated on Cisco UCS servers The Cisco UCS S3260 Storage Server originally designed for the data center together with Cloudian HyperStore is optimized for object storage...

Page 6: ...ta and to whatever data size It is essentially designed for Applications not direct end users This document describes the architecture and design procedures of Cloudian Hyperstore software on Cisco UCS S3260 M5 servers along with Cisco UCS C220 M5 Rack Mount servers for load balancing Audience The audience for this document includes but is not limited to sales engineers field consultants professio...

Page 7: ... Cisco UCS 6332 Series Fabric Interconnects Cisco UCS S3260 M5 storage servers Cisco S3260 system IO controller with VIC 1380 Cisco C220M5 servers with VIC 1387 Cisco Nexus C9332PQ Series Switches Cloudian HyperStore 7 1 2 Red Hat Enterprise Linux 7 5 ...

Page 8: ...ork Attached Storage NAS over the unified fabric By unifying the storage access Cisco Unified Computing System can access storage over Ethernet NFS or iSCSI Fibre Channel and Fibre Channel over Ethernet FCoE This provides customers with choice for storage access and investment protection In addition the server administrators can pre assign storage access policies for system connectivity to storage...

Page 9: ...vice IaaS The Cisco UCS XML API provides comprehensive access to all Cisco UCS Manager functions The API provides Cisco UCS system visibility to higher level systems management tools from independent software vendors ISVs such as VMware Microsoft and Splunk as well as tools from BMC CA HP IBM and others ISVs and in house developers can use the XML API to enhance the value of the Cisco UCS platform...

Page 10: ...re can be configured on ports 1 to 12 and ports 15 to 26 on the Cisco UCS 6332UP fabric interconnect Ports 17 to 34 on the Cisco UCS 6332 16 UP fabric interconnect support the breakout feature Cisco UCS 9332 Nexus Switches The Cisco Nexus 9000 Series Switches include both modular and fixed port switches that are designed to overcome these challenges with a flexible agile low cost application centr...

Page 11: ...ise class Redundant Array of Independent Disks RAID redundancy or be in a pass through mode This high density rack server comfortably fits in a standard 32 inch depth rack such as the Cisco R42610 Rack Server The Cisco UCS S3260 is deployed as a standalone server in both bare metal or virtualized environments Its modular architecture reduces TCO by allowing you to upgrade individual components ove...

Page 12: ...tions that help reduce customers TCO and increase their business agility The Cisco UCS C220 M5 server extends the capabilities of the Cisco UCS portfolio in a 1 Rack Unit 1RU form factor It incorporates the Intel Xeon Scalable processors supporting up to 20 percent more cores per socket twice the memory capacity 20 percent greater storage density and five times more PCIe NVMe Solid State Disks SSD...

Page 13: ...Cs or host bus adapters HBAs In addition the VIC supports Cisco Data Center Virtual Machine Fabric Extender VM FEX technology This technology extends the Cisco UCS Fabric Interconnect ports to virtual machines simplifying server virtualization deployment The Cisco UCS VIC 1387 provides the following features and benefits Stateless and agile platform The personality of the card is determined dynami...

Page 14: ...rganizations make a seamless transition to emerging datacenter models that include virtualization and cloud computing Red Hat Enterprise Linux includes support for major hardware architectures hypervisors and cloud providers making deployments across physical and different virtual environments predictable and secure Enhanced tools and new capabilities in this release enable administrators to tailo...

Page 15: ...alability while keeping it simple to manage Deploy as on premises storage or configure a hybrid cloud and automatically tier data to the public cloud All at 70 percent less TCO than conventional storage View system health manage users and groups and automate tasks with Cloudian s web based UI and REST API Manage your workload with a self service portal that lets users administer their own storage ...

Page 16: ...ators can set quotas and rate controls for individual members of the group Access control rights Read and write access controls are supported at per bucket and per object granularity Objects can also be exposed via public URLs for regular web access subject to configurable expiration periods Reporting and billing The Cloudian HyperStore system supports usage reporting on a system wide group wide o...

Page 17: ... administrator you can use the CMC to perform the following tasks Provisioning groups and users Managing quality of service QoS controls Creating and managing rating plans Generating usage data reports Generating bills Viewing and managing users stored data objects Setting access control rights on users buckets and stored objects Group administrators can perform a limited range of administrative t...

Page 18: ...ore in addition to being S3 compliant also offers the flexibility to be on premises object storage as well as hybrid tier to Amazon and Google clouds Integrated Billing Management and Monitoring The HyperStore system maintains comprehensive service usage data for each group and each user in the system This usage data which is protected by replication serves as the foundation for HyperStore service...

Page 19: ... treatment of designated source IP addresses so that the billing mechanism does not apply any data transfer charges for data coming from or going to these whitelisted domains Cloudian Features The following are the key Cloudian features Auto Tiering Programmable Multi Tenancy IAM User Support Broad Application Support Cloudian HyperStore Secret Sauce Distributed Peer To Peer Architecture Parallel ...

Page 20: ...able on Demand Cisco and Cloudian HyperStore offers on demand infinite scalability allowing you to grow your storage as needed As your demand grows you can seamlessly grow your storage across multiple DCs Security Cisco and Cloudian HyperStore takes safeguarding customer data very seriously Two server side encryption methods SSE SSE c KeySecure are implemented to ensure that data is protected at a...

Page 21: ...o UCS S3260 Storage Server Chassis each with two Cisco UCS S3260 M5 nodes configured as storage servers and one Cisco UCS C220 M5S rack server as Supervisor node The whole solution is connected to a pair of Cisco UCS 6332 Fabric Interconnects and a pair of upstream network Cisco Nexus 9332PQ switches The configuration is comprised of the following 2 x Cisco Nexus 9332PQ Switches 2 x Cisco UCS 6332...

Page 22: ...nterconnect UCS Manager 4 0 1a Kernel 5 0 3 N2 4 01a System 5 0 3 N2 4 01a Network Nexus 9332PQ BIOS 07 51 NXOS 7 0 3 I4 8b Software Red Hat Enterprise Linux Server 7 5 x86_64 Cloudian HyperStore 7 1 2 Hardware Requirements and Bill of Materials Component Model Quantity Comments Cloudian Storage Nodes Cisco UCS S3260 M5 Chassis 3 2 x UCS S3X60 M5 Server Nodes per Chassis Total 6nodes Per Server No...

Page 23: ... 2 1GHz 8 Cores 96GB RAM Cisco 12G SAS RAID Controller 2 x 600GB SAS for OS Dual port 40 Gbps VIC UCS Fabric Intercon nects Cisco UCS 6332 Fabric Intercon nects 2 Switches Cisco Nexus 9332PQ Switches 2 Physical Topology and Configuration The following sections describe the physical design of the solution and the configuration of each component ...

Page 24: ...lustrates the Network Topology used in the setup High Availability As part of the hardware and software resiliency the following tests will be conducted on the test bed The results of the tests will be included in the deployment guide to be published ...

Page 25: ...Solution Design 25 ...

Page 26: ...es Priority Industry Target Telco Mobile Operator Cable Operator SaaS Other Cloud Services Financial Services Media Entertainment Police Intelligence Agencies Hospitals and Medical Imaging Vendor Transportation Other Global 2000 non XaaS FIN M E Transp Hosp The following sections describe some considerations for the design of the Infrastructure and the Cloudian HyperStore Requirements The requirem...

Page 27: ...Store system lets you preconfigure one or more storage policies Users when creating a new storage bucket can choose which preconfigured storage policy to use to protect data in that bucket Users cannot create buckets until you have created at least one storage policy For each storage policy that you create you can choose from the following two data protection methods Replication With replication a...

Page 28: ...d across your datacenters for example you could have a storage policy that for each S3 object store 3 replicas of the object in each of your datacenters and a second storage policy that erasure codes objects and stores them in just one particular datacenter Supported Erasure Coding Configurations Cloudian HyperStore supports EC replicated EC and distributed EC configurations EC This configuration ...

Page 29: ... EC solution implements the new ISA L Erasure Codes that is vectored and fast ISA L is the Intel library containing functions to improve erasure coding The Cloudian Distributed Data Center with EC configuration requires a minimum of 3 data centers with 4 nodes each Data stored DC1 4 Dc2 4 DC3 4 Metadata stored Data stored DC1 4 DC2 4 DC3 3 Distributed EC configuration offers the same level of prot...

Page 30: ...k traffic passing through the upstream switches for inter node communication if any This unique feature of fabric interconnects and VIC s makes the design highly flexible and scalable Uplinks The uplinks from fabric interconnects to upstream switches like Nexus carry the traffic in case of FI failures or reboots A reboot for instance is needed during a firmware upgrade While there is complete high...

Page 31: ...t as your requirements change Cisco UCS 6332 Fabric Interconnects have 32 ports each Each server is connected to either of the FI s Leaving the uplinks and any other clients directly connected to the Fabrics 24 28 server nodes can be connected to FI pairs If more servers are required you should plan for a multi domain system Cisco UCS offers KVM management both in band and out of band In case out ...

Page 32: ...ated and applied on the newly added hardware Cisco UCS makes Infrastructure readiness extremely simple for any newly added storage nodes Rack the nodes connect the cables and then clone and apply the service profile When the nodes are ready you may have to follow the node addition procedure per the Cloudian documentation The simplified management of the infrastructure with Cisco UCS and well teste...

Page 33: ...ed with 6 x Cisco UCS S3260 storage nodes While a few functional tests were completed more in depth tests will follow The findings of these tests and the resulting best practices will be documented in the deployment guide which will be published at a later date ...

Page 34: ...d storage on premise Cloudian HyperStore software ensures unlimited scale multi data center storage fully automated data tiering and support for all S3 applications all behind your firewall Cloudian HyperStore software deployed on Cisco UCS S Series servers combines robust availability with system management control monitoring capabilities and reporting A host of features including hybrid cloud st...

Page 35: ...His current role includes building Cloud Computing Software defined Storage Automation and Management Converged and Hyper Converged Solutions on Cisco UCS platforms Muhammad also holds Cisco Internetwork Expert Data Center Certification CCIE DC Acknowledgements For their support and contribution to the design validation and creation of this Cisco Validated Design the authors would like to thank Ch...

Reviews: