background image

 

4-11

Data Center High Availability Clusters Design Guide

OL-12518-01

Chapter 4      FCIP over IP/MPLS Core

  Using FCIP Tape Acceleration

Simplified management—Provides a unified management environment independent of whether 
servers use FCIP to connect to the storage network.

Comprehensive security—Combines the ubiquitous IP security infrastructure with Cisco virtual 
SANs (VSANs), hardware-based zoning, and hardware-based access control lists (ACLs) to provide 
robust security.

Cisco FCIP

Cisco FCIP offers the following functions:

Simplifies data protection and business continuance strategies by enabling backup, remote 
replication, and disaster recovery over WAN distances using open-standard FCIP tunneling

Improves utilization of WAN resources for backup and replication by tunneling up to three virtual 
ISLs on a single Gigabit Ethernet port

Reduces SAN complexity by eliminating the need to deploy and manage a separate remote 
connectivity platform

Preserves the Cisco MDS 9000 Family enhanced capabilities including VSANs, advanced traffic 
management, and security across remote connections

Cisco MDS 9216i

The Cisco MDS 9216i Multilayer Fabric Switch is designed for building mission-critical enterprise 
storage networks that take advantage of the cost-effectiveness and ubiquity of IP for more robust 
business continuance services, leveraging both Fibre Channel and IP in a single module. The Cisco MDS 
9216i brings new capability to the fabric switch market. Sharing a consistent architecture with the Cisco 
MDS 9500 Series, the Cisco MDS 9216i integrates both Fibre Channel and IP storage services in a single 
system to allow maximum flexibility in user configurations.

With 14 2-Gbps Fibre Channel ports, two Gigabit Ethernet IP storage services ports, and a modular 
expansion slot, the Cisco MDS 9216i is ideally suited for enterprise storage networks that require high 
performance SAN extension or cost-effective IP storage connectivity. This level of integration gives 
Cisco MDS 9216i users the benefits of a multiprotocol system without sacrificing Fibre Channel port 
density. The expansion slot on the Cisco MDS 9216i allows for the addition of any Cisco MDS 9000 
Family module, so users can add additional Fibre Channel ports and additional IP ports. Alternatively, 
the expansion slot may be used for a variety of Cisco MDS 9000 Family services modules, thereby 
providing an unparalleled level of storage services in a single, highly available 3-rack unit system.

As the storage network expands further, Cisco MDS 9000 Family modules can be removed from Cisco 
MDS 9216i switches and migrated into Cisco MDS 9500 Series Multilayer Directors, providing smooth 
migration, common sparing, and outstanding investment protection.

The modular design of the Cisco MDS 9216i allows it to support current and future Cisco MDS 9000 
Family switching or services module.

Currently available modules include the following:

16-port and 32-port 2-Gbps Fibre Channel switching modules

IP Services Module supporting iSCSI and FCIP over both four and eight ports of Gigabit Ethernet

Multiprotocol Services Module supporting 14 ports of 2-Gbps Fibre Channel and 2 ports of Gigabit 
Ethernet that provide iSCSI and FCIP storage services

Advanced Services Module and Caching Services Module supporting integrated network-hosted 
application services

Summary of Contents for OL-12518-01

Page 1: ...ents offered by optical networks These applications can be easily transported over IP networks using FCIP interfaces The obvious advantage of transporting storage over IP is the ubiquitous nature of IP Disk replication is the primary type of application that runs over an extended SAN network for business continuance or disaster recovery The two main types of disk replication are array based provid...

Page 2: ...ity is considered for storage but is not currently installed because of cost constraints Most of the data at the home office location may be consolidated into a local SAN and the data at the satellite offices can be consolidated into small SAN islands This introduces the problem of storage connectivity between SAN islands for disaster recovery and business continuance There are several options to ...

Page 3: ... compressing data The LZS compression algorithm works by searching for redundant data strings in the input data stream and then replaces these strings with data tokens that are shorter in length than the original data A table is built of these string matches pointing to previous data in the input stream The net result is that future data is compressed based on previous data The more redundant the ...

Page 4: ...Mode 1 Equivalent to the high throughput ratio of SAN OS 1 3 Use Mode 1 for WAN paths up to 100 Mbps on the IPS 4 and IPS 8 and WAN paths up to 1 Gbps on the MPS 14 2 Mode 2 Higher compression ratio than Mode1 but applicable only to slow WAN links up to 25 Mbps Mode 3 Higher compression ratio than Mode 1 and slightly higher than Mode 2 Applicable to very slow WAN links up to 10 Mbps The following ...

Page 5: ...work through which an FCIP tunnel is created The following scenarios are possible Unauthorized Fibre Channel device gaining access to resources through normal Fibre Channel processes Unauthorized agents monitoring and manipulating Fibre Channel traffic that flows over physical media used by the IP network Security protocols and procedures used for other IP networks can be used with FCIP to safegua...

Page 6: ...200 VXR routers the SA VAM has a bandwidth cost of 300 bandwidth points The SA VAM has a maximum throughput of 140 Mps making it suitable for WAN links up to DS3 or E3 line rates The SA VAM2 is supported on the 7200 VXR routers with a minimum Cisco IOS version of 12 3 1 The SA VAM2 has a bandwidth cost of 600 bandwidth points The SA VAM2 has a maximum throughput of 260 Mps making it suitable for W...

Page 7: ...ation or write I O throughput and reduces I O response time in most situations particularly as the FCIP Round Trip Time RTT increases Each FCIP link can be filled with a number of concurrent or outstanding I Os These I Os can originate from a single replication source or a number of replication sources The FCIP link is filled when the number of outstanding I Os reaches a certain ceiling The ceilin...

Page 8: ...ions TCP implemented on traditional servers or hosts tends to overreact to packet drops The throttling back that occurs in the traditional TCP implementation is not acceptable to storage traffic The TCP stack implemented for FCIP in the Cisco MDS 9000 is optimized for carrying storage traffic by reducing the probability of drops and increasing the resilience to drops when they occur Fibre Channel ...

Page 9: ...r than the bandwidth of the slowest link in the FCIP link path For example if the FCIP link is mapped over a dedicated DS3 WAN link set the maximum bandwidth to 45 Mbps The TCP maximum bandwidth value is used as the bandwidth value in the bandwidth delay product calculation of the TCP MWS Observe the following guidelines when selecting a value for TCP maximum bandwidth Set the TCP maximum bandwidt...

Page 10: ... fabric switch market Sharing a consistent architecture with the Cisco MDS 9500 Series the Cisco MDS 9216 combines multilayer intelligence with a modular chassis making it the most intelligent and flexible fabric switch in the industry Starting with 16 2 1 Gbps auto sensing Fibre Channel ports the MDS 9216 expansion slot allows for the addition of any Cisco MDS 9000 Family modules for up to 48 tot...

Page 11: ...Fibre Channel and IP storage services in a single system to allow maximum flexibility in user configurations With 14 2 Gbps Fibre Channel ports two Gigabit Ethernet IP storage services ports and a modular expansion slot the Cisco MDS 9216i is ideally suited for enterprise storage networks that require high performance SAN extension or cost effective IP storage connectivity This level of integratio...

Page 12: ...s if there are two other port adapters on the same PCI bus Additionally when data is fast switched between port adapters it must traverse to and from the 7200 system memory over the PCI bus If the data is transmitted between two port adapters on the same bus the data must traverse the PCI bus twice effectively reducing the performance of the data path by three fourths Also consider the PCI bandwid...

Page 13: ... AVVID call control 4 FCIP this is just a recommendation a full analysis of the network and application is required 5 Voice transport 6 Video 7 SP network management and control Note The above DSCP values are just a recommendation a network administrator may choose another consistent set of numbers if desired Cisco MDS supports DSCP values for marking all IP packets in the type of service TOS fiel...

Page 14: ...ng must be designed and implemented so that no data is lost in case of a disaster To achieve this transactions must be written on both the main and backup sites synchronously to keep the databases consistent When an application writes data to disk that data is being replicated to the remote site before a write acknowledgement is sent back to the application The write I O is acknowledged on the ser...

Page 15: ... Remote tape vaulting Enables remote backup for disaster recovery using tape or disk Tape applications typically allow a single outstanding I O operation which limits throughput on long distance links Write Acceleration and optionally compression techniques can help to optimize throughput in these situations Host initiator to remote pooled storage Enables access to FC attached pooled storage array...

Page 16: ...as DWDM or SONET SDH for high density and reliability However because of disaster recovery requirements corporations might need a third site to protect all their business needs in case of a disaster FCIP is preferred to connect the secondary site to the third site as shown in Figure 4 7 FC Fabrics FC Fabrics 132427 IP MPLS Network FCIP Corporate HQ FC Fabric FC Fabrics FC FC FC FCIP Remote Sites C...

Page 17: ... host based mirroring based on iSCSI A typical network setup is shown in Figure 4 8 Figure 4 8 Host based Mirroring using iSCSI Table 4 1summarizes the possible service offerings an SP can provide to its customers FC Fabric Cisco MDS 9216 Multi layer Edge Switch Cisco MDS 9216 Multi layer Edge Switch 119870 SP MPLS Network FC Fabric FC Primary Site Optical FCIP Secondary Site Third Site 132428 SCS...

Page 18: ...mirroring Require no data loss High volume rev impact Finance banking brokerage Telecom federal defense Disaster recovery providers CLARiiON Symmetrix Hitachi And so on Ethernet Fibre Channel FICON DWDM SONET DWDM ONS 15530 ONS 15540 ONS 15454 Asynchronous Data replication Near real time ext distance mirroring Larger market Healthcare Life Sci Biomedical Engineering Symmetrix CLARiiON Ethernet Fib...

Page 19: ...VPN IPv4 prefixes among PE routers using MP BGP This ensures that the routes for a given VPN are learned only by other members of that VPN enabling members of the VPN to communicate with each other When a VPN route learned from a customer edge CE router is injected into BGP a list of VPN route target extended community attributes is associated with it Typically the list of route target community v...

Page 20: ... A derived CEF table A set of interfaces that use the forwarding table A set of rules and routing protocols that determine what goes into the forwarding table Testing Scenarios and Results Test Objectives This section describes the testing performed to simulate an IP MPLS network and to transport FCIP traffic across the simulated SP network The test objectives were as follows Transporting FCIP tra...

Page 21: ...ronization bgp log neighbor changes neighbor 10 200 0 105 remote as 65001 Remote PE neighbor 10 200 0 105 update source Loopback0 no auto summary address family vpnv4 neighbor 10 200 0 105 activate neighbor 10 200 0 105 send community extended exit address family address family ipv4 vrf storage redistribute connected redistribute the CE routes onto the storage VRF no auto summary no synchronizatio...

Page 22: ...arries the CE routing information The PE router learns an IP prefix from a CE router through a BGP session with the CE router router bgp 65001 no synchronization bgp log neighbor changes neighbor 10 200 0 106 remote as 65001 Remote PE neighbor 10 200 0 106 update source Loopback0 no auto summary address family vpnv4 neighbor 10 200 0 106 activate neighbor 10 200 0 106 send community extended exit ...

Page 23: ...ity of the core network to the end storage customer which means that the core network infrastructure including addressing and topology is not visible to the VPN customers Customer VPN routes that originate from other PE routers across the core network are associated with the BGP next hop address of the originating PE router The BGP next hop address of the PE router is not visible or reachable in t...

Page 24: ...re Channel data frames in typical storage applications have a payload of 2048 bytes plus 36 bytes in headers and trailers leaving a frame size of 2084 bytes The Cisco MDS 9000 optionally adds two headers to the Fibre Channel frame The EISL header is an eight byte field carrying VSAN tagging information that is only added if the FCIP interface is defined as a TE_Port If the EISL header is present i...

Page 25: ...ic headers Selective acknowledgement SACK is enabled by default in the FCIP profile and should not be turned off SACK enables the TCP receiver to identify to the sender the contiguous TCP blocks that have been successfully received The sender can then selectively retransmit only the missing blocks Figure 4 12 shows the result for the throughput testing of the MTU A full line rate is achieved with ...

Page 26: ...abit Ethernet traffic and is limited to around 35 percent of Gigabit Ethernet bandwidth This is ideal when the customer traffic is compressed and the requirement does not go beyond the capabilities of the Cisco 7500 As the traffic demand increases the Cisco 7500 can be replaced by high performing routers like the GSR Figure 4 15 Scenario 3 MDS 9216i Connected to 7500 PE GSR P 132433 0 200 400 600 ...

Page 27: ...ast Reroute Less than 50 ms In the case of an MPLS backbone Label Distribution Protocol LDP convergence also has to be taken into consideration The convergence of this protocol depends on the particular mode of operation that is being used frame mode or cell mode Scenario 5 Impact of Core Performance MPLS provides an efficient mechanism for supporting VPN VRFs With a VRF the traffic of a given ent...

Page 28: ...s the IP MTU size on the FCIP connection SAN Extension data packets are usually the maximum Fibre Channel size of 2148 bytes With an IP MTU of 1500 bytes the IPS must fragment large Fibre Channel frames into two IP frames which decreases overall performance Also the data compression occurs after the Fibre Channel frame is encapsulated into IP and compression works better with a larger frame to exa...

Page 29: ...t each application demands to obtain maximum efficiency Compensation Ratio 12 0 10 0 8 0 6 0 4 0 2 0 0 0 SA VAM SA VAM2 IPS high throughput IPS high compression MPS Mode 1 MPS Mode 2 MPS Mode 3 132437 alice29 txt asyoulik txt cp htm l fields c gram m er lsp kennedy xls lcet10 txt plrabn12 txt ptt5 sum N ull D ata Table 4 2 Application Requirements Application Bandwidth Latency Async or Sync Commen...

Page 30: ...ackup an essential component of business continuance and disaster recovery applications The efficiency is achieved by implementing buffering and error recovery mechanisms The concept is similar to spoofing even though the server and tape controller are separated by a large distance they behave as if they are co located The tape pipelining technique relaxes the design constraints of SAN extension t...

Reviews: