background image

C H A P T E R

 

4-1

Data Center High Availability Clusters Design Guide

OL-12518-01

4

FCIP over IP/MPLS Core

This chapter discusses the transport of Fibre Channel over IP (FCIP) over IP/Multiprotocol Label 
Switching (MPLS) networks and addresses the network requirements from a service provider (SP) 
perspective. This chapter also describes service architectures and storage service offerings using FCIP 
as a primary storage transport mechanism.

Overview

Storage extension solutions offer connectivity between disparate storage “islands,” and promote 
transport solutions that are specifically geared towards carrying storage area network (SAN) protocols 
over WAN and MAN networks. This emerging demand is providing a new opportunity for carriers. SPs 
can now deliver profitable SAN extension services over their existing optical (Synchronous Optical 
Network [SONET]/Synchronous Digital Hierarchy [SDH] and Dense Wavelength Division Multiplexing 
[DWDM]) or IP infrastructure. DWDM networks are ideal for high-bandwidth, highly resilient networks 
and are typically deployed within metro areas. Transporting storage traffic over the existing 
SONET/SDH infrastructure allows SPs to maximize the use of their existing SONET/SDH ring 
deployments. Some applications do not mandate stringent requirements offered by optical networks. 
These applications can be easily transported over IP networks using FCIP interfaces. The obvious 
advantage of transporting storage over IP is the ubiquitous nature of IP.

Disk replication is the primary type of application that runs over an extended SAN network for business 
continuance or disaster recovery. The two main types of disk replication are array-based (provided by 
EMC

2

 SRDF, Hitachi True Copy, IBM PPRC XD, or HP DRM, and host-based (for example, Veritas 

Volume Replicator). Both disk replication types run in synchronous and asynchronous modes. In 
synchronous mode, an acknowledgement of a host-disk write is not sent until a copy of the data to the 
remote array is completed. In asynchronous mode, host-disk writes are acknowledged before the copy 
of the data to the remote array is completed.

Applications that use synchronous replication are highly sensitive to response delays and might not work 
with slow-speed or high-latency links. It is important to consider the network requirements carefully 
when deploying FCIP in a synchronous implementation. Asynchronous deployments of FCIP are 
recommended in networks with latency or congestion issues. With FCIP, Fibre Channel SAN can be 
extended anywhere an IP network exists and the required bandwidth is available. FCIP can be extended 
over metro, campus, or intercontinental distances using MPLS networks. FCIP may be an ideal choice 
for intercontinental and coast-to-coast extension of SAN. 

Содержание OL-12518-01

Страница 1: ...ents offered by optical networks These applications can be easily transported over IP networks using FCIP interfaces The obvious advantage of transporting storage over IP is the ubiquitous nature of IP Disk replication is the primary type of application that runs over an extended SAN network for business continuance or disaster recovery The two main types of disk replication are array based provid...

Страница 2: ...ity is considered for storage but is not currently installed because of cost constraints Most of the data at the home office location may be consolidated into a local SAN and the data at the satellite offices can be consolidated into small SAN islands This introduces the problem of storage connectivity between SAN islands for disaster recovery and business continuance There are several options to ...

Страница 3: ... compressing data The LZS compression algorithm works by searching for redundant data strings in the input data stream and then replaces these strings with data tokens that are shorter in length than the original data A table is built of these string matches pointing to previous data in the input stream The net result is that future data is compressed based on previous data The more redundant the ...

Страница 4: ...Mode 1 Equivalent to the high throughput ratio of SAN OS 1 3 Use Mode 1 for WAN paths up to 100 Mbps on the IPS 4 and IPS 8 and WAN paths up to 1 Gbps on the MPS 14 2 Mode 2 Higher compression ratio than Mode1 but applicable only to slow WAN links up to 25 Mbps Mode 3 Higher compression ratio than Mode 1 and slightly higher than Mode 2 Applicable to very slow WAN links up to 10 Mbps The following ...

Страница 5: ...work through which an FCIP tunnel is created The following scenarios are possible Unauthorized Fibre Channel device gaining access to resources through normal Fibre Channel processes Unauthorized agents monitoring and manipulating Fibre Channel traffic that flows over physical media used by the IP network Security protocols and procedures used for other IP networks can be used with FCIP to safegua...

Страница 6: ...200 VXR routers the SA VAM has a bandwidth cost of 300 bandwidth points The SA VAM has a maximum throughput of 140 Mps making it suitable for WAN links up to DS3 or E3 line rates The SA VAM2 is supported on the 7200 VXR routers with a minimum Cisco IOS version of 12 3 1 The SA VAM2 has a bandwidth cost of 600 bandwidth points The SA VAM2 has a maximum throughput of 260 Mps making it suitable for W...

Страница 7: ...ation or write I O throughput and reduces I O response time in most situations particularly as the FCIP Round Trip Time RTT increases Each FCIP link can be filled with a number of concurrent or outstanding I Os These I Os can originate from a single replication source or a number of replication sources The FCIP link is filled when the number of outstanding I Os reaches a certain ceiling The ceilin...

Страница 8: ...ions TCP implemented on traditional servers or hosts tends to overreact to packet drops The throttling back that occurs in the traditional TCP implementation is not acceptable to storage traffic The TCP stack implemented for FCIP in the Cisco MDS 9000 is optimized for carrying storage traffic by reducing the probability of drops and increasing the resilience to drops when they occur Fibre Channel ...

Страница 9: ...r than the bandwidth of the slowest link in the FCIP link path For example if the FCIP link is mapped over a dedicated DS3 WAN link set the maximum bandwidth to 45 Mbps The TCP maximum bandwidth value is used as the bandwidth value in the bandwidth delay product calculation of the TCP MWS Observe the following guidelines when selecting a value for TCP maximum bandwidth Set the TCP maximum bandwidt...

Страница 10: ... fabric switch market Sharing a consistent architecture with the Cisco MDS 9500 Series the Cisco MDS 9216 combines multilayer intelligence with a modular chassis making it the most intelligent and flexible fabric switch in the industry Starting with 16 2 1 Gbps auto sensing Fibre Channel ports the MDS 9216 expansion slot allows for the addition of any Cisco MDS 9000 Family modules for up to 48 tot...

Страница 11: ...Fibre Channel and IP storage services in a single system to allow maximum flexibility in user configurations With 14 2 Gbps Fibre Channel ports two Gigabit Ethernet IP storage services ports and a modular expansion slot the Cisco MDS 9216i is ideally suited for enterprise storage networks that require high performance SAN extension or cost effective IP storage connectivity This level of integratio...

Страница 12: ...s if there are two other port adapters on the same PCI bus Additionally when data is fast switched between port adapters it must traverse to and from the 7200 system memory over the PCI bus If the data is transmitted between two port adapters on the same bus the data must traverse the PCI bus twice effectively reducing the performance of the data path by three fourths Also consider the PCI bandwid...

Страница 13: ... AVVID call control 4 FCIP this is just a recommendation a full analysis of the network and application is required 5 Voice transport 6 Video 7 SP network management and control Note The above DSCP values are just a recommendation a network administrator may choose another consistent set of numbers if desired Cisco MDS supports DSCP values for marking all IP packets in the type of service TOS fiel...

Страница 14: ...ng must be designed and implemented so that no data is lost in case of a disaster To achieve this transactions must be written on both the main and backup sites synchronously to keep the databases consistent When an application writes data to disk that data is being replicated to the remote site before a write acknowledgement is sent back to the application The write I O is acknowledged on the ser...

Страница 15: ... Remote tape vaulting Enables remote backup for disaster recovery using tape or disk Tape applications typically allow a single outstanding I O operation which limits throughput on long distance links Write Acceleration and optionally compression techniques can help to optimize throughput in these situations Host initiator to remote pooled storage Enables access to FC attached pooled storage array...

Страница 16: ...as DWDM or SONET SDH for high density and reliability However because of disaster recovery requirements corporations might need a third site to protect all their business needs in case of a disaster FCIP is preferred to connect the secondary site to the third site as shown in Figure 4 7 FC Fabrics FC Fabrics 132427 IP MPLS Network FCIP Corporate HQ FC Fabric FC Fabrics FC FC FC FCIP Remote Sites C...

Страница 17: ... host based mirroring based on iSCSI A typical network setup is shown in Figure 4 8 Figure 4 8 Host based Mirroring using iSCSI Table 4 1summarizes the possible service offerings an SP can provide to its customers FC Fabric Cisco MDS 9216 Multi layer Edge Switch Cisco MDS 9216 Multi layer Edge Switch 119870 SP MPLS Network FC Fabric FC Primary Site Optical FCIP Secondary Site Third Site 132428 SCS...

Страница 18: ...mirroring Require no data loss High volume rev impact Finance banking brokerage Telecom federal defense Disaster recovery providers CLARiiON Symmetrix Hitachi And so on Ethernet Fibre Channel FICON DWDM SONET DWDM ONS 15530 ONS 15540 ONS 15454 Asynchronous Data replication Near real time ext distance mirroring Larger market Healthcare Life Sci Biomedical Engineering Symmetrix CLARiiON Ethernet Fib...

Страница 19: ...VPN IPv4 prefixes among PE routers using MP BGP This ensures that the routes for a given VPN are learned only by other members of that VPN enabling members of the VPN to communicate with each other When a VPN route learned from a customer edge CE router is injected into BGP a list of VPN route target extended community attributes is associated with it Typically the list of route target community v...

Страница 20: ... A derived CEF table A set of interfaces that use the forwarding table A set of rules and routing protocols that determine what goes into the forwarding table Testing Scenarios and Results Test Objectives This section describes the testing performed to simulate an IP MPLS network and to transport FCIP traffic across the simulated SP network The test objectives were as follows Transporting FCIP tra...

Страница 21: ...ronization bgp log neighbor changes neighbor 10 200 0 105 remote as 65001 Remote PE neighbor 10 200 0 105 update source Loopback0 no auto summary address family vpnv4 neighbor 10 200 0 105 activate neighbor 10 200 0 105 send community extended exit address family address family ipv4 vrf storage redistribute connected redistribute the CE routes onto the storage VRF no auto summary no synchronizatio...

Страница 22: ...arries the CE routing information The PE router learns an IP prefix from a CE router through a BGP session with the CE router router bgp 65001 no synchronization bgp log neighbor changes neighbor 10 200 0 106 remote as 65001 Remote PE neighbor 10 200 0 106 update source Loopback0 no auto summary address family vpnv4 neighbor 10 200 0 106 activate neighbor 10 200 0 106 send community extended exit ...

Страница 23: ...ity of the core network to the end storage customer which means that the core network infrastructure including addressing and topology is not visible to the VPN customers Customer VPN routes that originate from other PE routers across the core network are associated with the BGP next hop address of the originating PE router The BGP next hop address of the PE router is not visible or reachable in t...

Страница 24: ...re Channel data frames in typical storage applications have a payload of 2048 bytes plus 36 bytes in headers and trailers leaving a frame size of 2084 bytes The Cisco MDS 9000 optionally adds two headers to the Fibre Channel frame The EISL header is an eight byte field carrying VSAN tagging information that is only added if the FCIP interface is defined as a TE_Port If the EISL header is present i...

Страница 25: ...ic headers Selective acknowledgement SACK is enabled by default in the FCIP profile and should not be turned off SACK enables the TCP receiver to identify to the sender the contiguous TCP blocks that have been successfully received The sender can then selectively retransmit only the missing blocks Figure 4 12 shows the result for the throughput testing of the MTU A full line rate is achieved with ...

Страница 26: ...abit Ethernet traffic and is limited to around 35 percent of Gigabit Ethernet bandwidth This is ideal when the customer traffic is compressed and the requirement does not go beyond the capabilities of the Cisco 7500 As the traffic demand increases the Cisco 7500 can be replaced by high performing routers like the GSR Figure 4 15 Scenario 3 MDS 9216i Connected to 7500 PE GSR P 132433 0 200 400 600 ...

Страница 27: ...ast Reroute Less than 50 ms In the case of an MPLS backbone Label Distribution Protocol LDP convergence also has to be taken into consideration The convergence of this protocol depends on the particular mode of operation that is being used frame mode or cell mode Scenario 5 Impact of Core Performance MPLS provides an efficient mechanism for supporting VPN VRFs With a VRF the traffic of a given ent...

Страница 28: ...s the IP MTU size on the FCIP connection SAN Extension data packets are usually the maximum Fibre Channel size of 2148 bytes With an IP MTU of 1500 bytes the IPS must fragment large Fibre Channel frames into two IP frames which decreases overall performance Also the data compression occurs after the Fibre Channel frame is encapsulated into IP and compression works better with a larger frame to exa...

Страница 29: ...t each application demands to obtain maximum efficiency Compensation Ratio 12 0 10 0 8 0 6 0 4 0 2 0 0 0 SA VAM SA VAM2 IPS high throughput IPS high compression MPS Mode 1 MPS Mode 2 MPS Mode 3 132437 alice29 txt asyoulik txt cp htm l fields c gram m er lsp kennedy xls lcet10 txt plrabn12 txt ptt5 sum N ull D ata Table 4 2 Application Requirements Application Bandwidth Latency Async or Sync Commen...

Страница 30: ...ackup an essential component of business continuance and disaster recovery applications The efficiency is achieved by implementing buffering and error recovery mechanisms The concept is similar to spoofing even though the server and tape controller are separated by a large distance they behave as if they are co located The tape pipelining technique relaxes the design constraints of SAN extension t...

Отзывы: