Chapter 5: Operating Modes and Configuration
messages are transmitted in the C-Bits of the frame and do not decrease bandwidth available for data.
PMDL Circuit ID messages facilitate confirmation of the data source when presented with a pair of
unlabeled BNC cables.
Packet Flow
Packet Order and Channel Bonding/Aggregation
On single-telecom converters, LAN packet delivery order is guaranteed with the exception that high-priority
802.1p CoS packets may by transmitted before lower priority packets. The remainder of this section applies
to dual-telecom units only.
Dual DS3/E3 converters have three settings available for DS3/E3 utilization:
●
Failover
●
Load-Balancing
●
Bonded (in firmware beginning March 2008)
Failover mode utilizes only one DS3/E3 channel and transmits only sync signals on the other DS3/E3
channel unless the first fails. This setting is generally not useful, as the other two settings will also revert to
a single operational channel if either fails.
The next two modes determine packet order preservation. While traveling across dual telecom links, packet
ordering can be lost as short packets arrive on one path before longer packets on the other or if telecom path
lengths differ. As an aside, to reduce latency, E3Switch converters dynamically compute telecom path
delays and attempt to use the shorter path first.
“Bonded” mode guarantees packet delivery order is preserved, although high-CoS traffic will bypass low-
CoS traffic. In “load-balancing” configuration, or for firmware shipped before March 2008, packet order is
generally maintained but not guaranteed.
The optimum settings for bandwidth, latency performance or data integrity is often determined only by
trying each setting in the field. “Bonded” is preferred from a data-integrity and interoperability standpoint;
however, “load-balancing” mode will generally deliver packets to their final destination faster, especially
for very high traffic loads of consistently small packets (<128 bytes).
Link aggregation (bonding) of the telecom channels allows packets to be delivered to the remote converter's
LAN port in the same order in which they were presented to the local converter's incoming LAN port. The
E3Switch converters do this in a highly efficient manner which allows full bandwidth of both telecom
channels to be utilized. 802.3ad link aggregation is not used and is a less efficient protocol which would
segregate specific “conversations” to specific telecom links, leaving the bandwidth of the second path
unavailable for utilization by a particular “conversation.”
Maintaining strict packet order can sometimes be detrimental. The TCP protocol already has some ability
to assemble incoming packets in their correct order. In such a case, waiting to deliver packet number three
until packet two has arrived at the converter simply introduces additional latency. If a packet is lost, this
latency can grow larger until the loss is recognized. E3Switch converters have the ability to immediately
recognize packet loss if successive packets are received from the same telecom channel, and in the worst
case will not delay longer than 10ms waiting for a missing packet. Maximum packet-per-second forwarding
rates can also be impacted if the bonding mode is required to re-order packets which are consistently < 128
bytes in length.
Maintaining packet order is most useful for non-TCP traffic such as financial UDP data where some
expectation may exist for the data to arrive in the correct order.
Port to Port Packet Flow
Packets typically pass between LAN and telecom ports and are typically not filtered by MAC address.
13
