251
Security protocols and encapsulation modes
Security protocols
IPsec comes with two security protocols, AH and ESP. They define how to encapsulate IP packets and the
security services that they can provide.
•
AH (protocol 51) defines the encapsulation of the AH header in an IP packet, as shown in
.
AH can provide data origin authentication, data integrity, and anti-replay services to prevent data
tampering, but it cannot prevent eavesdropping. Therefore, it is suitable for transmitting
non-confidential data. AH supports authentication algorithms HMAC-MD5 and HMAC-SHA1.
•
ESP (protocol 50) defines the encapsulation of the ESP header and trailer in an IP packet, as shown
in
. ESP can provide data encryption, data origin authentication, data integrity, and
anti-replay services. Unlike AH, ESP can guarantee data confidentiality because it can encrypt the
data before encapsulating the data to IP packets. ESP supports encryption algorithms such as DES,
3DES, and AES, and authentication algorithms HMAC-MD5 and HMAC-SHA1.
Both AH and ESP provide authentication services, but the authentication service provided by AH is
stronger. In practice, you can choose either or both security protocols. When both AH and ESP are used,
an IP packet is encapsulated first by ESP and then by AH.
Encapsulation modes
IPsec supports the following encapsulation modes:
•
Transport mode
—The security protocols protect the upper layer data of an IP packet. Only the
transport layer data is used to calculate the security protocol headers. The calculated security
protocol headers and the encrypted data (only for ESP encapsulation) are placed after the original
IP header. You can use the transport mode when end-to-end security protection is required (the
secured transmission start and end points are the actual start and end points of the data). The
transport mode is typically used for protecting host-to-host communications, as shown in
.
Figure 78
IPsec protection in transport mode
•
Tunnel mode
—The security protocols protect the entire IP packet. The entire IP packet is used to
calculate the security protocol headers. The calculated security protocol headers and the encrypted
data (only for ESP encapsulation) are encapsulated in a new IP packet. In this mode, the
encapsulated packet has two IP headers. The inner IP header is the original IP header. The outer IP
header is added by the network device that provides the IPsec service. You must use the tunnel
mode when the secured transmission start and end points are not the actual start and end points of
the data packets (for example, when two gateways provide IPsec but the data start and end points
are two hosts behind the gateways). The tunnel mode is typically used for protecting
gateway-to-gateway communications, as shown in
Figure 79
IPsec protection in tunnel mode
Host A
Host B
Data flow
IPsec tunnel
Host A
Host B
Gateway A
Gateway B
Data flow
IPsec tunnel
