IPSEC notes

This post contains short notes about VPN technology.

VPN types



IPSEC algorithms

IPsec is a framework of open standards (protocols) used for secure communications. The framework allows technologies to be replaced over time.

The following picture shows the algorithms used by IPSEC. The next sections will explain “where and when” the specific algorithms are involved within IPSEC process.

IPSEC algorithms

IPSEC algorithms


IPSEC uses IKE (Internet Key Exchange). IKE performs the following tasks:

  1. negotiates ISKAMP SA (security association [tunnel]) parameters
  2. authenticate peers
  3. generate encryption keys (and automatic key refresh)

There are two versions of IKE – IKEv1 and IKEv2

IKEv1 consist of two phases

  1. Phase 1 – ISKAMP SA (security association) is created. It is a bidirectional secure channel created over unsecured network. The channel is used to negotiate data channels (IPSEC SAs).
  2. Phase 2 – two unidirectional IPSEC  SAs are created. There are two IPSEC flavors: ESP and AH. Nowadays IPSEC ESP is mostly used.

IKEv2 – overcome some limitations of IKE

  1. speed
  2. NAT traversal
  3. support EAP for auth. VPN endpoints; EAP allows different authentication mechanisms to be used between VPN peers (a VPN client may authenticate the VPN server using the server’s digital certificate while the server authenticates the client via a preshared key).
  4. support L3 roaming (mobile users can change IP address without disconnecting IPSEC session)

Note that in many places these words are used interchangeably:
IKE phase 1 = ISKAMP
IKE phase 2 = IPSEC

IKEv1 – phase 1

The tasks performed by IKEv1 phase 1:

  1. negotiate ISKAMP SA security parameters (all parameters must match for peers except SA lifetime)
    • Encryption algorithm (ex. AES)
    • Hash algorithm (ex. sha256). This algorithm will be used by HMAC (Hashed Message Authentication Code). HMAC uses hash function in combination with a secret shared key to authenticate messages.
    • Diffie-Hellman group number (ex. group 5); the DH group indicates how strong (long) keys are used. DH alg. is responsible for creating secure channel over insecure network. In this channel shared secret keys can be generated and used for further encryption.
    • Peer authentication method (ex. PSK)
    • SA lifetime (ex. 43200 sec.)
  2. create secure association over insecure network
  3. authenticate peers

The steps used by IKEv1 phase 1:

  1. Negotiate security parameters (all must match except SA lifetime); the initiator sends a list of policy proposals to the responder (it accepts it or not)
  2. Diffie Hellman key agreement is implemented (“generate public/private nonces [integers] for both sides”); encrypted channel is created – after the DH key exchange is complete, shared cryptographic keys are provisioned.
  3. Peer authentication (PSK or RSA signatures [digital certs])

If the peer authentication succeeds, the ISAKMP SA provides a secure tunnel in which to negotiate Phase 2 IPsec SAs.

IKEv1 – phase 2

The tasks performed by IKEv1 phase 2:

  1. negotiates IPsec security parameters (IPsec transform sets); encryption and integrity alg. used for SAs
  2. creates IPsec SAs (and renegotiates it)
  3. optionally, it can performs its own  additional DH key exchange (it is called PFS – perfect forward secrecy)

IPSEC flavors

There are two IPSEC frameworks. It means that IPSEC tunnel (also konwn as IKE phase 2 tunnel or data SAs) can be implemented in two different ways

  1. AH
    • tunnel mode
    • transport mode
  2. ESP (mostly used nowadys)
    • tunnel mode
    • transport mode

ESP and AH operates in the Transport OSI layer. AH is IP protocol 51 and ESP is IP protocol 50.

AH features

  1. data integrity (has) and authentication (shared-secret key)
  2. anti-replay attack
  3. no encryption (that is why AH is not used rarely used now)
  4. AH is not compatible with NAT performed in the transmission path – NAT changes the IP addresses in the IP header, causing AH data integrity checks to fail.

The following pictures shows AS packet (transport mode) – IP header is not changed; AH header is put between original IP header and data payload.



ESP features

  1. data integrity
  2. anti-replay attack
  3. encryption – in transport mode only original data is encrypted; in tunnel mode new IP header is created (original data and old IP header are both encrypted)
  4. IP header is not changed in transport mode; new IP header is created in tunnel mode


Tunnel mode provides a new IP header. Transport mode maintains the original IP header.

Note: ESP transport mode can be used for site-to-site VPN if another technology, such as GRE tunneling, is used to provide the outer IP header.


IKEv2 creates SAs without phases. The process of creating of IKEv2 tunnel:

  1. IKE_SA_INIT – the peers establish secure channel; all further communication is encrypted
  2. IKE_AUTH – the peers authenticate each other; both peers build IPSEC SA
  3. CREATE_CHILD_SA – each line in the crypto ACL (match address) needs its own IPSEC SA; if there are multiple ACLs then additional SAs are created

Leave a Reply

Your email address will not be published. Required fields are marked *