Fortigate ECMP with BGP

It’s like clock work…around this time the seasons are changing..autumn colors are out, and the colder air reminds us that things are changing around here, embrace it! One thing that I’ve been working with is AWS Transit gateways, a common theme this year is all cloud! AWS Transit Gateways make it easier to move towards that!

AWS Transit gateways relatively new, (couple years old now) so instead of attaching to each AWS VPC environment independently we can connect to the transit gateway and attach VPCs to the gateway. If you enable BGP your routes will propagate. I’ve been working with ECMP, specifically with Fortigates.

There are two ways to have redundancy in a network when you have multiple paths to a destination. Similar routes that have a higher cost/weight then ones that don’t are only used when there is a problem with primary path. So you have a backup route on standby until the primary route fails. This could be a good option if the backup route isn’t the same bandwidth/media or really not equal to the primary link which is why different cost/weight is important to have.

However if you have multiple paths that were equal bandwidth/media we could/would want to use both of them at the same time so that traffic is load balanced across the network, this is called ECMP (Equal-cost multi-path routing)

Let’s walk though the steps on how to setup ECMP on a Fortigate using BGP. I’m already guessing you have an AWS account with an AWS transit gateway deployed that supports ECMP, as well as configured your customer gateway in AWS with BGP Autonomous system (AS) number.

In this example I have a Fortigate Firewall running version 6.4.2, I have set the BGP “AS” number to 65000 and the AWS “AS” number is 64512.

Amazon gives you a text file to download the configuration for these two tunnels but depending on your Fortigate setup we may have to tweak the file, (not a good idea to just paste it) In this example here is what I have for tunnel one:

# Configure Phase 1 of the tunnel, aka IKE

config vpn ipsec phase1-interface
edit "AWS-0615bcc6f-1"
set dpd on-idle
# Outside Interface to reach AWS
set interface outside
# External IP address using (RFC 5737) in this example
set local-gw 198.51.100.1
set dhgrp 2
set proposal aes128-sha1
set keylife 28800
# Remote Peer this is AWS Transit, using (RFC 5737) in example
set remote-gw 192.0.2.124
set psksecret ciscoskills.net
set dpd-retryinterval 10
next
end

# Configure Phase 2 of the Tunnel, aka IPSEC

config vpn ipsec phase2-interface
edit "AWS-0615bcc6f-1"
set phase1name "AWS-0615bcc6f-1"
set proposal aes128-sha1
set dhgrp 2
set pfs enable
set keylifeseconds 3600
end

# Build the tunnel interface

config system interface
edit "AWS-0615bcc6f-1"
set vdom "root"
set ip 169.254.150.142 255.255.255.255
set allowaccess ping
set type tunnel
set tcp-mss 1379
set remote-ip 169.254.150.141/32
set mtu enable
set mtu 1427
set interface "outside"
next
end

After tunnel one is created, AWS gives us configuration for the second tunnel, basically the same configuration with some differences, here is tunnel two.

# Configure Phase 1 of the tunnel, aka IKE

config vpn ipsec phase1-interface
edit "AWS-0615bcc6f-2"
set dpd on-idle
# Outside Interface to reach AWS
set interface outside
# External IP address using (RFC 5737) in this example
set local-gw 198.51.100.1
set dhgrp 2
set proposal aes128-sha1
set keylife 28800
# Remote Peer this is AWS Transit, using (RFC 5737) in example
set remote-gw 203.0.113.25
set psksecret ciscoskills.com
set dpd-retryinterval 10
next
end

# Configure Phase 2 of the Tunnel, aka IPSEC

config vpn ipsec phase2-interface
edit "AWS-0615bcc6f-2"
set phase1name "AWS-0615bcc6f-2"
set proposal aes128-sha1
set dhgrp 2
set pfs enable
set keylifeseconds 3600
end

# Build the tunnel interface
config system interface
edit "AWS-0615bcc6f-2"
set vdom "root"
set ip 169.254.157.90 255.255.255.255
set allowaccess ping
set type tunnel
set tcp-mss 1379
set remote-ip 169.254.157.89/32
set mtu enable
set mtu 1427
set interface "outside"
next
end

We now have two tunnels configured, these won’t show online until the Fortigate has a policy attached to them. So you could put firewall rules on each tunnel interface or better yet is to create a zone within the Fortigate and add the two tunnel interfaces into same zone. By creating a zone within Fortigate the same firewall rules apply to each interface instead of creating/copying rules to each tunnel interface. (This would be an example)

config system zone
edit AWS-0615bcc6f
set interface AWS-0615bcc6f-1
set interface AWS-0615bcc6f-2
set intrazone allow
next
end

Once those interfaces are either in a zone and in a Firewall policy you should be able to bring those tunnel interfaces online. The final thing is configure our Fortigate with BGP so that when you attach a VPC to the transit gateway that IP space gets advertised automatically. We also want to enable ECMP with BGP.

# Enable BGP on the Fortigate

config router bgp
# This is the number we set in AWS, so it should match what TG has
set as 65000
set router-id 198.51.100.1

# Since we are using "AS" Numbers, we tell the Fortigate to use
# External Multipath with out it BGP will use one route at time
# Active/Passive

set ebgp-multipath enable
config neighbor
edit "169.254.157.89"
set remote-as 64512
next
edit "169.254.150.141"
set remote-as 64512
next
end

If we attached a VPC to the transit gateway, and in this example I attached a VPC range of 10.0.0.0/16 to the transit gateway. That IP range would show up in the Fortigate, if we look at the routing table via get router info routing-table all

FW01 # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
- candidate default

Routing table for VRF=0
S* 0.0.0.0/0 [5/0] via 198.51.100.25, ppp1
B 10.0.0.0/16 [20/100] via 169.254.150.141, AWS-0615bcc6f-1, 03:07:57
[20/100] via 169.254.157.89, AWS-0615bcc6f-2, 03:07:57
C 198.51.100.1/32 is directly connected, ppp1
C 169.254.150.141/32 is directly connected, AWS-0615bcc6f-1
C 169.254.150.142/32 is directly connected, AWS-0615bcc6f-1
C 169.254.157.89/32 is directly connected, AWS-0615bcc6f-2
C 169.254.157.90/32 is directly connected, AWS-0615bcc6f-2

We can see that 10.0.0.0/16 is available to us via the VPN interfaces, we have equal paths as well, traffic will load balance between each tunnel interface. BGP with ECMP is alive and active! … That’s all I got for this post, like always I hope this information is helpful, got questions? Comment below!

Leave a Reply...

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s