Fortigate ECMP with BGP

Share on:

It's like clock work...around this time the seasons are changing..autumn colors are out, and the colder air reminds us that things are changing around here, embrace it! One thing that I've been working with is AWS Transit gateways, a common theme this year is all cloud! AWS Transit Gateways make it easier to move towards that!

AWS Transit gateways relatively new, (couple years old now) so instead of attaching to each AWS VPC environment independently we can connect to the transit gateway and attach VPCs to the gateway. If you enable BGP your routes will propagate. I've been working with ECMP, specifically with Fortigates.

There are two ways to have redundancy in a network when you have multiple paths to a destination. Similar routes that have a higher cost/weight then ones that don't are only used when there is a problem with primary path. So you have a backup route on standby until the primary route fails. This could be a good option if the backup route isn't the same bandwidth/media or really not equal to the primary link which is why different cost/weight is important to have.

However if you have multiple paths that were equal bandwidth/media we could/would want to use both of them at the same time so that traffic is load balanced across the network, this is called ECMP (Equal-cost multi-path routing)

Let's walk though the steps on how to setup ECMP on a Fortigate using BGP. I'm already guessing you have an AWS account with an AWS transit gateway deployed that supports ECMP, as well as configured your customer gateway in AWS with BGP Autonomous system (AS) number.

In this example I have a Fortigate Firewall running version 6.4.2, I have set the BGP "AS" number to 65000 and the AWS "AS" number is 64512.

Amazon gives you a text file to download the configuration for these two tunnels but depending on your Fortigate setup we may have to tweak the file, (not a good idea to just paste it) In this example here is what I have for tunnel one:

 1# Configure Phase 1 of the tunnel, aka IKE  
 2config vpn ipsec phase1-interface  
 3edit "AWS-0615bcc6f-1"  
 4set dpd on-idle  
 5
 6# Outside Interface to reach AWS  
 7set interface outside  
 8
 9# External IP address using (RFC 5737) in this example
10set local-gw 198.51.100.1  
11set dhgrp 2  
12set proposal aes128-sha1  
13set keylife 28800  
14
15# Remote Peer this is AWS Transit, using (RFC 5737) in example`  
16set remote-gw 192.0.2.124  
17set psksecret ciscoskills.net  
18set dpd-retryinterval 10  
19next  
20end  
21  
22# Configure Phase 2 of the Tunnel, aka IPSEC  
23config vpn ipsec phase2-interface  
24edit "AWS-0615bcc6f-1"  
25set phase1name "AWS-0615bcc6f-1"  
26set proposal aes128-sha1  
27set dhgrp 2  
28set pfs enable  
29set keylifeseconds 3600  
30end
31
32# Build the tunnel interface  
33config system interface   
34edit "AWS-0615bcc6f-1"  
35set vdom "root"  
36set ip 169.254.150.142 255.255.255.255  
37set allowaccess ping  
38set type tunnel  
39set tcp-mss 1379  
40set remote-ip 169.254.150.141/32  
41set mtu enable  
42set mtu 1427  
43set interface "outside"  
44next  
45end

After the tunnel one is created, AWS gives us configuration for the second tunnel, basically the same configuration with some differences, here is tunnel two.

 1# Configure Phase 1 of the tunnel, aka IKE  
 2  
 3config vpn ipsec phase1-interface  
 4edit "AWS-0615bcc6f-2"  
 5set dpd on-idle  
 6# Outside Interface to reach AWS  
 7set interface outside  
 8# External IP address using (RFC 5737) in this example  
 9set local-gw 198.51.100.1  
10set dhgrp 2  
11set proposal aes128-sha1  
12set keylife 28800  
13# Remote Peer this is AWS Transit, using (RFC 5737) in example  
14set remote-gw 203.0.113.25  
15set psksecret ciscoskills.com  
16set dpd-retryinterval 10  
17next  
18end  
19  
20# Configure Phase 2 of the Tunnel, aka IPSEC  
21  
22config vpn ipsec phase2-interface  
23edit "AWS-0615bcc6f-2"  
24set phase1name "AWS-0615bcc6f-2"  
25set proposal aes128-sha1  
26set dhgrp 2  
27set pfs enable  
28set keylifeseconds 3600  
29end  
30  
31# Build the tunnel interface  
32config system interface  
33edit "AWS-0615bcc6f-2"  
34set vdom "root"  
35set ip 169.254.157.90 255.255.255.255  
36set allowaccess ping  
37set type tunnel  
38set tcp-mss 1379  
39set remote-ip 169.254.157.89/32  
40set mtu enable  
41set mtu 1427  
42set interface "outside"  
43next  
44end

We now have two tunnels configured, these won't show online until the Fortigate has a policy attached to them. So you could put firewall rules on each tunnel interface or better yet is to create a zone within the Fortigate and add the two tunnel interfaces into same zone. By creating a zone within Fortigate the same firewall rules apply to each interface instead of creating/copying rules to each tunnel interface. (This would be an example)

1config system zone  
2edit AWS-0615bcc6f  
3set interface AWS-0615bcc6f-1  
4set interface AWS-0615bcc6f-2  
5set intrazone allow  
6next  
7end

Once those interfaces are either in a zone and in a Firewall policy you should be able to bring those tunnel interfaces online. The final thing is configure our Fortigate with BGP so that when you attach a VPC to the transit gateway that IP space gets advertised automatically. We also want to enable ECMP with BGP.

 1# Enable BGP on the Fortigate  
 2  
 3config router bgp  
 4# This is the number we set in AWS, so it should match what TG has  
 5set as 65000   
 6set router-id `198.51.100.1`  
 7  
 8# Since we are using "AS" Numbers, we tell the Fortigate to use    
 9# External Multipath with out it BGP will use one route at time  
10# Active/Passive  
11  
12set ebgp-multipath enable  
13config neighbor  
14edit "169.254.157.89"  
15set remote-as 64512  
16next  
17edit "169.254.150.141"  
18set remote-as 64512  
19next  
20end

If we attached a VPC to the transit gateway, and in this example I attached a VPC range of 10.0.0.0/16 to the transit gateway. That IP range would show up in the Fortigate, if we look at the routing table via get router info routing-table all

 1FW01 # get router info routing-table all  
 2Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP  
 3O - OSPF, IA - OSPF inter area  
 4N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2  
 5E1 - OSPF external type 1, E2 - OSPF external type 2  
 6i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area  
 7- candidate default  
 8  
 9Routing table for VRF=0  
10S* 0.0.0.0/0 [5/0] via `198.51.100.`25, ppp1  
11B 10.0.0.0/16 [20/100] via 169.254.150.141, AWS-0615bcc6f-1, 03:07:57  
12              [20/100] via 169.254.157.89, AWS-0615bcc6f-2, 03:07:57  
13C `198.51.100.1`/32 is directly connected, ppp1  
14C 169.254.150.141/32 is directly connected, AWS-0615bcc6f-1  
15C 169.254.150.142/32 is directly connected, AWS-0615bcc6f-1  
16C 169.254.157.89/32 is directly connected, AWS-0615bcc6f-2  
17C 169.254.157.90/32 is directly connected, AWS-0615bcc6f-2

We can see that 10.0.0.0/16 is available to us via the VPN interfaces, we have equal paths as well, traffic will load balance between each tunnel interface. BGP with ECMP is alive and active! ... That's all I got for this post, like always I hope this information is helpful, got questions? Comment below!