Fortigate ECMP with BGP
It's like clock work...around this time the seasons are changing..autumn colors are out, and the colder air reminds us that things are changing around here, embrace it! One thing that I've been working with is AWS Transit gateways, a common theme this year is all cloud! AWS Transit Gateways make it easier to move towards that!
AWS Transit gateways relatively new, (couple years old now) so instead of attaching to each AWS VPC environment independently we can connect to the transit gateway and attach VPCs to the gateway. If you enable BGP your routes will propagate. I've been working with ECMP, specifically with Fortigates.
There are two ways to have redundancy in a network when you have multiple paths to a destination. Similar routes that have a higher cost/weight then ones that don't are only used when there is a problem with primary path. So you have a backup route on standby until the primary route fails. This could be a good option if the backup route isn't the same bandwidth/media or really not equal to the primary link which is why different cost/weight is important to have.
However if you have multiple paths that were equal bandwidth/media we could/would want to use both of them at the same time so that traffic is load balanced across the network, this is called ECMP (Equal-cost multi-path routing)
Let's walk though the steps on how to setup ECMP on a Fortigate using BGP. I'm already guessing you have an AWS account with an AWS transit gateway deployed that supports ECMP, as well as configured your customer gateway in AWS with BGP Autonomous system (AS) number.
In this example I have a Fortigate Firewall running version 6.4.2, I have set the BGP "AS" number to 65000 and the AWS "AS" number is 64512.
Amazon gives you a text file to download the configuration for these two tunnels but depending on your Fortigate setup we may have to tweak the file, (not a good idea to just paste it) In this example here is what I have for tunnel one:
1# Configure Phase 1 of the tunnel, aka IKE 2config vpn ipsec phase1-interface 3edit "AWS-0615bcc6f-1" 4set dpd on-idle 5 6# Outside Interface to reach AWS 7set interface outside 8 9# External IP address using (RFC 5737) in this example 10set local-gw 198.51.100.1 11set dhgrp 2 12set proposal aes128-sha1 13set keylife 28800 14 15# Remote Peer this is AWS Transit, using (RFC 5737) in example` 16set remote-gw 192.0.2.124 17set psksecret ciscoskills.net 18set dpd-retryinterval 10 19next 20end 21 22# Configure Phase 2 of the Tunnel, aka IPSEC 23config vpn ipsec phase2-interface 24edit "AWS-0615bcc6f-1" 25set phase1name "AWS-0615bcc6f-1" 26set proposal aes128-sha1 27set dhgrp 2 28set pfs enable 29set keylifeseconds 3600 30end 31 32# Build the tunnel interface 33config system interface 34edit "AWS-0615bcc6f-1" 35set vdom "root" 36set ip 169.254.150.142 255.255.255.255 37set allowaccess ping 38set type tunnel 39set tcp-mss 1379 40set remote-ip 169.254.150.141/32 41set mtu enable 42set mtu 1427 43set interface "outside" 44next 45end
After the tunnel one is created, AWS gives us configuration for the second tunnel, basically the same configuration with some differences, here is tunnel two.
1# Configure Phase 1 of the tunnel, aka IKE 2 3config vpn ipsec phase1-interface 4edit "AWS-0615bcc6f-2" 5set dpd on-idle 6# Outside Interface to reach AWS 7set interface outside 8# External IP address using (RFC 5737) in this example 9set local-gw 198.51.100.1 10set dhgrp 2 11set proposal aes128-sha1 12set keylife 28800 13# Remote Peer this is AWS Transit, using (RFC 5737) in example 14set remote-gw 203.0.113.25 15set psksecret ciscoskills.com 16set dpd-retryinterval 10 17next 18end 19 20# Configure Phase 2 of the Tunnel, aka IPSEC 21 22config vpn ipsec phase2-interface 23edit "AWS-0615bcc6f-2" 24set phase1name "AWS-0615bcc6f-2" 25set proposal aes128-sha1 26set dhgrp 2 27set pfs enable 28set keylifeseconds 3600 29end 30 31# Build the tunnel interface 32config system interface 33edit "AWS-0615bcc6f-2" 34set vdom "root" 35set ip 169.254.157.90 255.255.255.255 36set allowaccess ping 37set type tunnel 38set tcp-mss 1379 39set remote-ip 169.254.157.89/32 40set mtu enable 41set mtu 1427 42set interface "outside" 43next 44end
We now have two tunnels configured, these won't show online until the Fortigate has a policy attached to them. So you could put firewall rules on each tunnel interface or better yet is to create a zone within the Fortigate and add the two tunnel interfaces into same zone. By creating a zone within Fortigate the same firewall rules apply to each interface instead of creating/copying rules to each tunnel interface. (This would be an example)
1config system zone 2edit AWS-0615bcc6f 3set interface AWS-0615bcc6f-1 4set interface AWS-0615bcc6f-2 5set intrazone allow 6next 7end
Once those interfaces are either in a zone and in a Firewall policy you should be able to bring those tunnel interfaces online. The final thing is configure our Fortigate with BGP so that when you attach a VPC to the transit gateway that IP space gets advertised automatically. We also want to enable ECMP with BGP.
1# Enable BGP on the Fortigate 2 3config router bgp 4# This is the number we set in AWS, so it should match what TG has 5set as 65000 6set router-id `198.51.100.1` 7 8# Since we are using "AS" Numbers, we tell the Fortigate to use 9# External Multipath with out it BGP will use one route at time 10# Active/Passive 11 12set ebgp-multipath enable 13config neighbor 14edit "169.254.157.89" 15set remote-as 64512 16next 17edit "169.254.150.141" 18set remote-as 64512 19next 20end
If we attached a VPC to the transit gateway, and in this example I attached a VPC range of 10.0.0.0/16 to the transit gateway. That IP range would show up in the Fortigate, if we look at the routing table via get router info routing-table all
1FW01 # get router info routing-table all 2Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP 3O - OSPF, IA - OSPF inter area 4N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 5E1 - OSPF external type 1, E2 - OSPF external type 2 6i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area 7- candidate default 8 9Routing table for VRF=0 10S* 0.0.0.0/0 [5/0] via `198.51.100.`25, ppp1 11B 10.0.0.0/16 [20/100] via 169.254.150.141, AWS-0615bcc6f-1, 03:07:57 12 [20/100] via 169.254.157.89, AWS-0615bcc6f-2, 03:07:57 13C `198.51.100.1`/32 is directly connected, ppp1 14C 169.254.150.141/32 is directly connected, AWS-0615bcc6f-1 15C 169.254.150.142/32 is directly connected, AWS-0615bcc6f-1 16C 169.254.157.89/32 is directly connected, AWS-0615bcc6f-2 17C 169.254.157.90/32 is directly connected, AWS-0615bcc6f-2
We can see that 10.0.0.0/16 is available to us via the VPN interfaces, we have equal paths as well, traffic will load balance between each tunnel interface. BGP with ECMP is alive and active! ... That's all I got for this post, like always I hope this information is helpful, got questions? Comment below!