Saturday, September 9, 2017

Fortigate FortiOS 5.6 Redundant Internet Connections without SD-WAN

The scenario:
ISP 1 is on wan1, your IP is 1.1.1.2, gateway is 1.1.1.1
ISP 2 is on wan2, your IP is 2.2.2.2, gateway is 2.2.2.1
This configuration will be set up in a primary/secondary failover mode, i will not be doing load balancing

1. Setup gateway detection on wan1

This section tells the firewall to ping an IP (in this case, our gateway) repeatedly to see if it's alive or not. For most cable and DSL type connections this will work fine, however if you are interfacing with another ISP's router that may continue responding to pings even if the circuit is down, then consider using an internet IP such as 4.2.2.2 or 8.8.8.8. You can do this on both wan interfaces if you want it to fail both directions.
config router gwdetect
    edit 1
        set interface "wan1"
        set server "1.1.1.1"
    next
end

On newer firmware, this is replaced with link monitoring:
config system link-monitor
    edit "wan1 check"
        set srcintf "wan1"
        set server "1.1.1.1"
        set gateway-ip 1.1.1.1
        set timeout 5
        set recoverytime 1
    next
end

2. Configure your interfaces

Most of this can be done in the web interface, the only critical piece here are the two fail-detect options on wan1. You can also turn on failure detection for wan2 (make sure you add a gwdetect for it as well), however i don't find it necessary here since if both wan1 and wan2 are down then i'm screwed anyways. Note that the fail-detect options are not necessary on newer firmware.
config system interface
    edit "wan1"
        set ip 1.1.1.2 255.255.255.248
        set fail-detect enable
        set fail-detect-option detectserver
    next
    edit "wan2"
        set ip 2.2.2.2 255.255.255.248
    next
end

3. Configure your static routes

These can be done in the web interface, the important piece is to set the distance on wan2 to a higher value than wan1 so that wan1 is always the preferred route
config router static
    edit 1
        set device "wan1"
        set gateway 1.1.1.1
    next
    edit 2
        set device "wan2"
        set distance 20
        set gateway 2.2.2.1
    next
end

4. Configure Policies

This doesn't need to be done in the CLI, you just need to have an internal to wan1 allow/NAT rule, as well as an internal to wan2 allow/NAT rule. It will choose the proper rule based on which port the traffic is being routed through (the active internet connection). In case you want to see it in CLI, here it is:
config firewall policy
    edit 1
        set srcintf "internal"
        set dstintf "wan1"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set nat enable
    next
    edit 2
        set srcintf "internal"
        set dstintf "wan2"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set nat enable
    next
end

5. Test and confirm

At this point you should be up and running. You can check the status of the gateway detection using get router info gwdetect. You can also check the routing table status using get router info routing-table all.
It is important that when you do your testing you simulate a logical failure of the circuit so your wan1 stays "up". The easy way to do this is to unplug the carrier side of the equipment (unplug the coax for cable, phone line for DSL, T1/loops for T1/EoC/EoDS1, etc.)

6. VPN setup

If you have IPSEC VPN tunnels and want them to also follow the redundancy here, then there is a little more work to do. This is assuming you already have the tunnel set up on wan1. You would just create the tunnel identically on wan2, selecting wan2 as the local interface. Make sure both tunnels have DPD (Dead Peer Detection) turned on.
After setting up the second tunnel, you will add an IPSEC policy from internal to wan2 using the new tunnel configuration. You will of course have to configure the secondary tunnel on the other end as well. After doing this, the VPN should follow the active interface

7. Using VIPs on backup links

One small quirk, if your default route points out wan1 for example, but you want to have servers using Virtual IPs on wan2 while wan1 is still active, you need to do this:
config system settings
  set asymroute enable
end
This is necessary because the Fortigate will think traffic coming in over wan2 is spoofed unless the default route is pointing there. It would be preferable to leave this feature disabled but this is really the only way to accomplish having both circuits live at the same time.