Configure SureLink active recovery to detect WAN failures

Problems can occur beyond the immediate WAN connection that prevent some IP traffic from reaching its destination. Normally this kind of problem does not cause the Connect EZ device to detect that the WAN has failed, because the connection continues to work while the core problem exists somewhere else in the network.

Using Digi SureLink, you can configure the Connect EZ device to regularly probe connections through the WAN to determine if the WAN has failed.

Required configuration items

Additional configuration items

To configure the Connect EZ device to regularly probe connections through the WAN:


  1. Log into the Connect EZ WebUI as a user with full Admin access rights.
  2. On the menu, click System. Under Configuration, click Device Configuration.

  3. The Configuration window is displayed.

  4. Click Network > Interfaces.
  5. Create a new WAN or select an existing one:
  6. After creating or selecting the WAN, click SureLink.

    Enable SureLink.
  7. (Optional) Change the Test interval between connectivity tests.

    Allowed values are any number of weeks, days, hours, minutes, or seconds, and take the format number{w|d|h|m|s}.

    For example, to set Interval to ten minutes, enter 10m or 600s.

    The default is 15 minutes.

  8. (Optional) If more than one test target is configured, for Success condition, select either:
    • One test passes: Only one test needs to pass for Surelink to consider an interface to be up.
    • All test pass: All tests need to pass for SureLink to consider the interface to be up.
  9. (Optional) For Pass threshold, type or select the number of times that the test must pass after failure, before the interface is determined to be working and is reinstated.
  10. (Optional) For Response timeout, type the amount of time that the device should wait for a response to a test attempt before considering it to have failed.

    Allowed values are any number of weeks, days, hours, minutes, or seconds, and take the format number{w|d|h|m|s}.

    For example, to set Response timeout to ten minutes, enter 10m or 600s.

    The default is 15 seconds.

  11. Click to expand Tests.

    By default, Test DNS servers configured for this interface is automatically configured and enabled. This test communication with DNS servers that are either provided by DHCP, or statically configured for this interface.

    1. Click .

      New tests are enabled by default. To disable, click to toggle off Enable.

    2. Type a Label for the test.

    3. Click to toggle on IPv6 if the test should apply to both IPv6 rather than IPv4.

    4. Select the Test type.

      • Ping test: Uses ICMP to determine connectivity.

        • Ping target: The type of target for the ping, one of:
          • Hostname or IP address of an external server.
            • Ping host: hostname or IP address of the server.
          • The Interface gateway. If Interface gateway is selected, an initial traceroute is sent to the hostname or IP address configured in the SureLink advanced settings, and then the first hop in that route is used for the ping test.
          • The Interface address.
          • The Interface DNS server.
        • Ping payload size: The number of bytes to send as part of the ping payload.

      • DNS test: Performs a DNS query to the named DNS server.

        • DNS server: The IP address of the DNS server.

      • HTTP test: Uses HTTP(s) GET requests to determine connectivity to the configured web server.

        • Web server: The URL of the web server.

      • Test DNS servers configured for this interface: Tests communication with DNS servers that are either provided by DHCP, or statically configured for this interface.

      • Test the interface status: Tests the current status of the interface. The test fails if the interface is down. Failing this test infers that all other tests fail.

        • Down time: The amount of time that the interface is down before the test can be considered to have failed.

          Allowed values are any number of weeks, days, hours, minutes, or seconds, and take the format number{w|d|h|m|s}.

          For example, to set Down time to ten minutes, enter 10m or 600s.

        • Initial connection time: The amount of time to wait for the interface to connect for the first time before the test is considered to have failed.

          • Allowed values are any number of weeks, days, hours, minutes, or seconds, and take the format number{w|d|h|m|s}.

          For example, to set Initial connection time to ten minutes, enter 10m or 600s.

      • Custom test: Tests the interface with custom commands.

        • The Commands to run to test.

      • TCP connection test: Tests that the interface can reach a destination port on the configured host.

        • TCP connect host: The hostname or IP address of the host to create a TCP connection to.
        • TCP connect port: The TCP port to create a TCP connection to.

      • Test another interface's status: Tests the status of another interface.

        • Test interface: The interface to test.
        • IP version: The type of IP connection, one of:
          • Any: Either the IPv4 or IPv6 connection must be up.
          • Both: Both the IPv4 or IPv6 connection must be up.
          • IPv4: The IPv4 connection must be up.
          • IPv6: The IPv6 connection must be up.
        • Expected status: The status required for the test to past.
          • Up: The test will pass only if the referenced interface is up and passing its own SureLink tests (if applicable).
          • Down: The test will pass only if the referenced interface is down or failing its own SureLink tests (if applicable).

    5. Repeat for each additional test.
  12. Add recovery actions:
    1. Click to expand Recovery actions.
    2. Click .

      New recovery actions are enabled by default. To disable, click to toggle off Enable.

    3. Type a Label for the recovery action.

    4. For Recovery type, select the type of recovery action. If multiple recovery actions are configured, they are performed in the order that they are listed.

      • Change default gateway: Increases the interface's metric to change the default gateway.

        • SureLink test failures: The number of failures for this recovery action to perform, before moving to the next recovery action.
        • Increase metric to change active default gateway: Increase the interface's metric by this amount. This should be set to a number large enough to change the routing table to use another default gateway. The default is 100.
        • Override wait interval before performing the next recovery action: The time to wait before the next test is run. If set to the default value of 0s, the Test interval is used.

      • Restart interface.

        • SureLink test failures: The number of failures for this recovery action to perform, before moving to the next recovery action.
        • Override wait interval before performing the next recovery action: The time to wait before the next test is run. If set to the default value of 0s, the Test interval is used.

      • Reset modem: This recovery action is available for WWAN interfaces only.

        • SureLink test failures: The number of failures for this recovery action to perform, before moving to the next recovery action.
        • Override wait interval before performing the next recovery action: The time to wait before the next test is run. If set to the default value of 0s, the Test interval is used.

      • Switch to alternate SIM: Switches to an alternate SIM. This recovery action is available for WWAN interfaces only.

        • SureLink test failures: The number of failures for this recovery action to perform, before moving to the next recovery action.
        • Override wait interval before performing the next recovery action: The time to wait before the next test is run. If set to the default value of 0s, the Test interval is used.

      • Reboot device.

        • SureLink test failures: The number of failures for this recovery action to perform, before moving to the next recovery action.
        • Override wait interval before performing the next recovery action: The time to wait before the next test is run. If set to the default value of 0s, the Test interval is used.

      • Execute custom Recovery commands.

        • SureLink test failures: The number of failures for this recovery action to perform, before moving to the next recovery action.
        • The Commands to run to recovery connectivity.
        • Override wait interval before performing the next recovery action: The time to wait before the next test is run. If set to the default value of 0s, the Test interval is used.

      • Powercycle the modem. This recovery action is available for WWAN interfaces only.

        • SureLink test failures: The number of failures for this recovery action to perform, before moving to the next recovery action.
        • Override wait interval before performing the next recovery action: The time to wait before the next test is run. If set to the default value of 0s, the Test interval is used.

    5. Repeat for each additional recovery action.

    1. Click to expand Advanced settings.

    2. For Delayed Start, type the amount of time to wait while the device is starting before SureLink testing begins. This setting is bypassed when the interface is determined to be up.

      Allowed values are any number of weeks, days, hours, minutes, or seconds, and take the format number{w|d|h|m|s}.

      For example, to set Delayed start to ten minutes, enter 10m or 600s.

      The default is 300 seconds.

    3. For Backoff interval, type the time to add to the test interval when restarting the list of actions. This option is capped at 15 minutes.

      Allowed values are any number of weeks, days, hours, minutes, or seconds, and take the format number{w|d|h|m|s}.

      For example, to set Backoff interval to ten minutes, enter 10m or 600s.

      The default is 300 seconds.

    4. Test interface gateway by pinging is used by the Interface gateway Ping test as the endpoint for traceroute to use to determine the interface gateway. The default is 8.8.8.8, and should only be changed if this IP address is not accessible due to networking issues.
  14. Click Apply to save the configuration and apply the change.

  1. Select the device in Remote Manager and click Actions > Open Console, or log into the Connect EZ local command line as a user with full Admin access rights.

    Depending on your device configuration, you may be presented with an Access selection menu. Type admin to access the Admin CLI.

  2. At the command line, type config to enter configuration mode:

    > config
(config)>

  3. Create a new WAN, or edit an existing one:
    • To create a new WAN, see Configure a LAN, Configure a Wide Area Network (WAN), or Configure a Wide Area Network (WAN) or Configure a Wireless Wide Area Network (WWAN).
    • To edit an existing WAN, change to the WAN's node in the configuration schema. For example, for a WAN named my_wan, change to the my_wan node in the configuration schema:
    (config)> network interface my_wan
(config network interface my_wan)>
  4. Enable SureLink.

    By default, SureLink is enabled for the preconfigured WAN. The default configuration tests the DNS servers configured for the interface.

    (config network interface my_wan)> surelink enable true
(config network interface my_wan)>

  5. By default, the Test DNS servers configured for this interface test is automatically configured and enabled. This tests communication with DNS servers that are either provided by DHCP, or statically configured for this interface.

    To add additional tests:

    1. Add a test:

      (config network interface my_wan)> add surelink tests end
(config network interface my_wan surelink tests 1)>

    2. New tests are enabled by default. To disable: 

      (config network interface my_wan surelink tests 1)> enable false
(config network interface my_wan surelink tests 1)>
    3. Create a label for the test:
      (config network interface my_wan surelink tests 1)> label string(config network interface my_wan surelink tests 1)>
    4. if the test should apply to both IPv6 rather than IPv4, enable IPv6:
      (config network interface my_wan surelink tests 1)> ipv6 true
(config network interface my_wan surelink tests 1)>

    5. Set the test type:

      (config network interface my_wan surelink tests 1)> test value
(config network interface my_wan surelink tests 1)>

      • ping: Uses ICMP to determine connectivity.

        • Set the ping_method: 

          (config network interface my_wan surelink tests 1)> ping_method value
(config network interface my_wan surelink tests 1)>

          where value is one of:

          • hostname: The hostname or IP address of an external server.

            • Set ping_host to the hostname or IP address of the server:

              (config network interface my_wan surelink tests 1)> ping_host hostname/IP_address
(config network interface my_wan surelink tests 1)>

          • interface_gateway. If set, an initial traceroute is sent to the hostname or IP address configured in the SureLink advanced settings, and then the first hop in that route is used for the ping test.

          • interface_address.

          • interface_dns: The interface's DNS server.

        • Set the number of bytes to send as part of the ping payload:

          (config network interface my_wan ipsec tunnel ipsec_example surelink tests 1)> ping_size int
(config network interface my_wan surelink tests 1)>

      • dns: Performs a DNS query to the named DNS server. 

        (config network interface my_wan surelink tests 1)> dns_server IP_address
(config network interface my_wan surelink tests 1)>

      • http: Uses HTTP(s) GET requests to determine connectivity to the configured web server. 

        • (config network interface my_wan surelink tests 1)> http url
(config network interface my_wan surelink tests 1)>

      • dns_configured: Tests communication with DNS servers that are either provided by DHCP, or statically configured for this interface.

      • interface_up: Tests the current status of the interface. The test fails if the interface is down. Failing this test infers that all other tests fail.

        • Set the amount of time that the interface is down before the test can be considered to have failed.

          (config network interface my_wan surelink tests 1)> interface_down_time value
(config network interface my_wan surelink tests 1)>

          where value is any number of weeks, days, hours, minutes, or seconds, and takes the format number{w|d|h|m|s}.

          For example, to set interface_down_time to ten minutes, enter either 10m or 600s:

          (config network interface my_wan surelink tests 1)> interface_down_time 600s
(config)>

        • Set the amount of time to wait for the interface to connect for the first time before the test is considered to have failed.

          (config network interface my_wan surelink tests 1)> interface_timeout value
(config network interface my_wan surelink tests 1)>

          where value is any number of weeks, days, hours, minutes, or seconds, and takes the format number{w|d|h|m|s}.

          For example, to set interface_timeout to ten minutes, enter either 10m or 600s:

          (config network interface my_wan surelink tests 1)> interface_timeout 600s
(config)>

      • custom_test: Tests the interface with custom commands. 

        • (config network interface my_wan  surelink tests 1)> custom_test_commands "string"
(config network interface my_wan surelink tests 1)>

      • tcp_connection: Tests that the interface can reach a destination port on the configured host.

        • Set the hostname or IP address of the host to create a TCP connection to:
          (config network interface my_wan surelink tests 1)> tcp_host hostname/IP_address
(config network interface my_wan surelink tests 1)>
        • Set the TCP port to create a TCP connection to.
          (config network interface my_wan surelink tests 1)> tcp_port port
(config network interface my_wan surelink tests 1)>

      • other: Tests the status of another interface.

        • Set the interface to test.
          2. Set the interface. For example:
            (config network interface my_wan surelink tests 1)> other_interface /network/interface/eth1
(config network interface my_wan surelink tests 1)>
        • Set the type of IP connection:
          (config network interface my_wan surelink tests 1)> other_ip_version value
(config network interface my_wan surelink tests 1)>

          where value is one of:

          • any: Either the IPv4 or IPv6 connection must be up.
          • both: Both the IPv4 or IPv6 connection must be up.
          • ipv4 The IPv4 connection must be up.
          • ipv6: The IPv6 connection must be up.
        • The status required for the test to past.
          (config network interface my_wan surelink tests 1)> other_status value
(config network interface my_wan surelink tests 1)>

          where value is one of:

          • up: The test will pass only if the referenced interface is up and passing its own SureLink tests (if applicable).
          • down: The test will pass only if the referenced interface is down or failing its own SureLink tests (if applicable).

    6. Repeat for each additional test.

  6. Add recovery actions:
    1. Type ... to return to the root of the configuration:
      (config network interface my_wan surelink tests 1)> ...
(config)>
    2. Add a recovery action:
      (config)> add network interface my_wan surelink actions end
(config network interface my_wan surelink actions 0)>

    3. New actions are enabled by default. To disable: 

      (config network interface my_wan surelink actions 0)> enable false(config network interface my_wan surelink actions 0)>
    4. Create a label for the action:
      (config network interface my_wan surelink actions 0)> label string
(config network interface my_wan surelink actions 0)>

    5. Set the type of recovery action. If multiple recovery actions are configured, they are performed in the order that they are listed. 

      (config network interface my_wan surelink actions 0)> action value
(config network interface my_wan surelink actions 0)>

      where value is one of:

      • update_routing_table: Increases the interface's metric to change the default gateway.

        • Set the number of failures for this recovery action to perform, before moving to the next recovery action:
          (config network interface my_wan surelink actions 0)> test_failures int
(config network interface my_wan surelink actions 0)>

          The default is 3.

        • Set the amount that the interface's metric should be increased. This should be set to a number large enough to change the routing table to use another default gateway.
          (config network interface my_wan surelink actions 0)> metric_adjustment_modem int
(config network interface my_wan surelink actions 0)>

          The default is 100.

        • Set the time to wait before the next test is run. If set to the default value of 0s, the test interval is used.
          (config network interface my_wan surelink actions 0)> override_interval int
(config network interface my_wan surelink actions 0)>

      • restart_interface.

          • (missing or bad snippet)(missing or bad snippet)

      • reboot_device.

        • Set the number of failures for this recovery action to perform, before moving to the next recovery action:
          (config network interface my_wan surelink actions 0)> test_failures int
(config network interface my_wan surelink actions 0)>

          The default is 3.

          • (missing or bad snippet)

      • custom_action: Execute custom recovery commands.

        • Set the number of failures for this recovery action to perform, before moving to the next recovery action:
          (config network interface my_wan surelink actions 0)> test_failures int
(config network interface my_wan surelink actions 0)>

          The default is 3.

        • Set the commands to run to attempt to recovery connectivity.
          (config network interface my_wan surelink actions 0)> custom_action_commands_modem "string"
(config network interface my_wan surelink actions 0)>
        • Set the time to wait before the next test is run. If set to the default value of 0s, the test interval is used.
          (config network interface my_wan surelink actions 0)> override_interval int
(config network interface my_wan surelink actions 0)>

    6. Repeat for each additional recovery action.
    1. Type ... to return to the root of the configuration:
      (config network interface my_wan surelink actions 0)> ...
(config)>
    2. Set the test interval between connectivity tests:
      (config)> network interface my_wan interval value(config)>

      where value is any number of weeks, days, hours, minutes, or seconds, and takes the format number{w|d|h|m|s}.

      For example, to set interval to ten minutes, enter either 10m or 600s:

      (config)> network interface my_wan surelink 600s
(config)>

      The default is 15m.

    3. If more than one test target is configured, set the success condition:
      (config)> network interface my_wan success_condition value 
(config)>

      where value is either:

      • one: Only one test needs to pass for Surelink to consider an interface to be up.
      • all: All tests need to pass for SureLink to consider the interface to be up.
    4. Set the number of times that the test must pass after failure, before the interface is determined to be working and is reinstated.
      (config)> network interface my_wan pass_threshold int
(config)>

      The default is 1.

    5. Set the amount of time that the device should wait for a response to a test attempt before considering it to have failed:
      (config)> network interface my_wan timeout value
(config)>

      where value is any number of weeks, days, hours, minutes, or seconds, and takes the format number{w|d|h|m|s}.

      For example, to set timeout to ten minutes, enter either 10m or 600s:

      (config)> network interface my_wan surelink timeout 600s
(config)>

      The default is 15s.

    6. Set the amount of time to wait while the device is starting before SureLink testing begins. This setting is bypassed when the interface is determined to be up.
      (config)> network interface my_wan advanced delayed_start value
(config)>

      where value is any number of weeks, days, hours, minutes, or seconds, and takes the format number{w|d|h|m|s}.

      For example, to set delayed_start to ten minutes, enter either 10m or 600s:

      (config)> network interface my_wan surelink advanced delayed_start 600s
(config)>

      The default is 300s.

    7. Set the time to add to the test interval when restarting the list of actions. This option is capped at 15 minutes.
      (config)> network interface my_wan advanced backoff_interval value
(config)>

      where value is any number of weeks, days, hours, minutes, or seconds, and takes the format number{w|d|h|m|s}.

      For example, to set backoff_interval to ten minutes, enter either 10m or 600s:

      (config)> network interface my_wan advanced backoff_interval 600s
(config)>

      The default is 300 seconds.

    8. The interface_gateway parameter is used by the Interface gateway Ping test as the endpoint for traceroute to use to determine the interface gateway. The default is 8.8.8.8, and should only be changed if this IP address is not accessible due to networking issues. To set to an alternate host:
      (config)> network interface my_wan advanced interface_gateway hostname/IP_address
(config)>
  8. Save the configuration and apply the change.
    (config network interface my_wan ipv4 surelink)> save
Configuration saved.
>
  9. Type exit to exit the Admin CLI.

    Depending on your device configuration, you may be presented with an Access selection menu. Type quit to disconnect from the device.