(Resolved) 2017-11-22 14:00-16:00 UTC: BGP Reachability alerts

Last updated: Thu Nov 30 20:42:56 GMT 2017

Issue Description
ThousandEyes platform issued several BGP reachability alerts for prefix reachability, and customers may have noticed dips in reachability due to missing routing tables for certain BGP monitors. This issue has occurred at the following time:

  • 2017-11-22 14:00-16:00 UTC 

Background
As described here, ThousandEyes syndicates data made available by the Route Views project to show prefix reachability from a number of global vantage points (BGP monitors). Data is extracted from full routing tables which are published every two hours, as well as updates published every 15 minutes. BGP view metrics for prefix reachability, updates and path changes are based on information extracted from these routing tables. When a Routing Information Base (RIB) file (either a full table or an update) is not available, reachability of monitored prefixes drops for each monitor contained in that RIB, since the absence of a routing table is interpreted to mean that the prefix is unreachable.

What happened
The rv/oreg endpoint would periodically publish a truncated RIB file in the full routing table dump which occurs every 2 hours.  During processing, we would identify certain prefixes as having no routes in the routing tables, therefore would show that those prefixes as having been withdrawn.  Alerts (if configured) would trigger based on lack of reachability.

Current Status
Our development team is evaluating the implementation of additional detection mechanisms when it comes to handling truncated RIB files from Route Views. We will keep this announcement updated for any decision taken on the matter.