F-Root DNS server moved to Beijing

Posted by Andree Toonk - October 3, 2011 - Hijack - 6 Comments

F-Root DNS server moved to Beijing
Systems such as DNS (root) servers often rely on anycast technology to improve availability and response time. The idea behind anycast is that the same prefix is announced from multiple geographically separated systems. As a result the client should always end-up at the closest (as seen from a BGP perspective) server.

Often these anycast nodes are deployed to only serve specific network and/or regional areas, making them only reachable from the networks where they are deployed. In the case of the F-root server, operated by ISC, there are quite a few local servers. See this page for a detailed list: https://www.isc.org/community/f-root/sites One of these servers is the F-root located in China.

This weekend for about 25 hours this particular root server was reachable not only from China, but from several places in the world when using IPv6. Normally this shouldn’t be a real problem, as the root servers return the same DNS response regardless of their geographical location. However in the case of China, all traffic typically passes through the great firewall where DNS responses for requests such as facebook.com are often rewritten to different false IP addresses. Please note that I am not aware of any data to confirm if this is what happened in this particular case!

Technical details
This weekend between 01 Oct 2011 17:56:12 and 02 Oct 2011 19:01:09 GMT a number of providers thought that the best IPv6 path to the F-root server (AS3557), was to the F-root in China.

The prefix for the F-root server is 2001:500:2f::/48 and is originated by the following AS numbers:
3557 ISC-F-ROOT Internet Systems Consortium, Inc
1280 ISC-AS1280 Internet Systems Consortium, Inc.
30132 ISC-AMS1 Internet Systems Consortium, Inc

By looking at the next hop AS number we can get a good feeling of where the server for this particular announcement resides. When analyzing the announcement for 2001:500:2f::/48 we saw one new AS we had not seen before outside of China, AS55439.

When looking at all announcements for the F-root in China, we saw the following constant in the ASpath:
6939 23911 18344 37944 24151 55439 3557

In this case AS55439 is the AS for ISC-PEK2 Internet Systems Consortium, (Beijing, China). Next the announcement was passed along to AS24151 (China Internet Network Information Center), AS37944 (CHINA SCIENCE AND TECHNOLOGY NETWORK), AS18344 (CNCGROUP-BACKBONE-NORTH), it then ended up at the AS23911, the China Next Generation Internet Beijing IX. That’s where it was learned by AS6939 Hurricane Electric, the world’s largest IPv6 provider. Hurricane electric then advertised it to its customers.

The dataset I used showed that numerous Hurricane electric customers in countries worldwide, including for example Germany, South Africa, Brazil and the US selected this path to China. Some of the large providers include AT&T, Telia and Interoute.

What does all this mean
It’s likely that DNS queries over IPv6 to the F-root server from clients worldwide were answered by the F-root in China. Although this is in itself is not necessarily a problem, it does expose this traffic to the Great Chinese Firewall with the risk of altering the responses.

This is not the first time this happens, last year Renesys reported about a similar event involving the I-root server and China.

The only way to make sure that DNS responses are not rewritten is to use DNSSEC, this will allow the resolver to verify the authenticity of the response.
With regards to BGP, in this case all that seems to have happened was a BGP leak. Things like this are typically the result of an operator mistake. Best practice is to monitor for these kinds of things, so you’ll be alerted as soon as it happens and as a result allowing operators to limit the effect of such an event. BGPmon.net provides users with several flexible ways to monitor BGP announcement and to check if they match your BGP policies.

6 comments

  • You do not mention the IPv6 specificities, while I believe they play an important part here. In the IPv4 Internet, the AS path length (the main decision factor of BGP, when it chooses a route) is more or less very roughly correlated to the physical distance. It does not change often so routes remain more or less stable.

    But the IPv6 Internet is much more flat. Everyone peers with Hurricane Electric, de facto hub of the v6 Internet and therefore the average AS path length is much shorter, leading to easier route switchings. It is quite possible that it is what happened here: in the flat IPv6 Internet, a very small change can dramatically alter the catchment of an anycast instance.

    See a RIPE survey about the difference of typical AS path lengths between IPv4 and IPv6 http://labs.ripe.net/Members/mirjam/interesting-graph-as-path-lengths

    Editor’s Note:
    Hi Stéphane, excellent comment.
    In this specific case though, the ASpath was really quite long, much longer than the average ASpath length. For some reason HE preferred the much longer path over the path they normally prefer: 6939 1280 3557. This is the path HE used before and after the event.
    I can only imagine that HE lost the path via AS1280, or for other reasons (local pref?) preferred this much longer path.
    You’re absolutely right that many providers rely (only) on HE for IPv6 connectivity, increasing the chances of using the host in Beijing.

  • Peter Losher says:

    Another thing was that the title is misleading – F-Root is served from over 55 sites worldwide, mostly serving local markets, with over a third of them IPv6 enabled. Stating in the tittle that F-Root “was moved to Beijing” is wrong – F-Root has been in Beijing for over five years serving the Chinese ISP community. This looks to have been a simple IPv6 route leak of F’s IPv6 route by one of the upstreams there which ISC has pulled and is investigating. Also, ISC has no evidence that there was any responses from the node from Beijing that were altered.

    Editor’s Note:
    please note that I explicitly stated in this blog that I am not aware of any data indicating that responses from the Beijing node were altered. I do highlight that given the fact that traffic likely passes through the ‘great firewall’ does leave these responses vulnerable to rewriting by the great Chinese firewall.

  • Peter Losher says:

    Also, more information on how ISC announces F can be found here: http://www.isc.org/community/blog/201008/f-root-routing-how-does-it-work-0

  • [...] post: F-Root DNS server moved to Beijing | BGPmon.net Blog Posted in 4, 55, and, china, dns, for, improve, in, internet, is, it, me, na, net, no, of, root, [...]

  • fredan says:

    In witch routing databases is AS3557, AS1280 and AS30132 allowed to announce the prefix 2001:500:2f::/48?

    Editor’s note:

    This is based on the data in the BGPmon.net databases. Looking at recent BGP announcements world wide, we see the prefix being originated from these 3 ASn’s. You can check this yourself using:


    $ whois -h whois.bgpmon.net 2001:500:2f::f

    % This is the BGPmon.net whois Service
    % You can use this whois gateway to retrieve information
    % about an IP adress or prefix
    % We support both IPv4 and IPv6 address.
    %
    % For more information visit:
    % http://bgpmon.net/bgpmonapi.php

    Prefix: 2001:500:2f::/48
    Prefix description: Internet Systems Consortium, Inc. aka ISC
    Country code: US
    Origin AS: 3557
    Origin AS Name: ISC-F-ROOT Internet Systems Consortium, Inc.
    RPKI status: No ROA found

    Prefix: 2001:500:2f::/48
    Prefix description: Internet Systems Consortium, Inc. aka ISC
    Country code: US
    Origin AS: 1280
    Origin AS Name: ISC-AS1280 Internet Systems Consortium, Inc.
    RPKI status: No ROA found

    Prefix: 2001:500:2f::/48
    Prefix description: Internet Systems Consortium, Inc. aka ISC
    Country code: US
    Origin AS: 30132
    Origin AS Name: ISC-AMS1 Internet Systems Consortium, Inc.
    RPKI status: No ROA found

  • Anand Buddhdev says:

    Another correction to this article is that the incident reported by Renesys last year was about I-root, not L-root.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>