Filter Anonymous Internet Requests
Most countries will happily agree that internet filtering is a good idea, and of course for illegal sites such as CSA – there is no issue. Unfortunately it rarely stops there and most countries and organisations quickly develop their own extensive URL lists to be blocked which contain all sorts of subjects. So how do these countries actually filter the internet feeds to their users?
Internet Filtering – Basic methods
There are generally two main methods used to filter and block access to specific internet pages. Both of these methods rely on the creation of a blacklist (sites that should be blocked).
The first method which is actually used quite extensively is of DNS poisoning, I have written about this subject before but it is a rather unsophisticated method which relies on modifying the DNS tables of your ISPs and redirecting requests for blocked pages to somewhere else. So when a user tried to access www.verybadsite.com their browser will not receive the correct IP address for that server and will be redirected to another web site.
The DNS poisoning method is actually used by lots of countries, most Scandinavian countries including Sweden, Norway, Finland and Denmark also used fairly extensively by ISPs in Germany and Holland.
It’s not a very desirable method of internet filtering partly because the method used is already commonly used by criminals – redirecting users to malware infected web sites by modifying DNS tables. But the biggest problem is that it is so easy to bypass, any user who knows how to modify his DNS settings can simply bypass this filtering method.
The other problem with DNS poisoning is that you cannot simply block a single page, you have to block the entire domain. This can be difficult when there are so many collaborative platforms and social media websites. If you Government decides they don’t like a particular YouTube video then they’d have to block access to millions of inoffensive ones to!
Then There’s the List, of the Internet Sites you Cannot See
The other blocking methods that are commonly used are more sophisticated, technologies such as BTs Cleanfeed, Optenet, and Netclean which actually block specific URLs. They all work in slightly different ways but essentially using the same concept, all URL request are directed through a central systems which check access compared to a defined list of ‘bad urls’. If you try to access one of the URLs listed on this black list, the web request is not forwarded.
Sounds simple again, of course the creation and management of the list itself is the concern for those of us who argue against censorship and promote freedom online. However the technologies can also cause issues as well – a recent report from Watchdog International highlighted several technical problems that can occur with these technologies.
Here’s a couple of their examples.
ACMA Test of YouTube Blocking
When the Australian Government where trialing the BGP filtering system called Netclean White Box, they included a couple of URLs from YouTube to be blocked. The problem happened because when the YouTube URL was added, all requests for this domain then get handled directly through the filter. Of course normally this would be quite low traffic if you’re just filtering illegal sites, but in this case millions of YouTube requests where being redirected. The result was that the Netscreen White box systems was overloaded.
IWF List contained Wikipedia image
The Internet Watch Foundation manages a very extensive black list of web sites across the internet. The list is used by many blocking companies as a master list of which sites to block. In this incident the IWF added the URL of an Image stored on Wikipedia. However a limitation of the BT Cleanfeed system ended up with Wikipedia being completely inaccessible. The problem was that the Cleanfeed system passes any filtered traffic through a proxy server, which then modifies the web request by replacing the customers IP address with it’s own. Of course when this happened in the UK, Wikipedia suddenly had a huge amount of traffic coming from a single IP address (the Netfeed proxy server installed at the ISP) – this of course looked like an attack so they blocked access.
The Internet Watch Foundation realised their mistake and removed the URL fairly quickly but it does highlight the potential problems when you start any mainstream censorship and Internet Filtering.
