Ad-blocking with Polipo

2010-06-16
, , , ,

Polipo >

See also Ad-blocking with Polipo and adzapper

I did not follow these notes. They're just here for reference.

Ad-blocking with Polipo ∞

http://rightfootin.blogspot.com/2009/11/ad-blocking-with-polipo.html

Polipo is a fast local proxy that does on-disk caching (by default, at least). Privoxy is another local proxy, with a focus on privacy and ad-blocking. Due to the nature and purpose of Privoxy, it has to buffer portions of the page (to check for content it should block) before serving it to the browser. This makes it a bit slower than Polipo. You could always use Polipo in front of Privoxy (see here, middle of the page), but that is a bit much if you don't need fancy filtering and simply want a domain/regex blocklist.

You could build up the blocklist yourself, by hand, but that would be a pain. Instead, we'll just convert an adblock filterset to a format that Polipo can understand. Since Polipo is blocking by matching the URL only, we don't have the same fine-grained control as Adblock rules, but I personally don't require that level of control.

Firstly, grab an adblock filterset (e.g., easylist.txt). Next, grab either adblock2polipo.py (python) or adblock2polipo.rb (ruby). Then run whichever script you downloaded with the filerset file as the first parameter. The script will dump the re-written rules to the console so they can be inspected or be redirected to the file polipo uses to load its blocking rules from ~/.polipo-forbidden or /etc/polipo/forbidden on *nix systems. Restart Polipo and the new blocking rules should take effect.

PS. If you're like me, you'd rather have Polipo serve up a blank page for blocked URLs rather than a 403 error page. To accomplish this you need to edit the Polipo config file and add a forbiddenUrl option pointing to an empty image, such as this one. One good option for this is the following: make sure localDocumentRoot is either set to a real path (such as /usr/share/polipo/www) or commented out completely. Then create an empty file in that directory:

sudo wget -O /usr/share/polipo/www/empty.gif \
http://upload.wikimedia.org/wikipedia/commons/4/4b/Empty.gif

Then point forbiddenUrl to http://127.0.0.1:8123/empty.gif. Restart Polipo. That's it. :)

NB. Firefox users will need to add port 8123 to the allowed ports list, or they'll get an equally annoying error message from Firefox instead of a blank page. To do this, you need to open about:config in a new tab/window. Right-click and go to New->String, and for the property name put network.security.ports.banned.override, and for the value put 8123. It should work properly after that.

adblock2polipo.py ∞

Spoiler

#!/usr/bin/python

# convert adblock ruleset into polipo-forbidden format

if __name__ == "__main__":

        import os
        import sys
        import re

        if len(sys.argv) == 1:
                sys.exit("Usage: %s <adblockrules>" % os.path.basename(sys.argv[0]))

        if not os.path.exists(sys.argv[1]):
                sys.exit("The rules file (%s) doesn't exist" % sys.argv[1])

        fhandle = file(sys.argv[1])
        lines = fhandle.readlines()
        fhandle.close()

        dollar_re = re.compile("(.*?)\$.*")

        for line in lines:
                if line:
                        if (line[0] in ("[", "!", "~", "#", "@") or
                            line.startswith("/adverti") or
                            "##" in line):
                                continue
                        line = dollar_re.sub(r"\1", line)
#                       line = line.replace("|http://", "")
                        line = line.replace("|", "")
                        line = line.replace("||", "")
                        line = line.replace(".", r"\.")
                        line = line.replace("*", ".*")
                        line = line.replace("?", r"\?")
                        line = line.replace("^", r"[\/:\.=&\?\+\-\ ]+")
#                       line = line.replace("&", r"\&")
#                       line = line.replace("+", r"\+")
#                       line = line.replace("-", r"\-")
#                       line = line.replace(";", r"\;")
#                       line = line.replace("=", r"\=")
#                       line = line.replace("/", r"\/")
                        print(line.strip())
        print("")
</pre>

== adblock2polipo.rb ==

<pre>
#!/usr/bin/ruby

# convert adblock ruleset into polipo-forbidden format

if __FILE__ == $0

        if ARGV.length == 0
                exit("Usage: #{File.basename($0)} <adblockrules>")
        end

        if not File.exist?(ARGV[0])
                exit("The rules file (#{ARGV[0]}) doesn't exist")
        end

        dollar_re = Regexp.new(/(.*?)\$.*/)

        File.readlines(ARGV[0]).each {
                | line |
                unless line.empty?
                        if (["[", "!", "~", "#", "@"].include?(line[0]) or
                            line[0, 8] == "/adverti" or
                            line.include?("##"))
                                next
                        end
                        line = line.gsub(dollar_re, "\\1")
#                       line = line.gsub("|http://", "")
                        line = line.gsub("|", "")
                        line = line.gsub("||", "")
                        line = line.gsub(".", "\\.")
                        line = line.gsub("*", ".*")
                        line = line.gsub("?", "\\?")
                        line = line.gsub("^", "[\\/:\\.=&\\?\\\\+\\-\\ ]+")
#                       line = line.gsub("&", "\\&")
#                       line = line.gsub("+", "\\+")
#                       line = line.gsub("-", "\\-")
#                       line = line.gsub(";", "\\;")
#                       line = line.gsub("=", "\\=")
#                       line = line.gsub("/", "\\/")
                        puts(line.strip)
                end
        }
        puts("")

end

Comment

Leave a Reply

Your email address will not be published. Required fields are marked *