To block ip addresses with Varnish edit the following file:

sudo nano /etc/varnish/default.vcl

And add the following. Of course replace the example ip addresses with the real ips you want to block.

acl forbidden {
    "111.11.111.111";
    "123.12.123.123";
    "456.123.12.72";
}

sub vcl_recv {
    # Block access from these ips
    if (client.ip ~ forbidden) {
        error 403 "Forbidden";
    }
}

After editing, execute:

sudo service varnish restart