Tags / proxy

Distributed forward proxy servers
to crawl Internet via multiple cloud instances.

squid haproxy proxy http setup distributed

Building a scalable distributed web crawler
which can perform both crawling and data extraction

web crawler scala akka kafka couchbase jsoup big data design distributed proxy