Tags / web crawler

Crawlpod - open source scalable web crawler
built on top of Scala and Akka framework.

crawlpod web crawler scala akka mongodb design open source

Building a scalable distributed web crawler
which can perform both crawling and data extraction

web crawler scala akka kafka couchbase jsoup big data design distributed proxy