Skip to content

Web crawler with very flexible crawling options. Can either use standalone or can be used with resque to perform clustered crawls.

License

Notifications You must be signed in to change notification settings

stewartmckee/cobweb

Error
Looks like something went wrong!

About

Web crawler with very flexible crawling options. Can either use standalone or can be used with resque to perform clustered crawls.

Resources

License

Stars

Watchers

Forks

Packages

No packages published