unarist 5c8ca024ef Improve error handling on LinkCrawlWorker (#5250) ...

* Improve error handling on LinkCrawlWorker

* Ignore TimeoutError and InvalidURIError too
* Record errors to debug log
* Enable dead job queue on LinkCrawlWorker

Since most of acceptable errors were already ignored, only our side issue should go to dead job queue.

* Ignore all http gem errors

2017-10-06 20:39:08 +02:00

263 B

Raw Blame History

View Raw