Avoiding panics

[urandom]

Would it be possible to introduce a new method alongside Crawl, or change the existing one, so that an error is returned as the last argument rather than panicing? It'd be nice to obtain the error on any stage of the crawl (fetching, parsing, etc) so it can be handled accordingly.

[flexd]

Looking at the code, there are a lot of places where errors should be returned instead of just logging or panic(). I agree that this is not very idiomatic, and should be fixed.