Platform rate limits

[iojw]

While the asynchronous code can handle a large number of queries, platforms have rate limits to control traffic on their sites and this leads to errors when too many queries are executed with socialscan. From my testing, GH, Reddit, Snapchat and Instagram tend to fail when handling a large number of queries in a short time.

There's support for proxy lists which can help bypass these, but it might be good to look into building a way of getting past these limits into the tool itself. (I think this has to be handled responsibly so as to not open the tool up for abuse)

[codeswhite]

I suggest to add an option to input a list of proxies that will be later used in rotation.

For example we query username CodesWhite with proxy list:

1.1.1.1:80
2.2.2.2:8008
3.3.3.3:8080

So the utilization of the proxies will be:

GitLab -> accessed trough 1.1.1.1:80
Last.fm -> accessed trough 2.2.2.2:8008
Tumbler -> accessed trough 3.3.3.3:8080
Twitter -> accessed trough 1.1.1.1:80
... etc ...

Now add to this platforms order randomization - so that different proxies are accessing different platforms every time, and we have a solution.

I once wrote a tool that I have used only by myself, I had no need for extending it beyond my requirements, but I think we can extend it to suit SocialScan's requirements.

Here: proxion