python multithreading

Requests 2.12.4 does not timeout on chunked stream  #angularjs #reactjs

  • While the threads are still active, the log messages within “with closing” are showing up.
  • sign up log in tour help
  • Once they’re hung, the last log message they published is always log 1.
  • Stack Overflow works best with JavaScript enabled
  • But by the end of the first hour running, all 20 of my threads are hung.

I have multi-threaded code that needs to fetch just the head of each webpage. By head here, I mean the part that is enclosed in the HTML head tag, not the http header. Streaming chunks until I find the end head tag increases my performance by a factor of 2-4. But by the end of the first hour running, all 20 of my threads are hung. The threads are still alive (I have a monitoring thread checking for dead threads and respawning), they’re just stuck.

@ng_real_ninja: Requests 2.12.4 does not timeout on chunked stream #angularjs #reactjs

I have multi-threaded code that needs to fetch just the head of each webpage. By head here, I mean the part that is enclosed in the HTML head tag, not the http header. Streaming chunks until I find the end head tag increases my performance by a factor of 2-4. But by the end of the first hour running, all 20 of my threads are hung. The threads are still alive (I have a monitoring thread checking for dead threads and respawning), they’re just stuck.

Here is a simplified version of my code:

While the threads are still active, the log messages within “with closing” are showing up. Once they’re hung, the last log message they published is always log 1.

I’ve read all the existing posts I can find about similar problems. Supposedly there was a bug in Requests like this that was fixed in 2.3.0, but since I’m using 2.12.4, that shouldn’t be the problem. So, I have two questions:

python multithreading

You might also like More from author

Comments are closed, but trackbacks and pingbacks are open.