[SOLVED] Python Cannot assign requested address

Python exception

Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/srv/sbproxy/sandboxproxytest/framework/multiprocess.py", line 39, in run
    res = self._function(*task)
.
.
.
    return requests.request(method, url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 455, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 558, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 378, in send
    raise ConnectionError(e)
ConnectionError: HTTPConnectionPool(host='172.31.37.5', port=8080): Max retries exceeded with url: /jobs/list?after=2015-09-27 (Caused by <class 'socket.error'>: [Errno 99] Cannot assign requested address)

Check netstat | grep -c tcp 

When the expection happens, the above commans show around 28000.

Reason can be seen here:

http://stackoverflow.com/questions/11190595/repeated-post-request-is-causing-error-socket-error-99-cannot-assign-reques
If the connections are in CONNECTION_WAIT then the server closed the connection earlier than should, but if the connection is in TIME_WAIT state, that is normal:

RFC 793 sets the TIME-OUT to be twice the Maximum Segment Lifetime, or 2MSL. Since MSL, the maximum time a packet can wander around Internet, is set to 2 minutes, 2MSL is 4 minutes. Since there is no ACK to an ACK, the active closer can't do anything but to wait 4 minutes if it adheres to the TCP/IP protocol correctly, just in case the passive sender has not received the ACK to its FIN (theoretically).

Some explanation:
http://serverfault.com/a/329846

Solution

Set the tcp_tw_recycle to 1

echo 1 >   /proc/sys/net/ipv4/tcp_tw_recycle

Or add to /etc/sysctl.conf:

net.ipv4.tcp_tw_recycle = 1

and reload the configuration like this:

sysctl -p 

Some documentation advise to use /proc/sys/net/ipv4/tcp_tw_reuse, but in my case it has not solved this issue.