TCP implementation of the transport layer.

The TCP implementation guarantees that only a single TCP connection (socket) will be used between endpoints, provided that the addresses specified are canonical. If A connects to B and reports its address as 192.168.0.1:8080 and B subsequently connects tries to connect to A as client1.local:http-alt then the transport layer will not realize that the TCP connection can be reused.

Applications that use the TCP transport should use withSocketsDo in their main function for Windows compatibility (see Network.Socket).

Goals

The TCP transport maps multiple logical connections between A and B (in either direction) to a single TCP connection:

+-------+                          +-------+
| A     |==========================| B     |
|       |>~~~~~~~~~~~~~~~~~~~~~~~~~|~~~\   |
|   Q   |>~~~~~~~~~~~~~~~~~~~~~~~~~|~~~Q   |
|   \~~~|~~~~~~~~~~~~~~~~~~~~~~~~~<|       |
|       |==========================|       |
+-------+                          +-------+

Ignoring the complications detailed below, the TCP connection is set up is when the first lightweight connection is created (in either direction), and torn down when the last lightweight connection (in either direction) is closed.

Connecting

Let A, B be two endpoints without any connections. When A wants to connect to B, it locally records that it is trying to connect to B and sends a request to B. As part of the request A sends its own endpoint address to B (so that B can reuse the connection in the other direction).

When B receives the connection request it first checks if it did not already initiate a connection request to A. If not it will acknowledge the connection request by sending ConnectionRequestAccepted to A and record that it has a TCP connection to A.

The tricky case arises when A sends a connection request to B and B finds that it had already sent a connection request to A. In this case B will accept the connection request from A if As endpoint address is smaller (lexicographically) than Bs, and reject it otherwise. If it rejects it, it sends a ConnectionRequestCrossed message to A. The lexicographical ordering is an arbitrary but convenient way to break the tie. If a connection exists between A and B when B rejects the request, B will probe the connection to make sure it is healthy. If A does not answer timely to the probe, B will discard the connection.

When it receives a ConnectionRequestCrossed message the A thread that initiated the request just needs to wait until the A thread that is dealing with B's connection request completes, unless there is a network failure. If there is a network failure, the initiator thread would timeout and return an error.

Disconnecting

The TCP connection is created as soon as the first logical connection from A to B (or B to A) is established. At this point a thread (#) is spawned that listens for incoming connections from B:

+-------+                          +-------+
| A     |==========================| B     |
|       |>~~~~~~~~~~~~~~~~~~~~~~~~~|~~~\   |
|       |                          |   Q   |
|      #|                          |       |
|       |==========================|       |
+-------+                          +-------+

The question is when the TCP connection can be closed again. Conceptually, we want to do reference counting: when there are no logical connections left between A and B we want to close the socket (possibly after some timeout).

However, A and B need to agree that the refcount has reached zero. It might happen that B sends a connection request over the existing socket at the same time that A closes its logical connection to B and closes the socket. This will cause a failure in B (which will have to retry) which is not caused by a network failure, which is unfortunate. (Note that the connection request from B might succeed even if A closes the socket.)

Instead, when A is ready to close the socket it sends a CloseSocket request to B and records that its connection to B is closing. If A receives a new connection request from B after having sent the CloseSocket request it simply forgets that it sent a CloseSocket request and increments the reference count of the connection again.

When B receives a CloseSocket message and it too is ready to close the connection, it will respond with a reciprocal CloseSocket request to A and then actually close the socket. A meanwhile will not send any more requests to B after having sent a CloseSocket request, and will actually close its end of the socket only when receiving the CloseSocket message from B. (Since A recorded that its connection to B is in closing state after sending a CloseSocket request to B, it knows not to reciprocate B reciprocal CloseSocket message.)

If there is a concurrent thread in A waiting to connect to B after A has sent a CloseSocket request then this thread will block until A knows whether to reuse the old socket (if B sends a new connection request instead of acknowledging the CloseSocket) or to set up a new socket.