Continuing on with the talk about socket connections and the like.

When we left off, we talked about making sure you check all the return codes of your method calls, then about making your sockets asynchronous so that they don't block and freeze your program. These things are all true even if you use TCP under some layers of abstraction. Maybe your network abstraction layer turns connection errors into exceptions. Fine, handle all of the exceptions. Maybe the socket functionality is exposed in synchronous and asynchronous versions. Very well, you use the asynchronous version. Even after you've done all that, you're still not done making your program robust in regards to dealing with TCP sockets.

What more could go wrong? Plenty! First, let's think about the delivery guarantees that TCP gets you, which is one of the primary reasons why you'd want to use TCP and not UDP. When your data gets sent over your connected socket, it reliably makes it over to its target. If a piece of data didn't make it to the target, it gets retransmitted automatically. Your data arrives in order at the other end and every piece makes it. That's our hopes and our dreams. If you've ever written a socket enabled application and you observed that not happening, you might feel a bit betrayed. Here's the skinny - TCP isn't letting you down, it's just not giving you the guarantee you think it's giving you. When your socket is open, your data is sure to make it to the target system. The key word there is "system." There's no guarantee that your data makes it to your target "application". If the data going over the wire makes it to your target system and then the target system is not able to receive it for some reason, then your packet of data is lost to the wind. If you want guarantees that your target application received all the data, you'll need to add it to your protocol.

It might work like this. Program one says, I'm about to send you w word, it's 5 letters. The recipient sends back, I received the word, it has this hash code, and the like. With some acknowledgement by the recipient, the sender has much more certainty that the other program was able to receive the data.

Next you'll have to deal with zombie socket. A zombie socket is a socket that looks like it's connected, but sends over it go into the void and nothing can be received on them. This can happen if there's a network component like a switch that has some physical damage. You can never assume that you won't have to deal with zombie sockets, whatever the situation. As long as you're dealing with TCP, that's a possibility that you will need to account for to make your networked program robust. The way to deal with this is again in the protocol. The only way you know for sure that a socket is working correctly is by receiving something on it. If it's been some time since you were able to receive something on the socket, it might have become a zombie. This is more of a problem with long-lived sockets than with short-lived ones, but a short-lived socket can also become a zombie. So what do you do when you legitimately don't have some data to send to the other side? That's where the PINGs come into play. Send a PING type message on one side, and when the other side receives the PING, have it send a PONG. That way, both sides have received something from the other, and both ends know that the socket is not a zombie.

Now see where we are. You checked all your return codes, you made your sockets asynchronous, your sends and your receives are asynchronous, you implemented some acknowledgement functionality in your data protocol so you know the other end received the message, you're dealing with segmentation correctly and piecing together fragmented messages, and you have keep-alive logic with PING/PONG and zombie detection. All good now? Almost, but not quite yet. There's still one more matter to attend do, and that's to do with network performance.

For good optimization, you really should take measurements to see what things can be improved. When measuring network performance, two important metrics to consider are goodput and latency. Goodput has to do with how long it takes you to transfer a certain set of data from one end to the other. Latency has to do with the time it takes for a particular piece of data to get from one end to the other. Whether you care more about the goodput or the latency really depends on what your application is doing. If you are communicating states that need to be updated in real time between one system and the other, then latency is going to play big role in how performant that is. If you are transferring files around, then goodput is the measurement you are aiming for - that's the effective data transfer rate.

If you're optimizing for goodput, one thing you might do is try to adjust your protocol to aggregate data. If, instead of calling send() 100 times with 10 bytes of data each, you make it able to call send() twice with 500 bytes each, then your throughput can be much more optimized.

If you're serious about network performance and your program is on Windows, you can make use of I/O Completion Ports. This is a system that facilitates optimal I/O handling on Windows systems. Other operating systems have similar mechanisms.

If you're writing code with AI, can you hope that it accounts for all these things? Maybe it can if you explicitly tell it to do so, but if you aren't telling it to write code that does all this, it won't do it. That's the thing with LLM AI bots, they write exactly what you tell them to, but nothing more really.