Asynchronous network I/O

Server software that processes multiple connections from different clients (such as web servers, mail servers, proxy servers etc.) can be developed using different programming paradigms. Here we will look at three such paradigms, each of which has its benefits and drawbacks.

• Synchronous
  This is the simplest method of handling communications and is often the most prevalent in amateur software development. It is really not ideal for large-scale software though because a single connection can hold up all other connections while it is waiting to send or receive data. Imagine a mail server with fifty clients connected to and one client is sending a message with large attachments. All the other clients have to wait until that message is finished sending in order to obtain access to the server. If you take into account network, disk and memory latency this is a very inefficient way of programming.
• Multi-threaded
  Most developers, as they get more confident, will attempt a multi-threaded approach to communications – usually starting a new thread for each incoming connection and then using that thread to process the connection in a synchronous way. Using multiple threads of execution certainly stops the main problem of using a single synchronous thread, in that each connection gets some time on the CPU but it has its own problems. There is an inherent overhead involved in creating threads, in terms of memory and CPU time. These resources are limited and as such there is a definite limit on the number of threads that can be started. Threads can also get quite complicated when it comes to synchronising access to shared resources.
• Asynchronous/event driven
  By far the best way of dealing with multiple connections is to use asynchronous or event driven communications. Apart from being a more elegant solution it is also more efficient. There is less chance of running out of resources on the server and no chance of one connection bringing the software to a halt. Completion routines can be implemented on worker threads which are more efficient and only exist for a short space of time thus reducing CPU load. This method scales so much better than the others and software using it can handle hundreds of connections without consuming too many resources. It can be a more complicated method of programming and requires a definite change in the way developers think about their software, but it is worth the extra investment in time and definitely results in better software.