15: Network programming

Historically, network programming has been error-prone, difficult, and complex.

The programmer had to know many details about the network and sometimes even the hardware. You usually needed to understand the various “layers” of the networking protocol, and there were a lot of different functions in each different networking library concerned with connecting, packing, and unpacking blocks of information; shipping those blocks back and forth; and handshaking. It was a daunting task.

However, the concept of networking is not so difficult. You want to get some information from that machine over there and move it to this machine here, or vice versa. It’s quite similar to reading and writing files, except that the file exists on a remote machine and the remote machine can decide exactly what it wants to do about the information you’re requesting or sending.

One of Java’s great strengths is painless networking. As much as possible, the underlying details of networking have been abstracted away and taken care of within the JVM and local machine installation of Java. The programming model you use is that of a file; in fact, you actually wrap the network connection (a “socket”) with stream objects, so you end up using the same method calls as you do with all other streams. In addition, Java’s built-in multithreading is exceptionally handy when dealing with another networking issue: handling multiple connections at once.

This chapter introduces Java’s networking support using easy-to-understand examples.

Of course, in order to tell one machine from another and to make sure that you are connected with the machine you want, there must be some way of uniquely identifying machines on a network. Early networks were satisfied to provide unique names for machines within the local network. However, Java works within the Internet, which requires a way to uniquely identify a machine from all the others in the world. This is accomplished with the IP (Internet Protocol) address that can exist in two forms:

In both cases, the IP address is represented internally as a 32-bit number[59] (so each of the quad numbers cannot exceed 255), and you can get a special Java object to represent this number from either of the forms above by using the static InetAddress.getByName( ) method that’s in java.net. The result is an object of type InetAddress that you can use to build a “socket” as you will see later.

As a simple example of using InetAddress.getByName( ), consider what happens if you have a dial-up Internet service provider (ISP). Each time you dial up, you are assigned a temporary IP address. But while you’re connected, your IP address has the same validity as any other IP address on the Internet. If someone connects to your machine using your IP address then they can connect to a Web server or FTP server that you have running on your machine. Of course, they need to know your IP address, and since it’s assigned each time you dial up, how can you find out what it is?

The following program uses InetAddress.getByName( ) to produce your IP address. To use it, you must know the name of your computer. It has been tested only on Windows 95, but there you can go to “Settings,” “Control Panel,” “Network,” and then select the “Identification” tab. “Computer name” is the name to put on the command line.

In my case, the machine is called “Colossus” (from the movie of the same name, because I keep putting bigger disks on it). So, once I’ve connected to my ISP I run the program:

I get back a message like this (of course, the address is different each time):

If I tell my friend this address, he can log onto my personal Web server by going to the URL http://199.190.87.75 (only as long as I continue to stay connected during that session). This can sometimes be a handy way to distribute information to someone else or to test out a Web site configuration before posting it to a “real” server.

The whole point of a network is to allow two machines to connect and talk to each other. Once the two machines have found each other they can have a nice, two-way conversation. But how do they find each other? It’s like getting lost in an amusement park: one machine has to stay in one place and listen while the other machine says, “Hey, where are you?”

The machine that “stays in one place” is called the server, and the one that seeks is called the client. This distinction is important only while the client is trying to connect to the server. Once they’ve connected, it becomes a two-way communication process and it doesn’t matter anymore that one happened to take the role of server and the other happened to take the role of the client.

So the job of the server is to listen for a connection, and that’s performed by the special server object that you create. The job of the client is to try to make a connection to a server, and this is performed by the special client object you create. Once the connection is made, you’ll see that at both server and client ends, the connection is just magically turned into an IO stream object, and from then on you can treat the connection as if you were reading from and writing to a file. Thus, after the connection is made you will just use the familiar IO commands from Chapter 10. This is one of the nice features of Java networking.

For many reasons, you might not have a client machine, a server machine, and a network available to test your programs. You might be performing exercises in a classroom situation, or you could be writing programs that aren’t yet stable enough to put onto the network. The creators of the Internet Protocol were aware of this issue, and they created a special address called localhost to be the “local loopback” IP address for testing without a network. The generic way to produce this address in Java is:

If you hand getByName( ) a null, it defaults to using the localhost. The InetAddress is what you use to refer to the particular machine, and you must produce this before you can go any further. You can’t manipulate the contents of an InetAddress (but you can print them out, as you’ll see in the next example). The only way you can create an InetAddress is through one of that class’s static member methods getByName( ) (which is what you’ll usually use), getAllByName( ), or getLocalHost( ).

You can also produce the local loopback address by handing it the string localhost:

or by using its dotted quad form to name the reserved IP number for the loopback:

All three forms produce the same result.

An IP address isn’t enough to identify a unique server, since many servers can exist on one machine. Each IP machine also contains ports, and when you’re setting up a client or a server you must choose a port where both client and server agree to connect; if you’re meeting someone, the IP address is the neighborhood and the port is the bar.

The port is not a physical location in a machine, but a software abstraction (mainly for bookkeeping purposes). The client program knows how to connect to the machine via its IP address, but how does it connect to a desired service (potentially one of many on that machine)? That’s where the port numbers come in as second level of addressing. The idea is that if you ask for a particular port, you’re requesting the service that’s associated with the port number. The time of day is a simple example of a service. Typically, each service is associated with a unique port number on a given server machine. It’s up to the client to know ahead of time which port number the desired service is running on.

The system services reserve the use of ports 1 through 1024, so you shouldn’t use those or any other port that you know to be in use. The first choice for examples in this book will be port 8080 (in memory of the venerable old 8-bit Intel 8080 chip in my first computer, a CP/M machine).

The socket is the software abstraction used to represent the “terminals” of a connection between two machines. For a given connection, there’s a socket on each machine, and you can imagine a hypothetical “cable” running between the two machines with each end of the “cable” plugged into a socket. Of course, the physical hardware and cabling between machines is completely unknown. The whole point of the abstraction is that we don’t have to know more than is necessary.

In Java, you create a socket to make the connection to the other machine, then you get an InputStream and OutputStream (or, with the appropriate converters, Reader and Writer) from the socket in order to be able to treat the connection as an IO stream object. There are two stream-based socket classes: a ServerSocket that a server uses to “listen” for incoming connections and a Socket that a client uses in order to initiate a connection. Once a client makes a socket connection, the ServerSocket returns (via the accept( ) method) a corresponding server side Socket through which direct communications will take place. From then on, you have a true Socket to Socket connection and you treat both ends the same way because they are the same. At this point, you use the methods getInputStream( ) and getOutputStream( ) to produce the corresponding InputStream and OutputStream objects from each Socket. These must be wrapped inside buffers and formatting classes just like any other stream object described in Chapter 10.

The use of the term ServerSocket would seem to be another example of a confusing name scheme in the Java libraries. You might think ServerSocket would be better named “ServerConnector” or something without the word “Socket” in it. You might also think that ServerSocket and Socket should both be inherited from some common base class. Indeed, the two classes do have several methods in common but not enough to give them a common base class. Instead, ServerSocket’s job is to wait until some other machine connects to it, then to return an actual Socket. This is why ServerSocket seems to be a bit misnamed, since its job isn’t really to be a socket but instead to make a Socket object when someone else connects to it.

However, the ServerSocket does create a physical “server” or listening socket on the host machine. This socket listens for incoming connections and then returns an “established” socket (with the local and remote endpoints defined) via the accept( ) method. The confusing part is that both of these sockets (listening and established) are associated with the same server socket. The listening socket can accept only new connection requests and not data packets. So while ServerSocket doesn’t make much sense programmatically, it does “physically.”

When you create a ServerSocket, you give it only a port number. You don’t have to give it an IP address because it’s already on the machine it represents. When you create a Socket, however, you must give both the IP address and the port number where you’re trying to connect. (On the other hand, the Socket that comes back from ServerSocket.accept( ) already contains all this information.)

This example makes the simplest use of servers and clients using sockets. All the server does is wait for a connection, then uses the Socket produced by that connection to create an InputStream and OutputStream. After that, everything it reads from the InputStream it echoes to the OutputStream until it receives the line END, at which time it closes the connection.

The client makes the connection to the server, then creates an OutputStream. Lines of text are sent through the OutputStream. The client also creates an InputStream to hear what the server is saying (which, in this case, is just the words echoed back).

Both the server and client use the same port number and the client uses the local loopback address to connect to the server on the same machine so you don’t have to test it over a network. (For some configurations, you might need to be connected to a network for the programs to work, even if you aren’t communicating over that network.)

Here is the server:

You can see that the ServerSocket just needs a port number, not an IP address (since it’s running on this machine!). When you call accept( ), the method blocks until some client tries to connect to it. That is, it’s there waiting for a connection but other processes can run (see Chapter 14). When a connection is made, accept( ) returns with a Socket object representing that connection.

The responsibility for cleaning up the sockets is crafted carefully here. If the ServerSocket constructor fails, the program just quits (notice we must assume that the constructor for ServerSocket doesn’t leave any open network sockets lying around if it fails). For this case, main( ) throws IOException so a try block is not necessary. If the ServerSocket constructor is successful then all other method calls must be guarded in a try-finally block to ensure that, no matter how the block is left, the ServerSocket is properly closed.

The same logic is used for the Socket returned by accept( ). If accept( ) fails, then we must assume that the Socket doesn’t exist or hold any resources, so it doesn’t need to be cleaned up. If it’s successful, however, the following statements must be in a try-finally block so that if they fail the Socket will still be cleaned up. Care is required here because sockets use important non-memory resources, so you must be diligent in order to clean them up (since there is no destructor in Java to do it for you).

Both the ServerSocket and the Socket produced by accept( ) are printed to System.out. This means that their toString( ) methods are automatically called. These produce:

Shortly, you’ll see how these fit together with what the client is doing.

The next part of the program looks just like opening files for reading and writing except that the InputStream and OutputStream are created from the Socket object. Both the InputStream and OutputStream objects are converted to Java 1.1 Reader and Writer objects using the “converter” classes InputStreamReader and OutputStreamWriter, respectively. You could also have used the Java 1.0 InputStream and OutputStream classes directly, but with output there’s a distinct advantage to using the Writer approach. This appears with PrintWriter, which has an overloaded constructor that takes a second argument, a boolean flag that indicates whether to automatically flush the output at the end of each println( ) (but not print( )) statement. Every time you write to out, its buffer must be flushed so the information goes out over the network. Flushing is important for this particular example because the client and server each wait for a line from the other party before proceeding. If flushing doesn’t occur, the information will not be put onto the network until the buffer is full, which causes lots of problems in this example.

When writing network programs you need to be careful about using automatic flushing. Every time you flush the buffer a packet must be created and sent. In this case, that’s exactly what we want, since if the packet containing the line isn’t sent then the handshaking back and forth between server and client will stop. Put another way, the end of a line is the end of a message. But in many cases messages aren’t delimited by lines so it’s much more efficient to not use auto flushing and instead let the built-in buffering decide when to build and send a packet. This way, larger packets can be sent and the process will be faster.

Note that, like virtually all streams you open, these are buffered. There’s an exercise at the end of the chapter to show you what happens if you don’t buffer the streams (things get slow).

The infinite while loop reads lines from the BufferedReader in and writes information to System.out and to the PrintWriter out. Note that these could be any streams, they just happen to be connected to the network.

When the client sends the line consisting of “END” the program breaks out of the loop and closes the Socket.

Here’s the client:

In main( ) you can see all three ways to produce the InetAddress of the local loopback IP address: using null, localhost, or the explicit reserved address 127.0.0.1. Of course, if you want to connect to a machine across a network you substitute that machine’s IP address. When the InetAddress addr is printed (via the automatic call to its toString( ) method) the result is:

By handing getByName( ) a null, it defaulted to finding the localhost, and that produced the special address 127.0.0.1.

Note that the Socket called socket is created with both the InetAddress and the port number. To understand what it means when you print out one of these Socket objects, remember that an Internet connection is determined uniquely by these four pieces of data: clientHost, clientPortNumber, serverHost, and serverPortNumber. When the server comes up, it takes up its assigned port (8080) on the localhost (127.0.0.1). When the client comes up, it is allocated to the next available port on its machine, 1077 in this case, which also happens to be on the same machine (127.0.0.1) as the server. Now, in order for data to move between the client and server, each side has to know where to send it. Therefore, during the process of connecting to the “known” server, the client sends a “return address” so the server knows where to send its data. This is what you see in the example output for the server side:

This means that the server just accepted a connection from 127.0.0.1 on port 1077 while listening on its local port (8080). On the client side:

which means that the client made a connection to 127.0.0.1 on port 8080 using the local port 1077.

You’ll notice that every time you start up the client anew, the local port number is incremented. It starts at 1025 (one past the reserved block of ports) and keeps going up until you reboot the machine, at which point it starts at 1025 again. (On UNIX machines, once the upper limit of the socket range is reached, the numbers will wrap around to the lowest available number again.)

Once the Socket object has been created, the process of turning it into a BufferedReader and PrintWriter is the same as in the server (again, in both cases you start with a Socket). Here, the client initiates the conversation by sending the string “howdy” followed by a number. Note that the buffer must again be flushed (which happens automatically via the second argument to the PrintWriter constructor). If the buffer isn’t flushed, the whole conversation will hang because the initial “howdy” will never get sent (the buffer isn’t full enough to cause the send to happen automatically). Each line that is sent back from the server is written to System.out to verify that everything is working correctly. To terminate the conversation, the agreed-upon “END” is sent. If the client simply hangs up, then the server throws an exception.

You can see that the same care is taken here to ensure that the network resources represented by the Socket are properly cleaned up, using a try-finally block.

Sockets produce a “dedicated” connection that persists until it is explicitly disconnected. (The dedicated connection can still be disconnected un-explicitly if one side, or an intermediary link, of the connection crashes.) This means the two parties are locked in communication and the connection is constantly open. This seems like a logical approach to networking, but it puts an extra load on the network. Later in the chapter you’ll see a different approach to networking, in which the connections are only temporary.

The JabberServer works, but it can handle only one client at a time. In a typical server, you’ll want to be able to deal with many clients at once. The answer is multithreading, and in languages that don’t directly support multithreading this means all sorts of complications. In Chapter 14 you saw that multithreading in Java is about as simple as possible, considering that multithreading is a rather complex topic. Because threading in Java is reasonably straightforward, making a server that handles multiple clients is relatively easy.

The basic scheme is to make a single ServerSocket in the server and call accept( ) to wait for a new connection. When accept( ) returns, you take the resulting Socket and use it to create a new thread whose job is to serve that particular client. Then you call accept( ) again to wait for a new client.

In the following server code, you can see that it looks similar to the JabberServer.java example except that all of the operations to serve a particular client have been moved inside a separate thread class:

The ServeOneJabber thread takes the Socket object that’s produced by accept( ) in main( ) every time a new client makes a connection. Then, as before, it creates a BufferedReader and auto-flushed PrintWriter object using the Socket. Finally, it calls the special Thread method start( ), which performs thread initialization and then calls run( ). This performs the same kind of action as in the previous example: reading something from the socket and then echoing it back until it reads the special “END” signal.

The responsibility for cleaning up the socket must again be carefully designed. In this case, the socket is created outside of the ServeOneJabber so the responsibility can be shared. If the ServeOneJabber constructor fails, it will just throw the exception to the caller, who will then clean up the thread. But if the constructor succeeds, then the ServeOneJabber object takes over responsibility for cleaning up the thread, in its run( ).

Notice the simplicity of the MultiJabberServer. As before, a ServerSocket is created and accept( ) is called to allow a new connection. But this time, the return value of accept( ) (a Socket) is passed to the constructor for ServeOneJabber, which creates a new thread to handle that connection. When the connection is terminated, the thread simply goes away.

If the creation of the ServerSocket fails, the exception is again thrown through main( ). But if it succeeds, the outer try-finally guarantees its cleanup. The inner try-catch guards only against the failure of the ServeOneJabber constructor; if the constructor succeeds, then the ServeOneJabber thread will close the associated socket.

To test that the server really does handle multiple clients, the following program creates many clients (using threads) that connect to the same server. Each thread has a limited lifetime, and when it goes away, that leaves space for the creation of a new thread. The maximum number of threads allowed is determined by the final int maxthreads. You’ll notice that this value is rather critical, since if you make it too high the threads seem to run out of resources and the program mysteriously fails.

The JabberClientThread constructor takes an InetAddress and uses it to open a Socket. You’re probably starting to see the pattern: the Socket is always used to create some kind of Reader and/or Writer (or InputStream and/or OutputStream) object, which is the only way that the Socket can be used. (You can, of course, write a class or two to automate this process instead of doing all the typing if it becomes painful.) Again, start( ) performs thread initialization and calls run( ). Here, messages are sent to the server and information from the server is echoed to the screen. However, the thread has a limited lifetime and eventually completes. Note that the socket is cleaned up if the constructor fails after the socket is created but before the constructor completes. Otherwise the responsibility for calling close( ) for the socket is relegated to the run( ) method.

The threadcount keeps track of how many JabberClientThread objects currently exist. It is incremented as part of the constructor and decremented as run( ) exits (which means the thread is terminating). In MultiJabberClient.main( ), you can see that the number of threads is tested, and if there are too many, no more are created. Then the method sleeps. This way, some threads will eventually terminate and more can be created. You can experiment with MAX_THREADS to see where your particular system begins to have trouble with too many connections.

The examples you’ve seen so far use the Transmission Control Protocol (TCP, also known as stream-based sockets), which is designed for ultimate reliability and guarantees that the data will get there. It allows retransmission of lost data, it provides multiple paths through different routers in case one goes down, and bytes are delivered in the order they are sent. All this control and reliability comes at a cost: TCP has a high overhead.

There’s a second protocol, called User Datagram Protocol (UDP), which doesn’t guarantee that the packets will be delivered and doesn’t guarantee that they will arrive in the order they were sent. It’s called an “unreliable protocol” (TCP is a “reliable protocol”), which sounds bad, but because it’s much faster it can be useful. There are some applications, such as an audio signal, in which it isn’t so critical if a few packets are dropped here or there but speed is vital. Or consider a time-of-day server, where it really doesn’t matter if one of the messages is lost. Also, some applications might be able to fire off a UDP message to a server and can then assume, if there is no response in a reasonable period of time, that the message was lost.

The support for datagrams in Java has the same feel as its support for TCP sockets, but there are significant differences. With datagrams, you put a DatagramSocket on both the client and server, but there is no analogy to the ServerSocket that waits around for a connection. That’s because there is no “connection,” but instead a datagram just shows up. Another fundamental difference is that with TCP sockets, once you’ve made the connection you don’t need to worry about who’s talking to whom anymore; you just send the data back and forth through conventional streams. However, with datagrams, the datagram packet must know where it came from and where it’s supposed to go. That means you must know these things for each datagram packet that you load up and ship off.

A DatagramSocket sends and receives the packets, and the DatagramPacket contains the information. When you’re receiving a datagram, you need only provide a buffer in which the data will be placed; the information about the Internet address and port number where the information came from will be automatically initialized when the packet arrives through the DatagramSocket. So the constructor for a DatagramPacket to receive datagrams is:

in which buf is an array of byte. Since buf is an array, you might wonder why the constructor couldn’t figure out the length of the array on its own. I wondered this, and can only guess that it’s a throwback to C-style programming, in which of course arrays can’t tell you how big they are.

You can reuse a receiving datagram; you don’t have to make a new one each time. Every time you reuse it, the data in the buffer is overwritten.

The maximum size of the buffer is restricted only by the allowable datagram packet size, which limits it to slightly less than 64Kbytes. However, in many applications you’ll want it to be much smaller, certainly when you’re sending data. Your chosen packet size depends on what you need for your particular application.

When you send a datagram, the DatagramPacket must contain not only the data, but also the Internet address and port where it will be sent. So the constructor for an outgoing DatagramPacket is:

This time, buf (which is a byte array) already contains the data that you want to send out. The length might be the length of buf, but it can also be shorter, indicating that you want to send only that many bytes. The other two arguments are the Internet address where the packet is going and the destination port within that machine.[60]

You might think that the two constructors create two different objects: one for receiving datagrams and one for sending them. Good OO design would suggest that these should be two different classes, rather than one class with different behavior depending on how you construct the object. This is probably true, but fortunately the use of DatagramPackets is simple enough that you’re not tripped up by the problem, as you can see in the following example. This example is similar to the MultiJabberServer and MultiJabberClient example for TCP sockets. Multiple clients will send datagrams to a server, which will echo them back to the same client that sent the message.

To simplify the creation of a DatagramPacket from a String and vice-versa, the example begins with a utility class, Dgram, to do the work for you:

The first method of Dgram takes a String, an InetAddress, and a port number and builds a DatagramPacket by copying the contents of the String into a byte buffer and passing the buffer into the DatagramPacket constructor. Notice the “+1” in the buffer allocation – this was necessary to prevent truncation. The getBytes( ) method of String is a special operation that copies the chars of a String into a byte buffer. This method is now deprecated; Java 1.1 has a “better” way to do this but it’s commented out here because it truncates the String. So you’ll get a deprecation message when you compile it under Java 1.1, but the behavior will be correct. (This bug might be fixed by the time you read this.)

The Dgram.toString( ) method shows both the Java 1.0 approach and the Java 1.1 approach (which is different because there’s a new kind of String constructor).

Here is the server for the datagram demonstration:

The ChatterServer contains a single DatagramSocket for receiving messages, instead of creating one each time you’re ready to receive a new message. The single DatagramSocket can be used repeatedly. This DatagramSocket has a port number because this is the server and the client must have an exact address where it wants to send the datagram. It is given a port number but not an Internet address because it resides on “this” machine so it knows what its Internet address is (in this case, the default localhost). In the infinite while loop, the socket is told to receive( ), whereupon it blocks until a datagram shows up, and then sticks it into our designated receiver, the DatagramPacket dp. The packet is converted to a String along with information about the Internet address and socket where the packet came from. This information is displayed, and then an extra string is added to indicate that it is being echoed back from the server.

Now there’s a bit of a quandary. As you will see, there are potentially many different Internet addresses and port numbers that the messages might come from – that is, the clients can reside on any machine. (In this demonstration they all reside on the localhost, but the port number for each client is different.) To send a message back to the client that originated it, you need to know that client’s Internet address and port number. Fortunately, this information is conveniently packaged inside the DatagramPacket that sent the message, so all you have to do is pull it out using getAddress( ) and getPort( ), which are used to build the DatagramPacket echo that is sent back through the same socket that’s doing the receiving. In addition, when the socket sends the datagram, it automatically adds the Internet address and port information of this machine, so that when the client receives the message, it can use getAddress( ) and getPort( ) to find out where the datagram came from. In fact, the only time that getAddress( ) and getPort( ) don’t tell you where the datagram came from is if you create a datagram to send and you call getAddress( ) and getPort( ) before you send the datagram (in which case it tells the address and port of this machine, the one the datagram is being sent from). This is an essential part of datagrams: you don’t need to keep track of where a message came from because it’s always stored inside the datagram. In fact, the most reliable way to program is if you don’t try to keep track, but instead always extract the address and port from the datagram in question (as is done here).

To test this server, here’s a program that makes a number of clients, all of which fire datagram packets to the server and wait for the server to echo them back.

ChatterClient is created as a Thread so that multiple clients can be made to bother the server. Here you can see that the receiving DatagramPacket looks just like the one used for ChatterServer. In the constructor, the DatagramSocket is created with no arguments since it doesn’t need to advertise itself as being at a particular port number. The Internet address used for this socket will be “this machine” (for the example, localhost) and the port number will be automatically assigned, as you will see from the output. This DatagramSocket, like the one for the server, will be used both for sending and receiving.

The hostAddress is the Internet address of the host machine you want to talk to. The one part of the program in which you must know an exact Internet address and port number is the part in which you make the outgoing DatagramPacket. As is always the case, the host must be at a known address and port number so that clients can originate conversations with the host.

Each thread is given a unique identification number (although the port number automatically assigned to the thread would also provide a unique identifier). In run( ), a message String is created that contains the thread’s identification number and the message number this thread is currently sending. This String is used to create a datagram that is sent to the host at its address; the port number is taken directly from a constant in ChatterServer. Once the message is sent, receive( ) blocks until the server replies with an echoing message. All of the information that’s shipped around with the message allows you to see that what comes back to this particular thread is derived from the message that originated from it. In this example, even though UDP is an “unreliable” protocol, you’ll see that all of the datagrams get where they’re supposed to. (This will be true for localhost and LAN situations, but you might begin to see some failures for non-local connections.)

When you run this program, you’ll see that each of the threads finishes, which means that each of the datagram packets sent to the server is turned around and echoed to the correct recipient; otherwise one or more threads would hang, blocking until their input shows up.

You might think that the only right way to, for example, transfer a file from one machine to another is through TCP sockets, since they’re “reliable.” However, because of the speed of datagrams they can actually be a better solution. You simply break the file up into packets and number each packet. The receiving machine takes the packets and reassembles them; a “header packet” tells the machine how many to expect and any other important information. If a packet is lost, the receiving machine sends a datagram back telling the sender to retransmit.

Now let’s consider creating an application to run on the Web, which will show Java in all its glory. Part of this application will be a Java program running on the Web server, and the other part will be an applet that’s downloaded to the browser. The applet collects information from the user and sends it back to the application running on the Web server. The task of the program will be simple: the applet will ask for the email address of the user, and after verifying that this address is reasonably legitimate (it doesn’t contain spaces, and it does contain an ‘@’ symbol) the applet will send the email address to the Web server. The application running on the server will capture the data and check a data file in which all of the email addresses are kept. If that address is already in the file, it will send back a message to that effect, which is displayed by the applet. If the address isn’t in the file, it is placed in the list and the applet is informed that the address was added successfully.

Traditionally, the way to handle such a problem is to create an HTML page with a text field and a “submit” button. The user can type whatever he or she wants into the text field, and it will be submitted to the server without question. As it submits the data, the Web page also tells the server what to do with the data by mentioning the Common Gateway Interface (CGI) program that the server should run after receiving this data. This CGI program is typically written in either Perl or C (and sometimes C++, if the server supports it), and it must handle everything. First it looks at the data and decides whether it’s in the correct format. If not, the CGI program must create an HTML page to describe the problem; this page is handed to the server, which sends it back to the user. The user must then back up a page and try again. If the data is correct, the CGI program opens the data file and either adds the email address to the file or discovers that the address is already in the file. In both cases it must format an appropriate HTML page for the server to return to the user.

As Java programmers, this seems like an awkward way for us to solve the problem, and naturally, we’d like to do the whole thing in Java. First, we’ll use a Java applet to take care of data validation at the client site, without all that tedious Web traffic and page formatting. Then let’s skip the Perl CGI script in favor of a Java application running on the server. In fact, let’s skip the Web server altogether and simply make our own network connection from the applet to the Java application on the server!

As you’ll see, there are a number of issues that make this a more complicated problem than it seems. It would be ideal to write the applet using Java 1.1 but that’s hardly practical. At this writing, the number of users running Java 1.1-enabled browsers is small, and although such browsers are now commonly available, you’ll probably need to take into account that a significant number of users will be slow to upgrade. So to be on the safe side, the applet will be programmed using only Java 1.0 code. With this in mind, there will be no JAR files to combine .class files in the applet, so the applet should be designed to create as few .class files as possible to minimize download time.

Well, it turns out the Web server (the one available to me when I wrote the example) does have Java in it, but only Java 1.0! So the server application must also be written using Java 1.0.

Now consider the server application, which will be called NameCollector. What happens if more than one user at a time tries to submit their email addresses? If NameCollector uses TCP/IP sockets, then it must use the multithreading approach shown earlier to handle more than one client at a time. But all of these threads will try to write to a single file where all the email addresses will be kept. This would require a locking mechanism to make sure that more than one thread doesn’t access the file at once. A semaphore will do the trick, but perhaps there’s a simpler way.

If we use datagrams instead, multithreading is unnecessary. A single datagram socket will listen for incoming datagrams, and when one appears the program will process the message and send the reply as a datagram back to whomever sent the request. If the datagram gets lost, then the user will notice that no reply comes and can then re-submit the request.

When the server application receives a datagram and unpacks it, it must extract the email address and check the file to see if that address is there already (and if it isn’t, add it). And now we run into another problem. It turns out that Java 1.0 doesn’t quite have the horsepower to easily manipulate the file containing the email addresses (Java 1.1 does). However, the problem can be solved in C quite readily, and this will provide an excuse to show you the easiest way to connect a non-Java program to a Java program. A Runtime object for a program has a method called exec( ) that will start up a separate program on the machine and return a Process object. You can get an OutputStream that connects to standard input for this separate program and an InputStream that connects to standard output. All you need to do is write a program using any language that takes its input from standard input and writes the output to standard output. This is a convenient trick when you run into a problem that can’t be solved easily or quickly enough in Java (or when you have legacy code you don’t want to rewrite). You can also use Java’s native methods (see Appendix A) but those are much more involved.

The job of this non-Java application (written in C because Java wasn’t appropriate for CGI programming; if nothing else, the startup time is prohibitive) is to manage the list of email addresses. Standard input will accept an email address and the program will look up the name in the list to see if it’s already there. If not, it will add it and report success, but if the name is already there then it will report that. Don’t worry if you don’t completely understand what the following code means; it’s just one example of how you can write a program in another language and use it from Java. The particular programming language doesn’t really matter as long as it can read from standard input and write to standard output.

This assumes that the C compiler accepts ‘//’ style comments. (Many do, and you can also compile this program with a C++ compiler.) If yours doesn’t, simply delete those comments.

The first function in the file checks to see whether the name you hand it as a second argument (a pointer to a char) is in the file. Here, the file is passed as a FILE pointer to an already-opened file (the file is opened inside main( )). The function fseek( ) moves around in the file; here it is used to move to the top of the file. fgets( ) reads a line from the file list into the buffer lbuf, not exceeding the buffer size BSIZE. This is inside a while loop so that each line in the file is read. Next, strchr( ) is used to locate the newline character so that it can be stripped off. Finally, strcmp( ) is used to compare the name you’ve passed into the function to the current line int the file. strcmp( ) returns zero if it finds a match. In this case the function exits and a one is returned to indicate that yes, the name was already in the list. (Note that the function returns as soon as it discovers the match, so it doesn’t waste time looking at the rest of the list.) If you get all the way through the list without a match, the function returns zero.

In main( ), the file is opened using fopen( ). The first argument is the file name and the second is the way to open the file; a+ means “Append, and open (or create if the file does not exist) for update at the end of the file.” The fopen( ) function returns a FILE pointer which, if it’s zero, means that the open was unsuccessful. This is dealt with by printing an error message with perror( ) and terminating the program with exit( ).

Assuming that the file was opened successfully, the program enters an infinite loop. The function call gets(buf) gets a line from standard input (which will be connected to the Java program, remember) and places it in the buffer buf. This is simply passed to the alreadyInList( ) function, and if it’s already in the list, printf( ) sends that message to standard output (where the Java program is listening). fflush( ) is a way to flush the output buffer.

If the name is not already in the list, fseek( ) is used to move to the end of the list and fprintf( ) “prints” the name to the end of the list. Then printf( ) is used to indicate that the name was added to the list (again flushing standard output) and the infinite loop goes back to waiting for a new name.

Remember that you usually cannot compile this program on your computer and load it onto the Web server machine, since that machine might use a different processor and operating system. For example, my Web server runs on an Intel processor but it uses Linux, so I must download the source code and compile using remote commands (via telnet) with the C compiler that comes with the Linux distribution.

This program will first start the C program above and make the necessary connections to talk to it. Then it will create a datagram socket that will be used to listen for datagram packets from the applet.

The first definitions in NameCollector should look familiar: the port is chosen, a datagram packet is created, and there’s a handle to a DatagramSocket. The next three definitions concern the connection to the C program: a Process object is what comes back when the C program is fired up by the Java program, and that Process object produces the InputStream and OutputStream objects representing, respectively, the standard output and standard input of the C program. These must of course be “wrapped” as is usual with Java IO, so we end up with a PrintStream and DataInputStream.

All the work for this program happens inside the constructor. To start up the C program, the current Runtime object is procured. This is used to call exec( ), which returns the Process object. You can see that there are simple calls to produce the streams from the Process object: getOutputStream( ) and getInputStream( ). From this point on, all you need to consider is sending data to the stream nameList and getting the results from addResult.

As before, a DatagramSocket is connected to a port. Inside the infinite while loop, the program calls receive( ), which blocks until a datagram shows up. When the datagram appears, its contents are extracted into the String rcvd. This is trimmed to remove white space at each end and sent to the C program in the line:

This is only possible because Java’s exec( ) provides access to any executable that reads from standard input and writes to standard output. There are other ways to talk to non-Java code, which are discussed in Appendix A.

Capturing the result from the C program is slightly more complicated. You must call read( ) and provide a buffer where the results will be placed. The return value for read( ) is the number of bytes that came from the C program, and if this value is -1 it means that something is wrong. Otherwise, the resultBuf is turned into a String and the spaces are trimmed off. This string is then placed into a DatagramPacket as before and shipped back to the same address that sent the request in the first place. Note that the sender’s address is part of the DatagramPacket we received.

Remember that although the C program must be compiled on the Web server, the Java program can be compiled anywhere since the resulting byte codes will be the same regardless of the platform on which the program will be running.

As mentioned earlier, the applet must be written with Java 1.0 so that it will run on the largest number of browsers, so it’s best if the number of classes produced is minimized. Thus, instead of using the Dgram class developed earlier, all of the datagram manipulations will be placed in line. In addition, the applet needs a thread to listen for the reply from the server, and instead of making this a separate thread it’s integrated into the applet by implementing the Runnable interface. This isn’t as easy to read, but it produces a one-class (and one-server-hit) applet:

The UI for the applet is quite simple. There’s a TextField in which you type your email address, and a Button to send the email address to the server. Two Labels are used to report status back to the user.

By now you can recognize the DatagramSocket, InetAddress, buffer, and DatagramPacket as trappings of the network connection. Lastly, you can see the run( ) method that implements the thread portion so the applet can listen for the reply sent back by the server.

The init( ) method sets up the GUI with the familiar layout tools, then creates the DatagramSocket that will be used both for sending and receiving datagrams.

The action( ) method (remember, we’re confined to Java 1.0 now, so we can’t use any slick inner listener classes) watches only to see if you press the “send” button. When the button is pressed, the first action is to check the Thread pl to see if it’s null. If it’s not null, there’s a live thread running. The first time the message is sent a thread is started up to watch for the reply. Thus, if a thread is running, it means this is not the first time the user has tried to send the message. The pl handle is set to null and the old listener is interrupted. (This is the preferred approach, since stop( ) is deprecated in Java 1.2 as explained in the previous chapter.)

Regardless of whether this is the first time the button was pressed, the text in l2 is erased.

The next group of statements checks the email name for errors. The String.indexOf( ) method is used to search for illegal characters, and if one is found it is reported to the user. Note that all of this happens without any network activity, so it’s fast and it doesn’t bog down the Internet.

Once the name is verified, it is packaged into a datagram and sent to the host address and port number in the same way that was described in the earlier datagram example. The first label is changed to show you that the send has occurred, and the button text is changed so that it reads “re-send.” At this point, the thread is started up and the second label informs you that the applet is waiting for a reply from the server.

The run( ) method for the thread uses the DatagramSocket that lives in NameSender to receive( ), which blocks until the datagram packet comes from the server. The resulting packet is placed into NameSender’s DatagramPacket dp. The data is retrieved from the packet and placed into the second label in NameSender. At this point, the thread terminates and becomes dead. If the reply doesn’t come back from the server in a reasonable amount of time, the user might become impatient and press the button again, thus terminating the current thread (and, after re-sending the data, starting a new one). Because a thread is used to listen for the reply, the user still has full use of the UI.

Of course, the applet must go inside a Web page. Here is the complete Web page; you can see that it’s intended to be used to automatically collect names for my mailing list:

The applet tag is quite trivial, no different from the first one presented in Chapter 13.

This certainly seems like an elegant approach. There’s no CGI programming and so there are no delays while the server starts up a CGI program. The datagram approach seems to produce a nice quick response. In addition, when Java 1.1 is available everywhere, the server portion can be written entirely in Java. (Although it’s quite interesting to see how easy it is to connect to a non-Java program using standard input and output.)

There are problems, however. One problem is rather subtle: since the Java application is running constantly on the server and it spends most of its time blocked in the Datagram.receive( ) method, there might be some CPU hogging going on. At least, that’s the way it appeared on the server where I was experimenting. On the other hand, there wasn’t much else happening on that server, and starting the program using “nice” (a Unix program to prevent a process from hogging the CPU) or its equivalent could solve the problem if you have a more heavily-loaded server. In any event, it’s worth keeping your eye on an application like this – a blocked receive( ) could hog the CPU.

The second problem is a show stopper. It concerns firewalls. A firewall is a machine that sits between your network and the Internet. It monitors all traffic coming in from the Internet and going out to the Internet, and makes sure that traffic conforms to what it expects.

Firewalls are conservative little beasts. They demand strict conformance to all the rules, and if you’re not conforming they assume that you’re doing something sinful and shut you out (not quite so bad as the Spanish Inquisition, but close). For example, if you are on a network behind a firewall and you start connecting to the Internet using a Web browser, the firewall expects that all your transactions will connect to the server using the accepted http port, which is 80. Now along comes this Java applet NameSender, which is trying to send a datagram to port 8080, which is way outside the range of the “protected” ports 0-1024. The firewall naturally assumes the worst – that someone has a virus – and it doesn’t allow the transaction to happen.

As long as your customers have raw connections to the Internet (for example, using a typical Internet service provider) there’s no problem, but you might have some important customers dwelling behind firewalls, and they won’t be able to use your program.

This is rather disheartening after learning so much Java, because it would seem that you must give up Java on the server and learn how to write CGI scripts in C or Perl. But as it turns out, despair is not in order.

One scenario is part of Sun’s grand scheme. If everything goes as planned, Web servers will be equipped with servlet servers. These will take a request from the client (going through the firewall-accepted port 80) and instead of starting up a CGI program they will start up a Java program called a servlet. This is a little application that’s designed to run only on the server. A servlet server will automatically start up the servlet to handle the client request, which means you can write all your programs in Java (further enabling the “100 percent pure Java initiative”). It is admittedly an appealing idea: once you’re comfortable with Java, you don’t have to switch to a more primitive language to handle requests on the server.

Since it’s only for handling requests on the server, the servlet API has no GUI abilities. This fits quite well with NameCollector.java, which doesn’t have a GUI anyway.

At this writing, a low-cost servlet server was available from java.sun.com. In addition, Sun is encouraging other Web server manufacturers to add servlet capabilities to their servers.

A Java program can send a CGI request to a server just like an HTML page can. As with HTML pages, this request can be either a GET or a POST. In addition, the Java program can intercept the output of the CGI program, so you don’t have to rely on the program to format a new page and force the user to back up from one page to another if something goes wrong. In fact, the appearance of the program can be the same as the previous version.

It also turns out that the code is simpler, and that CGI isn’t difficult to write after all. (An innocent statement that’s true of many things – after you understand them.) So in this section you’ll get a crash course in CGI programming. To solve the general problem, some CGI tools will be created in C++ that will allow you to easily write a CGI program to solve any problem. The benefit to this approach is portability – the example you are about to see will work on any system that supports CGI, and there’s no problem with firewalls.

This example also works out the basics of creating any connection with applets and CGI programs, so you can easily adapt it to your own projects.

In this version, the name and the email address will be collected and stored in the file in the form:

This is a convenient form for many mailers. Since two fields are being collected, there are no shortcuts because CGI has a particular format for encoding the data in fields. You can see this for yourself if you make an ordinary HTML page and add the lines:

This creates two data entry fields called name and email, along with a submit button that collects the data and sends it to a CGI program. Listmgr2.exe is the name of the executable program that resides in the directory that’s typically called “cgi-bin” on your Web server.[61] (If the named program is not in the cgi-bin directory, you won’t see any results.) If you fill out this form and press the “submit” button, you will see in the URL address window of the browser something like:

(Without the line break, of course). Here you see a little bit of the way that data is encoded to send to CGI. For one thing, spaces are not allowed (since spaces typically separate command-line arguments). Spaces are replaced by ‘+’ signs. In addition, each field contains the field name (which is determined by the HTML page) followed by an ‘=’ and the field data, and terminated by a ‘&’.

At this point, you might wonder about the ‘+’, ‘=,’ and ‘&’. What if those are used in the field, as in “John & Marsha Smith”? This is encoded to:

That is, the special character is turned into a ‘%’ followed by its ASCII value in hex.

Fortunately, Java has a tool to perform this encoding for you. It’s a static method of the class URLEncoder called encode( ). You can experiment with this method using the following program:

This takes the command-line arguments and combines them into a string of words separated by spaces (the final space is removed using String.trim( )). These are then encoded and printed.

To invoke a CGI program, all the applet needs to do is collect the data from its fields (or wherever it needs to collect the data from), URL-encode each piece of data, and then assemble it into a single string, placing the name of each field followed by an ‘=’, followed by the data, followed by an ‘&’. To form the entire CGI command, this string is placed after the URL of the CGI program and a ‘?’. That’s all it takes to invoke any CGI program, and as you’ll see you can easily do it within an applet.

The applet is actually considerably simpler than NameSender.java, partly because it’s so easy to send a GET request and also because no thread is required to wait for the reply. There are now two fields instead of one, but you’ll notice that much of the applet looks familiar, from NameSender.java.