Adding Socket Timeout to java.net.URLConnection (JDK 1.2)

found a bug , see "connected = true;" in public void connect() throws IOException {

---

Note: 05/11/01 Sam Found a patch for borland

As I got the email:

Just writing to inform you theis patch for 1.3 works with the 1.3 shipped with borland JBuilder 4 (not sure which excat version it is)

the only problems I had where that the code was a bit messed up, following are the changes made to it to make it work.

				
  public void SetTimeout(int i) 
    throws SocketException 
  { 
    this.timeout = i; // Should be i not -1   <------------ERROR 
    serverSocket.setSoTimeout(i) ; 
  } 

  public boolean parseHTTP(MessageHeader header, ProgressEntry entry) 
    throws java.io.IOException 
  { 
    if( this.timeout != -1 ) { 
      try { 
        serverSocket.setSoTimeout(timeout) ; // should be timeout not i <---------------ERROR 
      } catch( SocketException e ) { 
        throw new java.io.IOException("unable to set socket timeout!") ; 
      } 
    } 

    return super.parseHTTP(header, entry) ; 
  } 
Sam 

Under JDK 1.3, which is HTTP 1.1 compatible, the InterruptedIOException gets caught by the socket I/O routines and ignored. input is read in "chunks". I debugged the existing code under 1.3, the Timeout is getting set properly etc., but the exception gets caught in the underlying I/O routines, which have a single retry if any IOExceptions are thrown. Thanks a lot Sun....

3/22/01: Patch for JDK 1.3 unverified

Patch code for JDK 1.3 from Matt Ho (unverified)

				
----[ snip ]----

import sun.net.www.MessageHeader ;
import sun.net.ProgressEntry ;

.
.
.

  private int timeout = -1 ;

  public void SetTimeout(int i)
    throws SocketException
  {
    this.timeout = -1 ;
    serverSocket.setSoTimeout(i) ;
  }

  public boolean parseHTTP(MessageHeader header, ProgressEntry entry)
    throws java.io.IOException
  {
    if( this.timeout != -1 ) {
      try {
        serverSocket.setSoTimeout(i) ;
      } catch( SocketException e ) {
        throw new java.io.IOException("unable to set socket timeout!") ;
      }
    }

    return super.parseHTTP(header, entry) ;
  }

----[ snip ]----

On with the rest of the stuff

The BSD socket API supports a timeout option (the option is SO_TIMEOUT), which is also supported in java.net.socket. Unfortunately, java.net.URLConnection does not expose the underlying socket. So if you have a URL connection that attempts to connect to a dead URL (i.e., the URL is well formed and exists but the site is down), the socket will eventually timeout using the operating system's default timeout (420 seconds on Win NT). The timeout is a very long time, e.g., for spiders or URL checking.

The following files illustrate a technique to introduce a socket timeout to URL connection, based upon the actual java source code itself (see the open source community licensing at JavaSoft).

The Base classes, or URLConnection internals

Java's implementation of networking is protocol independent, as well as object oriented. Therefore the implementation is not as straightfoward as one might imagine.

URLConnection relies upon several internal classes using a client/server model as well as a "factory" design pattern. The client's base class is sun.net.www.http.HttpClient. This class is extended for the purpose of exposing the socket.

The default factory is URLStreamHandlerFactory, which indirectly "handles" the creation of an HTTP client by instantiating a class that is specific to the HTTP protocol: sun.net.www.protocol.http.Handler. The handler actually creates the client.

In practice, the factory is only necessary to mimic java's implementation, but only the Handler is really needed.

Derived Classes

We derive 4 classes so as to preserve the symmetry with the java source code:

HttpURLConnectionTimeout extends sun.net.www.protocol.http.HttpURLConnection
HttpTimeoutHandler extends sun.net.www.protocol.http.Handler
HttpTimeoutFactory implements java.net.URLStreamHandlerFactory
HttpClientTimeout extends sun.net.www.http.HttpClient

On with the source code.

HttpURLConnectionTimeout

// whatever package you want
import sun.net.www.http.HttpClient;
import java.net.*;
import java.io.*;
public class HttpClientTimeout extends HttpClient
{
    public HttpClientTimeout(URL url, String proxy, int proxyPort) throws IOException
	{
		super(url, proxy, proxyPort);
	}

    public HttpClientTimeout(URL url) throws IOException
	{
		super(url, null, -1);
    }

	public void SetTimeout(int i) throws SocketException { 
    	serverSocket.setSoTimeout(i); 
    } 

    /* This class has no public constructor for HTTP.  This method is used to
     * get an HttpClient to the specifed URL.  If there's currently an
     * active HttpClient to that server/port, you'll get that one.
	 *
	 * no longer syncrhonized -- it slows things down too much
	 * synchronize at a higher level
     */
    public static HttpClientTimeout GetNew(URL url)  
    throws IOException {
	/* see if one's already around */
	HttpClientTimeout ret = (HttpClientTimeout) kac.get(url);
	if (ret == null) {
	    ret = new HttpClientTimeout (url);  // CTOR called openServer()
	} else {
	    ret.url = url;
	}
	// don't know if we're keeping alive until we parse the headers
	// for now, keepingAlive is false
	return ret;
    }

	public void Close() throws IOException
	{
		serverSocket.close();
	}

	public Socket GetSocket()
	{
		return serverSocket;
	}


}

HttpTimeoutFactory

import java.net.*;

public class HttpTimeoutFactory implements URLStreamHandlerFactory
{
	int fiTimeoutVal;
	public HttpTimeoutFactory(int iT) { fiTimeoutVal = iT; }
	public URLStreamHandler createURLStreamHandler(String str)
	{
		return new HttpTimeoutHandler(fiTimeoutVal); 
	}

}

HttpTimeoutHandler

import java.net.*;
import java.io.IOException;

public class HttpTimeoutHandler extends sun.net.www.protocol.http.Handler
{
	int fiTimeoutVal;
	HttpURLConnectionTimeout fHUCT;
	public HttpTimeoutHandler(int iT) { fiTimeoutVal = iT; }

    protected java.net.URLConnection openConnection(URL u) throws IOException {
		return fHUCT = new HttpURLConnectionTimeout(u, this, fiTimeoutVal);
    }

    String GetProxy() { return proxy; }		// breaking encapsulation
    int GetProxyPort() { return proxyPort; }    // breaking encapsulation

	public void Close() throws Exception 
	{
		fHUCT.Close();
	}

	public Socket GetSocket()
	{
		return fHUCT.GetSocket();
	}
}

HttpURLConnectionTimeout

import java.net.*;
import java.io.*;
import sun.net.www.http.HttpClient;

public class HttpURLConnectionTimeout extends sun.net.www.protocol.http.HttpURLConnection
{
	int fiTimeoutVal;
    HttpTimeoutHandler fHandler;
	HttpClientTimeout fClient;
  	public HttpURLConnectionTimeout(URL u, HttpTimeoutHandler handler, int iTimeout) throws IOException
	{   
    	super(u, handler);
		fiTimeoutVal = iTimeout;
	}

	public HttpURLConnectionTimeout(URL u,  String host, int port) throws IOException
	{  
    	super(u, host, port);
	}

    public void connect() throws IOException {
	if (connected) {
	    return;
	}
	try {
	    if ("http".equals(url.getProtocol()) /* && !failedOnce <- PRIVATE */ ) {
		// for safety's sake, as reported by KLGroup
		synchronized (url)
		{
			http = HttpClientTimeout.GetNew(url);
		}
		fClient = (HttpClientTimeout)http;
   		((HttpClientTimeout)http).SetTimeout(fiTimeoutVal);
	    } else {
		// make sure to construct new connection if first
		// attempt failed
		http = new HttpClientTimeout(url, fHandler.GetProxy(), fHandler.GetProxyPort());
	    }
	    ps = (PrintStream)http.getOutputStream();
	} catch (IOException e) {
	    throw e;  }
		// this was missing from the original version
		connected = true;
	}

    /**
     * Create a new HttpClient object, bypassing the cache of
     * HTTP client objects/connections.
     *
     * @param url	the URL being accessed
     */
    protected HttpClient getNewClient (URL url)
    throws IOException {
		HttpClientTimeout client = new HttpClientTimeout (url, (String)null, -1);
		try {
		client.SetTimeout(fiTimeoutVal);
		} catch (Exception e) 
		{ System.out.println("Unable to set timeout value"); }
	return (HttpClient)client; 
    }

    /**
     * opens a stream allowing redirects only to the same host.
     */
    public static InputStream openConnectionCheckRedirects(URLConnection c)
	throws IOException
    {
        boolean redir;
        int redirects = 0;
        InputStream in = null;

        do {
            if (c instanceof HttpURLConnectionTimeout) {
                ((HttpURLConnectionTimeout) c).setInstanceFollowRedirects(false);
            }

            // We want to open the input stream before
            // getting headers, because getHeaderField()
            // et al swallow IOExceptions.
            in = c.getInputStream();
            redir = false;

            if (c instanceof HttpURLConnectionTimeout) {
                HttpURLConnectionTimeout http = (HttpURLConnectionTimeout) c;
                int stat = http.getResponseCode();
                if (stat >= 300 && stat <= 305 &&
                        stat != HttpURLConnection.HTTP_NOT_MODIFIED) {
                    URL base = http.getURL();
                    String loc = http.getHeaderField("Location");
                    URL target = null;
                    if (loc != null) {
                        target = new URL(base, loc);
                    }
                    http.disconnect();
                    if (target == null
                        || !base.getProtocol().equals(target.getProtocol())
                        || base.getPort() != target.getPort()
                        || !HostsEquals(base, target)
                        || redirects >= 5)
                    {
                        throw new SecurityException("illegal URL redirect");
		    }
                    redir = true;
                    c = target.openConnection();
                    redirects++;
                }
            }
        } while (redir);
        return in;
    }

    // Same as java.net.URL.hostsEqual


	static boolean HostsEquals(URL u1, URL u2) 
    { 
        final String h1 = u1.getHost();
        final String h2 = u2.getHost();

        if (h1 == null) {
            return h2 == null;
        } else if (h2 == null) {
            return false;
        } else if (h1.equalsIgnoreCase(h2)) {
            return true;
        }
            // Have to resolve addresses before comparing, otherwise
            // names like tachyon and tachyon.eng would compare different
        final boolean result[] = {false};

        java.security.AccessController.doPrivileged(
            new java.security.PrivilegedAction() {
            public Object run() {
            try {
                InetAddress a1 = InetAddress.getByName(h1);
                InetAddress a2 = InetAddress.getByName(h2);
                result[0] = a1.equals(a2);
            } catch(UnknownHostException e) {
            } catch(SecurityException e) {
            }
            return null;
            }
        });

        return result[0];
	}

	void Close() throws Exception
	{
		fClient.Close();
	}

	Socket GetSocket()
	{
		return fClient.GetSocket();
	}
}

Sample Usage #1

import java.net.*;
public class MainTest
{

	public static void main(String args[])
	{
	    int i = 0;
    	try {
			URL theURL = new URL((URL)null, "http://www.snowball.com", new HttpTimeoutHandler(150)); // timeout value in milliseconds

	        // the next step is optional
			theURL.setURLStreamHandlerFactory(new HttpTimeoutFactory(150));


			URLConnection theURLconn = theURL.openConnection();
	        theURLconn.connect();
    	    i = theURLconn.getContentLength();
        }
        catch (InterruptedIOException e)
        {
        	System.out.println("timeout on socket");
        }
        System.out.println("Done, Length:" + i);
	}
}

Sample Usage #2

		try
		{
			HttpTimeoutHandler xHTH = new HttpTimeoutHandler(10);	// timeout value in milliseconds
	        URL theURL = new URL((URL)null, "http://www.javasoft.com", xHTH);
	        HttpURLConnection theUC = theURL.openConnection();
			.
			.
			.
		}
		catch (InterruptedIOException e)
		{
			// socket timed out

		}

Some remarks: this code is thread safe.

More to come

来源:http://www.logicamente.com/sockets.html

     http://www.edevs.com/java-programming/15068/

Thanks Felipe!

If I understand information at http://www.logicamente.com/sockets.html correctly there are 2 problems with timeout when using HttpURLConnection in JDK 1.3:

1. HttpURLConnection does not allow changing the default timeout that is in order of few minutes.

2. If actual HTTP stream is chunked then HttpURLConnection ignores even the default timeout and tries to read what it perceives as a continued stream resulting in indefinite read wait.

The patch shown at the above URL, consisting of subclassing of 4 system classes (1 from java.net... and 3 from sun.net.www...), is aimed to resolve problem 1 above but does not help in problem 2.

My main problem is to have timeout when reading chunked stream (system default timeout will be ok to beginning with) and therefore the question is if this bug has been corrected in later versions of JDK? Thanks.

-----

I have seen much chat about this "problem", that is setSoTimeout not available or not working properly.

how about you write your own Timer (resettable) or 1.4 has Timer class

you just reset it anytime you detect network activity and close the Socket if the Timer finishes its cycle?