Using HTTPBuilder and getting "java.net.SocketException: No buffer space available (maximum connections reached?): connect"

Question

I'm using HTTPBuilder to build a client for the REST API of a NoSQL database. It works fine except when processing a large data set. After a large number of calls the client throws this exception:

java.net.SocketException: No buffer space available (maximum connections reached?): connect
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at  org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:446)
at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:417)
at groovyx.net.http.HTTPBuilder.request(HTTPBuilder.java:366)
at groovyx.net.http.HTTPBuilder$request.call(Unknown Source)
at scripts.groovy.prototypes.NeoClient.getRelationship(NeoClient.groovy:70)
at com.gsgenetics.genie.trait.Node$Edge.getTargetNode(Node.java:108)
at com.gsgenetics.genie.trait.Node$Edge.compareTo(Node.java:145)
at com.gsgenetics.genie.trait.Node$Edge.compareTo(Node.java:67)
at com.gsgenetics.genie.trait.Node.getEdges(Node.java:32)
at com.gsgenetics.genie.trait.Node.iterator(Node.java:384)
at com.gsgenetics.genie.GenomeStructuredGraph.add(GenomeStructuredGraph.java:131)
at com.gsgenetics.genie.GenomeStructuredGraph.add(GenomeStructuredGraph.java:197)
at com.gsgenetics.genie.GenomeStructuredGraph$add.call(Unknown Source)
at scripts.groovy.prototypes.read-ccds$_run_closure3.doCall(read-ccds.groovy:53)
at scripts.groovy.prototypes.read-ccds.run(read-ccds.groovy:50)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

It looks like some resources are not being released but I can' figure out what exactly and how to get them to release. I am running this on Windows 7 but I get a similar error when running it on Linux.

Here's the code where I am instantiating and using HTTPBuilder:

import groovyx.net.http.*
import net.sf.json.JSONArray
import net.sf.json.JSONObject

import static groovyx.net.http.ContentType.*
import static groovyx.net.http.Method.*

class NeoClient {

    static final URL = 'http://some IP address'
//    static def http = new HTTPBuilder( URL )
    static net.sf.json.JSONObject query(statement, params=null, success=null, error=null) {
        def http = new HTTPBuilder( URL )
        net.sf.json.JSONObject returnJson = []
        http.request( POST, JSON ) {
            uri.path = '/db/data/cypher/'
            headers.'X-Stream' = 'true'
            requestContentType = JSON
            body =  [ query : statement , params : params ?: [:] ]


            response.success = { resp, json ->
                if (success) success(json)
                else {
                    println "Status ${resp.statusLine} Columns ${json.columns}\nData: ${json.data}"
                }
                returnJson = json
            }

            response.failure = { resp, message ->
                def result=[status:resp.statusLine.statusCode,statusText:resp.statusLine.reasonPhrase]
                result.headers = resp.headers.collect { h -> [ (h.name) : h.value ] }
                result.message = message
                if (error) {
                    error(result)
                } else {
                    println "Status: ${result.status} : ${result.statusText} "
                    println "Headers: ${result.headers}"
                    println "Message: ${result.message}"
                }
                throw new Exception("Neo Client Exception");
            }
        }
    }
}

Note that I tried having a single instance of HTTPBuilder in my static class NeoClient, and also creating a new instance of HTTPBuilder for each call to "query", and I get the same exception in either case.

Any ideas would be greatly appreciated.

Thanks

You have to look at the server side to find out whether multiple connections are opened and not closed to the NoSQL db (say MongoDB). — dmahapatro
– dmahapatro, Commented Sep 18, 2013 at 13:28

Dror Bereznitsky · Accepted Answer · 2013-09-18 14:14:28Z

You should use the HTTPBuilder.shutdown method for releasing any system resources held by this instance.

In addition, creating a new HTTPBuilder for each request is not the best approach in terms of performance and resource management. A better approach can be having a single instance of HttpBuilder which handles multiple requests. To do so, you will need to override the default behavior of HttpBuilder to use a thread safe connection manager:

private static class CustomHttpBuilder extends HttpBuilder {
    protected AbstractHttpClient createClient( HttpParams params ) {
       def connManager = new MultiThreadedHttpConnectionManager()
       def connManagerParams = new HttpConnectionManagerParams()
       connManagerParams.maxTotalConnections = 50 // default is 20
       connManagerParams.defaultMaxConnectionsPerHost = 50 // default is 2
       connManager.params = connManagerParams
       new HttpClient(connManager)
   }
}

I got this code example from the Grails users forum. This will allow you to use a single instance of the CustomHttpBuilder for multiple requests.

Collectives™ on Stack Overflow

Using HTTPBuilder and getting "java.net.SocketException: No buffer space available (maximum connections reached?): connect"

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related