Connection reset while sending data from AWS Lambda to SQS queue

Question

I'm using AWS SDK for Java in which I'm sending data from AWS Lambda to SQS.

We are getting exception:

Caused by: java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:115)
at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:886)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:857)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124)
at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:160)
at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:113)
at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:120)
at org.apache.http.entity.StringEntity.writeTo(StringEntity.java:167)
at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)
at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:160)
at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)
at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doSendRequest(SdkHttpRequestExecutor.java:63)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1236)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)

Code:

 List<SendMessageBatchRequestEntry> sqsList= new LinkedList<SendMessageBatchRequestEntry>();
    int batchId = 0; //To send a unique batchId for each msg in a batch
    for (Metadata metadata: metadataList) {
        String jsonString = new Gson().toJson(metadata);
        sqsList.add(new SendMessageBatchRequestEntry(batchId + "", jsonString));
        batchId++;
    }
    amazonSqs.sendMessageBatch(new SendMessageBatchRequest(queueUrl, sqsList));

Background what we are trying to do:

We have a main Lambda function which create and initialize an SQS queue, and contain detail for every record that should be processed. Now SQS queue need to set up for creating batches of X number of messages from the queue and automatically invoke another SQS Lambda function for each of the batches.

Dejan Vasilevski · Accepted Answer · 2019-10-16 08:38:30Z

3

The maximum number of messages per batch is 10. You cannot fill the SQS queue with 20k at once and send that request. Try to break it into 10's.

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-limits.html#limits-queues

edited Oct 16, 2019 at 8:38

answered Jan 24, 2019 at 20:50

Dejan Vasilevski

464 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Raushan Over a year ago

Thanks Dejan. A simple tweak in code has worked. Adding code in the answer of this question.

Raushan · Accepted Answer · 2019-01-25 06:21:28Z

2

We can send it in the batch size of 10. Working code:

List<SendMessageBatchRequestEntry> sqsList= new LinkedList<SendMessageBatchRequestEntry>();
    int batchId = 1; //To send a unique batchId for each msg in a batch
    for (Metadata metadata: metadataList) {
        String jsonString = new Gson().toJson(metadata);
        if (sqsList.size() == 10) {
            amazonSqs.sendMessageBatch(new SendMessageBatchRequest(queueUrl, sqsList));
            sqsList.clear();
        } 
        sqsList.add(new SendMessageBatchRequestEntry(batchId + "", jsonString)); 
        batchId++;
    }
    if(sqsList.size()>0) {
        amazonSqs.sendMessageBatch(new SendMessageBatchRequest(queueUrl, sqsList));
    }

answered Jan 25, 2019 at 6:21

Raushan

3471 gold badge4 silver badges16 bronze badges

Comments

Matus Dubrava · Accepted Answer · 2019-01-24 10:06:36Z

0

Seems like your code is fine, and as far as I remember (I have seen this error myself several times), this happens from time to time when using SDK due to how SDK reuses HTTP connections. This error only tells you that your the Lambda, well, reseted the HTTP connection, but SDK has build in functionality to retry failed requests so you should be fine if you don't see this error on every single request.

answered Jan 24, 2019 at 10:06

Matus Dubrava

14.6k3 gold badges44 silver badges60 bronze badges

7 Comments

Raushan Over a year ago

Thanks Matus... Even i don't mind if the SDK has failover for this. But it happen for me every time. Total number of record which are trying to send is around 20k. Is the number of data which is creating a problem ?

Matus Dubrava Over a year ago

@Raushan Are you using FIFO or standard queue, and what is the size of a message that you are sending into the queue?

Raushan Over a year ago

It's a standard queue. And the size is 6800 KB :(. How to breakdown this ? I mean it should send in batches to the queue so that the other Lambda process it in batch. ( As we are restricted to 15 mins).

Matus Dubrava Over a year ago

@Raushan Max size of a message that you can send to queue is 256 KB, anything larger will be dropped. There are some workarounds for this but I would need to know the use case and contents of the messages. Why is the message so large?

Raushan Over a year ago

Well...We are moving our CRON jobs from EC2 instance to Lambda. One of the job which has 20k records and for each record need to collect their metadata from the service. So 15 mins is going to be short. Then we thought to push these data to queue and later handle this from another lambda which process X records in a single invocation. So SQS queue need to be created in batches of X message. That's why we are doing this. Hope I am going in right direction !!!

|

Collectives™ on Stack Overflow

Connection reset while sending data from AWS Lambda to SQS queue

3 Answers 3

1 Comment

Comments

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related