Java 17 runtime kinesis lambda code doesnt work localstack (easy code)

Question

Its very simple. This code works as jar and lambda (maven project):

package test2;

import java.util.Map;

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;

public class test implements RequestHandler<Map<String,String>, String>{

      @Override
      public String handleRequest(Map<String,String> event, Context context)
      {
        String prueba = "Prueba";
        return prueba;
      }
      public static void main(String[] args) {
        // TODO Auto-generated method stub
        System.out.println("Testing def");
      }
    }

Pom.xml with dependencies:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>test2</groupId>
  <artifactId>test2</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <build>
    <sourceDirectory>src</sourceDirectory>
    <plugins>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.8.1</version>
        <configuration>
          <release>17</release>
        </configuration>
      </plugin>
    </plugins>
  </build>
      <dependencies>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-core</artifactId>
            <version>1.0.0</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-events</artifactId>
            <version>3.11.2</version>
        </dependency>
    </dependencies>
</project>

I export it as a runable jar, with its run configuration: export; runnable jar file; runable configuration and package required libraries

Later I go to localstack:

aws --endpoint-url=http://localhost:4566 lambda create-function --function-name Test3 --runtime java17 --handler test2.test::handleRequest --role arn:aws:iam::000000000000:role/RolLambda --zip-file fileb://./test.jar

aws --endpoint-url=http://localhost:4566 lambda invoke --function-name Test3 output.json

Output: It works!

Now the code THAT DOESNT WORK as lambda (maven project too). I only added a kinesisclient object:

package main;

import java.net.URI;
import java.util.Map;

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;

import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.kinesis.KinesisClient;

public class Test implements RequestHandler<Map<String,String>, String>{

    public static void main(String[] args) {
        // TODO Auto-generated method stub
        System.out.println("Testing def");
    }
    @Override
    public String handleRequest(Map<String,String> event, Context context)
    {
        String Error = "kinesis_Error";
        String prueba = "Prueba";
        
        try {
            KinesisClient kinesis_client = KinesisClient.builder().region(Region.US_EAST_1).endpointOverride(URI.create("http://127.0.0.1:4566")).build();
        } catch (Exception e) {
            return Error;
        }
        
        return prueba;
    }
}

Pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>definitive_test</groupId>
  <artifactId>definitive_test</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <build>
    <sourceDirectory>src</sourceDirectory>
    <plugins>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.8.1</version>
        <configuration>
          <release>17</release>
        </configuration>
      </plugin>
    </plugins>
  </build>
      <dependencies>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-core</artifactId>
            <version>1.0.0</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-events</artifactId>
            <version>3.11.2</version>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>kinesis</artifactId>
            <version>2.20.135</version>
        </dependency>
        
    </dependencies>
</project>

I do the same steps: -runnable jar with the properly configuration set -create lambda and make the execution:

aws --endpoint-url=http://localhost:4566 lambda create-function --function-name Test3 --runtime java17 --handler main.Test::handleRequest --role arn:aws:iam::000000000000:role/RolLambda --zip-file fileb://./test.jar

But now I get the unhandled:

aws --endpoint-url=http://localhost:4566 lambda invoke --function-name Test3 output.json

{
    "StatusCode": 200,
    "FunctionError": "Unhandled",
    "ExecutedVersion": "$LATEST"
}

The output:

{"errorMessage":"software/amazon/awssdk/services/kinesis/KinesisClient","errorType":"java.lang.NoClassDefFoundError","stackTrace":["main.Test.handleRequest(Test.java:25)","java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)","java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)","java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)","java.base/java.lang.reflect.Method.invoke(Unknown Source)"],"cause":{"errorMessage":"software.amazon.awssdk.services.kinesis.KinesisClient","errorType":"java.lang.ClassNotFoundException","stackTrace":["java.base/java.net.URLClassLoader.findClass(Unknown Source)","java.base/java.lang.ClassLoader.loadClass(Unknown Source)","java.base/java.lang.ClassLoader.loadClass(Unknown Source)","main.Test.handleRequest(Test.java:25)","java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)","java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)","java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)","java.base/java.lang.reflect.Method.invoke(Unknown Source)"]}}

I don't know why this happens. I have already checked rol permissions. I think its ok (its the arn arn:aws:iam::000000000000:role/RolLambda)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "lambda:*",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::bucket1/*"
    },
    {
      "Effect": "Allow",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::bucket1/*"
    },
    {
      "Effect": "Allow",
      "Action": [
                "kinesis:PutRecord",
                "kinesis:PutRecords",
                "kinesis:GetRecords",
                "kinesis:GetShardIterator",
                "kinesis:DescribeStream",
                "kinesis:ListStreams"
            ],
            "Resource": "*"
    }
  ]
}

Any suggestions? Thanks for your time

smac2020 · Accepted Answer · 2023-09-28 19:24:03Z

I will test that code. However - i will make a few changes. 1st to ensure all of the dependencies are included in your JAR, I will create a FAT JAR using:

mvn package

Then to test it, I will upload the JAR to the Lambda AWS console and directly invoke the Lambda function using the AWS Console.

By the way - your issues is this:

errorMessage":"software/amazon/awssdk/services/kinesis/KinesisClient","errorType":"java.lang.NoClassDefFoundError","stackTrace":

The Kinesisclient cannot be found. This means you did not create the JAR properly.

UPDATE

As promised - here is the result. The code works when built with mvn package and deploying via AWS Lambda Console at https://us-east-1.console.aws.amazon.com/lambda/home?region=us-east-1#/functions

The Java Code.

package org.example;

import java.util.Map;

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.kinesis.KinesisClient;
import software.amazon.awssdk.services.kinesis.model.ListShardsRequest;
import software.amazon.awssdk.services.kinesis.model.ListShardsResponse;

public class TestHandler implements RequestHandler<Map<String,String>, String>{

    @Override
    public String handleRequest(Map<String,String> event, Context context)
    {
        String name = "LamDataStream";
        context.getLogger().log("*** Stream Name: " + name );
        KinesisClient kinesisClient = KinesisClient.builder()
            .region(Region.US_EAST_1)
            .build();

        ListShardsRequest request = ListShardsRequest.builder()
            .streamName(name)
            .build();

        ListShardsResponse response = kinesisClient.listShards(request);
        context.getLogger().log(name + " has " + response.shards());
        return name;
    }
}

I deployed it using the AWS Lambda console:

I tested it and it worked pefectly:

Here is output in logs:

Finally my POM (there may be a few extra dependencies):

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>TestLambda</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>software.amazon.awssdk</groupId>
                <artifactId>bom</artifactId>
                <version>2.20.45</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>org.apache.logging.log4j</groupId>
                <artifactId>log4j-bom</artifactId>
                <version>2.19.0</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>dynamodb</artifactId>
        </dependency>
        <dependency>
            <groupId>commons-logging</groupId>
            <artifactId>commons-logging</artifactId>
            <version>1.2</version>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>apache-client</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>commons-logging</groupId>
                    <artifactId>commons-logging</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>s3</artifactId>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk.crt</groupId>
            <artifactId>aws-crt</artifactId>
            <version>0.25.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-slf4j2-impl</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-1.2-api</artifactId>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-events</artifactId>
            <version>3.11.1</version>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>textract</artifactId>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>s3control</artifactId>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-core</artifactId>
            <version>1.2.2</version>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>translate</artifactId>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>comprehend</artifactId>
        </dependency>
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20230227</version>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>polly</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-booter</artifactId>
            <version>3.0.0-M3</version>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>lambda</artifactId>
        </dependency>
        <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
            <version>2.7</version>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>s3-transfer-manager</artifactId>
            <version>2.20.26</version>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.9.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.googlecode.json-simple</groupId>
            <artifactId>json-simple</artifactId>
            <version>1.1</version>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>kinesis</artifactId>
            <version>2.20.45</version>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.9.0</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.2</version>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.3.0</version>
                <configuration>
                    <createDependencyReducedPom>false</createDependencyReducedPom>
                </configuration>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

Thanks for replying! After changing src to "main\Test.java" (not "org\example\TestHandler.java") and aplying your code and your pom.xml I get this error (aws lambda unhandled functionError code 200 ): {"errorMessage":"Error loading method handleRequest on class main.Test","errorType":"java.lang.NoClassDefFoundError"}. The java code was modified to implement URI in kinesis object on line 19: KinesisClient kinesis_client = KinesisClient.builder().region(Region.US_EAST_1).endpointOverride(URI.create("http://127.0.0.1:4566")).build();
Your JAR is not being built successfully. How did you build it
I solved it! If you interested check the answer, many thanks.

Angel Sanchez · Accepted Answer · 2023-10-04 09:49:36Z

Finally the issue was the jar compilation. Here is the build of a basic pom.xml that put jar dependencies into final jar:

<?xml version="1.0"?>

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>Productor</groupId>
    <artifactId>Productor</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    
    <properties>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <build>
        <plugins>
            <!-- Build an executable JAR with dependencies -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>3.3.0</version>
                <configuration>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                    <archive>
                        <manifest>
                            <mainClass>productor.Productor</mainClass>
                        </manifest>
                    </archive>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <!-- Otras dependencias aquí -->
        <!-- Dependencia para el servicio de Amazon Kinesis -->
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>kinesis</artifactId>
            <version>2.20.135</version>
        </dependency>

        <!-- Dependencia para el servicio de Amazon S3 -->
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>s3</artifactId>
            <version>2.20.135</version>
        </dependency>

        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-core</artifactId>
            <version>1.0.0</version>
        </dependency>

        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-lambda-java-events</artifactId>
            <version>3.11.2</version>
        </dependency>
    </dependencies>
</project>

You can put any dependency you need in dependencies flag. And you need to input your package and main class in mainclass flag (in my case productor.Productor). Properties set java version and encoding. Now with @smac2020 command mvn package you can make JAR file correctly. Now I face a aws credential problem in a more complex code, but that's other history ':/

Notice that it's functional in Maven projects, other java projects weren't tested

Collectives™ on Stack Overflow

Java 17 runtime kinesis lambda code doesnt work localstack (easy code)

2 Answers 2

3 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related