NiFI custom processor ClassNotFoundException

Question

I am trying to load a custom NiFi processor, but cannot get NiFi to load all my .nar dependencies, despite trying various pom.xml configurations. I've come across some similar questions on SO but have not found the answer to this issue.

I package my processor as a .nar with mvn clean install, then copy the .nar to NIFI_HOME/lib. Currently I get java.lang.ClassNotFoundException: org.apache.http.conn.HttpClientConnectionManager. Current pom dependencies:

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.7</maven.compiler.source>
    <maven.compiler.target>1.7</maven.compiler.target>
    <nifi.version>1.1.2</nifi.version>
</properties>
<dependencies>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-api</artifactId>
        <version>${nifi.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-utils</artifactId>
        <version>${nifi.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-standard-services-api-nar</artifactId>
        <version>${nifi.version}</version>
        <type>nar</type>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-ssl-context-service</artifactId>
        <version>${nifi.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-ssl-context-service-api</artifactId>
        <version>${nifi.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-processor-utils</artifactId>
        <version>${nifi.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpclient</artifactId>
        <version>4.4.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpcore</artifactId>
        <version>4.4.1</version>
    </dependency>
    <dependency>
        <groupId>joda-time</groupId>
        <artifactId>joda-time</artifactId>
        <version>2.9.7</version>
    </dependency>
</dependencies>

My nar's NIFI_HOME/.../bundled-dependencies/ contains all specified jars:

bcpkix-jdk15on-1.55.jar                joda-time-2.9.7.jar
bcprov-jdk15on-1.55.jar                nifi-api-1.1.2.jar
commons-codec-1.9.jar                  nifi-processor-utils-1.1.2.jar
commons-io-2.5.jar                     nifi-security-utils-1.1.2.jar
commons-lang3-3.4.jar                  nifi-ssl-context-service-1.1.2.jar
commons-logging-1.2.jar                nifi-ssl-context-service-api-1.1.2.jar
httpclient-4.4.1.jar                   nifi-utils-1.1.2.jar
httpcore-4.4.1.jar

Full stack trace:

2017-03-13 11:11:23,146 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.alar.nifi.processors.GetIBright could not be instantiated
java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.alar.nifi.processors.GetIBright could not be instantiated
    at java.util.ServiceLoader.fail(ServiceLoader.java:232) ~[na:1.8.0_111]
    at java.util.ServiceLoader.access$100(ServiceLoader.java:185) ~[na:1.8.0_111]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384) ~[na:1.8.0_111]
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[na:1.8.0_111]
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[na:1.8.0_111]
    at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:116) ~[nifi-nar-utils-1.1.2.jar:1.1.2]
    at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:97) ~[nifi-nar-utils-1.1.2.jar:1.1.2]
    at org.apache.nifi.NiFi.<init>(NiFi.java:139) ~[nifi-runtime-1.1.2.jar:1.1.2]
    at org.apache.nifi.NiFi.main(NiFi.java:262) ~[nifi-runtime-1.1.2.jar:1.1.2]
Caused by: java.lang.NoClassDefFoundError: org/apache/http/conn/HttpClientConnectionManager
    at java.lang.Class.getDeclaredConstructors0(Native Method) ~[na:1.8.0_111]
    at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) ~[na:1.8.0_111]
    at java.lang.Class.getConstructor0(Class.java:3075) ~[na:1.8.0_111]
    at java.lang.Class.newInstance(Class.java:412) ~[na:1.8.0_111]
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) ~[na:1.8.0_111]
    ... 6 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.apache.http.conn.HttpClientConnectionManager
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_111]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_111]
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[na:1.8.0_111]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_111]
    ... 11 common frames omitted

Can you provide the stacktrace for the class not found exception? — Bryan Bende
– Bryan Bende, Commented Mar 13, 2017 at 16:29
@JustinMoore hope you have restarted NiFi server after placing the .jar files under lib folder — Pons
– Pons, Commented Mar 13, 2017 at 19:10
@Pons yep, I have restarted NiFi, the error occurs when it tries to load the new custom processor on startup. Also - it is a .nar file, not .jar — Justin Moore
– Justin Moore, Commented Mar 13, 2017 at 19:17
One thing that is not correct (but should not be related to this problem) is you shouldn't be including the controller service JARs inside your NAR. Your NAR pom should have a NAR dependency on the controller service API, and your processors pom should have a provided dependency on the API so that it is available at compile time but not included at build time. cwiki.apache.org/confluence/display/NIFI/… — Bryan Bende
– Bryan Bende, Commented Mar 13, 2017 at 19:55

Justin Moore · Accepted Answer · 2017-03-13 21:03:24Z

Resolved this by modifying my single pom.xml into separate build (nar) and compile (jar) pom.xml files, with a parent pom.xml. Not sure why it did not work before but I have included the pom.xml files below

parent pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <artifactId>nifi-ibright-bundle</artifactId>
    <packaging>pom</packaging>
    <groupId>com.alar.nifi</groupId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.7</maven.compiler.source>
        <maven.compiler.target>1.7</maven.compiler.target>
        <nifi.version>1.1.2</nifi.version>
    </properties>

    <modules>
        <module>nifi-ibright-processor</module>
        <module>nifi-ibright-nar</module>
    </modules>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>com.alar.nifi</groupId>
                <artifactId>nifi-ibright-processor</artifactId>
                <version>1.0-SNAPSHOT</version>
            </dependency>
        </dependencies>
    </dependencyManagement> 
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.nifi</groupId>
                <artifactId>nifi-nar-maven-plugin</artifactId>
                <version>1.0.0-incubating</version>
                <extensions>true</extensions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.15</version>
            </plugin>
        </plugins>
    </build>
</project>

nifi-ibright-nar/pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <groupId>com.alar.nifi</groupId>
        <artifactId>nifi-ibright-bundle</artifactId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.alar.nifi</groupId>
    <artifactId>nifi-ibright-processor</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>


    <dependencies>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
            <version>${nifi.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-utils</artifactId>
            <version>${nifi.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-ssl-context-service-api</artifactId>
            <version>${nifi.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-processor-utils</artifactId>
            <version>${nifi.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.4.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpcore</artifactId>
            <version>4.4.1</version>
        </dependency>
        <dependency>
            <groupId>joda-time</groupId>
            <artifactId>joda-time</artifactId>
            <version>2.9.7</version>
        </dependency>
    </dependencies>

</project>

Collectives™ on Stack Overflow

NiFI custom processor ClassNotFoundException

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related