How to use NiFi ExecuteScript processor with Python?

Question

I have a very basic setup of the ExecuteScript processor in Apache NiFi with a simple Python script (saved as a .py file) as shown here. In the Properties of the processor, I set the Script Engine to python and Script File to the path of this script.

import time

count = 0

while(count < 20):
    print "The counter says: ", count
    count = count + 1
    time.sleep(.1)

And this is the dataflow diagram I made:

I don't see anything outputted to the log or the PutFile. However, I do see the print statements appear in \nifi-0.6.1\logs\nifi-bootstrap.log. My knowledge of this is currently limited. I would appreciate answers from anyone who knows how to use the ExecuteScript processor, or even give me a better example than my current setup.

I wonder if you tried the ExecuteProcess. It should grab the stdout and route it as a flowfile: nifi.apache.org/docs/nifi-docs/components/… — andrew
– andrew, Commented May 5, 2016 at 18:30

Bryan Bende · Accepted Answer · 2016-05-04 23:56:06Z

17

Given your script, I think everything is functioning as expected. The script is not producing any FlowFiles which is why nothing is moving from ExecuteScript to the other processors, and anything sent to system out is captured in the bootstrap.log so that is why the print statement ends up there.

Script executing with in ExecuteScript get access to a few standard objects:

session
context
log
REL_FAILURE
REL_SUCCESS

In order to produce FlowFiles you would need call session.create() and take the resulting FlowFile and transfer it to REL_SUCCESS.

The best source of info on the scripting processors is Matt Burgess's blog, this page has some good background (uses Groovy):

http://funnifi.blogspot.com/2016/02/executescript-processor-hello-world.html

This one has a Jython example:

http://funnifi.blogspot.com/2016/03/executescript-json-to-json-revisited_14.html

answered May 4, 2016 at 23:56

Bryan Bende

18.7k1 gold badge33 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Michael M Over a year ago

Not sure what you mean "get access" to standard objects. Does this mean that don't need any special import statements in the .py file to reference these?

Eden Trainor Over a year ago

That's exactly what he means, they're global variables, so your script has access to them without having to define them elsewhere.

shzyincu Over a year ago

we can't use pandas or any other library with it :(

Collectives™ on Stack Overflow

How to use NiFi ExecuteScript processor with Python?

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related