Apache-NiFi Array Index Out Of Bounds Exception

Question

I use ExecuteGroovyScript processor just to extract only wanted columns for my further calculations.

Groovy Code :

def flowFile = session.get()

if(!flowFile) return

flowFile = session.write(flowFile, {inputStream, outputStream ->
    outputStream.withWriter("UTF-8"){ w ->
        inputStream.eachLine("UTF-8"){ line ->

                 def row = line.split(';',-1)

                 w << row[0,1,6,8,9,11].join(',') << '\n'

        }
    }
} as StreamCallback)

session.transfer(flowFile, REL_SUCCESS)

But for some csv, I get java.lang.ArrayIndexOutOfBoundsException.

My csv :

id,name,email,address
1,sachith,[email protected],{"Lane":"ABC Lane","No":"24"}
2,nalaka,[email protected],{"Lane":
"DEF Lane","No":"34"}

How can I get just 1 row and ignore other two rows? I have tried ValidateCSV processor for validating. But it can not capture this.

give the sample of csv file

Daeho Ro
– Daeho Ro

2020-02-13 12:51:46 +00:00
Commented Feb 13, 2020 at 12:51 — Daeho Ro
– Daeho Ro, Commented Feb 13, 2020 at 12:51

maxime G · Accepted Answer · 2020-02-13 22:40:27Z

1

I was able to use ValidateCSV processor to validate the row. this is tricky because the , in the middle of {"Lane":"ABC Lane","No":"24"} will consider this field as 2 differents fields.

every invalids rows will be routed to invalid :

ValidateCsv processor configuration :

Schema :

ParseInt(),StrNotNullOrEmpty(),StrNotNullOrEmpty(),StrRegex("\{.*"),StrRegex(".*\}")

You should look at this processor documentation for more detail if you want a better schema :

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.0/org.apache.nifi.processors.standard.ValidateCsv/additionalDetails.html

answered Feb 13, 2020 at 22:40

maxime G

1,7981 gold badge17 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Apache-NiFi Array Index Out Of Bounds Exception

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related