remove blank rows in the CSV file using NiFi

Question

I want to achieve a simple use case to remove any blank row found in the CSV file. How Can I achieve this using NiFi?

I have CSV File as follows: (Plz see Attached Screenshot showing which row needs to be removed)

I want to remove the first Blank Row in the csv just above the headers using NiFi. Please, any suggestion is much appreciated. Thank You!

@emsimpson92 In NiFi you can use regex or any other way to delete the blank row. I want to know ways to achieve my use case. — Shrads
– Shrads, Commented Jan 4, 2019 at 0:41
Please edit your question and provide the text of your flow file. Is it only first line empty? — daggett
– daggett, Commented Jan 4, 2019 at 5:49
@daggett I want to remove the first row of the CSV please see the screenshot attached. — Shrads
– Shrads, Commented Jan 4, 2019 at 18:30

Andy · Accepted Answer · 2019-01-08 20:30:08Z

4

You can use a ReplaceText processor which replaces \A\n|\n*\s*(?=\n) with '' (empty replacement value). The search regex looks for:

\A\n - beginning of the content immediately followed by a newline OR
\n*\s*(?=\n) - newline (0 or more) followed by whitespace (0 or more) followed by a newline (not captured using lookahead group)

Update

Not sure why this was downvoted or did not work for some user, as I just created a template and it worked exactly as described.

2019-01-08 12:25:27,642 INFO [Timer-Driven Process Thread-2] o.a.n.processors.standard.LogAttribute LogAttribute[id=2f22d047-0168-1000-47b0-9ec963e65367] logging for flow file StandardFlowFileRecord[uuid=6c9cc388-19c8-4b98-9970-6a6e3979e4ee,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1546979126561-1, container=default, section=1], offset=152, length=50],offset=0,name=6c9cc388-19c8-4b98-9970-6a6e3979e4ee,size=50]
--------------------------------------------------
Standard FlowFile Attributes
Key: 'entryDate'
    Value: 'Tue Jan 08 12:25:27 PST 2019'
Key: 'lineageStartDate'
    Value: 'Tue Jan 08 12:25:27 PST 2019'
Key: 'fileSize'
    Value: '50'
FlowFile Attribute Map Content
Key: 'filename'
    Value: '6c9cc388-19c8-4b98-9970-6a6e3979e4ee'
Key: 'path'
    Value: './'
Key: 'uuid'
    Value: '6c9cc388-19c8-4b98-9970-6a6e3979e4ee'
--------------------------------------------------
header1,header2,header3
A1,A2,A3
B1,B2,B3
C1,C2,C3

edited Jan 8, 2019 at 20:30

answered Jan 4, 2019 at 18:54

Andy

14.2k2 gold badges38 silver badges80 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Shrads Over a year ago

It went to failure. CAn you provide more detail? Like what will be the replacement strategy?

Andy Over a year ago

I've provided a template, screenshots of configuration and flow statistics, and log output verifying this works.

Andy Over a year ago

If this causes failure for you, there may be other characters on your "blank" line. Please provide the CSV file in plaintext format, not as a screenshot in Excel.

Shrads Over a year ago

Thanks for the detail explanation.

Collectives™ on Stack Overflow

remove blank rows in the CSV file using NiFi

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related