Customize parsing for string array in Java

Question

I have a an array of strings like this (from Twitter):

String str= "The Green New Deal is viable. It is the same vision that FDR had for his New Deal programs: nationwide mobilization http://94739 #thegreendeal #nationwide"

What I want is to 1) turn this string into an array and 2) remove stop words and include stemming 3) remove all characters except for '#' which indicates a term is a hashtag.

So I have tried to use this cool library https://github.com/uttesh/exude which does stemming and removes stop words, and lowercases and removes characters. The problem is this removes the hashtags. Code for this:

String tweetString = ExudeData.getInstance().filterStoppingsKeepDuplicates(str);

I have also tried this:

String[] wordArray = str.replaceAll("[^a-zA-Z ]", "").toLowerCase().split("\\s+");

But this also removes hashtags. Any workaround using either method to keep the hashtags? (I'd prefer to keep the exude library for this)

Extract the hashtags before processing. Append back in after processing if needed. — Andreas
– Andreas, Commented Mar 4, 2019 at 17:39

Andrianekena Moise · Accepted Answer · 2019-03-04 17:42:51Z

1

Using the regex method, you can try to add # in the list of characters that should not be removed like this :

        String[] wordArray = str.replaceAll("[^a-zA-Z #]", "").toLowerCase().split("\\s+");

answered Mar 4, 2019 at 17:42

Andrianekena Moise

1,0687 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Customize parsing for string array in Java

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related