java object querying ( applying SQL logic on List of java objects)

Question

I have a Employee object for example, Employee contains several fields (300+) like name, department, salary, age, account, etc. Entire Employee table data cached into java List object , which contains 2+ million records.

Requirement user can search on any filed presents in Employee object like employee name like Sehwag and age > 30 or salary > 100000, Based on user search we have to show filtered list of Employee list.

due to performance issues we are not querying DB, we want to apply the user search criteria on cached java List object earlier

is there way api / frameworks / any other solution where we can query on java objects?

below approach I am trying but I am feeling not a good approach

Iterating the Employee list and applying condition user search criteria on employee object, to know the user selected search criteria among 300 fields is challenging, written a lot enum mapping logic and some additional logic for every filed to make it work. with current requirement it may works but thinking to use api or framework or better way to solve the requirement!

thanks in advance for your help.

You could check Lucene framework to create index files and search over those files — Ivan
– Ivan, Commented Jul 5, 2018 at 21:00
Normally I would expect better performance by querying the database. — rgettman
– rgettman, Commented Jul 5, 2018 at 21:01
@rgettman specialized in-memory index can be very fast and avoid the I/O trip to the database. See CQEngine for instance. — lexicore
– lexicore, Commented Jul 5, 2018 at 21:09
tried CQEngine's SQL based queries suits to my requirement, thank you All for replies. code also much simpleminded, but getting error when 2nd time called below logic SQLParser<Exmployee> parser = SQLParser.forPojoWithAttributes(Exmployee.class, com.googlecode.cqengine.codegen.AttributeBytecodeGenerator.createAttributes(Exmployee.class)); — Ramesh Mandri
– Ramesh Mandri, Commented Jul 6, 2018 at 20:33

Thiago Primerano · Accepted Answer · 2018-07-05 21:16:57Z

1

First of all. If you either don't want or don't have chance to change the existing solution, take a look at querydsl. There is a querydsl-collection module which fits exactly with your need. See more at http://www.querydsl.com/ and http://www.querydsl.com/static/querydsl/latest/reference/html/ch02s08.html

However, if you have a chance to review/rebuild the solution, you should consider something more appropriate for large volume querying. I suggest you exploring more about nonsql databases (mongodb) or indexing tools such as lucene or elasticsearch which adds a RESTFul layer on top of lucene.

I hope it helps.

answered Jul 5, 2018 at 21:16

Thiago Primerano

1165 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Ramesh Mandri · Accepted Answer · 2018-07-06 20:41:10Z

0

tried CQEngine's SQL based queries https://github.com/npgall/cqengine suits to my requirement,

below are some useful links https://dzone.com/articles/getting-started-cqengine-linq

https://mvnrepository.com/artifact/com.googlecode.cqengine/cqengine

answered Jul 6, 2018 at 20:41

Ramesh Mandri

559 bronze badges

Collectives™ on Stack Overflow

java object querying ( applying SQL logic on List of java objects)

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related