8

oI have a table with 2 millions of registers, but it will grow much more soon. Basically this table contains points of interest of an image with respective descriptors. When I'm trying to execute query that selects points that are spatially near to the query points, total execution time takes too long. More precisely Duration / Fetch = 0.484 sec / 27.441 sec. And the query is quite simple, which returns only ~17000 rows.

My query is: 

SELECT fp.fingerprint_id, fp.coord_x, fp.coord_y, fp.angle,
fp.desc1, fp.desc2, fp.desc3, fp.desc4, fp.desc5, fp.desc6, fp.desc7, fp.desc8, fp.desc9, fp.desc10,
fp.desc11, fp.desc12, fp.desc13, fp.desc14, fp.desc15, fp.desc16, fp.desc17, fp.desc18, fp.desc19,
fp.desc20, fp.desc21, fp.desc22, fp.desc23, fp.desc24, fp.desc25, fp.desc26, fp.desc27, fp.desc28,
fp.desc29, fp.desc30, fp.desc31, fp.desc32
FROM fingerprint fp 
WHERE 
fp.is_strong_point = 1 AND 
(coord_x BETWEEN 193-40 AND 193+40) AND (coord_y BETWEEN 49-15 AND 49+15 ) 
LIMIT 1,1000000;

That is what I've done.

  1. I've tried to change key_buffer_size in my.ini, but didn't see much changes.
  2. In addition I've tried to set coord_x and coord_y as indexes, but query time became slower.
  3. The table is partitioned by range of coord_x field, which gave me better results.

How I can reduce the Fetch time? Is it possible to reduce it to milliseconds?

Improve this question

5 Answers 5

Reset to default
18

I faced slow fetch issue too (MySQL, InnoDB). Finally I found that innodb_buffer_pool_size is set to 8MB by default for my system which is not enough to handle the query. After increasing it to 1GB performance seems fine:

                    Duration / Fetch
353 row(s) returned 34.422 sec / 125.797 sec (8MB innodb buffer)
353 row(s) returned 0.500 sec / 1.297 sec (1GB innodb buffer)

UPDATE:

To change innodb_buffer_pool_size add this to your my.cnf

innodb_buffer_pool_size=1G

restart your mysql to make it effect

Reference: How to change value for innodb_buffer_pool_size in MySQL on Mac OS?

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Hey, thanks for pointing out the innodb buffer pool size. But how can you identify the ideal size for this variable> Because I have similar kind of issue. My table have almost 100M rows, And I am trying to fetch 100K rows from it. Our system have 24Gigs of RAm and Buffer pool size almost 10G.
0

If i am right the query is really fast but what is slow is the fetching of the data from your database. It takes 27 seconds to load the 170000 results from your storage.

It looks like you use the wrong database type. Try switching the table from one database engine to another.

For maximum speed you can use the MEMORY engine. The only drawback would be that you would have to store a copy of that table in another engine if you have to do dynamic changes to it and after any change you would have to reload the differences or the entire table.

Also you would have to make a script that fires when you restart your server so that your memory table would be loaded on startup of your mysql server

See here for the doc

Improve this answer

16 Comments

I know that in Oracle it should be faster, but I must use MySQL in this project. I was thinking that there could be some tricks in MySQL configurations? Thanks for your answer.
@Andriy I hope that you also know that MySQL can also use more than one DB Engine. MyISAM and innodb for instance...
@Andriy if you are using a tool like phpmyadmin then you will be able to see what kind of database engine your table is using.
Ok, it is configured to default-storage-engine=INNODB, I'll try to use MyISAM.
views are just some kind of aliass and you mask the fetching of your rows in the duration. So what your view basically does is it executes, fetches the stuff and then applies your "select... from view" and this select is faster in case of fetching because everything is already in your memory because the view fetched it.
|
-1

Increasing my buffer size make myquery faster. But you need to open the my.ini file as notepad++ because it will some hex data if you open it as notepad.

Improve this answer

Comments

-3

I found a Fix, Just disable your AVG or any antivuris in your system and then restart your workbench

Improve this answer

Comments

-3

Make sure that the line is not written in your pom.xml.

<property name="hbm2ddl.auto">create</property>

If it is written than remove it.

Improve this answer

1 Comment

You should refer what does the line mentioned do, in order to make the asker and also the community able to improve their knowledge

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.