Timeline for Poor performance for signal processing
Current License: CC BY-SA 3.0
14 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 21, 2018 at 12:52 | answer | added | Maarten Fabré | timeline score: 1 | |
| Mar 21, 2018 at 8:26 | comment | added | Mathieu | @three_pineapples The lists aren't that large. 10 to 20 items top. The issue is that the function is called 50K times or 5M times... Anyway, Niklas proposed a different approach using numpy that is really more efficient on SO: stackoverflow.com/questions/49380594/… | |
| Mar 20, 2018 at 23:43 | comment | added | three_pineapples |
Hmm, I guess maybe you aren't actually appending to the list that often? For a large number of appends I would have expected the numpy version to be faster. How big are those lists that get created? Also, it needed to be one 2D array which you set via myarray[i,j]=1 to reproduce the logic you had previously, not two 1D arrays (you overwrite elements in one of the arrays in each iteration of the inner loop).
|
|
| Mar 20, 2018 at 13:48 | comment | added | Mathieu | In fact it even slowdowns the process compared to list. | |
| Mar 20, 2018 at 12:05 | history | edited | Mathieu | CC BY-SA 3.0 |
added 400 characters in body
|
| Mar 20, 2018 at 11:51 | history | edited | Mathieu | CC BY-SA 3.0 |
added 284 characters in body
|
| Mar 20, 2018 at 11:29 | answer | added | David L | timeline score: 4 | |
| Mar 20, 2018 at 10:52 | comment | added | Mathieu | @three_pineapples I just tried, see EDIT, but it didn't improve the performance :/ | |
| Mar 20, 2018 at 10:52 | history | edited | Mathieu | CC BY-SA 3.0 |
added 1777 characters in body
|
| Mar 20, 2018 at 10:13 | comment | added | three_pineapples |
I would consider avoiding the use of lists in duo_overlap and instead preallocate a numpy array with dimensions i,j using numpy.zeros. Elements that match your condition can then be set to 1 inside the loops. Afterwards, the indices containing nonzero elements found with numpy.nonzero() which should be equivalent to the lists you produce now (feel free to convert to a Python list at that point)
|
|
| S Mar 20, 2018 at 9:44 | history | suggested | yuri | CC BY-SA 3.0 |
Add additional tags
|
| Mar 20, 2018 at 9:43 | review | Suggested edits | |||
| S Mar 20, 2018 at 9:44 | |||||
| Mar 20, 2018 at 9:39 | review | First posts | |||
| Mar 20, 2018 at 9:43 | |||||
| Mar 20, 2018 at 9:38 | history | asked | Mathieu | CC BY-SA 3.0 |