Is there some mechanism to make find with -exec use multiple jobs?
Lets use this command for example
find ./* -exec flac --best {} \;
Is there some way to set the number of concurrent -exec commands to run easier than something like this solution:
run find ./* and save the results to a file/pipe them for another command to split into 16 parts (or however many threads you want) then run commands on these?
You can set the number of workers if you use xargs
find . -type f -print0|xargs -0 -P 16 -I % opusenc --bitrate 256 % %.opus
xargs -P 16
(since I struggled to find more details online or understand the man page section)
It does your command on 16, or the number you pass to -P, of the inputs at a time.
For my command I was trying to convert an organized folders of cd music files [artist/album/music file] to opus. This was a copied folder so I wanted to delete the originals. So a better command than the one I used would be
find . -type f -print0|xargs -0 -P 16 -I % sh -c 'opusenc --bitrate 256 % %.opus && rm %'
After conversion the original file is removed.
&& so you don't go around removing source files that failed to convert!
find . -type f -print0|xargs -0 -P 16 -I % sh -c 'opusenc --bitrate 256 % %.opus > %.log.txt && rm %' I suspect you may also want stderr though; use find . -type f -print0|xargs -0 -P 16 -I % sh -c 'opusenc --bitrate 256 % %.opus > %.log.txt 2>&1 && rm %' to do that
-Pfind ./* -type f -print0|xargs -0 -P 16 -I % opusenc --bitrate 256 % %.opus./*. A dot would be enough, and that would not blow up if you have many names in the current directory. I would also suggest a-nametest to get names you want and not everything (and-type fto avoid runningflac(oropusenc) on directories). If you have one single directory, you shouldn't needfindat all, and if you're usingbashorzsh, you could use**instead offind.