2

I have to perform CPU intensive analysis on data in DigitalMicrographs's DM-script language. I noticed that only a single CPU core maxes out during processing. Is there a way to achieve better performance by multi-threading in DM scripting? Simple examples would be appreciated.

4
  • First google result looks promising, no? Do your research. (i dont know much about this topic, so i wont cast a dup-vote here): stackoverflow.com/questions/25500016/… Commented Dec 13, 2021 at 12:23
  • @Zabuzard No, I don't think this is a duplicate. This is specific about speed improvements and there are little to no examples of this in DM-scripting I know of. I'm actually onto this with and answer right now... Commented Dec 13, 2021 at 12:31
  • Fair. But the linked answer shows how to use threads, no? Basically answering "how to use multi-threading in DM scripting". Commented Dec 13, 2021 at 12:35
  • 1
    @Zabuzard: Sure, but the question is literally "Is it possible to speed up scripts", not, is it possible to have multiple threads. And this is indeed a more complicated question to answer. Commented Dec 13, 2021 at 13:25

1 Answer 1

1

Multi-threading is certainly possible in DM-scripting, and it is documented in the F1 help here:

F1 help on threading

Whether or not speed improvement can be achieved depends on various things, most importantly whether or not the individual threads need access to the same resource (like the same data, or some GMS resource which is only available via the main-thread - f.e. the UI).

Also, a lot of data-processing is already multi-threaded internally when you use commands on image-expressions. You might achieve a lot more speedup by rephrasing your analytical processing in a way that doesn't requires for-loops in the scripting language but uses image-expressions instead.

Finally, doing things multi-threaded is a great way of introducing bugs and unexpected behavior which can be really hard to debug. Don't be frustrated if you run into those things while learning stuff.

That said, the below example demonstrates (at least on my machine) a speed improvement by "chunking" some data-analysis over multiple parallel background threads.

// Example showing the explicit use of multi-threading in DM scripting 

class CMultiThreadtest
{
    image data, keep
    number sx,sy,sz 
    number nChunks, chunksize, lastChunk, doneChunk 
    
    object SetData(object self, image img, number nChunks_) 
    { 
        if ( img.imagegetNumdimensions() != 3 ) throw( "only 3D data for testing please")
        img.ImageGetDimensionSizes(sx,sy,sz)
        nChunks = nChunks_
        if ( sz % nChunks != 0 ) Throw( "Z-size needs to be integer multiple of nChunks for this test.")
        chunksize = sz / nChunks
        
        data:=img
        keep = data
        
        return self
    }
    
    void CompareResult(object self){
        image dif := keep*2-data
        number ok = 0==sum(dif)
        Result("\n\t Result is " + (ok?"correct":"wrong"))
    }
    
    void RunOnData(object self){
        
        // For extra-caution of thread safety, the two lines below shoud be guarded with critical sections
        // but given the near-atomic nature of the call, this is omitted here.
        number chunkIndex = lastChunk
        lastChunk++
        
        image work := data.slice3(0,0,chunkIndex*chunksize, 0,sx,1, 1,sy,1, 2,chunksize,1)
        number startp = GetHighresTickCount()       
        for( number z=0;z<chunksize;z++)
            for( number y=0;y<sy;y++ )
                for( number x=0;x<sx;x++ ){
                        work[x,y,z] *= 2
        }
        number endp = GetHighresTickCount()     
        Result("\n\t\t Process (chunk "+chunkIndex+") done with " + sx*sy*chunksize + " steps in " + (endp-startp)/GetHighResTicksPerSecond())
        
        // For extra-caution of thread safety, the line below shoud be guarded with critical sections
        // but given the near-atomic nature of the call, this is omitted here.
        doneChunk++
    }
    
    void RunWithSubsets(object self, image src, number nChunks_, number inbackground){
        self.SetData(src, nChunks_)
        lastChunk = 0
        doneChunk = 0
        Result("\n.....\n Running with "+nChunks+" chunks of size "+chunksize+ " on " + (inbackground?" multiple background threads":" single main thread") +":")
        number startp = GetHighresTickCount()   
        for( number i=0; i<nChunks; i++){
            if ( inbackground )
                self.StartThread("runondata")
            else
                self.RunOnData()
        }   
        
        while( doneChunk != nChunks ){
            if ( ShiftDown() ){
                Throw("abort")
            doEvents()
            }
        }
        number endp = GetHighresTickCount()     
        Result("\n Total duration:" + (endp-startp)/GetHighResTicksPerSecond())
        self.CompareResult();
        Result("\n.....")
        
    }
};

void Test(){
    image img := RealImage("test cub",4,50,50,10)
    img = random()
    clearresults()
    object tester = Alloc(CMultiThreadtest)
    tester.RunWithSubsets(img, 1, 0)
    tester.RunWithSubsets(img, 1, 1)
    tester.RunWithSubsets(img, 5, 0)
    tester.RunWithSubsets(img, 5, 1)
}
test()
Sign up to request clarification or add additional context in comments.

3 Comments

Thanks a lot, exactly what I was looking for.
How do we do this? "two lines below should be guarded with critical sections". Can you give an example code?
Please refer to the F1 help documentation at Scripting > Objects >Threading, there is an example. If that doesn't help you enough, please post this as a separate question on site.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.