Understanding multithreading with a for loop Java

Question

I've got some code that I don't think is able to be multithreaded, perhaps I'm wrong. I'd like to make execute this code on a clustered system but I'm unsure of how to scale it for such a deployment.

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.PrintStream;
import java.text.DecimalFormat;
import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;

    public class Coord {
        public int a,b,c,d,e,f;


    public static void main(String[] args) throws IOException {
        FileOutputStream out = new FileOutputStream("/Users/evanlivingston/2b.txt");
        PrintStream pout = new PrintStream(out);
        Scanner sc = new Scanner(new File("/Users/evanlivingston/1.txt"));
        List<Coord> coords = new ArrayList<Coord>();{


            // for each line in the file
            while(sc.hasNextLine()) {
                String[] numstrs = sc.nextLine().split("\\s+"); 

                Coord c = new Coord();


                c.a = Integer.parseInt(numstrs[1]);
                c.b = Integer.parseInt(numstrs[2]);
                c.c = Integer.parseInt(numstrs[3]);
                c.d = Integer.parseInt(numstrs[4]);
                c.e = Integer.parseInt(numstrs[5]);
                c.f = Integer.parseInt(numstrs[6]);

                coords.add(c);

            }
// now you have all coords in memory
            {
for(int i=0; i<coords.size(); i++ ) 
    for( int j=0; j<coords.size(); j++) 
    {
        Coord c1 = coords.get(i);
        Coord c2 = coords.get(j);
        double foo = ((c1.a - c2.a) * (c1.a - c2.a)) *1 ;
        double goo = ((c1.b - c2.b) * (c1.b - c2.b)) *1 ;
        double hoo = ((c1.c - c2.c) * (c1.c - c2.c)) *2 ;
        double joo = ((c1.d - c2.d) * (c1.d - c2.d)) *2 ;
        double koo = ((c1.e - c2.e) * (c1.e - c2.e)) *4 ;
        double loo = ((c1.f - c2.f) * (c1.f - c2.f)) *4 ;
        double zoo = Math.sqrt(foo + goo + hoo + joo + koo + loo);

        DecimalFormat df = new DecimalFormat("#.###");
        pout.println(i + " " + j + " " + df.format(zoo));
        System.out.println(i);

    }
    pout.flush();
    pout.close();
            }
        }
    }   
}

I appreciate any help anyone can offer.

You couldn't run this on a clustered system in its current state. Not even close. Need something like OpenMPI like mpj or roll a custom solution maybe using RMI. — Chris Dennett
– Chris Dennett, Commented Apr 19, 2011 at 23:43

Tim Bender · Accepted Answer · 2011-04-20 00:04:24Z

Splitting the inner for loop into separate tasks looks like a good candidate for where to make this process multithreaded. Here is one way this could be done with an ExecutorService and Futures

    final ExecutorService executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
    final List<Future<String>> results = new LinkedList<Future<String>>();
    // now you have all coords in memory
    for (int i = 0; i < coords.size(); i++) {
        final int index = i;
        final Coord c1 = coords.get(index);
        results.add(executor.submit(new Callable<String>() {
            public String call() {
                final StringBuilder stringBuilder = new StringBuilder();
                for (int j = 0; j < coords.size(); j++) {
                    final Coord c2 = coords.get(j);
                    final double foo = ((c1.a - c2.a) * (c1.a - c2.a)) * 1;
                    final double goo = ((c1.b - c2.b) * (c1.b - c2.b)) * 1;
                    final double hoo = ((c1.c - c2.c) * (c1.c - c2.c)) * 2;
                    final double joo = ((c1.d - c2.d) * (c1.d - c2.d)) * 2;
                    final double koo = ((c1.e - c2.e) * (c1.e - c2.e)) * 4;
                    final double loo = ((c1.f - c2.f) * (c1.f - c2.f)) * 4;
                    final double zoo = Math.sqrt(foo + goo + hoo + joo + koo + loo);

                    final DecimalFormat df = new DecimalFormat("#.###");
                    stringBuilder.append(index + " " + j + " " + df.format(zoo));
                    System.out.println(index);
                }
                return stringBuilder.toString();
            }
        }));
    }
    for (Future<String> result : results) {
        pout.print(result.get());
    }
    pout.flush();
    pout.close();
    executor.shutdown();

For clustering, I think Hazelcast offers a good solution that will allow you to define a shared ExecutorService and shared Collections. You would need two flavors of nodes, the single node responsible for all I/O and creating the list of Coords as well as submitting the tasks. And a processing node which simply executes the tasks. That is all my opinion of how I might do it. However, if your dataset is small enough to fit in memory it is likely not worth the effort to split up the processing this much.

Daniel Lubarov · Accepted Answer · 2011-04-19 23:45:44Z

1

It looks very parallelizable to me. Why don't you have threads process one row of data at a time? You could use an AtomicInteger to keep a count of how many rows have been claimed by worker threads. Each thread would do a counter.getAndIncrement to get a row to work on (if it returns coords.size() or higher, the thread should terminate), then do all the math for that row, and repeat.

The printing would be out of order, but you could instead fill some buffers with the results, then quickly print everything at the end.

answered Apr 19, 2011 at 23:45

Daniel Lubarov

7,9341 gold badge39 silver badges57 bronze badges

Collectives™ on Stack Overflow

Understanding multithreading with a for loop Java

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related