Questions tagged [parallel-programming]
For questions about programming in such a way that a task can be divided among multiple threads/workers which work simultaneously ('in parallel') on the subtasks, leading to performance gains.
74 questions
0
votes
0
answers
88
views
How to connect to SFTP using Apache Spark 3.5 with Scala 2.12 for parallel file transfers?
I am working on a project where I need to transfer thousands of files (each sized between 50-60 MB) every hour from an SFTP server to local storage or AWS S3. I am using Apache Spark 3.5 with Scala 2....
- 11
0
votes
0
answers
45
views
What are the typical properties of dependency graphs?
I want to run a complex algorithm consisting of many subtasks in parallel on a number of processors. These subtasks depend on one another, so that this algorithm is characterized by a DAG (Directed ...
- 113
0
votes
0
answers
106
views
Optimizing the runtime of greedy scheduling for parallel computing
I have a situation where I want to run a complex algorithm consisting of many subtasks in parallel on a number of processors. These subtasks depend on one another, so that this algorithm is ...
- 113
1
vote
1
answer
222
views
Distribution of processing across cores between a LINQ and PLINQ query
I am trying to create a simple demonstration of using 'parallel LINQ' (PLINQ). I have two versions of my task, in C#:
var result = Enumerable.Range(1,1000000).Where(x => IsPrime(x)).ToList();
...
- 113
-3
votes
2
answers
141
views
How are futures and speculations evaluated differently?
In Practical Foundation of Programming Languages
38 Futures and Speculations
A future is a computation that is performed before it is value is
needed. Like a suspension, a future represents a value ...
- 5,555
1
vote
2
answers
1k
views
What are the differences between memory coherence and cache coherence?
https://en.wikipedia.org/wiki/Memory_coherence says:
Memory coherence is an issue that affects the design of computer
systems in which two or more processors or cores share a common area
of memory.[1]...
- 5,555
0
votes
0
answers
57
views
What is the most fitting thread model to redesign a sequential linear search algorithm?
Let's say that there is a simple sequential algorithm that performs a linear search of a unique key in an array:
public class SearchSeq {
public static int search(int[] a, int key) {
for (int i ...
3
votes
3
answers
2k
views
How can I improve the speed of scanning multiple directories recursively at the same?
So I am trying to speed up my program by using concurrency and/or multi-threading and/or process parallelism. The topics are pretty complex and I am sort of new to them so I am still trying to figure ...
- 263
4
votes
3
answers
2k
views
Alternating between Java streams and parallel streams at runtime
This is a question I constantly ask myself when designing a data intensive application: When is it appropriate to use stream() over parallelStream()? Would it make sense to use both? How do I quantify ...
- 263
10
votes
1
answer
452
views
How to design a good generic tiled image downloader?
Tiled images
Tiled images are large images that have been split in smaller square tiles.
There are several tiled image formats, with different ways of organizing
the tile files.
A tiled image on ...
- 209
1
vote
0
answers
61
views
MPI Derived Types and Portability
When using derived types in MPI for communication of data which is not contiguous in memory, the sequence of datatype-displacement pairs defining the derived type seems to be assumed to be the same ...
- 11
0
votes
1
answer
188
views
Can Kubernetes help with providing more processing power for the same request?
I am fairly new to Docker and Kubernetes and I have a question for which I could not figure out the answer myself. I am working on an application that does string matching on data extracted from ...
2
votes
1
answer
165
views
Relation Between Flynn's Taxonomy and Concurrency
I have known about the Flynn taxonomy for a while (SIMD, MIMD, etc.), as well as models of concurrency (actor model, petri nets, etc.). However I am wondering now how they overlap and interrelate. If ...
- 2,787
7
votes
3
answers
2k
views
How do I know if a set of code is a good candidate for parallelization?
How can I tell in an application if there are parts that will benefit from using the C# Task library (I believe that is the Parallel processing library.)
Given other optimizations have been done, how ...
- 3,679
7
votes
1
answer
877
views
Immediately awaiting an asynchronous call [duplicate]
While working on an inherited project, I noticed the original dev(s) created many asynchronous functions that never seem to take advantage of being, well, asynchronous. For example:
// The async ...
- 1,389
1
vote
2
answers
308
views
Parallel algorithm: calculations on graph
I have general parallel programming question.
Suppose there is a directed graph with cycles. Let’s assume that each node has fairly small amount of incoming edges ~ from 0 to 20 and potentially ...
- 19
0
votes
1
answer
284
views
Implement an actor-based concurrent language in ruby?
How would one implement an actor-based concurrent language in ruby? My thought is that only the correct way of creating programs is using actors, but I'm not entirely sure on how this could be done, ...
- 377
1
vote
0
answers
91
views
What specific attributes make code able to be executed in parallel? [closed]
List specific programming concepts that should code adhere to that it will run in parallel. For example, if a block of code does not change shared state, it should be able to be done on another thread....
3
votes
2
answers
3k
views
Parallel Image Processing Best Practices
When doing (possibly heavy) pixel processing on a large image, multithreading becomes a must. The standard practice is to initiate a loop whose indices are partitioned into multiple threads within a ...
- 219
1
vote
1
answer
1k
views
parallel programming memory usage
I am starting to learn parallel programming in c++, and I have a program like this:
for i = 1:N
do something time and memory intensive
end
If I program this in parallel, and my processors have ...
- 111
1
vote
4
answers
2k
views
Best algorithm to multi-thread this application?
I define an algorithm to be best if it minimizes the total run time over commodity hardware (e.g. normal desktop and server PCs).
I have sets A and B. I also have function f that works like f(a, b, n)...
- 73
1
vote
1
answer
446
views
One producer and one consumer vs inconsistent shared resource state
I was reading about implementation of producer-consumer problem with one producer and one consumer. Looking at this paragraph I see the implementation where shared resource access is not synchronized ...
- 212
5
votes
1
answer
957
views
Partially parallel producer-consumer pattern with internal state
I need to implement a producer-consumer pattern for reading, processing and saving electrical values. I have to implement this in C# .NET 4.6.1.
I try to describe this in great detail, so that there ...
- 358
0
votes
1
answer
209
views
Raspberry pi computer cluster question?
I am wanting to have/ or make a program that runs on a (raspberry pi)computer cluster with one pi executing only video content while the other only handles music, etc under a main program like an AI. ...
- 101
50
votes
5
answers
19k
views
What is it about functional programming that makes it inherently adapted to parallel execution? [duplicate]
I've been reading over and over that functional languages are ideal (or at least very often useful) for parallelism. Why is this? What core concepts and paradigms are typically employed and which ...
- 2,208