Questions tagged [distributed-computing]
Distributed computing refers to architectures where software components of a same system run on several processes distributed across a network of computers and somehow coordinated.
161 questions
2
votes
2
answers
151
views
For distributed workflows - tradeoffs having stateless workers operate on central state, versus stateful workers?
I'm working on a problem right now that processes incoming data at a very high rate. Each event that flows in has an association ID, and each group of associated events will affect behaviour over time ...
- 141
1
vote
2
answers
334
views
Generating unique number in distributed environment
I have little app that i plan to run behind load balancer, with up to 5-8 instances.
Inside the app has service, that needs to generate unique number that has up to 10 digits ( so 1-10 digits) each ...
- 227
3
votes
4
answers
1k
views
Giving multiple components access to a single database
This is in context of a distributed computing. There is a Service A which owns a database and hosts APIs for updating an entities in this database.
As time went by, the service has evolved and we are ...
- 41
0
votes
1
answer
78
views
What are the approaches for joining data in distributed processing
Let's suppose I have a processing workflow of:
Where:
The workflow can have many instances running at the same time.
A1 -> B1, C1 -> D1
A2 -> B2, C2 -> D2
and so on...
Each node is an ...
- 119
-1
votes
1
answer
143
views
Pinning Stateful Tasks in Stateless microservice cluster
We have a microservice whose multiple instances are running for High Availablity and Scaling. But although we have multipe stateless microservice there are some tasks which needs to be singleton.
For ...
- 97
2
votes
2
answers
612
views
Is sequential consistency equivalent to performing memory accesses by a processes in program order and performing each memory access atomically?
In Fundamentals of Parallel Multicore Architecture, by Yan Solihin, p304 defines sequential consistency memory model:
Overall, we can express programmers’ implicit expectation of memory
access ...
- 5,555
3
votes
1
answer
264
views
Database consistency for a distributed system
I'm working on a phone call monitoring project. The aim is to have one row in the calls table for each ongoing call. Each call instance may be updated by many different servers/threads as more ...
- 141
1
vote
1
answer
88
views
Ordering of analytical events
My team is working on an analytics system for the web app. The web app has a service-oriented architecture, and all events are gathered in one centralized analytics service. The system is sending ...
- 77
0
votes
1
answer
122
views
Implementing the microservice pattern
We are developing a system that runs a certain kind of simulation for our customers:
Every simulation runs on it's own server.
We typically have a two digit number of simulations running.
A client can ...
- 233
0
votes
2
answers
294
views
Cancelling a background service from another microservice in load balanced environment
I have 2 microservices [A], and [B]. These microservices run in a load balanced environment and there may be N instances if each microservice running. The microservices communicate via a message ...
- 491
0
votes
1
answer
424
views
what is the difference between "distributed data processing" and "distributed computing"?
While going through learning hadoop and spark, I came across "distributed data processing" and "distributed computing".
Could you let me know if they both are same or referring to different concepts?
- 123
-4
votes
1
answer
661
views
What difference and relation are between fault tolerance and (high) availability?
From Coulouris' Distributed Systems 5ed
Chapter 18 Replication
18.1 Introduction
Increased availability: Users require services to be highly available. That is, the proportion ...
- 5,555
-3
votes
2
answers
1k
views
What is the difference between masking and tolerating failures?
Distributed Systems 5ed by Coulouris says on p21-22
1.5.5 Failure handling
Detecting failures: Some failures can be detected. For example, checksums can be used to detect corrupted data in a ...
- 5,555
1
vote
1
answer
869
views
Does the producer-consumer problem appear in both shared memory and distributed memory architectures?
I saw some books on operating system concepts mention the producer-consumer problem in the context of synchronizing concurrent accesses to shared resources. All seem to be in shared memory ...
- 5,555
-2
votes
1
answer
872
views
How to understand two-tier and three-tier solutions for web applications?
From Distributed Systems by Coulouris:
Tiered architectures are complementary to layering. Whereas
layering deals with the vertical organization of services into layers
of abstraction, tiering ...
- 5,555
1
vote
1
answer
299
views
Designing frontend experience to support backend async CQRS operations
I'm developing a microservice with CQRS and Event Sourcing. When an event is saved to the event store, the service currently also saves the updated aggregate root as a JSON object in a separate table. ...
- 127
0
votes
1
answer
276
views
Consistency and Availability in distributed hashing Key value store
During the last semester of my college, we were asked to read the (chord paper) and implement it without fault tolerance. We built it with the following feature:
A new node can join.
A client can ...
- 135
4
votes
2
answers
2k
views
Using distributed locks in microservice environment
I'm developing distributed system and trying to use best practices of microservice architecture.
I was faced with a situation when I think I need something like distributed locks. Since I have not so ...
3
votes
0
answers
519
views
Is the concept of Rx debounce possible in a distributed environment?
The issue
I am building a server-side system that handles streams of time-series events from multiple users and I'd like to perform an action after a certain quiet period has elapsed in the event ...
- 139
1
vote
1
answer
887
views
How to design a high-scale, reliable, distributed and periodic task (cronjob) execution service?
Assume you are tasked with implementing a "cronjob as a service" where you are supposed to be able to run potentially millions of periodical tasks (to simplify, making HTTP requests to URLs) with ...
- 113
0
votes
1
answer
2k
views
Distribute jobs evenly among a set of servers
I have the need to distribute a set of long running jobs across a cluster of containers in ECS. These jobs essentially would need to open a socket connection with a remote server and begin streaming ...
- 9
3
votes
2
answers
833
views
Architecting a distributed file processing system with leadership election
I'm in the planning phases of trying to build a distributed file processing system in Java and I'm looking for feedback and advice:
Problem : There are a large number of files continuously posted on ...
- 139
5
votes
2
answers
1k
views
How can we keep sight of business flows in event driven architectures?
I am planning to set up an event driven architecture using Spring Boot apps that publish and read messages from a Kafka broker.
Let's suppose it were an e-commerce application with the usual events (...
- 427
2
votes
0
answers
199
views
Testing an algorithm for safety in mutual exclusion
I am attempting to write a code to test Lamport's Mutual Exclusion algorithm for safety as a correctness measure. I am running the alogrithm on a single core cpu machine with multiple processes ...
- 181
1
vote
2
answers
494
views
How to avoid clashes in distributed computing when polling for work to do?
A big data job is split up into X partitions. The partitions are stored in a database. Status on each partition is also stored in the database and is used to ensure that each partition is only ...
- 61