1

I am reaching out for ideas on the following topic. Imagine that we have 1 application. This application needs to execute an interval (cron) job. It doesn't make sense to execute however on all instances of the app. It needs to elect just one of X and execute the task, then on the next execution it would again pick one (like load-balancing).

I know that Kubernetes offers Jobs and CronJobs, but those serve only a single purpose of executing a task and then turning off, which would add more complexity to the solution.

An option is of course to have an external coordinator doing this, but I was wondering what is the Kubernetes way to handle such scenario.

Any ideas welcome, thanks.

Improve this question

2 Answers 2

Reset to default
1

Job pattern is what you are looking for. There are many examples in official Kubernetes Documentation/Tasks/Run Jobs

Your need seems to match with working with a queue. In this example, as each pod is created, it picks up one unit of work from a task queue, completes it, deletes it from the queue, and exits.

  1. Start a message queue service. In this example, we use RabbitMQ, but you could use another one. In practice you would set up a message queue service once and reuse it for many jobs.
  2. Create a queue, and fill it with messages. Each message represents one task to be done. In this example, a message is an integer that we will do a lengthy computation on.
  3. Start a Job that works on tasks from the queue. The Job starts several pods. Each pod takes one task from the message queue, processes it, and repeats until the end of the queue is reached.
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

I've read the docs, but this is not what I am looking for. A queue is an overkill, since I have the same task, repeating over time. The job is in general not a feasible option, as I don't want this container to only serve for cron and exit afterwards. It's just a regular app that should ALSO do a cron job.
Without having a shared source of messages, it can be a queue, you cannot do it. Or as you stated in the question, you will need an extra application/coordinator to do it..
0

Just wanted to share what I eventually did.

Since the recommended Kubernetes way is using the CronJobs type, I have modified my apps to be able to run in "cron mode", by specifying some config values (e.g. mode: cron/regular, cron: name of specific cron to support multiple per app).

This allows me to run the same container both as a cron on interval and as regular app for the long-running process.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.