Sidekiq job lost when deploying to Elastic BeanStalk

Question

I have a Sidekiq Enterprise version. My app is running on Elastic BeanStalk (web and worker env). Also I use ElastiCache service for Redis. Before this, the application was running on a Linux system. Now I have switched to the Docker platform.

After switching to the Docker platform, I started having problems with sidekiq (I use sidekiqswarm). If jobs are being performed and we start deploying the application, then after a successful deployment of web and worker (sidekiq here) env, the jobs simply disappear, the cloudwatch logs are empty. On the sidekick panel itself, I see that I have 2 processes, after deployment 2 new ones appear, the old ones are destroyed and along with this, the jobs disappear. Maybe someone also has a the same problem?

Dockerfile:

RUN chmod +x ./entrypoint.sh
ENTRYPOINT ["tini", "--"]
CMD ["./entrypoint.sh"]

entrypoint.sh:

#!/bin/bash

case $DEPLOYMENT_TYPE in
"web")
  bundle exec rails db:migrate;
  bundle exec puma; ;;
"worker")
  SIDEKIQ_PRELOAD= bundle exec sidekiqswarm -C ./config/sidekiq.yml; ;;
*)
  echo "DEPLOYMENT_TYPE is invalid" 1>&2;
  exit 1; ;;
esac

I suspect the problem is with docker. I have docker + docker-compose file. It turns out that with each deployment the image is rebuilt and maybe because of this the jobs are completely lost?

services:
  my-app:
    restart: always
    env_file:
      - .env
    build:
      context: .
      args:
        ....
    expose:
      - ....
    volumes:
      - ...
    .....
  

nginx-proxy:
    restart: always
    image: nginx:latest
    depends_on:
       - my-app
    ..

Sidekiq is running on an in-process memory store, rebuilding the image destroys the process and hence the memory. Move the backing memory store to its own instance or since you're in AWS you can use the ElastiCache service. — dbugger
– dbugger, Commented May 15 at 19:01
Sorry, I forgot to write about this important clarification. I use ElastiCache service for Redis. Maybe you have any other suggestions? — Maxim Gorbachevskiy
– Maxim Gorbachevskiy, Commented May 18 at 9:37

Mike Perham · Accepted Answer · 2025-05-19 16:18:19Z

0

It's because you are using a bash script within the Docker container to start the processes but you aren't forwarding signals to the child process so Sidekiq doesn't ever see SIGTERM, only SIGKILL after 30 seconds.

https://unix.stackexchange.com/a/196053

You want to use exec to replace the script in memory:

exec bundle exec ...

https://github.com/sidekiq/sidekiq/wiki/Problems-and-Troubleshooting#sidekiq-loses-jobs-on-deployment

edited May 19 at 16:18

answered May 19 at 16:10

Mike Perham

22.4k6 gold badges64 silver badges67 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Maxim Gorbachevskiy May 20 at 10:03

Thanks for your comment. I read about it and tried to do this. But in my case it still didn't help me (exec bundle exec sidekiqswarm ... or exec env SIDEKIQ_PRELOAD= bundle exec sidekiqswarm ....). In addition to the official documentation, I also used this source dev.betterdoc.org/docker/linux/container/signals/pid1/2021/06/…. Maybe you have any other suggestions?

Maxim Gorbachevskiy May 20 at 10:14

I also read that Elastic BeanStalk uses docker kill for containers and there is no way to change the behavior to docker stop (so that the container terminates correctly). This is most likely also the reason for such behavior with the sidekiq.

Maxim Gorbachevskiy · Accepted Answer · 2025-05-22 12:58:29Z

0

I found resolution. Need's to use exec ....(as Mike stated in the documentation) and for container in docker compose file need to insert stop_grace_period:.. and stop_signal: SIGTERM. In default config docker compose use SIGINT and this signal kills all processes with interruption so our jobs are lost. You need to explicitly set stop_signal: SIGTERM so that the sidekiq ends with gracefully shut down.

answered May 22 at 12:58

Maxim Gorbachevskiy

12 bronze badges

1 Comment

Community May 22 at 17:23

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

Collectives™ on Stack Overflow

Sidekiq job lost when deploying to Elastic BeanStalk

2 Answers 2

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related