After I spent an entire afternoon banging my head against the preverbal office wall, I FINALLY figured out how to deploy a worker process in Heroku when using a docker container.
A bit of background: our team has a Ruby on Rails app that’s deployed using Heroku’s “container” stack. This approach is nice because our app has several custom Linux tools like FFMPEG that are much easier to specify and deploy using a Dockerfile instead of Heroku buildpacks.
The problem came when I needed to add a sidekiq worker to our app. I followed the normal Heroku docs, which suggested adding something like this to the Procfile:
web: bundle exec puma -C config/puma.rb worker: bundle exec sidekiq -C config/sidekiq.yml
But whenever I tried to deploy, Heroku would refuse to recognize that it needed to spin up a worker dyno. I did a lot of digging to see what the issue was. Several people suggested that it could be that my Procfile wasn’t named correctly or that there was something wrong with the formatting inside the file. After confirming everything was correct and still running into the issue, I finally figured it out!
The solution was quite simple once I found it buried in the Heroku docs.
- Rename my Dockerfile to Dockerfile.web.
- Create a
Dockerfile.worker
that will run the sidekiq job. Mine looks something like this:
# Use Ruby 3.2.2 as the base image FROM ruby:3.2.2 # Set the working directory to /app WORKDIR /app # Copy the Gemfile and Gemfile.lock from the app directory to the container COPY Gemfile Gemfile.lock ./ # Install dependencies RUN bundle install # Used for video transcoding RUN apt-get update -y && apt-get install -y ffmpeg # Copy the rest of the application code to the container COPY . . # Set environment variables for build step ENV AWS_ACCESS_KEY_ID="" ENV AWS_SECRET_ACCESS_KEY="" ENV AWS_REGION=us-east-1 ENV AWS_BUCKET="test-bucket" # Start sidekiq CMD ["bundle", "exec", "sidekiq", "-C", "config/sidekiq.yml"]
3. Create or update your heroku.yml file to look like this:
build: docker: web: Dockerfile.web worker: Dockerfile.worker
4. Now deploy! Heroku will use the
heroku.yml
contents to deploy a dyno for each entry within the “docker” section. It should look something like this in your Heroku project’s “Resources” tab:
NOTE: I was having trouble with sidekiq not processing my jobs even after everything was deployed correctly. It turns out Heroku was kind to me and didn’t toggle the worker dyno to the “on” position so that I wouldn’t get charged until I explicitly flipped the switch and set the number of dynos I wanted. Make sure you check this before you try to start processing jobs.
I hope this helped save someone else the same headaches I experienced. As you can see, it’s simple to deploy a worker dyno using a Docker container once you know what you’re doing!
Austin Aldrich
Unosquare Lead Software Development Professional