Deploying Workers with Heroku Containers

After I spent an entire afternoon banging my head against the preverbal office wall, I FINALLY figured out how to deploy a worker process in Heroku when using a docker container.
A bit of background: our team has a Ruby on Rails app that’s deployed using Heroku’s “container” stack. This approach is nice because our app has several custom Linux tools like FFMPEG that are much easier to specify and deploy using a Dockerfile instead of Heroku buildpacks.
The problem came when I needed to add a sidekiq worker to our app. I followed the normal Heroku docs, which suggested adding something like this to the Procfile:

web: bundle exec puma -C config/puma.rb
worker: bundle exec sidekiq -C config/sidekiq.yml

But whenever I tried to deploy, Heroku would refuse to recognize that it needed to spin up a worker dyno. I did a lot of digging to see what the issue was. Several people suggested that it could be that my Procfile wasn’t named correctly or that there was something wrong with the formatting inside the file. After confirming everything was correct and still running into the issue, I finally figured it out!

The solution was quite simple once I found it buried in the Heroku docs.

1. Rename my Dockerfile to Dockerfile.web.
2. Create a Dockerfile.worker that will run the sidekiq job. Mine looks something like this:

# Use Ruby 3.2.2 as the base image
FROM ruby:3.2.2
# Set the working directory to /app
WORKDIR /app
# Copy the Gemfile and Gemfile.lock from the app directory to the container
COPY Gemfile Gemfile.lock ./
# Install dependencies
RUN bundle install
# Used for video transcoding
RUN apt-get update -y && apt-get install -y ffmpeg
# Copy the rest of the application code to the container
COPY . .
# Set environment variables for build step
ENV AWS_ACCESS_KEY_ID=""
ENV AWS_SECRET_ACCESS_KEY=""
ENV AWS_REGION=us-east-1
ENV AWS_BUCKET="test-bucket"
# Start sidekiq
CMD ["bundle", "exec", "sidekiq", "-C", "config/sidekiq.yml"]

3. Create or update your heroku.yml file to look like this:

build:
docker:
web: Dockerfile.web
worker: Dockerfile.worker

4. Now deploy! Heroku will use the heroku.yml contents to deploy a dyno for each entry within the “docker” section. It should look something like this in your Heroku project’s “Resources” tab:

NOTE: I was having trouble with sidekiq not processing my jobs even after everything was deployed correctly. It turns out Heroku was kind to me and didn’t toggle the worker dyno to the “on” position so that I wouldn’t get charged until I explicitly flipped the switch and set the number of dynos I wanted. Make sure you check this before you try to start processing jobs.

I hope this helped save someone else the same headaches I experienced. As you can see, it’s simple to deploy a worker dyno using a Docker container once you know what you’re doing!

Austin Aldrich

Unosquare Lead Software Development Professional