This post assumes you have some experience with Docker.
Example Application: Ruby on Rails, MySQL, Redis
We rarely use Rails by itself these days. For things like plugins, sure, but most real applications use at least one database layer. So let's have in mind that we're always working with multiple frameworks. But even if we aren't, I believe we should leverage on the benefits of Docker Compose to set some stuff for us, like ports, volumes and etc, without having to type long commands.
For the above stack, the common way is to create one container for each.
Let's start with Ruby on Rails.
As there is quite some stuff we need to do to create this container properly, we need a Dockerfile. I prefer to save all my dockerfiles inside /config/docker, but you can save them wherever you want.
So my Dockerfile (which I placed in /config/docker/Dockerfile-web) starts like this:
FROM ruby:2.3.3
ENV LANG C.UTF-8
ENV APP_DIR /app
RUN mkdir -p $APP_DIR
WORKDIR ${APP_DIR}
CMD rails s -p 3000
Most guides out there will add Bundler commands to the above file, such as bundle install, but as we're working with development, I prefer to have that outside the scope of building. More on that in a bit.
The above Dockerfile should be enough to get our development process started. We can create the docker-compose.yml file now:
version: "2"
services:
web:
build:
context: ./
dockerfile: ./config/docker/Dockerfile-web
ports:
- "3000:3000"
volumes:
- ./:/app
Then we can make our first image build with:
docker-compose build
It will build a "web" image with Ruby 2.3.3, as defined in the Dockerfile. Once building is done, we can "enter" the container with:
docker-compose run --rm web bash
Inside the container, we can run all the commands we know, such as bundle install, rails console, and so on. We can install gems at this point, and they will be saved in /usr/local/bundle (inside the container, and not in your host machine).
This is where things get tricky.
Once you exit the container, all new data created in that session will be gone, including the gems we just installed in /usr/local/bundle. This is by design, so we just need to understand what's going on.
A container works like an instance of an image. So once you exit/close the container, that instance will be gone. When you start the container again, all the data will be based on the original image, which means the /usr/local/bundle directory will be empty.
So we need a way to persist certain data, like bundled gems. One way (and mentioned by most guides) is to add them to the building process. In such cases, your Dockerfile will end up like this:
FROM ruby:2.3.3
ENV LANG C.UTF-8
ENV APP_DIR /app
RUN mkdir -p $APP_DIR
WORKDIR ${APP_DIR}
RUN gem install bundler --no-ri --no-rdoc
COPY Gemfile* ./
RUN bundle install
CMD rails s -p 3000
With the new commands, the build process will copy the Gemfile (and .lock) from your application folder to the to-be-generated-image and use them to build all gems into /usr/local/bundle (again, inside the image). This process makes the image ready to run immediately and is perfect for deployment, because it turns the image into a distributable package.
But it may not be suitable for some development workflows.
If your Gemfile changes, you will have to build the image again. This isn't usually a problem, except that every time you have to rebuild (whenever your Gemfile changes), all gems will be reinstalled. Depending on how many gems you have, this can take a long time. If you're changing gems a lot, or, say, you're switching around repository branches with different Gemfiles, this can hinder your productivity.
This is how I deal with this: I don't bundle install in my Dockerfile. I do it manually after I build the image, entering the container and running the commands myself. This is what I'd do if I weren't running Docker after all, so nothing changes in my development routine.
Now, how to deal with gem persistence when you bundle install outside of the building context, given that containers delete new data when they are closed?
The most recommended answer I've seen is to use a data container, and it's been working well in my experience. You can add this new container in your docker-compose.yml:
data:
image: tianon/true
volumes:
- /usr/local/bundle
Then use it from your "web" container:
web:
build:
context: ./
dockerfile: ./config/docker/Dockerfile-web
ports:
- "3000:3000"
volumes:
- ./:/app
volumes_from:
- data
This will redirect all data saved in /usr/local/bundle to the data container, and persist that data when you close them. Docker will handle storage internally.
After you build, then it's a matter of entering the container and using the commands you know and love to get your app ready to run.
We can set up MySQL now.
This one does not require a Dockerfile, so all we have to do is add the following new container in docker-compose.yml:
mysql:
image: mysql:5.6
ports:
- "33000:3306"
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: "yes"
MYSQL_DATABASE: myapp
MYSQL_USER: root
volumes_from:
- data
Now, regarding working with MySQL in development, I prefer to have direct access to it using a client such as Sequel Pro. To allow that, we have to expose a port to the host machine. In the above configuration, that port is 33000. You can pick any port you want, but it must be free in your machine. I don't personally recommend you use the default MySQL port (3306) for this, because if you happen to run multiple docker environments in the future, and more than one of them have MySQL trying to expose the same port to the host machine, you will run into errors. So I pick an arbitrary port and make sure they're different from other projects.
In regards to ports that other containers need to use, that's what the second number does: it exposes the default MySQL port to them (and not to the host). This is important because this is the port you will use in your database.yml file (and not 33000).
While at it, you also need to change the "host" setting in your database.yml to "mysql", which is the container name itself. So if you name your MySQL container to something else, you need to use that name in your configuration. Also set the database name, which must match what you define as MYSQL_DATABASE in the docker-compose.yml file (in the example above, it's myapp).
And finally, you must persist MySQL data, otherwise data will be gone when you close it. So we use the same technique we used with Bundler, adding an additional volume to our data container:
data:
image: tianon/true
volumes:
- /usr/local/bundle
- /var/lib/mysql
You can use another data container for MySQL alone, but the result is virtually the same.
Next up: setting up Redis. This is easier as no volumes are involved, so just add a new container to docker-compose.yml:
redis:
image: redis:3.0-alpine
Then you need to change your Redis configuration file in your application to point to "redis" as the host. The default port is 6379. If you want access to your Redis instance from your host (if you have a client or something), you will need to apply the same technique done to the MySQL container, exposing the ports in the docker-compose.yml file.
After you've created both MySQL and Redis containers, you need to make them accessible to your "web" container. This is done by adding them as "links" in the docker-compose.yml file:
web:
...
links:
- mysql
- redis
You will be able to see it working for yourself if you enter the "web" container and run ping mysql for example.
Before we bring the whole thing up, let's get over a few things, as they're related to development workflow.
Entering the container should be common now. Again, the command for that is:
docker-compose run --rm web bash
In practice, it runs "bash" in the container named "web" which gives us a shell interface. You want the --rm flag because otherwise the container will remain in your system when you exit it, taking up space.
While inside the container, the work you will commonly do is:
- bundle install / update
- rake db:create / migrate / seed / etc
- run other rake tasks
- rails generators
- and so on.
For running the application itself, it's best to leave it for Docker Compose to handle, instead of running it yourself inside the container.
When you docker-compose up, Docker Compose will bring all containers up and output all log on the screen. Then you can use Ctrl+C to close all containers. You can add -d to make them run in the background. In such case, you have to use docker-compose stop to close them.
Since running the containers this way will bring everything up, and down, it is sometimes counter-productive to shut down MySQL and Redis every time you need to restart your Rails application. So here's what I do:
First, I run anything, that's not the application itself, in the background:
docker-compose up -d mysql redis
Then, I bring the application up in synchronous mode:
docker-compose up web
That way, when I Ctrl+C, only the web container goes down. Then I just repeat the command to bring it back up. When you're developing, it is recommended that you do not run the application in background if you need to do some debugging (hello binding.pry).
From this point on, you should be able to develop as you usually do. You can even open another Terminal tab and enter the "web" container if you want to run commands while the application is running.
In closing, here are a few observations I've gathered.
I tried to map /usr/local/bundle to a folder in my machine, like tmp/bundle. This resulted in extremely slow bundle installs and application boot. That's why I used a volume to save gem data. Thankfully, having your own application files in your machine does not slow things down.
On the other hand, you can map your MySQL data folder (/var/lib/mysql) to a folder in your machine if you want, such as tmp/mysql. In my experience, this did not result in decreased performance.
From this point on, you should be able to develop as you usually do. You can even open another Terminal tab and enter the "web" container if you want to run commands while the application is running.
In closing, here are a few observations I've gathered.
I tried to map /usr/local/bundle to a folder in my machine, like tmp/bundle. This resulted in extremely slow bundle installs and application boot. That's why I used a volume to save gem data. Thankfully, having your own application files in your machine does not slow things down.
On the other hand, you can map your MySQL data folder (/var/lib/mysql) to a folder in your machine if you want, such as tmp/mysql. In my experience, this did not result in decreased performance.
In your Gemfile, if you are using gems that are hosted in private repositories that require SSH keys to download, here's a little trick I did: map the container's .ssh folder to your machine's:
web:
...
volumes:
- ./:/ecommerce
- ~/.ssh:/root/.ssh
You can then even use git commands inside your container like you were in your own machine. :)
I hope that helps someone out there. You can reach me on Twitter if you have questions or comments.
I hope that helps someone out there. You can reach me on Twitter if you have questions or comments.
