All Ruby on Rails Node JS Android iOS React Native Frontend

End to End Tests on CircleCI with Docker - Rails, Capybara, Selenium

The web development world is changing fast. I feel that we are moving towards splitting monolith web applications in two, by extracting backend part into an API. This solution, even though it has many pros, it brings in some difficulties as well. One of which are end to end tests.

It may be really tricky to execute those tests in a proper way, especially when we don't have that much of a control over the process on external hosts, for example during CircleCI build.

I decided to share with you what I've learned - how to set end to end tests on CircleCI. The instructions below can be easily applied to any framework combination other than Rails and React. However, article, in current form, refers to configuration like so:

  • two separate applications for backend (Rails) and frontend (React - the type doesn't really matter) on separate repos

  • tests in Rails are executed by RSpec with Capybara + Selenium + ChromeDriver

  • CircleCI configuration assume incorporating Docker and docker-compose

  • setup process was conducted on existing project with current CircleCI setup and dockerized app for staging and production

I'd like to fit into the form of a simple article, yet I'll try cover as many "What If's" and "Wait, what's" as possible, that may come up during the setup. Most of the lines in showed files that are essential to this setup are commented and explained.The big picture here is to allow writing integration tests (located in spec/features) on backend, test them locally without Docker and run during frontend app build on CircleCI.

Local tests - Backend

Environment setup

Make sure to add proper gems to Gemfile:

Then after bundle install let's add configuration for Capybara and Selenium in two files:

Since our tests are going to be placed in: spec/features our test suite on CircleCI is going to fail. We don't want to configure e2e tests on backend's CircleCI. This is why we need to override test command, in circle.yml:

Local tests - Frontend

Environment setup

This one is simple - almost none setup needed.

The only thing you have to do is to point requests to specific backend address (in our case it's http://localhost:5001 as stated in rails_spec.rb). This is because we want the frontend app to use the same database as is used by rspec during tests - only in this case tests are viable.

This step depends on the technology you use in your project. It's safe to say that most of the javascript projects have package.json file. Usually we started local server by command:

$ yarn start

since, we don't wan't to mess up development environment we can add custom start-test command in package.json:

Our frontend app depends on the env variable REACT_APP_API_BASE_URL which is used to build reference links to rails API. With the above we can now use:

$ yarn start-test

In your project you must figure out how to set this up. Possibilities:

  • any other env variable consumed at some point by the app - which sets proper environment in which server is started ex.: APP_ENV=test or NODE_ENV=test

  • setup .env file with environment variable same as above

  • some kind of if clause in API related service / component

Is it alive?

If you've set everything correctly at this point you are able to run e2e tests locally. Simple is it working? test case:

After we started frontend's local server like stated in previous section we are going to run specs manually just by:

$ rspec spec/features

So now let's move to the Crème de la crème - CircleCI setup!

CircleCI - Prerequisites

Let's start with identifying the files you will be working on:

Dockerfile - file used by docker which contains every information and command that is needed to build specific container by docker (containers are like little environments with their own dependencies, variables and configurations). Look at your Dockerfile now. It does not have any extension. IMPORTANT check if your current Dockerfile has two (or more) FROM instructions. If yes, then this is a problem in terms of CircleCI. See, this is called multi-stage build, which is great (it uses one image, takes something from it, adds something from other image and builds one image from which you set up your container. In single-stage you have to build separate images from the scratch - they take up a lot more space and time), but not for CircleCI. Multi-stage build requires Docker v17.05 and above (standard Docker on Circle 1.0 is much older). We can force that version, but only on CircleCI v2.0. So basically, if you have CircleCI v1.0 and multi-stage build Dockerfile - you have a problem to solve. Dockerfile reference

circle.yml - if you have file named like this then you use CircleCi v1.0. This version is way easier to setup. For v2.0 there is .circleci/config.yml file and it's structure is different. If you want you can migrate from 1.0 to 2.0 following steps in: Migrating from 1 to 2 but I personally don't recommend unless your really know what you are doing. However, using 2.0 have many pros - newer version of Docker and docker-compose which support commands like exec which might be useful. Circle.yml reference

docker-compose.yml - file used by, surprisingly, docker-compose tool. You can think of this file as of list of containers (specified by reference to Dockerfile or by image from Docker Hub) with respective config. This is the most important file in which you have to place every part of setup you need: rails, front, postgres, redis, etc, etc. Docker-compose.yml reference.

CircleCI - Backend

Setup Environment

In my opinion best approach here is to add specific environment to rails app, tailored specifically for end-to-end tests on CircleCI. I picked name: e2e. It's just convenient, it can be virtually any name. So to start:

  • Duplicate config/environments/test.rb as config/environments/e2e.rb

  • In Gemfile to each group specified for :test add :e2e as well in the group definition.

  • If you have secrets then you should make a namespace for e2e in config/secrets.yml (copy definitions from development) and add proper secret_key_base in encrypted keys (in our project we edit secrets by EDITOR=nano rails secrets:edit)

  • Remember to have in mind that in places where you use Rails.env.test? or something similar to setup / check anything you might want to specify how to behave in case of e2e environment as well

  • If you use any kind of requests mocking / suppressing / mimicking (like webmock or vcr) you will need to whitelist containers names in virtual network for example frontend, elasticsearch, postgres, redis etc.

Create new file config/database.e2e.yml:

Create Dockerfile.e2e(or duplicate other Dockerfile you have in project). Our is placed in docker/Dockerfile.e2e. Your Dockerfile may look like the one below. You can also create one from scratch using Dockerfile reference.

Add e2e specific configuration to rails_helper.rb locally, ENV is 'test', while on CircleCI it's 'e2e'. We don't want to start test server automatically. It will be started in a different way:

It's very good idea to keep all commands you need to start specs within one script. We can create one in docker/e2e.sh. This bash script will be started in container by a docker-compose.

That's it! We're done with backend.

CircleCI - Frontend

Setup Environment

Create new docker-compose.ci.yml file in project's main directory or just duplicate and rename current one for ex. docker-compose-staging.yml . You will need to setup few things: proper dependencies for backend, volumes, ports, aliases etc. Everything is explained below. In the end this file should more or less look like this:

If you need another service (in container) just add it to the list in a similar way, watch out for the order.

Now to the frontend Dockerfile. Same as before - copy current Dockerfile or create one in project's root. Unfortunately we had multi-stage build Dockerfile and CircleCI v1.0 so I had to convert multi-stage to single-stage. Let me use an example. At first we had this file:

Now it have to be split up in two files:

If you look carefully you see that I deleted COPY --from=builder /app/build /app/dist - this will be done in a different way.

The above change will force two new steps. In circle.yml we have to add in dependencies/pre after pyenv rehash:

In case your config is:

  • multi-stage + v1.0 - convert multi-stage to single-stage
  • multi-stage + v2.0 - make sure that you force proper Docker version and Edge build of CircleCI
  • single-stage + v2.0 - do nothing, just use that Dockerfile
  • single-stage + v1.0 - do nothing, just use that Dockerfile

The last step is to configure circle.yml and setup precedence of commands and indicate usage of Docker. In the end circle.yml should look like:

This is it, at this point everything should be safe and sound. If not and your build is failing ssh to CircleCI and play around.

Useful commands

Useful links

docker-compose run reference

docker cp reference

article about multi-stage build

another article about multi-stage build

yet another article about multi-stage build

understanding volumes in containers

copy data between containers

FAQ

Is it hard to setup?

It may be. Everything depends on CircleCi version you have, which limits interactions with Docker (old versions) and how many dependent containers you need for your backend. Some of them might be really tricky to setup correctly.

Should I consider migrating to CircleCI v2.0?

Definitely! This build takes much time to execute. With newer version of Circle you might speed it up significantly and use new Docker (less bugs, some commands actually work, not just fail).

What if I have CircleCI v2.0 file?

Then you should be happy. Everything you see above is still valid, only that you don't have to worry about multi-staging. Just follow this guide Multi-stage docker build with with CircleCI v2.0 and rewrite our circle.yml additional lines using 2.0 style CircleCI v2.0 reference

Summary

I hope that I've managed to describe everything to enable you to set end to end tests on your own in your project.

Please feel free to ask if anything is unclear. Maybe article is unclear or needs improvements? Point it out as well. I believe the best way we learn is by a mistake, we can both benefit from it.

Later, I'm planning to prepare a follow-up for CircleCI v2.0, so stay tuned!

We're building our future. Let's do this right - join us
New Call-to-action
READ ALSO FROM CircleCi
Read also
Need a successful project?
Estimate project or contact us