Step by step ECS Fargate setup from scratch using Ansible part 1

This second article from a series of 3. It focuses on how to create the same infrastructure we created in the first segment with awscli. But this time doing it with Ansible. The purpose of this article isn’t to introduce ansible to a real newbie but to help setup the project ahead of the actual work. However I will try to explain how I use ansible and the rational behind some of my choices. We still need to have an AWS enabled terminal i.e have awscli with a profile configured. To be sure we start from similar setup, I will go through the process of installing Ansible on a mac/linux.

Getting Ready with the prerequisites

I am currently using a M1 powered macbook pro and I use pyenv to manage my python installation and virtualenvs. In most tutorials, I see how people show the installation steps without really saying why they do it the way they do. I will attempt to throw more light on the installation choices.

Since ansible depends on python, there is a requirement to have python installed. To install python, I like many others prefer doing it with pyenv so they can use different versions when needed. With that, it’s a matter of setting up a particular version as the default version or creating an virtualenv. I use pip to install most python libraries, including ansible. That way it’s contained in the version of python used at that point in time. This allows then to have multiple versions of ansible and awscli installed on the same machine. Below are the commands I used, and you can learn more about pyenv and how to install it on your machine at https://github.com/pyenv/pyenv#installation

In the snippet above, we installed the python of flavour pypy v3.9. On the line 2 we create a vitual environment venv-pypy3.9-7 . That is based on the version of the python we installed with pyenv on the line 1. We created our project directory. Then we cd into it and activate our virtual environment. On line 5 we installed our first requirements: awscli, boto3 and ansible.

It will be a good idea to start the project by adding it to git with : git init . in ansible root folder ansible_project_ecs.

Let’s create the necessary files and a bit of a project file structure

In the commands above we have created the following:

• ansible.cfg: Our local ansible configuration file . If anything at all, this tells ansible where we want to store our roles and collections downloaded from ansible galaxy. It overrides the global ansible config if any.
• collection folder: it’s a folder which holds all the downloaded collections from ansible galaxy.
• roles folder: Similarly to collections folder , we created ansible roles folder to hold our roles as well.
• requirements.yml just like to python projects, this file contains the dependencies (roles and collections) and their specific versions.
• .gitignore: file to prevent the downloaded collections from being checked in git repositories.
• playbook_vars: there folder were we keep our variables by environments(test/uat/prod). For now let’s say we are creating the infrastructure for the test environment.
• global_vars: the global variable file contains all the shared variables(not across environment but per environment). Global variables file is included in all the playbooks.

In the config above we have defined a number of settings that would be useful to explain in case some are not known to you.

Most settings are in the defaults section of the config file and below are the explanations to settings configured:

• interpreter_python: This helps ansible find the python installed on your machine. It can be a path to the interpreter but can also be set to auto for auto discovery
• host_key_checking: This is set to false so it tells ansible to skip the host key checking while connecting to a host using ssh. If set t true, this can be the mother of all your problems if you are connecting to new hosts you have not priorly connected and accepted the keys manually.
• roles_path: This informs ansible the path, ie my current roles folder, where to find the roles
• collections_paths: This informs ansible the path, ie my current collections folder, where to find the collections. I hope you are aware that starting ansible 2.9 or so collections are moved out of ansible core right. You can learn more about it as as well about the requirement.yml file below at https://docs.ansible.com/ansible/5/user_guide/collections_using.html

For most python guy this is literally the same as the requirements.txt. It helps track the roles and collections and their versions needed for this current ansible project. These are the various collects our playbooks in this project are going to use or refer to. Most of our collections from the official ansible distribution hub Ansible Galaxy.

Above is our .gitignore files. It could seem obvious but it’s important to remember to keep the files checked in the version control light without the libraries. This is explained further in the portability section.

Now that we have the base skeleton, let’s pull the collections so we can start producing something meaningful. Please execute the following from the main ansible_project_ecs folder:

Portability

Before we dive into writing the playbooks for our setup, it’s important we call out the reasoning behind the project structure and choices made for having a local ansible configuration etc.

The reason is simply portability. We need to have a self contained project that can be safely checked into git repository and contributed to by a team. The collections and roles won’t depend on system level place. The other factor I would put under portability is the fact that we have total freedom on the version of python were using through the virtualenv setup.

Multi environments

The astute reader may have also noticed that under the playbook_vars folder we have multiple stage or environment. So each environment holds all the variables files needed for that environment ie

• dev
• test
• stage
• uat
• preprod
• prod

This has one visible downside which is code duplication for the variables. All the files per environment should be mirrored and this could get worse if we add different git branches to it. But I still do prefer that duplication to complex git conflicts (especially at the ansible vault files) as I am not particularly fan of crazy merge conflicts where the git branch represents a specific environment.

In this setup, the playbooks are directly in the root document and are shared by all environments, only the variable files have specific variable pertaining to one environment or the other.

In the next part of the tutorial, we will dive into the work itself and explain as much as possible the reasoning between some choices. Next => ECS Fargate with Ansible part 2.