Step by step ECS Fargate setup from scratch using Ansible part 2

Welcome to the part 2 of our attempt to deploy to ECS using ansible. This is the continuation of the ansible setup we did on in the part 1. In there in part 1, we have set up the project using a specific structure that allows for multi environments and allows for the use of collections from ansible galaxy. In this part 2, we will go through the implementation of ansible to create various AWS services necessary for us to reach our goal. This post is likely to be long based on the objective. There will be a part 3 where we hope to finalise on the usage of ansible for our setup. For those who want to follow along or who are just like me and sometimes just want to checkout the code and read , you can get it from my gitlab account


In the very first part of this series where we used strictly awscli to achieve the same setup, the usage of environment variables to hold certain values can be argued to have some level of reusability. That can be explained further with the fact that there aren’t too much hard coded values, especially for the outputs. It can easily be copied and pasted and executed.

The whole point of having ansible playbooks broken into a set reusable files is to promote reusability. After many years of using ansible , breaking playbooks into smaller pieces turns out to be one of the practices that had improved our operations in many ways, stability, security etc. The reason behind that is very simple, it boils down to trust issues 😀 . Even though there is a promise of idempotency in ansible, one should not always run a playbook that (re)-creates a whole VPC for example, when the deployment is just for one application many others. Especially when no change is required on the VPC section. So each time there is a deployment of a new update or for a brand new application/service, only the playbook that deploys that service should be the one that runs. This reduces greatly the radius blast when we run into issue or things go south. This post focussing on how to deploy to ECS using ansible shows how to achieve that

Setting up Base infrastructure

Let’s start with the playbook dedicated to the setup of the AWS VPC, its various subnets, NAT gateway and Internet gateways as well. The first playbook we are going to work on will be called aws_setup_base_infra_playbooks.yml.The scope of this base infrastructure playbook will cover the following:

  • Creating the VPC
  • Creating all the subnets
  • Creating the Internet Gateway
  • Creating the NAT Gateway
  • Mapping the public and the private subnets
  • Creating CloudWatch Log group

VPC & network components

The playbook aws_setup_base_infra_playbook.yml will make use of the following variable files:

  • playbook_vars/<environment>/aws_setup_base_infra_vars.yml: this will hold specific variables for the base infra.
  • playbook_vars/<environment>/global_vars.yml: This will hold all the common variables that cuts across multiple playbooks. For example any playbook that needs the id of the vpc will definitely lookup its name in the global_vars and then perform the use the ec2_vpc_net_info based on tag name and retrieve any necessary info returned.

Let’s look into the first iteration of the aws_setup_base_infra_playbooks.yml file

All the variables used in the playbook are from eitheraws_setup_base_infra_vars.yml or global_vars.yml . As explained earlier, the global_vars.yml mainly contains names and values that are likely to be needed by more then just the vpc playbook.

The setup base infra variables on the other hand, directly influences the way the base infra playbook should run. It allows to toggle from true to false certain aspect , enabling or disabling the execution of some steps.

There are number of elements from the playbook that I would like to call out specifically. In order to create the subnets, we used 2 different methods. For the subnets that are supposed to be public, I have used a loop construct and for the private one, a regular construct .

To execute this playbook, from the ansible folder , run the following:

  • AWS_PROFILE being yevi-test , the same as the one we used before. It’s a combination of the following <product>-<environment>. Yevi being our fiction product and test being the environment of interest right now.
  • aws_setup_base_infra_playbook being the name of the playbook
  • -e being the sub command for the extra variables. This has the highest precedence in case any variable defined here has the same key but different value. The extra variable one always overrides whichever one defined in a variable files.

After executing the playbook above a number of time, it became pretty apparent that each time we run it, to test the idempotency , it create a new Elastic IP and therefore a new NAT Gateway, that’s not what we want. After googling without much pointers , I upgraded my python from pypy3.9-7.3.9 to pypy3.10-7.3.12 so I can install the latest version of the ansible ansible [core 2.15.1].I have also upgraded the collections and my requirements.yml file looks now like the the following:

None of that seems to want to fix the broken idempotency. Later on, I narrowed it to the creation of the EIP. Once the EIP creation has been isolated (i.e created manually outside of ansible), the rest of the play became idempotent again.

This little issue has brought forward a practice that I was going to introduce later. This is directly related to our quest to reusability. The fact that files are broken down to a set of concerns in a playbook makes sharing of ids or results used by other playbooks, difficult .To solve that, we will need to introduce a file responsible of collecting facts and info from components already created (by previous playbooks, obviously) and make cross playbooks variables sharing a bit more manageable. I surprisingly named the file aws_setup_fact_gatherer_playbook.yml . Let’s dive into its content below

The change above will need an update of the following to the global_vars.yml . The EIP has been created manually using the same tags Name: <product>-<environment>-<name_sufx> i.e. yevi-test-ngw-eip . The set_fact module is exporting into a global ansible execution context, the value of the variables declared under it.

Now that we have the idempotency fixed. It’s time to work on the second iteration of our aws_setup_base_infra_playbooks.yml file. I apologise for creating or posting the same playbook twice. That has contributed to the length of this tutorial but , it’s a necessary step to show how in real world, we change or adjust our code base and adapt to new solutions.

There are a few things I would like to draw your attention to.

The first is that we have included the fact gatherer playbook into our base infrastructure playbook at its top. This means that the fact gatherer playbook will execute before the rest of the base infrastructure. It is at this point exporting exported_ngw_eip_info .

the second is that, At the line 155 of aws_setup_base_infra_playbooks.yml, allocation_id: "{{ hostvars.localhost.exported_ngw_eip_info.addresses.0.allocation_id }}" we are retrieving the value of the allocation_id from the exported exported_ngw_eip_info that was pushed by the fact gatherer. This allocation_id is crucial to the creation of our NAT Gateway.

Setting up container and permission resources

In this section of the post will focus on setting up resources such as IAM policies and roles as as well security groups. To deploy to ECS using ansible, we have to setup the container registry, since it’s all about containers in ECS (just in case we have not noticed 😀 ). Our containers will need security groups, execution roles etc. The file aws_setup_container_permission_playbook.yml will create all the aforementioned resources and will make use of the global variable file, the ansible vault variable file test_variables.vault as well as the aws_setup_cont_and_perm_vars.yml variable file.

Container resources

There are situations where a container needs to be passed environment variable securely. In AWS that can be done using 2 services. Parameter Store and Secrets manager. We will assume that our applications will require to use database credentials from the secrets manager for example (another service from AWS). We will also create the container registry that will keep our images from which we pull ,create and run containers.

The credentials will have to be stored somewhere in our ansible structure and eventually checked into our git repository. This, on the face value is just the right amount of recipe for a disaster. Anyone with access to your repo will also have access to the creds. To avoid that we will need to leverage the ansible vault utility. Let’s get started with the vault. We will need a place to store the vault file and a place for the password file as well. Some may decide not to use the password file and rather just use some password preferably stored in a environment variable on your computer. For now, let’s say we store our file inside a Dropbox folder.

Let’s create our password file. It can be generated with openssl as follows. But then I have a little utility pwgen that I love using for this kind of stuff

you could use openssl as the following to create the password file

But if you prefer the pwgen like I do then you could do it like the following

We will create the ansible vault file location and the vault file. From the root folder do the following. the ansible-vault command will want to use a cli text editor. I can’t remember the prompt very well but I used vim. So it will open up the vim editor for you to add the first variables in the vault

Now we can create the playbook that will make use of the ansible vault in order to create the aws secrets manager item from its content. We will also add the creation of the ECR to it.

It’s corresponding variable file is aws_setup_cont_and_perm_vars.yml shows as below but as well it’s worth noting that we need to update the global_vars.yml

Because we have introduced the vault file, to execute the playbook now, it has to include the vault file as shown in the following

Permissions (Security Groups and IAM)

We know we want to deploy to ECS using ansible, also know that we will need to create a set of security groups and at least an ecs task execution role with enough privileges to either pull the image from the ECR or read the secrets from the secrets manager.


We will use the same playbook to compete the rest of our requirements. Interestingly to achieve the goal we will have to introduce the concept of files and templates. In our root directory we will be creating the following 2 folders:

  • files : This will hold static files that don’t change or the resulting file out of the execution of a template fed with data.
  • templates: this is where we keep the jinja2 template files

We will put in the files, the assume role json document that we have to create for ECS

We will leverage the template file to create the IAM json policy we need the ECS task to have. Below is how the template looks like

In the template above we are created the following access to our ecs tacks:

  • Ability to retrieve images from the ECR
  • Ability to write logs in to the cloudwatch streams
  • Ability to read from parameter stores and secrets manager

We need the arn of the secrets manager to add the policy but it turns out that we won’t be able to get that from the returned output from it’s creation. We will need to compose the full arn of the secrets manager. We will need to know the AWS account id for that. It also turns out that we will need to find the arn of the log group to allow the write permission to. But we have already created that in the previous section. We will have to update the aws_setup_fact_gatherer_playbook.yml to get the missing information. We will also sneak in a way of retrieving the vpc id that we will pretty much soon need.

Putting everything together about the IAM we will create the aws_setup_container_permission_playbook.yml with the following content

It’s corresponding variable file aws_setup_cont_and_perm_vars.yml needs to be updated with the following, with all previous content hidden of course to prevent repetition.

The global_vars file also needs to be updated with the following

You can choose to execute the file using the same command involving the ansible vault as shown earlier or wait till the end of the next section to do it.

Security Groups

We already know the flow of the traffic from internet to our containers. We know that for serving our microservices ,we will the help of a load balancer. It will be an application load balancer. When the load balancer receives the traffic it needs to pass it on to the containers. Therefore the containers also need to allow traffic from the load balancer to themselves . Based on that, the following security groups are ones we will create:

  • A security group for the load balancer to accept request from the internet
  • A security group for the ecs task to accept request from the load balancer

To add the 2 security groups, let’s edit the aws_setup_container_permission_playbook.yml file its associated aws_setup_cont_and_perm_vars.yml variable file and of course the global_var.yml

The same ansible playbook with vault command applies here as well to execute this final section of the part 2 of our tutorial.

The snapshot of this particular section is tagged as base_infra_and_security in gitlab:

To checkout this tag, you will have to run the following:

We have arrived at the end of the part 2 of our post. I hope I have been able to show you how to deploy to ECS using ansible and other few things that can help you better structure your ansible project and even build on that. Feel free to share any alternative to ideas discussed here.

Leave a Reply

Your email address will not be published. Required fields are marked *

captcha * Time limit is exhausted. Please reload the CAPTCHA.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Scroll to top