Creating and maintaining an inventory file is one of the common tasks we have to deal with when working with Ansible. When dealing with a large number of hosts, it can be complex to handle this task manually.
There are some plugins available to automatically generate inventory files by interacting with most cloud providers’ APIs. In this post, however, I am going to show you how to generate and maintain an inventory file directly from within the Terraform code.
To illustrate this, we will generate an Ansible inventory file to deploy a MongoDB sharded cluster on the Google Cloud Platform. If you want to know more about Ansible provisioning, you might want to check out my talk at Percona Live Online 2021.
The Ansible Inventory
The format of the inventory file is closely related to the actual Ansible code that is used to deploy the software. The structure we need is defined as follows:
[cfg] tst-mongo-cfg00 mongodb_primary=True tst-mongo-cfg01 tst-mongo-cfg02 [shard0] tst-mongo-shard00svr0 mongodb_primary=True tst-mongo-shard00svr1 tst-mongo-shard00svr2 [shard1] tst-mongo-shard01svr0 mongodb_primary=True tst-mongo-shard01svr1 tst-mongo-shard01svr2 [mongos] tst-mongo-router00
We have to specify groups for each shard’s replicaset (shardN), as well as the config servers (cfg) and mongos routers (mongos).
The mongodb_primary tag is used to designate a primary server by giving it higher priority in the replicaset configuration.
Now that we know the structure we need, let’s start by provisioning our hardware resources with Terraform.
Creating Instances in GCP
Here is an example of a Terraform script to generate some instances for the shards. The Terraform count directive is used to create many copies of a given resource, so we define it dynamically.
resource "google_compute_instance" "shard" {
name = "tst-mongo-shard0${floor(count.index / var.shardsvr_replicas )}svr${count.index % var.shardsvr_replicas}"
machine_type = var.shardsvr_type
zone = data.google_compute_zones.available.names[count.index % var.shardsvr_replicas]
count = var.shard_count * var.shardsvr_replicas
labels = {
ansible-group = floor(count.index / var.shardsvr_replicas ),
ansible-index = count.index % var.shardsvr_replicas,
}
boot_disk {
initialize_params {
image = lookup(var.centos_amis, var.region)
}
}
network_interface {
network = google_compute_network.vpc-network.id
subnetwork = google_compute_subnetwork.vpc-subnet.id
}
The above code shuffles the instances through the available zones within a region for high availability. As you can see, there are some expressions used to generate the name and the labels. I will get to this shortly. We can use similar scripts for creating the instances for the config servers and the mongos nodes.
This is an example of the variables file:
variable "centos_amis" {
description = "CentOS AMIs by region"
default = {
northamerica-northeast1 = "centos-7-v20210420"
}
variable "shardsvr_type" {
default = "e2-small"
description = "instance type of the shard server"
}
variable "region" {
type = string
default = "us-east1"
}
variable "shard_count" {
default = "2"
description = "Number of shards"
}
variable "shardsvr_replicas" {
default = "3"
description = "How many replicas per shard"
}
}
Identifying the Resources
We need an easy way to identify our resources to work with them effectively. One way is to define labels on our Terraform code that is used to create the instances. We could also use instance tagging instead.
For the shards, the key part is that we need to generate a group for each shard dynamically, depending on how many shards we are provisioning. We need some extra information:
- the group names (i.e. shard0, shard1) for each host
- which member within a group we are working with
We define two labels for the purpose of iterating through the resources:
labels = {
ansible-group = floor(count.index / var.shardsvr_replicas ),
ansible-index = count.index % var.shardsvr_replicas,
}
For the ansible-group, the variable shardsvr_replicas defines how many members each shard’s replica set has (e.g. 3). So for 2 shards, the expression above gives us the following output:
floor(count.index / 2 ) 0 0 0 1 1 1
These values will be useful for matching each host with the corresponding group.
Now let’s see how to generate the list of servers per group. For the ansible-index, the expression gives us the following output:
count.index % 3 0 1 2 0 1 2
For mongos and config servers, it is easier:
labels = {
ansible-group = "mongos"
}
labels = {
ansible-group = "cfg"
}
So with the above, we should have every piece of information we need.
Creating an Output File
Terraform provides the local_file resource to generate a file. We will use this to render our inventory file, based on the contents of the inventory.tmpl template.
We have to pass the values we need as arguments to the template, as it is not possible to reference outside variables from within it. In addition to the counters we had defined, we need the actual hostnames and the total number of shards. This is what it looks like:
resource "local_file" "ansible_inventory" {
content = templatefile("inventory.tmpl",
{
ansible_group_shards = google_compute_instance.shard.*.labels.ansible-group,
ansible_group_index = google_compute_instance.shard.*.labels.ansible-index,
hostname_shards = google_compute_instance.shard.*.name,
ansible_group_cfg = google_compute_instance.cfg.*.labels.ansible-group,
hostname_cfg = google_compute_instance.cfg.*.name,
ansible_group_mongos = google_compute_instance.mongos.*.labels.ansible-group,
hostname_mongos = google_compute_instance.mongos.*.name,
number_of_shards = range(var.shard_count)
}
)
filename = "inventory"
}
The output will be a file called inventory, stored on the machine where we are running Terraform.
Template Generation
On the template, we need to loop through the different groups and print the information we have in the proper format.
To figure out whether to print the mongodb_primary attribute, we test the loop index for the first value and print the empty string otherwise. For the actual shards, we can generate the group name easily using our previously defined variable, and check if a host should be included in the group.
Anything between %{} contains a directive, while the ${} are used to substitute the arguments we fed the template. The ~ is used to remove the extra newlines.
[cfg]
%{ for index, group in ansible_group_cfg ~}
${ hostname_cfg[index] } ${ index == 0 ? "mongodb_primary=True" : "" }
%{ endfor ~}
%{ for shard_index in number_of_shards ~}
[shard${shard_index}]
%{ for index, group in ansible_group_shards ~}
${ group == tostring(shard_index) && ansible_group_index[index] == "0" ? join(" ", [ hostname_shards[index], "mongodb_primary=True\n" ]) : "" ~}
${ group == tostring(shard_index) && ansible_group_index[index] != "0" ? join("", [ hostname_shards[index], "\n" ]) : "" ~}
%{ endfor ~}
%{ endfor ~}
[mongos]
%{ for index, group in ansible_group_mongos ~}
${hostname_mongos[index]}
%{ endfor ~}
Final Words
Terraform and Ansible together is a powerful combination for infrastructure provisioning and management. We can automate everything from hardware deployment to software installation.
The ability to generate a file from Terraform is quite handy for the purpose of creating our Ansible inventory. Since it is responsible for the actual provisioning, Terraform has all the information we need, although it can be a bit tricky to get the templating right.
Please join Autodesk’s Senior Database Engineer, Vineet Khanna, and Percona’s Sr. MySQL DBA, Tate McDaniel as they present Migrating to Aurora and Monitoring with PMM on Thursday, August 9th, 2018, at 10:00 AM PDT (UTC-7) / 1:00 PM EDT (UTC-4).
Vineet Khanna, Senior Database Engineer, Autodesk