Lately I have been writing a lot of Terraform code for some personal projects. After copying certain files around for the 5th or 6th time, I decided to make a skeleton repository for Terraform projects to make things a little easier.
Randompassword.password0.result and randompassword.password1.result for your result values (randompassword.password without an index is treated like any other map in terraform when foreach or count is used for recursion). I set the count to 2 in the example, but this can be any int.
When I first started with Terraform a few years ago, I wanted to have my repositories structured in a logical fashion, but nothing I found seemed to fit what I was looking for. I searched the internet to find out what HashiCorp recommended and what other people are doing, but nothin I saw really felt right to me. In examining what other people were doing, however, I was able to take the best parts from various layouts and meld it into a structure that worked for me and I have been using it ever since.
- A 1Password Secrets Manager terraform module. Used to fetch secrets from 1Password. Read more master. Switch branch/tag. Find file Select Archive Format. Download source code. Zip tar.gz tar.bz2 tar. Clone Clone with SSH Clone with HTTPS Open in your IDE Visual Studio Code Copy HTTPS clone URL.
- 1Password Connect Terraform Provider. Use the 1Password Connect Terraform Provider to reference, create, or update items in your 1Password Vaults. To build the 1Password Connect Terraform provider run the following.
I expect that many will look at my layout and feel the same way I did about many of the ones I looked at. Hopefully, you will be able to take a way at least a nugget or two to help you with your structure. You can check out the GitHub repo here.
Directory Layout
The directory layout is pretty straight forward. It contains a directory for all the project variables and Terraform files as well as a Makefile and Jenkinsfile for automation. The file tree of the directory looks like this:
Projects
The projects directory stores the .tfvars file for each project. The skeleton repo contains two directories. The globals directory is run each time Terraform is run. It contains variables that are constant across deployments. The template directory is an example of an individual project file. When you use Makefile or Jenkinsfile to run the Terraform command, it will run with the global variables as well as the variables of the defined project. I think of a project as an individual instantiation of the Terraform state. It could be an environment (development/staging/production), accounts (aws1, aws2, etc), or even regions (us-west-1, eu-east-1, etc).
Makefile
I do not remember where I came across the idea to use a Makefile for running my Terraform commands, but it has been extremely useful. It allows me to run multiple commands at once without typing long command lines. Prior to running the make file, you need to set two environmental variables. The BUCKET variable is used in the terraform init</code> command to set the S3 bucket used to store state. The PROJECT</code> variable is the project that you want to run the terraform for. This variable is used in the name of the Terraform state file as well as to choose which project variables to run.
Jenkinsfile
The Jenkinsfile is used to run the terraform commands from Jenkins. It runs a a Jenkins pipeline that includes 4 stages: Checkout, Initialize the Backend, Terraform Plan, and Terraform Apply/Destroy. As requires 5 parameters to run the job: The name of the S3 bucket, the project name, the Git credentials to use, the AWS credentials to use, and a dropdown to apply or destroy the project.
Terraform Files
Rather than cramming everything into a single file, I tend to use more files rather than less for readability. To that end I generally have 5 .tf files that I use when working with Terraform.
backend.tf
The backend.tf file contains information about which backend to use (S3 in my case).
provider.tf
The provider.tf file contains which provider to use. My directory defaults to the AWS provider, but I have used Azure and GCP as well.
main.tf
This is where I define which modules I want to use. Now that Terraform has a module registry, I try to use that as much as possible, but occasionally I will write my own.
variables.tf
The variables.tf file is used to initialize all the variables that I want to pass in via my projects file.
outputs.tf
The outputs.tf file is for storing any outputs that you may want to make available to other Terraform projects at a later time.
Errata
The README.md and LICENSE file are self explanitory.
Starting a New Project
Terraform 1password Tutorial
Now when I start a new project, it is relatively easy for me to . Since I use GitHub, all my commands will be tailored for that platform.
Create an Empty Repository on GitHub
Terraform 1password Extension
Start by creating an empty repository on GitHub. You can do this through the web interface, or if you have a GitHub token you can create it through the API with the following command:
This will return the SSH URL for the newly created repo.
Clone the Skeleton Repo
Next you can clone the skeleton repository to your local machine and rename it to your new repo:
Change the Origin
Once you have the skeleton repository checked out, you can update the origin and push the code back up to GitHub:
That’s it. I am now ready to start developing my new Terraform project.
UPDATE: I’ve moved my skeleton repo to be a GitHub Template. You can find more information here
Password specialist 1Password has acquired SecretHub, a secrets management platform aimed at IT engineers, and made a new service called Secrets Automation, previously in beta, generally available.
The proliferation of passwords and SSH keys in modern IT has brought with it a tricky management problem, not only for people but also for machine-to-machine communications. Developers may struggle to keep secrets such as database logins secure, when their code will not function without them.
In 2019 researchers at North Carolina State University scanned code publicly committed to GitHub and found [PDF] that 'not only is secret leakage pervasive — affecting over 100,000 repositories — but that thousands of new, unique secrets are leaked every day.' In June 2020, security researcher Craig Hays deliberately leaked server credentials in a GitHub repository and observed an unauthorised login just 34 minutes later.
Secrets Automation uses a Connect Server, delivered as a Docker container, which users deploy in their environment. This provides a REST API which applications and services call to get the credentials they need.
These requests are authenticated with an access token, unique to each application or service. 1Password provides API client libraries for Go, Node.js and Python, and there are plugins for tools including Terraform, Kubernetes, Hashicorp Vault, and Ansible.
There is also an upcoming integration with GitHub; VP of partner engineering Dana Lawson said that 'with the upcoming GitHub and 1Password Secrets Automation integration, teams will be able to fully automate all of their infrastructure secrets,' but no further details are available yet. GitHub also has its own Secrets API as part of its Actions DevOps service.
Developers and admins still have the task of managing the access tokens themselves, though these can be stored in 1Password. We presume that embedding them in code is a bad idea, even though the 1Password sample code for Node looks like this:
1Password's chief product officer, Akshay Bhargava, acknowledged that Secrets Automation does not fix this part of the problem, telling us that 'we've purposely designed Secrets Automation to allow customers using tokens to narrow the scope of access to the secrets needed by each part of their infrastructure. It does mean that the token now has that access, so deploying it as a protected secret in your infrastructure is important.
'This could be a Kubernetes secret, an environment variable, or a managed secret in the various cloud platform stacks, etc. This isn't about delivering secure credentials for the connect server to the application. But instead it is about delivering infrastructure secrets through 1Password to the applications securely. More things will be ported over, but we are sunsetting the SecretHub product.'
The price of the new service is based on the number of tokens issued and the number of vaults they access. A free tier offers three credits per month, then pricing starts at $29 per month for 25 credits.
Secrets Automation was developed by 1Password; what will happen to the existing SecretHub product following the acquisition? 'There are going to be some key features from SecretHub that will make their way into the Secrets Automation product,' Bhargava told The Register.
Former SecretHub CEO Marc Mackenbach, who is now joining 1Password, said that existing users 'can continue to use SecretHub as you currently do until January 1st, 2022.' ®