How to Manage Remote Instances with Vagrant & Terraform

Improve collaboration, composability, and security by managing your instances with infrastructure code

This guide will use Terraform, Vagrant, and AWS to manage a Javascript application. Terraform and Vagrant are tools we can use to implement infrastructure as code (IaC) and config as code (CaC). IaC and CaC allow us to track our changes in git and automates error-prone steps. This has several positive knock-on effects, such as improved collaboration, composability, security, and more. Vagrant also automates many typical dev chores making working with remote machines less complicated.

We will be running our instance on a remote computer with AWS. Moving our instance onto a cloud provider like AWS allows us to offload resource-intensive VMs from our local computer and provides several other benefits such as:

• Persistent Stable Compute
• Access to VPC
• Accessible Everywhere
• Custom / Specialized Compute (GPU)

Setup

First, let’s clone this guide's git repository

$ git clone https://github.com/theFong/vagrant-guide.git

Next, let’s install the required tools and software

$ ./brev/setup.sh # tested on Ubuntu 20.04

This installs the following:

• AWS CLI
• Terraform
• Vagrant
• Vagrant AWS Provider

AWS Authentication

This guide requires an AWS account. Configure keys and set the default region to the one you want to use.

$ aws configure # set keys and default region

Create the Resources and Vagrantfile

Run the following command to generate a Vagrantfile. This command initializes and applies our terraform.

$ make generate # name used is "my-example"

This does the following:

• Initializes terraform
• Creates a public/private key pair
• Create sa security group to allow SSH connections
• Find the right Ubuntu 20.04 AMI
• Generate the proper Vagrantfile given parameters

Use Vagrant

Create Instance

Your Vagrantfile may look something the following:

require 'vagrant-aws'
  Vagrant.configure('2') do |config|
    config.vm.box = 'dummy'
    config.vm.provision "shell", path: ".brev/setup.sh"
    config.vm.provision "file", source: "~/.gitconfig", destination: ".gitconfig"
    config.vm.provision "file", source: "~/.ssh/id_rsa", destination: "/tmp/id_rsa"
    config.vm.provision "shell", inline: "mv /tmp/id_rsa .ssh/id_rsa && rm -rf /tmp/id_rsa"
    config.vm.provider 'aws' do |aws, override|
     aws.keypair_name = 'my-example-ssh-key'
     aws.instance_type = 't2.micro'
     aws.ami = 'ami-040a251ee9d7d1a9b'
     aws.aws_profile = 'default'
     aws.security_groups = ['my-example']
     override.ssh.username = 'ubuntu'
     override.ssh.private_key_path = 'keys/my-example-ssh-key.pem'
   end
 end

Let’s create our instance! We will use Ubuntu 20.04.4 LTS with 1GB of RAM and 1 CPU. Resources and OS can be configured with [aws.ami] and [aws.instance_type] respectively.

$ vagrant up # creates and sets up the instance

Our Javascript instance is now ready to use. The project dependencies are defined using a Vagrant “provisioner” found in the Vagrantfile. The provisioner runs our setup script [.brev/setup.sh], which installs all the necessary tools.

SSH

Let’s enter our instance.

$ vagrant ssh # creates new shell
$ vagrant ssh-config --host vagrant-guide >> ~/.ssh/config # made in original shell
$ ssh vagrant-guide # creates new shell # requires above ssh-config command to be run

Shared / Synced Directory

The project folder (the folder with the Vagrantfile) is synced uni-directionally from the original computer to the remote instance at path /vagrant. With the Vagrant AWS plugin, making changes to the files on the original computer can be propagated manually and automatically with rsync.

$ # local -> remote
$ vagrant rsync # syncs one time
$ vagrant rsync-auto # blocks -> watch for file changes and syncs

Developing the Application

Now we can finally develop our Javascript application!

$ vagrant shell # new shell
$ cd /vagrant
$ make setup-app run-app

In a new shell, let's verify our Node.js app is running

$ vagrant shell # new shell
$ curl localhost:3000 # Hello Node!

Now, let’s make a modification. We can either edit the code within the instance or modify the code on the original computer and sync it over. Let’s modify [app/index.js:6] on the original computer.

const server = http.createServer((req, res) => {
     res.statusCode = 200;
-    const msg = 'Hello Node!\\n'
+    const msg = 'Hello World!\\n'
     res.end(msg);
});

Next, let’s sync over the change.

$ vagrant rsync # could also run rsync-auto
$ vagrant shell # new shell
$ make run-app
$ curl localhost:3000 # Hello World!

Making code edits on the original computer has the added benefit that you do not need to transfer over your git credentials to the remote computer to commit and push. Working entirely in a remote instance with your credentials can be achieved with file provisioners.

Handling Config and Credentials

We use a file provisioner to set up our git config and credentials on our remote machine. Find this in [Vagrantfile:5-7].

config.vm.provision "file", source: "~/.gitconfig", destination: ".gitconfig"
config.vm.provision "file", source: "~/.ssh/id_rsa", destination: "/tmp/id_rsa"
config.vm.provision "shell", inline: "mv /tmp/id_rsa .ssh/id_rsa && rm -rf /tmp/id_rsa"

You can now make modifications within the instance and push them to your repo. This is also useful for secrets like AWS credentials.

Cleanup/Reset

Sometimes you may want to clean up your instance.

$ vagrant destroy

Limitations

We can make several improvements with this instance configuration— the SSH server is exposed to the public internet, uni-directional file sync is limiting, and the Vagrantfile makes it tough to collaborate with multiple users.

The SSH server exposed to the public internet poses a security risk which can be mitigated by closing off the port and using either a jump box/bastion server or a VPN.

Uni-directional file sync makes moving files from the remote computer to your local computer tedious. To solve this, you can use a tool like Mutagen to enable bi-directional sync opening up new workflow possibilities.

The Vagrantfile has user-specific fields, such as the SSH key pair. These fields committed to a project version control system make collaborating difficult. To solve this, we can move the user-specific configurations to another Vagrantfile and merge them. Vagrant automatically combines the Vagrantfile found in ~/.vagrant.d; let’s move all the user-specific fields there. Check out the branch collaborate and run make generate to see how to implement this.

Unfortunately, the AWS Vagrant provider has been abandoned and is no longer actively maintained. The most glaring of issues have workarounds; however, some sharp edges make working with the Vagrant AWS provider painful. Notably, the provider only supports a subset of Vagrant features such as snapshot, network, or share features.