A group of diverse gophers courtesy of gopherize.me

I have been using Go since early 2017 and have only grown to love the language and the community more and more over time. There is one thing I have always found frustrating about Go, however, which is the very manual process of installing a new version.

In general, the Go community is very good about updating their Go version as new releases come out. This leads to an issue, however, when the version of Go supported by ubiquitous package managers like brew or apt are not always up to date with the bleeding edge version. Sometimes I’ve seen it take weeks before a new version of Go is released on brew and have had to move to manual installation of a new version.

I decided to change that.

If you want to jump straight to installing and using GVM click here.

Table Of Contents

Starting on my Environment Automation Journey

In March 2020, I decided that I was tired of constantly rebuilding my development environments manually from memory. I started to create a list / blueprint of different software elements and configurations that I used regularly so that I could easily rebuild my environment on new systems or re-imaged assets. The goal was to create consistent and repeatable environments across my development workflows and systems.

The more I thought about building a “list” of programs the more I realized that this was a perfect use case for building out a script to do the work for me. My first step was to build a bash script to reproduce my current development environment. Since the inception of my env script, I have added support for MacOS, Linux, and Windows (WSL) on both arm64 and amd64 architectures.

Since the first version, I have been using it to keep my development systems in sync and application versions up to date across multiple operating systems and architectures with great success.

Back To Top

Lessons Learned

I quickly realized how much more effective I was at ramping up in new environments when I could reliably re-create my environment from scratch with a single command. Each time I added a new program, or configuration, I added it to the script and ran the script.

I went from what would usually take several days of work re-building my entire environment to a few minutes and a couple commands.

The more I built my environment script the more proficient I became with bash. I started added more complex logic and began creating helper scripts for different repetitive tasks within my developer workflow (like running long test commands or executing a persistent filtered ping to check if my internet was down).

Go Generics

At Gophercon 2021 the Go team made the official announcement that they would be introducing Generics into the language as part of Go 1.18 in February 2022. Generics are a new feature in Go that allows you to define a generic type and use it in a function, or in a struct by passing a type as a parameter. This is in some ways very similar to the concept of generics in other languages like Java or C#.

I wanted to be able to start building out libraries with the new generics feature and started going through the process of installing Go from source since a beta release had not yet been made available. I did not know until later about gotip1, but I’ll address why that does not matter for GVM in the Installing alternate Go versions section.

I became frustrated that there was not a simple way to install Go from source or manage different versions of Go which I had been doing through my environment script for the past 18 months. I decided it was time to create a tool to manage the different versions of Go I use. I took a weekend and started scripting out a proof of concept for a Go version manager basing the tool roughly on the Node Version Manager (NVM).

Back To Top

Building GVM (Go Version Manager)

What is a Version Manager?

A version manager is a piece of software that allows you to install and use different software versions quickly without having to go through the manual process of installation and configuration. Great examples of this are the Node Version Manager (NVM), and the Ruby Version Manager (RVM).

Here’s an example of NVM in action:

1# install or use (if already installed) node version 17
2nvm use 17
3
4# install or use (if already installed) the
5# latest long term support version of node
6nvm use --lts

NVM’s first time use experience is fantastic. It’s a great way to get started with a new version of node without having to manually install it yourself. It’s quick, to the point, and just works. I wanted to emulate the same experience with Go.

Why build a Version Manager for Go? What are you solving for?

The first question I got asked after I shared GVM was:

Why would you build this? Doesn’t the Go team does this for you. The Go team provides different methods for installing Go versions alongside an existing installation of Go including the current development version (known as gotip).

XKCD Automation
comic Firstly, it is my firm belief that open source development allows me to build whatever I want, whether it already exists or not. Secondly, the method of installing Go, whether through the use of the recommended version install process2 or through the use of gotip has a couple of requirements and side effects that I wasn’t happy with.

Here’s an example of installing a Go version other than the one already installed:

1# Installing Go 1.10.7
2go install golang.org/dl/go1.10.7@latest
3go1.10.7 download
4
5# Installing the development version of Go
6go install golang.org/dl/gotip@latest
7gotip download

It is very important to note here that an EXISTING installation of Go is required for either of these to work. Why is this necessary? Shouldn’t I be able to install a second, or third, version of Go without having to install Go first? What if I want to start with a completely clean slate?

Installing alternate Go versions

These are the issues I found with these methods of installing alternate Go versions.

  1. Both methods require a pre-existing installation of Go.
  2. Neither method allows direct use of the go command but rather a version specific command such as go1.10.7.
  3. There is no simple way of updating your entire environment to the alternate Go version.
  4. The process of installing vanilla Go remains the same.
  5. First time use experience is not great.
  6. Unless package managers like brew or apt are updating to new versions of Go quickly you are required to install manually anyways.
  7. Manual installation as recommended by the Go team requires administrative privileges (/usr/local/ is a system directory).

So what’s the solution?

Build a tool to do it all for you.

Goals of GVM

  1. No requirement for an existing installation of Go.
  2. No need to install Go manually.
  3. go command should be set to the active GVM version of Go.
  4. No dependency on package managers.
  5. Installation should not require administrative privileges.
  6. Non destructive (i.e. no side effects or removal of existing installations).
  7. Easy to use.
  8. GVM can automatically update itself.

Writing GVM

Taking a weekend and building out a tool to manage the different versions of Go was a fun project for me personally. I wanted to play with generics and the idea of going through a continual process of pulling latest from the Go repository manually and re-installing it, as new commits were pushed, seemed really cumbersome.

I built out a script to manage pulling of the latest development version of from the Go git repository and the execution of the commands to install Go from source. It was a simple enough script given the work I had done with my environment script and I was happy with it.

Once I finished installing Go from source I was annoyed that it did not overwrite the existing version of Go in my $PATH so that I could “seamlessly” use the new version.

I wanted it to just work.

Down the Rabbit Hole

A photo of Morpheus from the Matrix talking about the rabbit
hole This is when I took drastic steps. I spent some time thinking about the install process for Go and how I could improve and automate the majority of manual steps. I realized that on Unix based systems it was not necessary to install Go into a system directory like /usr/local/go, but that anywhere in the $PATH would suffice.

User Profile Installs

I decided that the tool should install Go into the user’s home directory ($HOME) rather than a system directory to avoid unnecessary administrator privileges. This is actually an important requirement because in organizations with heavy regulatory burden, or stringent security requirements, engineering teams do NOT have administrative rights on their machines. But if a user did have administrator privledges and wished to map their existing usr/local/go installation to the new $HOME/.gvm/go installation, they could.

To accomplish this I setup the script to create a .gvm directory in the user’s home directory. This hidden directory is where the tool installs each Go version and manages a symbolic link to the active version as specified by the user.

This satisfied my “Installation should not require administrative privileges.” requirement, and comes with some unique benefits.

Go maintains an set of environment configurations. These environment variables are configured for each installed version. So if a user needs different environment variables for one version of Go than another, they can easily configure them and when they swap versions with a gvm command, like gvm 1.17.5, the environment variable changes remain with the other Go version. Obviously, a user may with to share these variables across the different versions, but as of now this is not supported.

Pointing to the Active Version

Installing to the .gvm directory was successful, but it didn’t solve the issue of pointing to the active version of Go in the $PATH. I decided it was not a good idea to constantly update the $PATH and instead created a symbolic link inside of the .gvm directory which would point to the active version of Go instead. This symlink is easy to update and can be re-pointed without requiring constant PATH updates or reloading the shell.

So we have a .gvm directory with a symlink to the active version of Go.

No existing installation of Go

This requirement was pretty easy to solve for since GVM is built on bash there is no inherent dependency on Go itself. I just stole my existing installation code from my environment script and retrofitted it into the GVM script.

All I had to do was install Go into the .gvm/<version> directory rather than /usr/local, ensure that the $PATH was pointing to the new symlink, and that the symlink was pointing at the correct folder for the desired Go version.

1# Example of the `.gvm` directory on my local system
2➜  ~ ls -al ~/.gvm
3total 0
4drwxr-xr-x Dec 16 13:45 1.16.4
5drwxr-xr-x Dec 16 13:44 1.17.5
6drwxr-xr-x Dec 16 13:55 1.18beta1
7lrwxr-xr-x Dec 25 14:43 go -> /Users/benji/.gvm/next/go
8drwxr-xr-x Dec 16 13:45 next

In the example above, you can see that I have four versions of Go installed (1.16.4, 1.17.5, 1.18beta1, and the development branch which I’m calling next) and my currently active version is the next folder which is the current development version built directly from source on my machine. The go symlink points to the next folder (go -> /Users/benji/.gvm/next/go), and the below example shows the $PATH after the installation of gvm.

1➜  ~ echo $PATH
2/Users/benji/.gvm/go/bin:/usr/bin:/bin:...

The .gvm directory is added to the beginning of the $PATH so that it can properly override any existing versions of Go that may be installed without an destructive action.

Back To Top

GVM Prerequisites

There are currently a few prerequisites to install GVM, but I think that the installation process is pretty straightforward. In the future, I intend for the process to be more automated and more similar to the installation process for NVM which gives a command for either wget or curl based on your systems pre-existing configuration.

These prerequisites are:

NOTE: git is only required if you desire to install Go from source using the gvm next command.

If you meet these requirements click here to skip directly to installation.

Setting up your $HOME/bin directory

To determine if you have an existing $HOME/bin directory, you can use the following command:

1  ls $HOME | grep bin

If you do not get a result then create it by running the following command:

1  mkdir $HOME/bin

Adding $HOME/bin to the $PATH

  • Open your shell rc file in your text editor
    • Most likely you will be using $HOME/.bashrc or $HOME/.zshrc
  • Add export PATH=$PATH:$HOME/bin to the end of the file
  • Save the file, and reload your shell
    • Either source ~/.bashrc or source ~/.zshrc
    • Or just simply quit and restart your shell

Installing & Using GVM

Once you’ve met the prerequisites, install GVM with the following command:

1curl -L https://github.com/devnw/gvm/releases/download/latest/gvm \
2    > $HOME/bin/gvm && chmod +x $HOME/bin/gvm

This will install the gvm script into your user’s home directory bin folder and make it executable.

Once it’s installed it’s as easy as typing gvm <version> in your shell to get started.

1  gvm 1.17.5 # Installs Go 1.17.5

To use the latest development branch of go use the gvm next command.

Click here for the source and instructions on Github

Special notes about “gvm next”

Because gvm next is installing from source I decided not to re-compile Go every time gvm next is run. Instead if it already built it will only update the symlink. If you wish to get the latest development version and force a rebuild use the gvm next --update command instead and it will recompile Go.

Back To Top

Language support with gopls

NOTE: The version of Go which compiled gopls is what provides language server support for your editor.

If you want to use gvm next effectively you must execute the go install golang.org/x/tools/gopls@latest command after running gvm next to ensure it’s recompiled with updated support.

For example these are the tools I use in VSCode:

 1  gopkgs
 2  go-outline
 3  gotests
 4  gomodifytags
 5  impl
 6  goplay
 7  dlv
 8  dlv-dap
 9  golangci-lint
10  gopls

With VSCode however it’s easy to re-install all of these tools with either Ctrl + Shift + P on Windows or Linux or Cmd + Shift + P on MacOS. Then typing Go: Install/Update Tools, hit Enter and you’ll see a list of tools that are installed. Check the ones you want to install/update and hit Enter again.

In a future release I plan to add support for installing and updating tools automatically. Track the issue here.

Back To Top

Auto Updating

GVM uses the installed tag for gvm to determine if there is a new version of gvm available. If there is a new version of gvm available it will prompt you to update and overwrite the current version and re-run itself with the same flags.


BEST PRACTICE:
Use the latest release tag when installing gvm

How does GVM Auto Update?

The auto-update feature of gvm came about from my frustration with having to constantly re-run the install command while debugging for an issue filed by a community member. I realized that if I published a sha256 hash of the current version of gvm to the GitHub Releases latest Tag I could compare the released shasum hash to the current shasum hash and if they did not match then I know either a new version of gvm is available or the current version is broken.

All I have to do at that point is prompt the user to update, alerting them that a new update is available, and re-run the gvm command with the original flags.

Voila! GVM Auto Update!

Back To Top

Why is a version manager for Go important?

The introduction of generics is one of the single greatest changes to the Go language both in terms of syntax and semantics. This introduction is also one which creates a disparity between versions of Go. Previous releases of Go had API changes but they were generally backwards compatible, but with generics there will be a great number of new Go modules which require a minimum version of 1.18.

With this in mind it will be important for Go developers to be able to move between Go versions with ease as needed.

Back To Top

Future Improvements

The current version of GVM is not perfect or endstate by any means but it is working for me. I will be adding more features and fixing bugs the more I use it. If you wish to see what is in the works you can take a look at the current issues. My goal for GVM is that it becomes much more like NVM and has a simpler install story for the script itself.

If you’re interested in contributing to GVM, feel free to fork the repo and submit a pull request!