I have been using Go since early 2017 and have only grown to love the language and the community more and more over time. There is one thing I have always found frustrating about Go, however, which is the very manual process of installing a new version.
In general, the Go community is very good about updating their Go version as new
releases come out. This leads to an issue, however, when the version of Go
supported by ubiquitous package managers like brew or apt are not always up
to date with the bleeding edge version. Sometimes I’ve seen it take weeks before
a new version of Go is released on brew and have had to move to manual
installation of a new version.
I decided to change that.
If you want to jump straight to installing and using GVM click here.
Table Of Contents
Starting on my Environment Automation Journey
In March 2020, I decided that I was tired of constantly rebuilding my development environments manually from memory. I started to create a list / blueprint of different software elements and configurations that I used regularly so that I could easily rebuild my environment on new systems or re-imaged assets. The goal was to create consistent and repeatable environments across my development workflows and systems.
The more I thought about building a “list” of programs the more I realized that
this was a perfect use case for building out a script to do the work for me. My
first step was to build a bash script to reproduce my current development
environment. Since the inception of my
env script, I have added support for MacOS, Linux, and Windows (WSL) on both
arm64 and amd64 architectures.
Since the first version, I have been using it to keep my development systems in sync and application versions up to date across multiple operating systems and architectures with great success.
Lessons Learned
I quickly realized how much more effective I was at ramping up in new environments when I could reliably re-create my environment from scratch with a single command. Each time I added a new program, or configuration, I added it to the script and ran the script.
I went from what would usually take several days of work re-building my entire environment to a few minutes and a couple commands.
The more I built my environment script the more proficient I became with bash.
I started added more complex logic and began creating helper scripts for
different repetitive tasks within my developer workflow (like running long test
commands or executing a persistent filtered ping to check if my internet was
down).
Go Generics
At Gophercon 2021 the Go team made the official announcement that they would be introducing Generics into the language as part of Go 1.18 in February 2022. Generics are a new feature in Go that allows you to define a generic type and use it in a function, or in a struct by passing a type as a parameter. This is in some ways very similar to the concept of generics in other languages like Java or C#.
I wanted to be able to start building out libraries with the new generics
feature and started going through the process of installing Go from source since
a beta release had not yet been made available. I did not know until later about
gotip1, but I’ll address why that does not matter for
GVM in the Installing alternate Go
versions section.
I became frustrated that there was not a simple way to install Go from source or manage different versions of Go which I had been doing through my environment script for the past 18 months. I decided it was time to create a tool to manage the different versions of Go I use. I took a weekend and started scripting out a proof of concept for a Go version manager basing the tool roughly on the Node Version Manager (NVM).
Building GVM (Go Version Manager)
What is a Version Manager?
A version manager is a piece of software that allows you to install and use different software versions quickly without having to go through the manual process of installation and configuration. Great examples of this are the Node Version Manager (NVM), and the Ruby Version Manager (RVM).
Here’s an example of NVM in action:
1# install or use (if already installed) node version 17
2nvm use 17
3
4# install or use (if already installed) the
5# latest long term support version of node
6nvm use --lts
NVM’s first time use experience is fantastic. It’s a
great way to get started with a new version of node without having to manually
install it yourself. It’s quick, to the point, and just works. I wanted to
emulate the same experience with Go.
Why build a Version Manager for Go? What are you solving for?
The first question I got asked after I shared GVM was:
Why would you build this? Doesn’t the Go team does this for you. The Go team provides different methods for installing Go versions alongside an existing installation of Go including the current development version (known as
gotip).
Firstly, it is my firm belief that open source development allows me to build
whatever I want, whether it already exists or not. Secondly, the method of
installing Go, whether through the use of the recommended version install
process2 or through the use of gotip has a couple of requirements and side
effects that I wasn’t happy with.
Here’s an example of installing a Go version other than the one already installed:
1# Installing Go 1.10.7
2go install golang.org/dl/go1.10.7@latest
3go1.10.7 download
4
5# Installing the development version of Go
6go install golang.org/dl/gotip@latest
7gotip download
It is very important to note here that an EXISTING installation of Go is required for either of these to work. Why is this necessary? Shouldn’t I be able to install a second, or third, version of Go without having to install Go first? What if I want to start with a completely clean slate?
Installing alternate Go versions
These are the issues I found with these methods of installing alternate Go versions.
- Both methods require a pre-existing installation of Go.
- Neither method allows direct use of the
gocommand but rather a version specific command such asgo1.10.7. - There is no simple way of updating your entire environment to the alternate Go version.
- The process of installing vanilla Go remains the same.
- First time use experience is not great.
- Unless package managers like
breworaptare updating to new versions of Go quickly you are required to install manually anyways. - Manual installation as recommended by the Go team requires administrative
privileges (
/usr/local/is a system directory).
So what’s the solution?
Build a tool to do it all for you.
Goals of GVM
- No requirement for an existing installation of Go.
- No need to install Go manually.
gocommand should be set to the active GVM version of Go.- No dependency on package managers.
- Installation should not require administrative privileges.
- Non destructive (i.e. no side effects or removal of existing installations).
- Easy to use.
- GVM can automatically update itself.
Writing GVM
Taking a weekend and building out a tool to manage the different versions of Go was a fun project for me personally. I wanted to play with generics and the idea of going through a continual process of pulling latest from the Go repository manually and re-installing it, as new commits were pushed, seemed really cumbersome.
I built out a script to manage pulling of the latest development version of from the Go git repository and the execution of the commands to install Go from source. It was a simple enough script given the work I had done with my environment script and I was happy with it.
Once I finished installing Go from source I was annoyed that it did not
overwrite the existing version of Go in my $PATH so that I could “seamlessly”
use the new version.
I wanted it to just work.
Down the Rabbit Hole
This is when I took drastic steps. I
spent some time thinking about the install process for Go and how I could
improve and automate the majority of manual steps. I realized that on Unix based
systems it was not necessary to install Go into a system directory like
/usr/local/go, but that anywhere in the $PATH would suffice.
User Profile Installs
I decided that the tool should install Go into the user’s home directory
($HOME) rather than a system directory to avoid unnecessary administrator
privileges. This is actually an important requirement because in organizations
with heavy regulatory burden, or stringent security requirements, engineering
teams do NOT have administrative rights on their machines. But if a user did
have administrator privledges and wished to map their existing usr/local/go
installation to the new $HOME/.gvm/go installation, they could.
To accomplish this I setup the script to create a .gvm directory in the user’s
home directory. This hidden directory is where the tool installs each Go version
and manages a symbolic link to the active version as specified by the user.
This satisfied my “Installation should not require administrative privileges.” requirement, and comes with some unique benefits.
Go maintains an set of environment configurations. These environment variables
are configured for each installed version. So if a user needs different
environment variables for one version of Go than another, they can easily
configure them and when they swap versions with a gvm command, like gvm 1.17.5, the environment variable changes remain with the other Go version.
Obviously, a user may with to share these variables across the different
versions, but as of now this is not supported.
Pointing to the Active Version
Installing to the .gvm directory was successful, but it didn’t solve the issue
of pointing to the active version of Go in the $PATH. I decided it was not a
good idea to constantly update the $PATH and instead created a symbolic link
inside of the .gvm directory which would point to the active version of Go
instead. This symlink is easy to update and can be re-pointed without requiring
constant PATH updates or reloading the shell.
So we have a .gvm directory with a symlink to the active version of Go.
No existing installation of Go
This requirement was pretty easy to solve for since GVM is built on bash
there is no inherent dependency on Go itself. I just stole my existing
installation code from my environment
script and retrofitted it into the GVM
script.
All I had to do was install Go into the .gvm/<version> directory rather than
/usr/local, ensure that the $PATH was pointing to the new symlink, and that
the symlink was pointing at the correct folder for the desired Go version.
1# Example of the `.gvm` directory on my local system
2➜ ~ ls -al ~/.gvm
3total 0
4drwxr-xr-x Dec 16 13:45 1.16.4
5drwxr-xr-x Dec 16 13:44 1.17.5
6drwxr-xr-x Dec 16 13:55 1.18beta1
7lrwxr-xr-x Dec 25 14:43 go -> /Users/benji/.gvm/next/go
8drwxr-xr-x Dec 16 13:45 next
In the example above, you can see that I have four versions of Go installed
(1.16.4, 1.17.5, 1.18beta1, and the development branch which I’m calling next)
and my currently active version is the next folder which is the current
development version built directly from source on my machine. The go symlink
points to the next folder (go -> /Users/benji/.gvm/next/go), and the below
example shows the $PATH after the installation of gvm.
1➜ ~ echo $PATH
2/Users/benji/.gvm/go/bin:/usr/bin:/bin:...
The .gvm directory is added to the beginning of the $PATH so that it can
properly override any existing versions of Go that may be installed without an
destructive action.
GVM Prerequisites
There are currently a few prerequisites to install GVM, but I think that the
installation process is pretty straightforward. In the future, I intend for the
process to be more automated and more similar to the installation process for
NVM which gives a command
for either wget or curl based on your systems pre-existing configuration.
These prerequisites are:
- MacOS, Linux, or BSD
- amd64 or arm64 architecture
- An existing
$HOME/bindirectory (click here for instructions) $HOME/binis in the$PATH(click here for instructions)- POSIX compliant shell3
curl4git5
NOTE:
gitis only required if you desire to install Go from source using thegvm nextcommand.
If you meet these requirements click here to skip directly to installation.
Setting up your $HOME/bin directory
To determine if you have an existing $HOME/bin directory, you can use the
following command:
1 ls $HOME | grep bin
If you do not get a result then create it by running the following command:
1 mkdir $HOME/bin
Adding $HOME/bin to the $PATH
- Open your shell rc file in your text editor
- Most likely you will be using
$HOME/.bashrcor$HOME/.zshrc
- Most likely you will be using
- Add
export PATH=$PATH:$HOME/binto the end of the file - Save the file, and reload your shell
- Either
source ~/.bashrcorsource ~/.zshrc - Or just simply quit and restart your shell
- Either
Installing & Using GVM
Once you’ve met the prerequisites, install GVM with the following command:
1curl -L https://github.com/devnw/gvm/releases/download/latest/gvm \
2 > $HOME/bin/gvm && chmod +x $HOME/bin/gvm
This will install the gvm script into your user’s home directory bin folder
and make it executable.
Once it’s installed it’s as easy as typing gvm <version> in your shell to get
started.
1 gvm 1.17.5 # Installs Go 1.17.5
To use the latest development branch of go use the gvm next command.
Click here for the source and instructions on Github
Special notes about “gvm next”
Because gvm next is installing from source I decided not to re-compile Go
every time gvm next is run. Instead if it already built it will only update
the symlink. If you wish to get the latest development version and force a
rebuild use the gvm next --update command instead and it will recompile Go.
Language support with gopls
NOTE: The version of Go which compiled
goplsis what provides language server support for your editor.
If you want to use gvm next effectively you must execute the
go install golang.org/x/tools/gopls@latest command after running gvm next
to ensure it’s recompiled with updated support.
For example these are the tools I use in VSCode:
1 gopkgs
2 go-outline
3 gotests
4 gomodifytags
5 impl
6 goplay
7 dlv
8 dlv-dap
9 golangci-lint
10 gopls
With VSCode however it’s easy to re-install all of these tools with either
Ctrl + Shift + P on Windows or Linux or Cmd + Shift + P on MacOS. Then
typing Go: Install/Update Tools, hit Enter and you’ll see a list of tools
that are installed. Check the ones you want to install/update and hit Enter
again.
In a future release I plan to add support for installing and updating tools automatically. Track the issue here.
Auto Updating
GVM uses the installed tag for gvm to determine if there is a new version of
gvm available. If there is a new version of gvm available it will prompt you
to update and overwrite the current version and re-run itself with the same
flags.
BEST PRACTICE:
Use thelatestrelease tag when installinggvm
How does GVM Auto Update?
The auto-update feature of gvm came about from my frustration with having to
constantly re-run the install command while debugging for an issue filed by a
community member. I realized that if I published a sha256 hash of the current
version of gvm to the GitHub Releases latest
Tag I could compare the
released shasum hash to the current shasum hash and if they did not match
then I know either a new version of gvm is available or the current version is
broken.
All I have to do at that point is prompt the user to update, alerting them that
a new update is available, and re-run the gvm command with the original flags.
Voila! GVM Auto Update!
Why is a version manager for Go important?
The introduction of generics is one of the single greatest changes to the Go language both in terms of syntax and semantics. This introduction is also one which creates a disparity between versions of Go. Previous releases of Go had API changes but they were generally backwards compatible, but with generics there will be a great number of new Go modules which require a minimum version of 1.18.
With this in mind it will be important for Go developers to be able to move between Go versions with ease as needed.
Future Improvements
The current version of GVM is not perfect or endstate by any means but it is working for me. I will be adding more features and fixing bugs the more I use it. If you wish to see what is in the works you can take a look at the current issues. My goal for GVM is that it becomes much more like NVM and has a simpler install story for the script itself.
If you’re interested in contributing to GVM, feel free to fork the repo and submit a pull request!