Cover image

Go 1.16 – embed | io/fs | retract | Apple M1 support

Click Here to View the Presentation Slides

Table Of Contents

Build Improvements

Go Builds are up to 25% faster and use as much as 15% less memory.1

Architectures

  • Official Apple M1 Support with darwin/arm642
  • Other Ports
    • netbsd/arm64
    • openbsd/mips64

Retracting Module Versions

Module versions can now be marked `retracted` in the go.mod file3

Example go.mod

 1module atomizer.io/base
 2
 3go 1.16 
 4
 5require (
 6  atomizer.io/cmd v0.0.5
 7  atomizer.io/engine v1.0.4 // indirect 
 8) 
 9
10// Remote-triggered crash in package foo. See CVE-2021-01234.
11retract v1.0.5

This alerts users who use version v1.0.5 in this case with a warning message when checking for updates or upgrading a dependent package.

This may also include the comment in the go.mod file associated with the retract directive.4

1$ go list -m -u all
2example.com/lib v1.0.0 (retracted)
3$ go get .
4go: warning: example.com/[email protected]: retracted by module author:
5    Remote-triggered crash in package foo. See CVE-2021-01234.
6go: to switch to the latest unretracted version, run:
7    go get example.com/lib@latest

Example Retraction

1retract v1.0.0
2retract [v1.0.0, v1.9.9]
3retract (
4    v1.0.0
5    [v1.0.0, v1.9.9]
6)

Example with Comments

1retract (
2    v1.0.0 // Published accidentally.
3    v1.0.1 // Contains retractions only.
4)

Purpose

A severe security vulnerability has been identified.

A severe incompatibility or bug was discovered.

The version was published accidentally or prematurely.

Click Here to Visit the Playground Example

Go Tool Changes

Modules by Default

  • New Default: GO111MODULE=on3
  • This setting is now default with or without a go.mod file present
  • To Update: go env -w GO111MODULE=auto
  • go build and go test no longer modify go.mod / go.sum
1$ go build
2example.go:3:8: no required module provides package golang.org/x/net/html; to
3add it:
4    go get golang.org/x/net/html
5$ go get golang.org/x/net/html
6$ go build

Install & Get

  • go install now accepts versions3
    • go install golang.org/x/tools/[email protected]
    • Without the @<version> suffix go install functions as expected using go.mod version
  • go get -insecure deprecation for non-secure (HTTP) paths3
    • Use the new GOINSECURE environment variable instead
  • go get example.com/mod@patch now requires a version of the dependency to already be registered in the go.mod of the main module

GOVCS

New GOVCS environment variable alters which version control systems the go tool uses.3

This can be setup as module specific based on module path.

Example VSC Change

1GOVCS=github.com:git,evil.com:off,*:git|hg

This allows git for Github, only module proxy for evil.com and all other paths can use git or hg for pulling source.

Supported VCS (Version Control System) Types

  • git
  • hg
  • svn
  • bzr
  • fossil

Public / Private Modules

Keywords private and public can be used to setup specific settings3

1GOVCS=public:git|hg,private:all

Private is defined as those modules which are listed in the GOPRIVATE environment variable

The all Pattern

If a module declares itself as go version 1.16 or higher there is a new command option known as the all pattern.5

Examples:

1go get all

Will update the go.mod with transitive (//indirect) module dependencies

1go test all

Will execute tests for the current module as well as all dependencies (and transitive dependencies?)

1go build all

Build app and all dependencies

io/ioutil Deprecation

The io/ioutil package has been deprecated in version 1.16 and the functions in the package have been mapped internally to the io or os packages where applicable. The io/ioutil package will remain in the stdlib but point to the following packages.6

OriginalReplacement
ioutil.Discardio.Discard
ioutil.NopCloserio.NopCloser
ioutil.ReadAllio.ReadAll
ioutil.ReadDiros.ReadDir
ioutil.ReadFileos.ReadFile
ioutil.TempDiros.MkdirTemp
ioutil.TempFileos.CreateTemp
ioutil.WriteFileos.WriteFile

ReadDir includes an API change which “returns a slice of os.DirEntry rather than a slice of fs.FileInfo“6

Notes about Deprecation in Go

Deprecation in Go does not mean the loss of functionality normally expected as commented on by Russ Cox.

This goes along with similar conversations regarding the plugin package which is only deprecated because there is not a resource on the go team which is dedicated to maintaining it.

New Packages in stdlib

embed – Embedding Files

Possibly one of the most exciting new features in 1.16 is the ability to embed files into the compiled binary of your go application.7

Addition of a new compiler directive for embedding files

1//go:embed

This directive can be used several different ways.

NOTE: The application MUST import the embed package even if embed.FS is not used directly.

Example 1

 1import "embed"
 2
 3//go:embed image/* template/*
 4//go:embed html/index.html
 5//go:embed hello.txt
 6var f embed.FS
 7
 8data, _ := f.ReadFile("hello.txt")
 9
10print(string(data))

This example shows the use of the embed packages filesystem struct which allows for embedding of multiple files then accessing those files by filename using ReadFile.

Example 2

1import _ "embed"
2
3//go:embed hello.txt
4var s string
5
6print(s)

This example shows the use of embed by setting a compiler directive above a string variable which embeds the hello.txt file into that string for direct reference by print.

Supported Types

  • string
  • []byte
  • embed.FS

These embed directives: “must immediately precede a line containing the declaration of a single variable. Only blank lines and // line comments are permitted between the directive and the declaration.”7

Files beginning in . or _ when using the embed directive

If a directory (such as image) is embedded using the following directive:7

1// content is our static web server content.
2//go:embed image template html/index.html
3var content embed.FS

It will NOT include files beginning with . or _

If however the directive uses a wildcard like this:7

1// content holds our static web server content.
2//go:embed image/* template/*
3//go:embed html/index.html
4var content embed.FS

io/fs Package

  • Defines a new fs.FS interface for Read-Only trees of files8
  • Implemented by the embed package embed.FS
  • New http.FS function converts the fs.FS interface to an http.FileSystem for serving content
  • html/template and text/template have new ParseFS functions for serving templates provided by the fs.FS interface

runtime/metrics

Provides a stable interface for accessing metrics exported by the Go runtime.9

  • The interface accepts key values as strings
  • Supported metrics keys are returned by metrics.All()
  • Similar API to the reflect package

Example use of metrics package

1// Name of the metric we want to read.
2const myMetric = "/memory/classes/heap/free:bytes"
3
4// Create a sample slice for the metric.
5sample := []metrics.Sample{metrics.Sample{Name:myMetric}}
6
7// Sample the metric.
8metrics.Read(sample)

Docs Example

https://golang.org/pkg/runtime/metrics/#example_Read_readingOneMetric

Coming Soon

Go Generics Announcement

Expected: Go 1.18 release in early 202210

New Proposal to make Fuzzing in Go a First Class Citizen

https://go.googlesource.com/proposal/+/master/design/draft-fuzzing.md

  1. https://blog.golang.org/go1.16 ↩︎

  2. https://golang.org/doc/go1.16#ports ↩︎

  3. https://blog.golang.org/go116-module-changes ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎

  4. https://golang.org/ref/mod#go-mod-file-retract ↩︎

  5. https://golang.org/doc/go1.16#go-command ↩︎

  6. https://golang.org/doc/go1.16#ioutil ↩︎ ↩︎

  7. https://golang.org/pkg/embed/#FS ↩︎ ↩︎ ↩︎ ↩︎

  8. https://golang.org/doc/go1.16#fs ↩︎

  9. https://golang.org/pkg/runtime/metrics/ ↩︎

  10. https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md ↩︎