Go Embedding

Lately, I’ve been playing around with Go a bit. I remember playing around with it early on and enjoying the language and thinking it has some neat features. Since it’s been a while, I decided to take another look.

One of my favorite features of Go is that it compiles down to a single, static library. This makes it very easy to work with and deploy applications written in Go. It seems I managed to look back into Go at just the correct time as in Go 1.16 also added the ability to embed static resources directly into the executable . This method was described in a tutorial linked in the 1.16 release post . I thought this was neat enough to spend some time looking into and playing with. This post will also be looking at Go build tags , the io/fs package , and init functions .

In this post, I’m going to be looking at building a web server that uses files that are either from the disk or from the embedded resources based on a compile-time flag. Running it with the local files will allow you to edit those files and see the changes without restarting the server. This should hopefully make it easier to iterate over the files during development. When ready to release to production, the files can be run from the embedded executable so that there is still only a single file to deploy.

The complete code for this post can be found here .

Base Web Server

We’ll start with a simple server serving files from the content directory in our main.go file:

package main

import (
	"net/http"
	"os"
)

func main() {
	osDir := os.DirFS("content")
	httpDir := http.FS(osDir)
	fileServer := http.FileServer(httpDir)

	err := http.ListenAndServe(":8080", fileServer)

	if err != nil {
		panic(err)
	}
}

Nothing too crazy going on here. The slightly interesting portion is the difference between the osDir and the httpDir parameter. In order to abstract away the file system or the embedded resources, we use the fs.FS type from the io/fs package which was also added in go 1.16. The http.FileServer has existed for much longer and uses a http.FileSystem type from net/http package. To provide a compatibility mechanism between the two, the http.FS method takes an fs.FS and returns a http.FileSystem that can be used with the http server.

Build Tags and Embedding

We will need to separate our files into several other files because the build tags are done at the file level. This is going to result in three new files: fs.go, fs_embed.go and fs_noembed.go.

First, we’ll look at the fs.go file:

package main

import (
	"io/fs"
)

var files fs.FS

All this file will do is hold the value that will be populated by the other two files, depending on the build tags. This file isn’t strictly necessary as the variable could be put into the main.go file. I decided to put it into its own file to make it stand out and group it with the files with the build tags.

The next file is fs_noembed.go:

// +build !embed

package main

import (
	"os"
)

func init() {
	Files = os.DirFS("content")
}

This is the first file with a build tag, which is on the first line. If the embed build tag is not provided, it will build this file. The init function is run automatically when this package is first imported. All this does is assign the Files variable to the os.DirFS returned for the content.

The final file is fs_embed.go:

// +build embed

package main

import (
	"embed"
	"io/fs"
)

//go:embed content/*
var content embed.FS

func init() {
	Files, _ = fs.Sub(content, "content")
}

This file has a bit more going on. First, we have build tag that this will be used when we want to embed the files into the executable. After the imports, the embed directive and the variable that will hold the contents that are embedded. In this case, we are embedding the content directory and all files in it into the content variable. After that, we use an init function to store the embedded filesystem in the Files system. With the embedded filesystem, the files are stored as the full path, such as content/test.txt. In order to only make the content directory available as the root of the server, we use the fs.Sub method move into that sub-directory.

Lastly, we update the main.go file to use the Files variable by changing:

	osDir := os.DirFS("content")
	httpDir := http.FS(osDir)

to:

	httpDir := http.FS(Files)

That should be it.

Running the Application

Run the project with the command go run . to use the files from disk. You can change the contents of files in the content directory and the changes will be reflected without needing to change the executable.

Run the project with the command go run -tags embed . to use the files embedding during compilation. Any changes to the files in the content directory will not be reflected because, well, the files aren’t being read from there. They are being read from the executable! It’s worth noting that the -tags embed parameter must come before the . or they will be sent as parameters to the program, rather than the compiler.

You can also run go build -tags embed . to build an executable and run that from any directory and it will still serve the embedded files.

Conclusion

In the demo repository, it serves a single text file. That could be replaced as web resources such as css and js or go template files to keep all the assets in a single file for deployment.

Thanks for reading!