Storing Go Modules

Let me write a short article about storing Go modules here. An article about how to create a Go module, can be found here, Go Module.

When we worked on Go in our local, sometimes we did these:

All these stuff will cache those Go modules to $GOPATH/pkg/mod/cache/download. You have no $GOPATH? Make sure your Go installation is correct, Install Go. Once you have them, your text editor's LSP (Language Server Protocol) will recognize them.

Let me compare this Go module mechanism with PHP vendors and NodeJS modules (node modules). In PHP, we install libraries using Composer, which stores them in the vendor/ folder. In NodeJS, we install libraries using npm (Node Package Manager), which stores them in the node_modules/ folder. See the difference? The Go centralizes the libraries by the system. PHP and NodeJS centralize the libraries per project. That's why we put the vendor/ folder into the .gitignore for the PHP and the node_modules/ for the NodeJS.

There is a case when we build the Go app on the CI/CD pipeline and the runners have a slow or no internet connection. Here are how to work around this.

go mod vendor

Let's try to run go mod vendor. It will put the required Go modules in the vendor/ folder. It is a similar folder to PHP's vendor/. The difference is, that we don't need to put the vendor/ folder in the .gitignore for Go.

Pros

Cons

Go module cache

Let's say you don't prefer the go mod vendor, you can install Go to the runners and bring your $GOPATH/pkg/mod/cache/download.

Pros

Cons

Go module proxy

Let's say you are using Docker to build the Go app. It is always fresh for every pipeline job. Fresh without any Go library. You can set up your own Go module proxy that runners can reach. Maybe it is another computer in the intranet or the runner itself (which runs the Docker/job).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

package main

import (
    "log"
    "net/http"
    "os"
)

func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        http.ServeFile(w, r, os.Getenv("GOPATH") + "/pkg/mod/cache/download" + r.URL.Path)
    })
    log.Panic(http.ListenAndServe(":8080", nil))
}

Go GET http://localhost:8080/github.com/aristorinjuang/fibonacci/@v/list for example, to test whether it works or not. It will show the available versions if it works.

Then the script is GOPROXY=http://your-go-proxy-host go mod download or:

  1. go env -w GOPROXY="http://your-go-proxy-host"
  2. go mod download

Pros

Cons

I love any feedback. Feel free to comment.

Related Articles