Storing Go Modules

September 3, 2024

Let me write a short article about storing Go modules here. An article about how to create a Go module, can be found here, Go Module.

When we worked on Go in our local, sometimes we did these:

• We put libraries into your project using a go get command. Eg; go get github.com/aristorinjuang/fibonacci.
• We downloaded or updated the libraries for the project by using go mod tidy.
• We downloaded the libraries for the project and avoided the modification by using go mod download.

All these stuff will cache those Go modules to $GOPATH/pkg/mod/cache/download. You have no $GOPATH? Make sure your Go installation is correct, Install Go. Once you have them, your text editor's LSP (Language Server Protocol) will recognize them.

Let me compare this Go module mechanism with PHP vendors and NodeJS modules (node modules). In PHP, we install libraries using Composer, which stores them in the vendor/ folder. In NodeJS, we install libraries using npm (Node Package Manager), which stores them in the node_modules/ folder. See the difference? The Go centralizes the libraries by the system. PHP and NodeJS centralize the libraries per project. That's why we put the vendor/ folder into the .gitignore for the PHP and the node_modules/ for the NodeJS.

There is a case when we build the Go app on the CI/CD pipeline and the runners have a slow or no internet connection. Here are how to work around this.

go mod vendor

Let's try to run go mod vendor. It will put the required Go modules in the vendor/ folder. It is a similar folder to PHP's vendor/. The difference is, that we don't need to put the vendor/ folder in the .gitignore for Go.

Pros

• Simple
• Fast

Cons

• The Git will be a little messy and heavy because the libraries are tracked.

Go module cache

Let's say you don't prefer the go mod vendor, you can install Go to the runners and bring your $GOPATH/pkg/mod/cache/download.

Pros

• Fast, because it is already in the cache.
• No vendor/ in Git.

Cons

• Maintain the libraries' versions. The pipeline will fail if it has no required version and fails to download the libraries due to no internet connection.

Go module proxy

Let's say you are using Docker to build the Go app. It is always fresh for every pipeline job. Fresh without any Go library. You can set up your own Go module proxy that runners can reach. Maybe it is another computer in the intranet or the runner itself (which runs the Docker/job).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

package main

import (
    "log"
    "net/http"
    "os"
)

func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        http.ServeFile(w, r, os.Getenv("GOPATH") + "/pkg/mod/cache/download" + r.URL.Path)
    })
    log.Panic(http.ListenAndServe(":8080", nil))
}

Go GET http://localhost:8080/github.com/aristorinjuang/fibonacci/@v/list for example, to test whether it works or not. It will show the available versions if it works.

Then the script is GOPROXY=http://your-go-proxy-host go mod download or:

1. go env -w GOPROXY="http://your-go-proxy-host"
2. go mod download

Pros

• Same as the Go module cache.
• You have your own Go module proxy.

Cons

• Same as the Go module cache.
• Efforts to set up.

I love any feedback. Feel free to comment.

Related Articles

• Behavior-Driven Development with Go
• Install Go
• Concurrency in Go (Part 2)
• Deploy a Go App to AWS EC2
• A Simple Blog with Go

• Share
• Tweet

Comments