Go best practices

Our platform is written in Go. Creating models requires writing basic Go code. This page provides some basic tips and tricks for using Go effectively in the context of Nextmv.

Go provides extensive docs which we have no intention of replicating. Here, we focus on common patterns and tricks for those who are new to Go and need it to work with Hop. It also includes advice on structuring model source.

If you are new to Go, we recommend that you check out "A Tour of Go" and "How to write Go code" on the Go website.

Why Go

Briefly, we chose Go because of performance, simplicity, and interoperability.

• Performance. Go performance is frequently in line with Java, and one or more orders of magnitude faster than Python (see various benchmarks). Go helps us build an efficient core, and lets you improve model performance once that matters.
• Simplicity. Go has structures and methods, but no inheritance or polymorphism. Its elegant use of interfaces makes decoupling software components convenient. We find we can write performant Go code faster than in other languages.
• Interoperability. Go simplifies model deployment and interoperability with software services. Hop models are usually atomic binary artifacts that read and write JSON data. That is made possible partly by our use of Go.

Coding Patterns

In the sections that follow, we provide a few pointers on using Go effectively. Each example is runnable as a standalone program.

Initializing Variables

Variables are declared with var, or declared and initialized with :=. Use := most of the time. Use var if the initial value of a variable is not important because it will be overwritten soon.

package main

import "fmt"

func main() {
    var x int // equivalent to: x := 0
    y := 42.0 // equivalent to: var y float64 = 42.0

    fmt.Println(x, y)
}

Error Handling

Many Go functions return multiple values, with the last one being either an error or a boolean indicating some condition. You should capture these values and act on them immediately.

Errors indicate something is wrong, but not unrecoverable. Create an error from a string using errors.New. Insert variable values into an error string with fmt.Errorf.

package main

import (
    "fmt"
    "math"
    "os"
)

func unimaginarySqrt(x float64) (float64, error) {
    if x < 0 {
        return 0, fmt.Errorf("%v < 0", x)
    }
    return math.Sqrt(x), nil
}

func main() {
    root, err := unimaginarySqrt(-10)
    if err != nil {
        fmt.Println("error:", err)
        os.Exit(1)
    }
    fmt.Println("root:", root)
}

Boolean return values indicate an expected condition. For instance, if a map key is not present, that is a condition instead of an error.

package main

import "fmt"

func main() {
    m := map[string]int{}
    if _, ok := m["foo"]; !ok {
        m["foo"] = 10
    }
    if _, ok := m["foo"]; !ok {
        m["foo"] = 20
    }
    fmt.Println(m)
}

Function Receivers

Any user-defined type in Go can have methods. The (r record) before the method name is a "receiver." Note that fmt.Println calls record.String for us.

package main

import "fmt"

type record struct {
    number int
    name   string
}

func (r record) String() string {
    return fmt.Sprintf("number: %v, name: %q", r.number, r.name)
}

func main() {
    r := record{number: 3, name: "Ender"}
    fmt.Println(r)
}

Go passes variables by value. This means that the value of a receiver inside a method call is a copy of the original variable. Structures can mutate themselves using pointer receivers. Go handles the details of pointer manipulation for us.

package main

import "fmt"

type record struct {
    number int
    name   string
    score  float64
}

func (r record) copyAndWrite(score float64) record {
    r.score = score
    return r
}

func (r *record) mutate(score float64) {
    r.score = score
}

func main() {
    r1 := record{number: 3, name: "Ender"}
    r2 := r1.copyAndWrite(100)
    fmt.Println(r1, r2)

    r1.mutate(99)
    fmt.Println(r1, r2)
}

Modifying Slices & Maps

Slices and maps are pointer values. Changing them modifies the original variable. If you want to copy and modify a slice, there are two ways to do that. The first one uses make to pre-allocate memory for the new slice. You can also do this in the second function using make([]int, 0, len(x)+1).

package main

import "fmt"

func copyAndAppend1(x []int, y int) []int {
    z := make([]int, len(x)+1)
    copy(z, x)
    z[len(x)] = y
    return z
}

func copyAndAppend2(x []int, y int) []int {
    z := []int{}
    for _, v := range x {
        z = append(z, v)
    }
    z = append(z, y)
    return z
}

func main() {
    x1 := []int{1, 2, 3}
    x2 := copyAndAppend1(x1, 4)
    x3 := copyAndAppend2(x2, 5)
    fmt.Println(x1, x2, x3)
}

A similar pattern applies to maps.

package main

import "fmt"

func copyAndSet(x map[string]int, key string, val int) map[string]int {
    z := make(map[string]int, len(x)+1)
    for k, v := range x {
        z[k] = v
    }
    z[key] = val
    return z
}

func main() {
    x1 := map[string]int{"foo": 7, "bar": 11}
    x2 := copyAndSet(x1, "baz", 13)
    fmt.Println(x1, x2)
}

Note that Go intentionally randomizes the order of map keys when using range. This prevents subtle errors caused by relying on such an order, but can be disorienting the first time you see it. If you want your Hop models to execute deterministically, don't use a map to keep track of unmade decisions.

Working with JSON

Go provides convenient facilities for reading and writing JSON to and from structures. Exported fields on structures are automatically marshaled. The json annotation changes the name of the field in the output so it's "number" instead of "Number".

The JSON package only accesses the exported fields of struct types (those that begin with an uppercase letter). Therefore only the exported fields of a struct will be present in the JSON output. Visit this page for more information.

package main

import (
    "encoding/json"
    "fmt"
)

type record struct {
    Number int `json:"number"`
    name   string
}

func main() {
    r := record{Number: 3, name: "Ender"}
    b, err := json.Marshal(r)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(b))
}

To provide custom marshaling, add a MarshalJSON method.

package main

import (
    "encoding/json"
    "fmt"
    "strings"
)

type record struct {
    number int
    name   string
}

func (r record) MarshalJSON() ([]byte, error) {
    m := map[string]interface{}{
        "number": r.number,
        "name":   strings.ToUpper(r.name),
    }
    return json.Marshal(m)
}

func main() {
    r := record{number: 3, name: "Ender"}
    b, err := json.Marshal(r)
    if err != nil {
        panic(err)
    }
    fmt.Println(string(b))
}

Working with Time Durations

Go provides syntax for working with time durations. For example, 90 seconds is 1m30s, 20 seconds is 20s, and 100 milliseconds is 100ms.

Organizing Model Source

Modules & Packages

Modules are a recent addition to Go, and are sometimes confused with packages. A package is a directory containing .go files. A module is a collection of one or more packages with a go.mod file specifying the module name, the required version of Go, and any dependencies.

For example, the root directory of Hop is a module which contains three packages. These then contain various sub-packages.

hop/          module:  github.com/nextmv-io/code/hop
|-- model/    package: github.com/nextmv-io/code/hop/model
|-- run/      package: github.com/nextmv-io/code/hop/run
|-- solve/    package: github.com/nextmv-io/code/hop/solve

If a module has dependencies, Go stores their checksums in a go.sum file. You should commit both go.mod and go.sum to your source control repository so your builds are repeatable.

To create a new module in a directory, run go mod init. Running go get will scan your module's packages for dependencies and add these to your go.mod and go.sum, defaulting to current releases.

To upgrade (or downgrade) to a different version of Hop or any other dependency, use go get.

go get github.com/nextmv-io/code/hop@v0.8.0

This will update your go.mod and go.sum files with the new version. Optionally, you can run go mod tidy to remove unused versions of any dependencies. Be sure to commit your go.mod and go.sum files to save the updated versions.

Project Structure

Go code is easy to organize once you understand packages and modules. Simple models or simulations may not need anything more than a main package.

github.com/you/simple-model/
|-- go.mod
|-- go.sum
|-- main.go

This is easy to build and deploy. Run go build in the root of the project to get a binary named simple-model.

As projects become more complex, it is advantageous to structure the source. The example below has multiple binaries that share modeling code and types: one for the CLI and one for AWS Lambda. Main packages live under cmd/ by convention.

github.com/you/complex-model/
|-- cmd/
|   |-- cli/
|   |   |-- main.go
|   |-- lambda/
|   |   |-- main.go
|-- data/
|   |-- input1.json
|   |-- input2.json
|-- model.go
|-- go.mod
|-- go.sum

Use the -o flag to go build the two main packages into binaries with names that are not cli and lambda, respectively.

cd complex-model/cli/
go build -o complex-model-cli
cd ../lambda
go build -o complex-model-lambda

Building and Testing projects

To build a Go project, use go build and to test a Go project, use go test.

For example, if you are using the source distribution, you can build from the code/hop/, code/dash/, and code/engines directories with the following go command.

go build ./...

And to run tests, the following command can be invoked in test/hop (to run tests for Hop, for example).

go test ./...

Note, API testing code for hop and engines is separated into its own modules in order to reduce dependencies, keep build times short, and keep binaries small for production.