How I Learned Go

Introduction and Motivation

Earlier this year I joined Zemanta, which as a Native Demand Side Platform operates in the real time bidding business. We bid on auctions for native ad slots when users are consuming content. When we win an auction, our client’s ad is shown to a user in a specific slot on a web page. You can imagine we process a lot of impressions per second. Zemanta chose Go to build the whole infrastructure around real time bidding, instead of high speed enterprise languages such as Java or C++/C#. Go performs well compared to Java in terms of performance, plus its GC pause is shorter on average. Technical specifications were not the main point why Zemanta went with Go, but rather because of its fast and simple development process.

Go is a relatively new programming language. As many new tech inventions, Go was created as an experiment. The goal of its creators was to come up with a language that would resolve bad practices of others while keeping the good things. It was first released in March 2012. Since then Go has attracted many developers from all fields and disciplines.

This short introduction of Go should serve as motivation for further reading. The rest of this blog post will focus on what were my steps in learning Go and some interesting patterns Go uses to efficiently solve specific problems.

Getting Started

For me learning Go started by checking Go’s official webpage. There I found installation files, tutorials, documentation, etc. I had no clue where to start, so I asked a couple of experienced Go coders. All suggested me to start with A Tour of Go. This interactive Go tutorial is great to get familiar with Go’s core ideas and syntax. I didn’t even have to install anything on my computer. All exercises can be done in an interactive web “interpreter”. I finished this tutorial in a couple of days and with that I got an idea of how things like variable initialization, pointers, structs, lists, etc. are done in Go.

I really couldn’t say I understood Go very well at that point. Still, together with my experience from other programming languages, it gave me enough knowledge to do my first task at Zemanta. I had to refactor a Hadoop MapReduce job. MapReduce is made of two functions (mapper and reducer), each accepting lines on stdin and printing lines to stdout. You can pass a mapper and a reducer as binaries to Hadoop infrastructure and it will run a MapReduce job. Go is perfect for this since it compiles into a single binary, plus you can compile it for any platform from any platform. I am working on a Mac and Hadoop usually runs on Linux, so I just had to type

GOOS=linux go build

in my terminal and upload the files to our Hadoop cluster. Cross platform build? With all the dependencies and infrastructure to run the program in one binary file? “I could get used to that”, was my first thought after checking other Go tools that come out of the box when you install Go.

No More Exceptions

By this point, I got familiar with the syntax, as well as another important part of writing big IT systems: error handling. Go’s simple approach to error handling looked too good to be true at first. In Go, functions usually return a value and an error (functions can return multiple values), so handling errors is as simple as checking the second return value. This means errors get passed around until they are finally logged, printed or cause another action. This gives a good way to chain errors from one place to another. Sounds nice, but sometimes it just doesn’t look clean if 80% of your code checks for errors. On the other hand, it forces developers to handle errors, which can avoid some critical runtime errors.

func mapper(w *io.Writer, r io.Reader) {
	in := bufio.NewReader(r)

	for {
		line, err := in.ReadBytes('\n')

		if err == io.EOF {
			break
		} else if err != nil {
			log.Error(err)
		}
		if len(line) == 0 {
			continue
		}

		// do something with the line

		err = w.Write(line)
		if err != nil {
			log.Error(err)
		}
	}
}

After checking all of the official resources, I started searching for other posts about Go programming patterns. I came across Go by Example which is really similar to A Tour of Go, but with a bit more line by line explanations and is organized really well. Every programming language has many ways of expressing something, similarly there are many tutorials explaining the same thing in a different way. Go by Example helped me a lot with real life examples and how to solve them. Another similar resource I came across at the same time, which is actually part of the official documentation, was Effective Go. Both helped me make my code look more “Goish”.

Implicit is Great

Go by Example explained some, at that time, strange patterns, one of them being the use of interfaces I was seeing everywhere. Why was everyone so excited about them? For example, why do we need a reader interface with only read method signature?

type Reader interface {
    Read(p []byte) (n int, err error)
}

A waste of code in my opinion. Our struct that wants to read something will obviously implement read method. Then it finally clicked.

Let’s see why this is so neat on a practical (maybe a bit long) example.

func printContent(r io.Reader) error {
	contentBytes, err := ioutil.ReadAll(r)
	if err != nil {
		return err
	}
	fmt.Println(string(contentBytes))
	return nil
}

func main() {
	f1, err := os.Open("./test")
	if err != nil {
		// do error handling
	}
	defer f1.Close()

	response, err := http.Get("http://www.zemanta.com/")
	if err != nil {
		// do error handling
	}
	defer response.Body.Close()

	f2, err := os.Open("./testZip")
	if err != nil {
		// do error handling
	}
	defer f2.Close()

	z, err := gzip.NewReader(f2)
	if err != nil {
		// do error handling
	}
	defer z.Close()

	printContent(f1)
	printContent(response.Body)
	printContent(z)
}

When you understand interfaces in Go, you understand interfaces in general. I don’t think interfaces are hard to understand, it is just that in Go they are used to do exactly what they were designed to do in any programming language - logically group code that has similar behavior, not to achieve less code, which is a consequence. In the above example, we can pass any struct that implements a ReadAll(r io.Reader) method to printContent(r io.Reader) instead of writing a handler for each specific case. We also don’t need to explicitly say those structs are implementing reader interface. This is great for us humans, but not for our IDE which needs to figure out what methods should a struct implement.

Two-value Assignment

Another cool thing about interfaces is that every type in go implements at least zero methods, which means each type implements a special zero interface{}. This is useful when we do not know the exact type of a variable.

var data interface{}
if err := json.Unmarshal(raw_bytes, &data); err != nil {
	log.Error(err)
}

How do we get a type from this? With type assertion. Type assertion in Go is a simple statement. If assertion fails, then the assigned value will have the default value of the type. Another way to do it is to assign a second value in type assertion statement which will hold a boolean telling you if assertion was successful. If your value can be the same as default for this type, two-value assignment is the only way to properly do type assertion.

jsonStruct := data.(map[string]interface{}) // this can fail
num := jsonStruct["num"].(int)
str, ok := jsonStruct["str"].(string)
if !ok {
	log.Error("Couldn't convert to string")
}

The same applies for retrieving values from maps.

m := make(map[string]int)
m["Answer"] = 42
v, ok := m["Answer"]
if !ok {
	log.Info("Could not get value for key Answer")
}

The above example implies how we use sets in Go. Since there is no special set struct defined, we make use of maps.

idSet := make(map[string]struct{})
idSet["id1"] = struct{}{}
idSet["id2"] = struct{}{}

if _, ok := idSet["id1"]; ok {
	log.Println("id1 is in idSet")
} 

if _, ok := idSet["id3"]; !ok {
	log.Println("id3 is not in idSet")
}

The Great Go Book

This brings me to the last resource I used on my way to Go fluency - The Little Go Book. It is written for readers who are already comfortable with Go syntax. The book gave me another perspective on how to write idiomatic Go. It was exactly what I needed to sink in the knowledge that I accumulated over the past few weeks. It reassured me I was on the right path. While reading this book I had a lot of “Ahaaaa” moments, usually around the topics described above, and after reading it, the real fun began.

Conclusion

The whole process of going through tutorials, examples, reading posts and books took me around 5 weeks. I had programming experience before I started learning Go and I already worked with pointers and interfaces in other programming languages. During those 5 weeks I figured out a lot of general programming concepts I already heard of but never quite understood and it’s all thanks to Go. I encourage the reader to learn Go not because it is really cool and hype right now, but because you will learn a lot about programming and software development in general. Just think about the fact that Go was built on 40 years of programming experience and best practices. Technology has changed a lot in those years and so should an interface we use to manipulate it.