Building microservices with Golang and Go kit

Microservice architecture is a design pattern that structures applications as a collection of services. In a world powered by cloud computing, microservices have become one of, if not the most, popular architecture design pattern. Big companies like Netflix and Amazon have been using it for years and every day, more companies make an effort to migrate from big, monolithic architectures to microservices.  Some of the advantages of microservices are:

• Highly maintainable and testable
• Loosely coupled
• Independently deployable
• Organized around business capabilities
• Owned by a small group of people

Microservices are easier to maintain/test, scale more efficiently, use fewer resources, and are focused on specific processes that produce value to your company. 

 

Golang is a programming language created ten years ago and was designed by team members at Google. Its main features are a syntax similar to C, being strongly typed, and having a garbage collector. It also has excellent handling of concurrency and parallel programming which makes it work wonders with microservices. 

“As of today, applications use a number of external services: databases, caches, search and message queues. However, more and more specialists use microservices solutions due to its collection of separated components. While coding in Go, developers can use asynchronous Input, output (asynchronous I/O), so that an application can interact with any number of services without blocking web requests. “

Another relevant Golang trait is that the core of the language functionalities is kept small. This is not a language packed with pre-built functions you may be used to, like Python or Ruby. Instead, you are given a small set of basic capabilities that you can then use to create complex ones. So does this mean you will need to manually write and prepare all the specialized support for your microservice? Evidently, the answer is no.

 

Go kit is a programming toolkit for building microservices in Go. It was created to solve common problems in distributed systems and applications and allow developers to focus on the business part of programming. It’s basically a group of co-related packages that, together, form a framework for constructing large SOAs. Its main transport layer is RPC (Remote Procedure Call), but it can also use JSON over HTTP and it’s easy to integrate with the most common infrastructural components, to reduce deployment friction and promote interoperability with existing systems. 

Let’s create a minimal Go kit service starting with our business logic. Following the Go kit examples, we will create our own string service that allows us to manipulate strings in certain ways. We take advantage of Golang’s interface type to define our logic. This allows us to easily swap implementations according to our needs. 

//File: service.go

package mystr

import (
"errors"
"strings"
"github.com/go-kit/kit/log"
)

type Service interface {
IsPal(string) error
Reverse(string) string
}

type myStringService struct {
log log.Logger
}

func (svc *myStringService) IsPal(s string) error {
reverse := svc.Reverse(s)
if strings.ToLower(s) != reverse {
  return errors.New("Not palindrome")
}
return nil
}

func (svc *myStringService) Reverse(s string) string {
rns := []rune(s) // convert to rune
for i, j := 0, len(rns)-1; i ‹ j; i, j = i+1, j-1 {

  // swap the letters of the string,
  // like first with last and so on.
  rns[i], rns[j] = rns[j], rns[i]
}

// return the reversed string.
return strings.ToLower(string(rns))
}

 

 

Requests and Responses

In Go kit, the primary messaging pattern is RPC. This means that every method in our interface should be modeled as a client-server interaction. The requesting program is a client, and the service is the server. This allows us to specify the parameters and return types of each method. 

 

//requests.go

package mystr

type IsPalRequest struct {
Word string `json:"word"`
}

type ReverseRequest struct {
Word string `json:"word"`
}

 

responses.go

package mystr

type IsPalResponse struct {
Message string `json:"message"`
}

type ReverseResponse struct {
Word string `json:"reversed_word"`
}

 

 

Endpoints

This is where we introduce one of the main functionalities of Go kit: endpoints. These are abstractions provided by Go kit that work pretty much like an action or handler on a controller. It’s also the place to add safety and security logic. Each endpoint represents a single method in our service interface.

 

package mystr

import (
"context"
"errors"

"github.com/go-kit/kit/endpoint"
"github.com/go-kit/kit/log"
"github.com/go-kit/kit/log/level"
)

type Endpoints struct {
GetIsPalindrome endpoint.Endpoint
GetReverse      endpoint.Endpoint
}

func MakeEndpoints(svc Service, logger log.Logger, 
					middlewares []endpoint.Middleware) Endpoints {
return Endpoints{
  GetIsPalindrome: wrapEndpoint(makeGetIsPalindromeEndpoint(svc, logger), middlewares),
  GetReverse:      wrapEndpoint(makeGetReverseEndpoint(svc, logger), middlewares),
}
}

func makeGetIsPalindromeEndpoint(svc Service, logger log.Logger) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
  req, ok := request.(*IsPalRequest)
  if !ok {
    level.Error(logger).Log("message", "invalid request")
    return nil, errors.New("invalid request")
  }
  msg := svc.IsPal(ctx, req.Word)
  return &IsPalResponse{
    Message: msg,
  }, nil
}

}

func makeGetReverseEndpoint(svc Service, logger log.Logger) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
  req, ok := request.(*ReverseRequest)
  if !ok {
    level.Error(logger).Log("message", "invalid request")
    return nil, errors.New("invalid request")
  }
  reverseString := svc.IsPal(ctx, req.Word)
  return &ReverseResponse{
    Word: reverseString,
  }, nil
}
}

func wrapEndpoint(e endpoint.Endpoint, middlewares []endpoint.Middleware) endpoint.Endpoint {
for _, m := range middlewares {
  e = m(e)
}
return e
}

One of the structures provided by Go kit to ensure security is the middlewares. Middlewares work the same way they do in any other language. They are methods executed over the request before it reaches its handler. Here you can add functionality such as logging, load balancing, tracing, etc.

Logging

In the previous code snippet, we introduced two important Go kit components. The first one is, of course, the endpoint package. The other one is the logger. We’re all familiar with logs (that useful tool that lets us find out where that annoying bug is happening). Go kit comes with its own logger package, which allows us to write structured messages for easy consumption either from humans or other computers. They implement a key-value structure that allows us to log several pieces of information together without having to write multiple logs.



On a similar note, we also make use of the log level gokit package. This package provides an extra layer of information. The usual error, debug, info, etc. type of log we’re used to. 

 

Transport

The transport file is meant to represent the matching layer of the OSI model. It’s the file that contains every piece of code responsible for the end to end communication strategy. Go kit supports many transports out of the box. For simplicity, we will work with JSON over HTTP. 

The transport consists of a series of Server objects, one for each endpoint of our service, and it receives four parameters. These are:

• An endpoint: The request handler. Given the separation of structures and responsibilities, we can use the same structure across multiple transport implementations.
• A decode function: This function receives the external requests and translates it to Golang code.
• An encode function: The exact opposite of the decode function. This function translates the Golang response to the corresponding output for the selected transport.
• A set of server options: A set of options that could be credentials, codec, keep parameters alive, etc. These provide extra capabilities to our transport layer.

package mystr

import (
"context"
"encoding/json"
"errors"
"net/http"

"github.com/go-kit/kit/endpoint"
httptransport "github.com/go-kit/kit/transport/http"
)

func GetIsPalHandler(ep endpoint.Endpoint, options []httptransport.ServerOption) *httptransport.Server {
return httptransport.NewServer(
  ep,
  decodeGetIsPalRequest,
  encodeGetIsPalResponse,
  options...,
)
}

func decodeGetIsPalRequest(_ context.Context, r *http.Request) (interface{}, error) {
var req IsPalRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
  return nil, err
}
return req, nil
}

func encodeGetIsPalResponse(_ context.Context, w http.ResponseWriter, response interface{}) error {
resp, ok := response.(*IsPalResponse)
if !ok {
  return errors.New("error decoding")
}
return json.NewEncoder(w).Encode(resp)
}

func GetReverseHandler(ep endpoint.Endpoint, options []httptransport.ServerOption) *httptransport.Server {
return httptransport.NewServer(
  ep,
  decodeGetReverseRequest,
  encodeGetReverseResponse,
  options...,
)
}

func decodeGetReverseRequest(_ context.Context, r *http.Request) (interface{}, error) {
var req ReverseRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
  return nil, err
}
return req, nil
}

func encodeGetReverseResponse(_ context.Context, w http.ResponseWriter, response interface{}) error {
resp, ok := response.(*ReverseResponse)
if !ok {
  return errors.New("error decoding")
}

return json.NewEncoder(w).Encode(resp)
}

 

Putting it All Together

Let’s put our work to the test. We have all we need for our microservice to start working. Now we just need to start listening for requests.

 

package main

import (
"net/http"
"os"

"bitbucket.org/aveaguilar/stringsvc1/pkg/mystr"
"github.com/go-kit/kit/endpoint"
kitlog "github.com/go-kit/kit/log"
"github.com/go-kit/kit/log/level"
httptransport "github.com/go-kit/kit/transport/http"
"github.com/gorilla/mux"
)

func main() {
var logger kitlog.Logger
{
  logger = kitlog.NewLogfmtLogger(os.Stderr)
  logger = kitlog.With(logger, "ts", kitlog.DefaultTimestampUTC)
  logger = kitlog.With(logger, "caller", kitlog.DefaultCaller)
}

var middlewares []endpoint.Middleware
var options []httptransport.ServerOption
svc := mystr.NewService(logger)
eps := mystr.MakeEndpoints(svc, logger, middlewares)
r := mux.NewRouter()
r.Methods(http.MethodGet).Path("/palindrome").Handler(mystr.GetIsPalHandler(eps.GetIsPalindrome, options))
r.Methods(http.MethodGet).Path("/reverse").Handler(mystr.GetReverseHandler(eps.GetReverse, options))
level.Info(logger).Log("status", "listening", "port", "8080")
svr := http.Server{
  Addr:    "127.0.0.1:8080",
  Handler: r,
}
level.Error(logger).Log(svr.ListenAndServe())
}

Here, we’re just making use of the core Golang HTTP package in order to start an HTTP server listening to port 8080. We use the mux router package to have easier HTTP method handling. Any error will be captured by the Go kit logger we initialize. 

 

Remember, since we’re creating a very simple service, we don’t use any middleware or server options. But these are available to provide extra functionality and security to our application. 

So, we get our service going: 

$go run cmd/stringsvc1/main.go
level=info ts=2020-07-26T00:44:15.447990239Z caller=main.go:30 status=listening port=8080

 

And start making requests!

$curl -XGET '127.0.0.1:8080/palindrome' -d'{"word": "palindrome"}'
{"message":"Is not palindrome"}

 

$curl -X GET '127.0.0.1:8080/palindrome' -d'{"word": "1234554321"}'
{"message":"Is palindrome"}

 

$curl -X GET '127.0.0.1:8080/reverse' -d'{"word": "microservice"}'
{"reversed_word":"ecivresorcim"}

 

And that’s it! Although we skipped a few core packages, this is a good example of the bare minimum you will need to create your own service. With just a few files and lines of code, you will have a working application. Go kit makes it really easy.

If you’re interested in learning more, take a look at the Go kit website. It has more complex examples and good articles to expand your understanding of the kit. And of course, as always, GoDoc is an awesome resource to rely on when getting into the world of Golang.