Kubernetes sidecar pattern: nginx ssl proxy for nodejs

I learn about sidecar pattern from Kubernetes documentation and later from blog post by Brendan Burns The distributed system toolkit. Sidecar is very useful pattern and work nice with Kubernetes.
In the tutorial I want to demonstrate how “legacy” application can be extend with https support by using  sidecar pattern based on Kubernetes.

Problem

We have legacy application which doesn’t have HTTPS support. We also don’t want to send plain text traffic over network. We don’t want to make any changes to legacy application, but good thing that it is containerised.

Solution

We will use sidecar pattern to add HTTPS support to “legacy” application.

Overview

Main application
For our example main application I will use Nodejs Hello World service (beh01der/web-service-dockerized-example)
Sidecar container 
To add https support I will use Nginx ssl proxy (ployst/nginx-ssl-proxy) container

Deployment

TLS/SSL keys
First we need to generate TLS certificate keys and add them to Kubernetes secrets. For that I am using script from nginx ssl proxy repository which combine all steps in one:
git clone https://github.com/ployst/docker-nginx-ssl-proxy.git
cd docker-nginx-ssl-proxy
./setup-certs.sh /path/to/certs/folder

Adding TLS files to Kubernetes secrets

cd /path/to/certs/folder
kubectl create secret generic ssl-key-secret --from-file=proxykey=proxykey --from-file=proxycert=proxycert --from-file=dhparam=dhparam

Kubernetes sidecar deployment

In following configuration I have defined main application container “nodejs-hello” and nginx container “nginx”. Both containers run in the same pod and share pod resources, so in that way implementing sidecar pattern. One thing you want to modify is hostname, I am using not existing hostname appname.example.com for this example.
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name: nodejs-hello
  labels:
    app: nodejs
    proxy: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nodejs-hello
  template:
    metadata:
      labels:
        app: nodejs-hello
    spec:
      containers:
      - name: nodejs-hello
        image: beh01der/web-service-dockerized-example
        ports:
        - containerPort: 3000
      - name: nginx
        image: ployst/nginx-ssl-proxy
        env:
        - name: SERVER_NAME
          value: "appname.example.com"
        - name: ENABLE_SSL
          value: "true"
        - name: TARGET_SERVICE
          value: "localhost:3000"
        volumeMounts:
          - name: ssl-keys
            readOnly: true
            mountPath: "/etc/secrets"          
        ports:
        - containerPort: 80
          containerPort: 443
      volumes:
      - name: ssl-keys
        secret:
          secretName: ssl-key-secret

Save this file to deployment.yaml and create deployment Kubernetes object:

kubectl create -f deployment.yaml

Wait for pods to be Read:

kubectl get pods

NAME                            READY     STATUS    RESTARTS   AGE
nodejs-hello-686bbff8d7-42mcn   2/2       Running   0          1m

Testing

For testing I setup two port forwarding rules. First is for application port and second for nginx HTTPS port:

kubectl -n test port-forward <pod> 8043:443
#and in new terminal window run
kubectl -n test port-forward <pod> 8030:3000

First lets validate that application respond on http and doesn’t respond on https requests

#using http
curl -k -H "Host: appname.example.com" http://127.0.0.1:8030/ 
Hello World! 
I am undefined!

#now using https
curl -k -H "Host: appname.example.com" https://127.0.0.1:8030/ 
curl: (35) Server aborted the SSL handshake

Note: SSL handshake issue is expected as our “legacy” application doesn’t support https and even if it would it must serve https connection on different port than http. The test goal was to demonstrate the response.

Time to test connection through sidecar nginx ssl proxy

curl -k  -H "Host: appname.example.com" https://127.0.0.1:8043/
Hello World!
I am undefined!

Great! We have got expected output through https connection.

Conclusions

  • Nginx extended nodejs app with https support with zero changes to any of containers
  • Sidecar pattern modular structure provide great re-use of containers, so teams can be focused on application development
  • Ownership of containers can be split between teams as there is no dependency between containers
  • Scaling might not be very efficient, because sidecar container have to scale with main container

calculating md5 sum using go lang

Hi guys,
this is day 18 out of 100 days of code.

To get md5 sum of a file you crypto/md5 library and io/ioutil for reading files.

package main

import (
	"crypto/md5"
	"fmt"
	"io/ioutil"
	"log"
	"os"
)

func main() {
	if len(os.Args) <= 1 {
		log.Fatal("expected filename as parameter")
	}
	filename := os.Args[1]

	data, err := ioutil.ReadFile(filename)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("MD5 (%s) = %x\n", filename, md5.Sum(data))
}

Thanks!

Slow algorithm to insert strings into indexed file in alphabetical order

Hi guys,
this is day 17 out of 100 days of code in go lang.

Today I have played with files, sorting, bytes and strings. The goal was to create a kind of indexed file where all lines are arranged alphabetically.
What I got is very slow algorithm which took 15 seconds to insert 10K strings in an empty file and 49 seconds to insert another 10K strings to the same file. So, it slowdown very quickly as size grow.

func writeLineToFileSorted(newdata string) {
	indexData, err := ioutil.ReadFile("db.txt")
	if err != nil {
		log.Fatal(err)
	}
	var newIndexData []string
	for _, line := range bytes.Split(indexData, []byte("\n")) {
		//fmt.Printf("%s\n", line)
		newIndexData = append(newIndexData, string(line))
	}
	newIndexData = append(newIndexData, string(newdata))
	sort.Strings(newIndexData)

	bytesData := []byte(strings.Join(newIndexData, "\n"))
	ioutil.WriteFile("db.txt", bytesData, 0644)
}

I have to convert between strings and bytes which was kind of annoying, but maybe I am doing it wrong? Please let me know in comments.

If you want to load test it here is how:

func main() {
	rand.Seed(time.Now().UnixNano())
	var newdata string
	for i := 0; i < 10000; i++ {
		newdata = randStringRunes(10)
		writeLineToFileSorted(newdata)
		fmt.Println(i)
	}

}

var letterRunes = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")

func randStringRunes(n int) string {
	b := make([]rune, n)
	for i := range b {
		b[i] = letterRunes[rand.Intn(len(letterRunes))]
	}
	return string(b)
}

Source code at Github

Happy coding!

Random image file download with go

Hi guys,
this is day 16 out of 100 days of code.

With help of http and ioutil packages file download and storage is quite easy task.

package main

import (
	"io/ioutil"
	"log"
	"net/http"
)

func main() {
	url := "https://picsum.photos/200/300/?random"

	resp, err := http.Get(url)
	if err != nil {
		log.Fatal(err)
	}
	defer resp.Body.Close()

	image, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}
	err = ioutil.WriteFile("./myimage.gif", image, 0644)
	if err != nil {
		log.Fatal(err)
	}
}

In the example I am using Lorem Picsum service to get random image every run.

Happy coding!

Github issues reader in go lang

Hi guys,
this is day 15 out of 100 days of go land coding.

I continue github subject and today it is github issue reader. Example demonstrate how API results which come in json format convert into struct type.

package main

import (
	"encoding/json"
	"flag"
	"fmt"
	"log"
	"net/http"
)

func main() {
	repo := flag.String("repo", "", "github owner/repo e.g. golang/go")
	id := flag.Int("id", -1, "issue id")
	flag.Parse()

	if *id == -1 || *repo == "" {
		log.Fatal("--repo and --id parameters must be provided")
	}
	issue, _ := read(*repo, *id)
	fmt.Print(issue.Title)
}

// IssueData - specify data fields for new github issue submission
type IssueData struct {
	Title string `json:"title"`
	Body  string `json:"body"`
}

func read(ownerRepo string, id int) (*IssueData, error) {
	apiURL := fmt.Sprintf("https://api.github.com/repos/%s/issues/%d", ownerRepo, id)
	resp, err := http.Get(apiURL)
	if err != nil {
		log.Fatal(err)
		return nil, err
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		log.Fatal(err)
		return nil, err
	}

	var result *IssueData
	if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
		resp.Body.Close()
		return nil, err
	}

	return result, nil
}

Happy coding!

Create github issue ticket with golang

Hi guys,
today is day 14 out of 100 days of golang coding.

This time I was playing with github issues api and made small script which create new issues in given repository. However to use the code you would need to obtain github personal token key.

The interesting part was to make http post request with custom headers.

package main

import (
	"bytes"
	"encoding/json"
	"flag"
	"fmt"
	"io/ioutil"
	"log"
	"net/http"
)

func main() {
	token := flag.String("token", "", "github auth token")
	repo := flag.String("repo", "", "github owner/repo e.g. golang/go")
	title := flag.String("title", "", "title for new issue")
	body := flag.String("body", "", "body for new issue")
	flag.Parse()

	if *token == "" || *title == "" || *repo == "" {
		log.Fatal("--token, --repo and --title parameters must be provided")
	}
	create(*repo, *title, *body, *token)
}

// NewIssue - specify data fields for new github issue submission
type NewIssue struct {
	Title string `json:"title"`
	Body  string `json:"body"`
}

func create(ownerRepo, title, body, token string) {
	apiURL := "https://api.github.com/repos/" + ownerRepo + "/issues"
	//title is the only required field
	issueData := NewIssue{Title: title, Body: body}
	//make it json
	jsonData, _ := json.Marshal(issueData)
	//creating client to set custom headers for Authorization
	client := &http.Client{}
	req, _ := http.NewRequest("POST", apiURL, bytes.NewReader(jsonData))
	req.Header.Set("Authorization", "token "+token)
	resp, err := client.Do(req)
	if err != nil {
		log.Fatal(err)
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusCreated {
		fmt.Printf("Response code is is %d\n", resp.StatusCode)
		body, _ := ioutil.ReadAll(resp.Body)
		//print body as it may contain hints in case of errors
		fmt.Println(string(body))
		log.Fatal(err)
	}
}

Example usage is as follow:

go build github_issue.go
./github_issue --token="5daf49b235c41d53ba6fsfasdfasdfasfsad" --repo="vorozhko/go-tutor" --title="my new issue 3" --body="test"

Source code at Github.

 

github issue tracker with categories by date in go

Hi guys,
today is day 13 out of 100 days of code in go!

This time it is example of fetching issues from github and categorizing by created date.
To run example you need to install github issue searcher package from The Go Progamming Language book

go get gopl.io/ch4/github

Full code

package main

import (
	"fmt"
	"log"
	"os"
	"time"

	"gopl.io/ch4/github"
)

func main() {
	result, err := github.SearchIssues(os.Args[1:])
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("%d issues:\n", result.TotalCount)
	var dateformat string
	for _, item := range result.Items {
		days := int(time.Since(item.CreatedAt).Hours()) / 24
		if days < 30 {
			dateformat = "less than a month old"
		} else if days < 365 {
			dateformat = "less than a year old"
		} else {
			dateformat = "more than a year old"
		}
		fmt.Printf("%s, #%-5d %9.9s %.55s\n", dateformat, item.Number, item.User.Login, item.Title)
	}
}

Happy coding!

Sort map by it’s keys in go

Hi guys,
this is day 12 out of 100 days of code. I didn’t post for a while, because I did a small break for hackaton, but now I am back on track.

When you print a map using range function it will access elements in randomized order. So, if you want to print map in sorted order you have to sort keys first. See how:

package main

import (
	"fmt"
	"sort"
)

func main() {
	m := map[string]int{"A": 1, "B": 2, "AA": 1, "C": 3, "BBB": 2}

	//reverse keys and values of m map
	var mapKeys []string
	fmt.Print("Unsorted:\n")
	for k, v := range m {
		fmt.Printf("%s -> %d\n", k, v)
		mapKeys = append(mapKeys, k)
	}

	sort.Strings(mapKeys)
	fmt.Print("Sorted:\n")
	for _, v := range mapKeys {
		fmt.Printf("%s -> %d\n", v, m[v])
	}
}

Solution is to use additional array for keys and sort it in wished order.

Happy coding!

how to sort map values in go

Hi guys,
this is day 11 out of 100 days of go coding.

This time it is very short demo of go maps and simple sort by map values. Note that the example below will not work correctly if values has duplicates.

package main

import (
	"fmt"
	"sort"
)

func main() {
        //init a map with strings keys and int values
	var m = make(map[string]int)
	m["Sun"] = 7
	m["Sat"] = 6
	m["Fri"] = 5
	m["Thu"] = 4
	m["Mon"] = 1
	m["Tue"] = 2
	m["Wed"] = 3

	//reverse keys and values of m map
	var days = make(map[int]string)
	//array of indexes for sorting
	var daykeys = make([]int, len(m))

	fmt.Print("Unsorted days of week\n")
	counter := 0
	for k, v := range m {
		fmt.Printf("%s -> %d\n", k, v)
		days[v] = k
		daykeys[counter] = v
		counter++
	}
	//sort indexes array here
	sort.Ints(daykeys)

	fmt.Print("Sorted days of week\n")
	for _, v := range daykeys {
		fmt.Printf("%s\n", days[v])
	}
}

I believe this code has a lot of room for optimization, so if you know how please write me in comments or ping me in twitter with your version.

Happy coding everyone!