I have been working with Kubernetes in production for 2 years now. Over this time we have migrated 3 or 4 times clusters, changed techinques and tools of deployments, changed habits or pushed back to some of the hype and trends coming up every day on the kubernetes scene. I guess nothing new here, this is what almost everybody does.
My personal take on this ever changing scene is to keep things simple – as much as possible, especially while your underlying infrastructure is changing from version to version. Kubernetes 1.4 (the first version I used in production) has the same basic constructs as version 1.7 and 1.10 currently in production, but many other other things come and go or change. When you are adapting or trying to adopt kubernetes on your org you need to rely on these simple and basic constructs that will make your transition from one version to the other painless, through with time you master more advanced and `cryptic` features. Anyway the point is, keep it simple.
On the early adoption days, we followed what felt natural based on our previous experiences. We were (still are) creating services spreading them around accomodating for the different business needs and features. Kubernetes is our golden cage and creating new services / apps never felt so easy. Going back to our previous experiences, while creating a new app/ service you feel the urge to `expose` it somehow, to your private network, to some namespace, to some context. Using the service type ` LoadBalancer` while in AWS spinning ELB’s felt like total magic! At first you have a couple of services, talking to each other, exposing some public API and it was good, you were provisioning your ELB’s, you could related to the old concept a DNS entry for a specific service. Old school I know but it felt good.
Time goes by the 2 services are 15 now, they talk to each other, they integrate. Creating some many ELB’s or Services becomes a messy and error prone business! Even within the kubernetes namespace context – dealing with some many different service entries is challenging – especially when you follow point to point integration. It was obvious that you need some kind of proxy. A step further
- You might need a public Inbound Proxy (Ingress)
- You might need internal proxies for your `service` mesh! (or mess haha)
Personally I always thought that the proposed Ingress API / Solution from kubernetes despite the `good` intentions – was too much, overcomplicated for my over simplifying mind in its ever ending battle to keep things simple. Also, I never felt good with the idea to have 1 everything under the same ingress hose. So despite having examples of things like Traefik or
Nginx as the reference Ingress controllers, personally I thought that I was increasing my platform’s configuration entropy and complexity. Maybe a bit spoiled by all the nice things abstracted by Kubernetes after all this time, adding some serious config for doing something like simple never made it to my book. So yes to Ingresses and Proxies but most probably multiple and independent Ingresses within the same cluster and not one.
On the other end internal proxies within the kubernetes network – was always something OK to do, but kubernetes with its DNS service was always easing the pain- at the end of the day everything is just a label in a `service.yml` definition. Internal service names change or move easily.
One night I end up reading this article by R.Li (CEO of Datawire) and I was like `yes!! I totally agree with you`. This is how I decided to give Ambassador a go. What I really liked was the fact that
Ambassador which is a kubernetes wrapper around Envoy proxy (a bit oversimplified definition apologies), takes away all the config that you would have to do using the Kubernetes Ingress resources, glues almost instantly on your existing deployment – being aware of your existing services and then leaving you only the task to configure its proxying rules and proxying behavior.
So in 1 place, you get to spin a horizontally scaled L7 proxy, no news here, the same way you scale your pods you would scale Ambassador, and either you can feed it your proxying config or you can enhance your existing service definitions with metadata that Ambassaor will detect (since it talks and understands the Kubernetes API). Currently I prefer (the first) – adding directly to the Ambassador deployment all the configs instead of adding labels and metadata to our existing services. In away our Services and the pods behind our services do not have a hint that are being proxied and this gives us great freedom to play around and change configs in 1 place without risking errors or mistakes to existing pieces of the puzzle!
Configuring Ambassador was really easy
- Pick your RBAC or NOT Helm deployment
- Provide on the same Helm Chart the service definitions that will make Ambassador visible to the users + its routing proxying table (see below)
Service definition
---
apiVersion: v1
kind: Service
metadata:
name: {{.Chart.Name}}-{{.Values.env}}
annotations:
service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
getambassador.io/config: |
---
apiVersion: ambassador/v0
kind: Mapping
name: user-service # name of the mapping
prefix: /user-service/ # this is the url prefix on the main proxy url e.g my.proxy.aws/user-service/api
service: user-service-preprod # this is the internal kubernetes service
---
apiVersion: ambassador/v0
kind: Mapping
name: logging service
prefix: /logging/
service: logging-service-prod
timeout_ms: 10000 #you can define origin time outs (default is 3sec)
spec:
type: LoadBalancer
ports:
- name: http
port: 8080
targetPort: 8080
- name: admin
port: 6666
targetPort: 6666
selector:
service: {{.Chart.Name}}
Overall
- I like that I don’t have to deal with the Ingress resources of Kubernetes especially when it comes to internal use scenarios and shadowing internal services.
- Installing Ambassador is dead simple – especially the Helm sub charts can be easily adapted or templated for your own case.
- All you have to do is decide if
- You will make the proxying mapping in 1 single file (like I did on the example)
- Or you will enhance your existing service definitions (kubernete services) with the same config that we added above, so that Ambassador can `discover` them through the Kubernetes API
- Once you are up and running and you start doing some basic proxying things get more interesting
- Canary deployments
- Weighted deployments controlling – the traffic weight for each service
- Re-play production or pre-production traffic to new services (shadowing etc)
If you are looking for a really easy L7 proxy, kubernetes ready then invite Ambassador to your cluster you will be amazed with him!
| Published on System Code Geeks with permission by Paris Apostolopoulos, partner at our SCG program. See the original article here: its always good to have one Ambassador in your Kubernetes cluster @getambassadorio
Opinions expressed by System Code Geeks contributors are their own. |
