I’m not quite sure what you are getting at… Are you implying that I’m autistic because I only have 10 pods in a Kubernetes cluster?
Presently our clusters run roughly 1400 pods, and at this scale there certainly are benefits to using something like Kubernetes.
If your project is small enough to make sense on Heroku, then that’s awesome, but at some point Heroku stops making sense… both for managing at scale, and costs. Heroku already seems to be 2-4x as expensive as AWS on-demand. Presently we’re investigating moving out of AWS and into a datacenter, as it seems that we can reduce our costs by at least an order of magnitude.
No. I didn’t mean to attack you in particular in any way. I apologize if it came off as such. I just dislike the blind copying of what Google and Facebook do. Docker is another atrocity that everyone seems to feel obligated to use.
Heroku supports moderately large amounts of requests. It’s less expensive than having a proper sysops team in most cases.
With 1,400 it’s probably worth it to move away from AWS. Something like self-hosted Triton (descendant of Solaris) cluster would be far more elegant than kube and lxc.
Apology accepted, and thank you for not name calling.
And yeah, if you can save the ops team salaries by picking Heroku, then it certainly might offset the costs.
When you talk about Triton, do you mean this? Because funnily enough one of their bigger features seems to be that you can run Kubernetes on top of it. It looks pretty cool though, but I must say it was quite hard to find proper info on it.
Triton also seem to push for containerization quite heavily, and especially Docker… So when you talk about Triton are you suggesting to use the Infrastructure Containers or Virtual Machines instead?
Triton and SmartOS are still Sun Microsystems people. Solaris zones preceded Linux containers by almost a decade. They lost the popularity contest, so they have to provide compatibility with kube and docker. I think infrastructure containers are zones with extra whistles.
For even better performance, I’d go with Xen and Linux unikernel. The startup time can be just 1s with such minimum overhead. It also greatly reduces the attack surface. I highly doubt you’ll be allowed for such a drastic change though.
My team is constantly looking for new technologies to make sure we’re not turning ourselves into dinosaurs. We all know that Kubernetes won’t last forever, something better will come along some day.
That being said I don’t really see the full value of Triton or Xen with unikernels… They might have a bit less performance overhead if used correctly, but then again Kubernetes on bare metal also has very little overhead.
Kubernetes is certainly comes with a learning curve, and you need to know how to manage it, but once you have Kubernetes there’s a ton of nifty benefits that appear due to the thriving community.
Need to autoscale based on some kind of queue? Just install the Keda helm chart
Running in the cloud and want the cluster to autoscale the nodes? Just install cluster-autoscaler helm chart
Want to pick up all of your logs and ship them somewhere? Just install the promtail helm chart
Need a deployment tool? Just install the ArgoCD helm chart
Need your secrets injected from some secret management solution? Just install the external-secrets helm chart
Need to vulnerability scan all the images you are using in your cluster? Just install the trivy-operator helm chart
Need a full monitoring stack? Just install the kube-prometheus-stack helm chart
Need a logging solution? Just install the loki helm chart
Need certificates? Just install the cert-manager helm chart
The true benefit of Kubernetes isn’t Kubernetes itself, but all the it’s and pieces the community has made to add value to Kubernetes.
That’s what I meant. It’s a standard, and the sunk cost and network effect make it practical. Same as HTTP and SSL compared to SCTP and IPSec. Isn’t it sad when you see web devs preferring native apps even more than the general public?
Packaging is very practical, but that’s a boring dead end that unfortunately lasted 20+ years. I mentioned Triton in the context of Sun folks who have done containers better and earlier, yet failed to market because Sun went bankrupt. Your long list that was packaged is nothing that can’t be developed in-house better with far less complexity. Triton provides no benefit to you, but who knows maybe you’ll need dtrace one day.
Unikernels and Xen allow function as a service (I assume you don’t need that as well). FaaS is the future, as it’s a progression of the micro services trend. Startup time and hardening are not of interest to you as well, so no reason to switch.
I’m not completely against packaging. It’s great for open source desktop apps. When it’s out of sight away from the user, it turns into a boring game.
I’m not quite sure what you are getting at… Are you implying that I’m autistic because I only have 10 pods in a Kubernetes cluster?
Presently our clusters run roughly 1400 pods, and at this scale there certainly are benefits to using something like Kubernetes.
If your project is small enough to make sense on Heroku, then that’s awesome, but at some point Heroku stops making sense… both for managing at scale, and costs. Heroku already seems to be 2-4x as expensive as AWS on-demand. Presently we’re investigating moving out of AWS and into a datacenter, as it seems that we can reduce our costs by at least an order of magnitude.
No. I didn’t mean to attack you in particular in any way. I apologize if it came off as such. I just dislike the blind copying of what Google and Facebook do. Docker is another atrocity that everyone seems to feel obligated to use.
Heroku supports moderately large amounts of requests. It’s less expensive than having a proper sysops team in most cases.
With 1,400 it’s probably worth it to move away from AWS. Something like self-hosted Triton (descendant of Solaris) cluster would be far more elegant than kube and lxc.
Apology accepted, and thank you for not name calling.
And yeah, if you can save the ops team salaries by picking Heroku, then it certainly might offset the costs.
When you talk about Triton, do you mean this? Because funnily enough one of their bigger features seems to be that you can run Kubernetes on top of it. It looks pretty cool though, but I must say it was quite hard to find proper info on it.
Triton also seem to push for containerization quite heavily, and especially Docker… So when you talk about Triton are you suggesting to use the Infrastructure Containers or Virtual Machines instead?
Triton and SmartOS are still Sun Microsystems people. Solaris zones preceded Linux containers by almost a decade. They lost the popularity contest, so they have to provide compatibility with kube and docker. I think infrastructure containers are zones with extra whistles.
For even better performance, I’d go with Xen and Linux unikernel. The startup time can be just 1s with such minimum overhead. It also greatly reduces the attack surface. I highly doubt you’ll be allowed for such a drastic change though.
My team is constantly looking for new technologies to make sure we’re not turning ourselves into dinosaurs. We all know that Kubernetes won’t last forever, something better will come along some day.
That being said I don’t really see the full value of Triton or Xen with unikernels… They might have a bit less performance overhead if used correctly, but then again Kubernetes on bare metal also has very little overhead.
Kubernetes is certainly comes with a learning curve, and you need to know how to manage it, but once you have Kubernetes there’s a ton of nifty benefits that appear due to the thriving community.
Need to autoscale based on some kind of queue? Just install the Keda helm chart
Running in the cloud and want the cluster to autoscale the nodes? Just install cluster-autoscaler helm chart
Want to pick up all of your logs and ship them somewhere? Just install the promtail helm chart
Need a deployment tool? Just install the ArgoCD helm chart
Need your secrets injected from some secret management solution? Just install the external-secrets helm chart
Need to vulnerability scan all the images you are using in your cluster? Just install the trivy-operator helm chart
Need a full monitoring stack? Just install the kube-prometheus-stack helm chart
Need a logging solution? Just install the loki helm chart
Need certificates? Just install the cert-manager helm chart
The true benefit of Kubernetes isn’t Kubernetes itself, but all the it’s and pieces the community has made to add value to Kubernetes.
That’s what I meant. It’s a standard, and the sunk cost and network effect make it practical. Same as HTTP and SSL compared to SCTP and IPSec. Isn’t it sad when you see web devs preferring native apps even more than the general public?
Packaging is very practical, but that’s a boring dead end that unfortunately lasted 20+ years. I mentioned Triton in the context of Sun folks who have done containers better and earlier, yet failed to market because Sun went bankrupt. Your long list that was packaged is nothing that can’t be developed in-house better with far less complexity. Triton provides no benefit to you, but who knows maybe you’ll need dtrace one day.
Unikernels and Xen allow function as a service (I assume you don’t need that as well). FaaS is the future, as it’s a progression of the micro services trend. Startup time and hardening are not of interest to you as well, so no reason to switch.
I’m not completely against packaging. It’s great for open source desktop apps. When it’s out of sight away from the user, it turns into a boring game.