Introduction
Welcome to loxilb documentation
Background
loxilb started as a project to ease deployments of cloud-native/kubernetes workloads for the edge. When we deploy services in public clouds like AWS/GCP, the services becomes easily accessible or exported to the outside world. The public cloud providers, usually by default, associate load-balancer instances for incoming requests to these services to ensure everything is quite smooth.
However, for on-prem and edge deployments, there is no service type - external load balancer provider by default. For a long time, MetalLB was the only choice for the needy. But edge services are a different ball game altogether due to the fact that there are so many exotic protocols in play like GTP, SCTP, SRv6 etc and integrating everything into a seamlessly working solution has been quite difficult.
loxilb dev team was approached by many people who wanted to solve this problem. As a first step to solve the problem, it became apparent that networking stack provided by Linux kernel, although very solid, really lacked the development process agility to quickly provide support for a wide variety of permutations and combinations of protocols and stateful load-balancing on them. Our search led us to the awesome tech developed by the Linux community - eBPF. The flexibility to introduce new functionality into the OS Kernel as a safe sandbox program was a complete fit to our design philosophy. It also does not need any dedicated CPU cores which makes it perfect for designing energy-efficient edge architectures.
What is loxilb
loxilb is an open source cloud-native load-balancer based on GoLang/eBPF with the goal of achieving cross-compatibity across a wide range of on-prem, public-cloud or hybrid K8s environments. loxilb is being developed to support the adoption of cloud-native tech in telco, mobility, and edge computing.
Kubernetes with loxilb
Kubernetes defines many service constructs like cluster-ip, node-port, load-balancer, ingress etc for pod to pod, pod to service and outside-world to service communication.
All these services are provided by load-balancers/proxies operating at Layer4/Layer7. Since Kubernetes's is highly modular, these services can be provided by different software modules. For example, kube-proxy is used by default to provide cluster-ip and node-port services. For some services like LB and Ingress, no default is usually provided.
Service type load-balancer is usually provided by public cloud-provider(s) as a managed entity. But for on-prem and self-managed clusters, there are only a few good options available. Even for provider-managed K8s like EKS, there are many who would want to bring their own LB to clusters running anywhere. Additionally, Telco 5G and edge services introduce unique challenges due to the variety of exotic protocols involved, including GTP, SCTP, SRv6, SEPP and DTLS, making seamless integration particularly challenging. loxilb provides service type load-balancer as its main use-case. loxilb can be run in-cluster or ext-to-cluster as per user need.
loxilb works as a L4 load-balancer/service-proxy by default. Although L4 load-balancing provides great performance and functionality, an equally performant L7 load-balancer is also necessary in K8s for various use-cases. loxilb also supports L7 load-balancing in the form of Kubernetes Ingress implementation which is enhanced with eBPF sockmap helpers. This also benefit users who need L4 and L7 load-balancing under the same hood.
Additionally, loxilb also supports:
- [x] kube-proxy replacement with eBPF(full cluster-mesh implementation for Kubernetes)
- [x] Ingress Support
- [x] Kubernetes Gateway API
- [ ] Kubernetes Network Policies (in-progress)
Telco-Cloud with loxilb
For deploying telco-cloud with cloud-native functions, loxilb can be used as a SCP(service communication proxy). SCP is a communication proxy defined by 3GPP and aimed at optimizing telco micro-services running in cloud-native environment. Read more about it here.
Telco-cloud requires load-balancing and communication across various interfaces/standards like N2, N4, E2(ORAN), S6x, 5GLAN, GTP etc. Each of these present its own unique challenges which loxilb aims to solve e.g.:
- N4 requires PFCP level session-intelligence
- N2 requires NGAP parsing capability(Related Blogs - Blog-1, Blog-2, Blog-3)
- S6x requires Diameter/SCTP multi-homing LB support(Related Blog)
- MEC use-cases might require UL-CL understanding(Related Blog)
- Hitless failover support might be essential for mission-critical applications
- E2 might require SCTP-LB with OpenVPN bundled together
- SIP support is needed to enable cloud-native VOIP
- N32 requires support for Security Edge Protection Proxy(SEPP)
Why choose loxilb?
Performs
much better compared to its competitors across various architectures- Utitlizes ebpf which makes it
flexible
as well ascustomizable
- Advanced
quality of service
for workloads (per LB, per end-point or per client) - Works with
any
Kubernetes distribution/CNI - k8s/k3s/k0s/kind/OpenShift + Calico/Flannel/Cilium/Weave/Multus etc - Extensive support for
SCTP workloads
(with multi-homing) on k8s - Dual stack with
NAT66, NAT64
support for k8s - k8s
multi-cluster
support (planned 🚧) - Runs in
any
cloud (public cloud/on-prem) orstandalone
environments
Overall features of loxilb
- L4/NAT stateful loadbalancer
- NAT44, NAT66, NAT64 with One-ARM, FullNAT, DSR etc
- Support for TCP, UDP, SCTP (w/ multi-homing), QUIC, FTP, TFTP etc
- High-availability support with hitless/maglev/cgnat clustering
- Extensive and scalable end-point liveness probes for cloud-native environments
- Stateful firewalling and IPSEC/Wireguard support
- Optimized implementation for features like Conntrack, QoS etc
- Full compatibility for ipvs (ipvs policies can be auto inherited)
- Policy oriented L7 proxy support - HTTP1.0, 1.1, 2.0 etc (planned 🚧)
Components of loxilb
- GoLang based control plane components
- A scalable/efficient eBPF based data-path implementation
- Integrated goBGP based routing stack
- A kubernetes agent kube-loxilb written in Go
Architectural Considerations
- Understanding loxilb modes and deployment in K8s with kube-loxilb
- Understanding High-availability with loxilb
Getting started with different K8s distributions & tools
loxilb as ext-cluster pod
- K8s : loxilb ext-mode
- K3s : loxilb with default flannel
- K3s : loxilb with calico
- K3s : loxilb with cilium
- K0s : loxilb with default kube-router networking
- EKS : loxilb ext-mode
loxilb as in-cluster pod
- K8s : loxilb in-cluster mode
- K8s : loxilb in-cluster mode with cilium
- K3s : loxilb in-cluster mode
- K0s : loxilb in-cluster mode
- MicroK8s : loxilb in-cluster mode
- EKS : loxilb in-cluster mode
- RedHat OCP : loxilb in-cluster mode
loxilb as service-proxy
loxilb as Kubernetes Ingress
loxilb in standalone mode
Advanced Guides
- How-To : Service-group zones with loxilb
- How-To : Access end-points outside K8s
- How-To : Deploy multi-server K3s HA with loxilb
- How-To : Deploy loxilb with multi-AZ HA support in AWS
- How-To : Deploy loxilb with multi-cloud HA support
- How-To : Deploy loxilb with ingress-nginx
- How-To : Run loxilb in-cluster with secondary networks
- How-To : Kubernetes virtual cluster setup with k3k and loxilb
- How-To : Kubernetes service sharding with loxilb
- How-To : loxilb L4/L7 Load-Balancing with Kubernetes Gateway API
- How-To : Use proxy protocol v2 with loxilb
Knowledge-Base
- What is eBPF
- What is k8s service - load-balancer
- Architecture in brief
- Code organization
- eBPF internals of loxilb
- What are loxilb NAT Modes
- loxilb load-balancer algorithms
- Manual steps to build/run
- Debugging loxilb
- loxicmd command-line tool usage
- Developer's guide to loxicmd
- Developer's guide to loxilb API
- HTTPS guide for loxilb API
- API Reference - loxilb web-Api
- Performance Reports
- Development Roadmap
- Contribute
- System Requirements
- Frequenctly Asked Questions- FAQs
Blogs
- Building a resilient EKS Cluster with in-cluster auto-scaled LoxiLB
- K8s - Elevating cluster networking
- eBPF - Map sync using Go
- K8s in-cluster service LB with LoxiLB
- K8s - Introducing SCTP Multihoming with LoxiLB
- Load-balancer performance comparison on Amazon Graviton2
- Hyperscale anycast load balancing with HA
- Getting started with loxilb on Amazon EKS
- K8s - Deploying "hitless" Load-balancing
- Oracle Cloud - Hitless HA load balancing
- Ipv6 migration in Kubernetes made easy
Community Posts
- 5G SCTP LoadBalancer Using loxilb
- 5G Uplink Classifier Using loxilb
- K3s - Using loxilb as external service lb
- K8s - Bringing load-balancing to multus workloads with loxilb
- 5G SCTP LoadBalancer Using LoxiLB on free5GC
- Kubernetes Services: Achieving optimal performance is elusive
- LoxiLB: Another Load Balancer