As of 14 June 2023, PROXY protocol is supported for Ingress Controllers in Red Hat OpenShift on IBM Cloud clusters hosted on VPC infrastructure.
Modern software architectures often include multiple layers of proxies and load balancers. Preserving the IP address of the original client through these layers is challenging, but might be required for your use cases. A potential solution for the problem is to use PROXY Protocol.
Starting with Red Hat OpenShift on IBM Cloud version 4.13, PROXY protocol is now supported for Ingress Controllers in clusters hosted on VPC infrastructure.
Setting up PROXY protocol for OpenShift Ingress Controllers
When using PROXY protocol for source address preservation, all proxies that terminate TCP connections in the chain must be configured to send and receive PROXY protocol headers after initiating L4 connections. In the case of Red Hat OpenShift on IBM Cloud clusters running on VPC infrastructure, we have two proxies: the VPC Application Load Balancer (ALB) and the Ingress Controller.
On OpenShift clusters, the Ingress Operator is responsible for managing the Ingress Controller instances and the load balancers used to expose the Ingress Controllers. The operator watches IngressController resources on the cluster and makes adjustments to match the desired state.
Thanks to the Ingress Operator, we can enable PROXY protocol for both of our proxies at once. All we need to do is to change the
endpointPublishingStrategy configuration on our
endpointPublishingStrategy: type: LoadBalancerService loadBalancer: scope: External providerParameters: type: IBM ibm: protocol: PROXY
When you apply the previous configuration, the operat,or switches the Ingress Controller into PROXY protocol mode and adds the
service.kubernetes.io/ibm-load-balancer-cloud-provider-enable-features: "proxy-protocol" annotation to the corresponding
LoadBalancer typed Service resource, enabling PROXY protocol for the VPC ALB.
In this example, we deployed a test application in a single-zone Red Hat OpenShift on IBM Cloud 4.13 cluster that uses VPC generation 2 compute. The application accepts HTTP connections and returns information about the received requests, such as the client address. The application is exposed by the
default-router created by the OpenShift Ingress Operator on the
Client information without using PROXY protocol
By default, the PROXY protocol is not enabled. Let’s test accessing the application:
$ curl https://echo.example.com Hostname: test-application-cd7cd98f7-9xbvm Pod Information: -no pod information available- Server values: server_version=nginx: 1.13.3 - lua: 10008 Request Information: client_address=172.24.84.165 method=GET real path=/ query= request_version=1.1 request_scheme=http request_uri=http://echo.example.com:8080/ Request Headers: accept=*/* forwarded=for=10.240.128.45;host=echo.example.com;proto=https host=echo.example.com user-agent=curl/7.87.0 x-forwarded-for=10.240.128.45 x-forwarded-host=echo.example.com x-forwarded-port=443 x-forwarded-proto=https Request Body: -no body in request-
As you can see, the address in the
10.240.128.45 does not match your address. That is the worker node’s address that received the request from the VPC load balancer. That means we can not recover the original address of the client:
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
10.240.128.45 Ready master,worker 5h33m v1.26.3+b404935
10.240.128.46 Ready master,worker 5h32m v1.26.3+b404935
Enabling PROXY protocol on the default ingress controller
First, edit the Ingress Controller resource:
oc -n openshift-ingress-operator edit ingresscontroller/default
In the Ingress controller resource, find the
spec.endpointPublishingStrategy.loadBalancer section and define the following
endpointPublishingStrategy: loadBalancer: providerParameters: type: IBM ibm: protocol: PROXY scope: External type: LoadBalancerService
Then, save and apply the resource.
Client information using PROXY protocol
Wait until the
default-router pods are recycled and test access to the application again:
$ curl https://echo.example.com Hostname: test-application-cd7cd98f7-9xbvm Pod Information: -no pod information available- Server values: server_version=nginx: 1.13.3 - lua: 10008 Request Information: client_address=172.24.84.184 method=GET real path=/ query= request_version=1.1 request_scheme=http request_uri=http://echo.example.com:8080/ Request Headers: accept=*/* forwarded=for=192.0.2.42;host=echo.example.com;proto=https host=echo.example.com user-agent=curl/7.87.0 x-forwarded-for=192.0.2.42 x-forwarded-host=echo.example.com x-forwarded-port=443 x-forwarded-proto=https Request Body: -no body in request-
This time, you can find the actual client address
192.0.2.42 in the request headers, which is the actual public IP address of the original client.
The PROXY protocol feature on Red Hat OpenShift on IBM Cloud is supported for only VPC generation 2 clusters that run 4.13 OpenShift version or later.
The post Know your real clients: Use PROXY protocol on Red Hat OpenShift on IBM Cloud appeared first on IBM Blog.