Blog

Know your real clients: Use PROXY protocol on Red Hat OpenShift on IBM Cloud

Posted on by Lucian Systems


As of 14 June 2023, PROXY protocol is supported for Ingress Controllers in Red Hat OpenShift on IBM Cloud clusters hosted on VPC infrastructure.

Introduction

Modern software architectures often include multiple layers of proxies and load balancers. Preserving the IP address of the original client through these layers is challenging, but might be required for your use cases. A potential solution for the problem is to use PROXY Protocol.

Starting with Red Hat OpenShift on IBM Cloud version 4.13, PROXY protocol is now supported for Ingress Controllers in clusters hosted on VPC infrastructure.

If you are interested in using PROXY protocol for Ingress Controllers on IBM Cloud Kubernetes Service clusters, you can find more information in our previous blog post.

Setting up PROXY protocol for OpenShift Ingress Controllers

When using PROXY protocol for source address preservation, all proxies that terminate TCP connections in the chain must be configured to send and receive PROXY protocol headers after initiating L4 connections. In the case of Red Hat OpenShift on IBM Cloud clusters running on VPC infrastructure, we have two proxies: the VPC Application Load Balancer (ALB) and the Ingress Controller.

On OpenShift clusters, the Ingress Operator is responsible for managing the Ingress Controller instances and the load balancers used to expose the Ingress Controllers. The operator watches IngressController resources on the cluster and makes adjustments to match the desired state.

Thanks to the Ingress Operator, we can enable PROXY protocol for both of our proxies at once. All we need to do is to change the endpointPublishingStrategy configuration on our IngressController resource:

endpointPublishingStrategy: type: LoadBalancerService loadBalancer: scope: External providerParameters: type: IBM ibm: protocol: PROXY

When you apply the previous configuration, the operat,or switches the Ingress Controller into PROXY protocol mode and adds the service.kubernetes.io/ibm-load-balancer-cloud-provider-enable-features: "proxy-protocol" annotation to the corresponding LoadBalancer typed Service resource, enabling PROXY protocol for the VPC ALB.

Example

In this example, we deployed a test application in a single-zone Red Hat OpenShift on IBM Cloud 4.13 cluster that uses VPC generation 2 compute. The application accepts HTTP connections and returns information about the received requests, such as the client address. The application is exposed by the default-router created by the OpenShift Ingress Operator on the echo.example.com domain.

Client information without using PROXY protocol

By default, the PROXY protocol is not enabled. Let’s test accessing the application:

$ curl https://echo.example.com Hostname: test-application-cd7cd98f7-9xbvm Pod Information: -no pod information available- Server values: server_version=nginx: 1.13.3 - lua: 10008 Request Information: client_address=172.24.84.165 method=GET real path=/ query= request_version=1.1 request_scheme=http request_uri=http://echo.example.com:8080/ Request Headers: accept=*/* forwarded=for=10.240.128.45;host=echo.example.com;proto=https host=echo.example.com user-agent=curl/7.87.0 x-forwarded-for=10.240.128.45 x-forwarded-host=echo.example.com x-forwarded-port=443 x-forwarded-proto=https Request Body: -no body in request-

As you can see, the address in the x-forwarded-for header 10.240.128.45 does not match your address. That is the worker node’s address that received the request from the VPC load balancer. That means we can not recover the original address of the client:

$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
10.240.128.45 Ready master,worker 5h33m v1.26.3+b404935
10.240.128.46 Ready master,worker 5h32m v1.26.3+b404935

Enabling PROXY protocol on the default ingress controller

First, edit the Ingress Controller resource:

oc -n openshift-ingress-operator edit ingresscontroller/default

In the Ingress controller resource, find the spec.endpointPublishingStrategy.loadBalancer section and define the following providerParameters values:

endpointPublishingStrategy: loadBalancer: providerParameters: type: IBM ibm: protocol: PROXY scope: External type: LoadBalancerService

Then, save and apply the resource.

Client information using PROXY protocol

Wait until the default-router pods are recycled and test access to the application again:

$ curl https://echo.example.com Hostname: test-application-cd7cd98f7-9xbvm Pod Information: -no pod information available- Server values: server_version=nginx: 1.13.3 - lua: 10008 Request Information: client_address=172.24.84.184 method=GET real path=/ query= request_version=1.1 request_scheme=http request_uri=http://echo.example.com:8080/ Request Headers: accept=*/* forwarded=for=192.0.2.42;host=echo.example.com;proto=https host=echo.example.com user-agent=curl/7.87.0 x-forwarded-for=192.0.2.42 x-forwarded-host=echo.example.com x-forwarded-port=443 x-forwarded-proto=https Request Body: -no body in request-

This time, you can find the actual client address 192.0.2.42 in the request headers, which is the actual public IP address of the original client.

Limitations

The PROXY protocol feature on Red Hat OpenShift on IBM Cloud is supported for only VPC generation 2 clusters that run 4.13 OpenShift version or later.

More information

For more information, check out our official documentation about exposing apps with load balancers, enabling PROXY protocol for Ingress Controllers or the Red Hat OpenShift documentation.

The post Know your real clients: Use PROXY protocol on Red Hat OpenShift on IBM Cloud appeared first on IBM Blog.

Lucian Systems