Fixing Proxmox SSL Certificates Behind HAProxy: Step-by-Step Tutorial

This tutorial chronicles my troubleshooting and resolution process for SSL certificate issues with Proxmox when running behind an OpenWRT router with HAProxy. It includes all intermediate steps, challenges, and reasoning, as well as interactions with my assistant, which helped guide the solution.

---

1. Network Context and Initial Problem

My setup was as follows:

ISP Router -> OpenWRT (DMZ) with HAProxy -> Proxmox

• Proxmox listens on its default HTTPS port 8006.
• Other internal web servers are served through HAProxy, which handles TLS for domains like aurispetreus.net.
• My goal was to have valid, trusted TLS certificates for Proxmox while maintaining other web services on the same HAProxy instance.

Problem Observed

When attempting to access Proxmox through its domain (for example, using the Proxmox Android app), I encountered SSL protocol errors. Initially, I wasn’t sure whether:

1. Proxmox was failing to pick up the manually uploaded certificates.
2. HAProxy configuration was interfering with TLS termination.
3. File permissions or certificate formats were incorrect.

---

2. Generating and Copying Certificates

Proxmox could not directly request Let’s Encrypt certificates because it wasn’t publicly accessible from the internet. Therefore, I generated certificates on a separate Debian server (debian) that could reach Let’s Encrypt:

certbot certonly -d da3.aurispetreus.net

This created:

/etc/letsencrypt/archive/da3.aurispetreus.net/fullchain1.pem
/etc/letsencrypt/archive/da3.aurispetreus.net/privkey1.pem

I then manually copied these files to Proxmox (proxmox A) and renamed them to match Proxmox’s expected filenames:

/etc/pve/local/pve-ssl.pem       # fullchain
/etc/pve/local/pve-ssl.key       # private key

I also set proper permissions for the private key:

chmod 600 /etc/pve/local/pve-ssl.key

At this point, I suspected that the certificates might not be correctly paired, so I decided to verify them.

---

3. Verifying Certificate and Key Match

To ensure the certificate and private key matched, I used OpenSSL to check the modulus:

openssl x509 -noout -modulus -in fullchain1.pem | openssl md5
openssl rsa -noout -modulus -in privkey1.pem | openssl md5

Initially, I received:

Not an RSA key

This indicated a misuse of commands or possibly a non-RSA key (like EC). After adjusting the method (taking into account the correct type of key generated by Certbot), I confirmed that the modulus of the certificate and the private key matched, verifying they belonged together.

---

4. HAProxy Configuration for Proxmox

Since Proxmox sits behind HAProxy, it was important that TLS traffic be forwarded correctly. I started with the following principles:

1. Use TCP mode on HAProxy for port 443 to allow end-to-end TLS (Proxmox manages its own certificates).
2. Route traffic using SNI to direct specific domains to their respective backend servers.
3. Keep the standard HTTPS port (443) public, without binding Proxmox’s internal port 8006 on HAProxy.

Here’s the relevant HAProxy frontend for HTTPS:

frontend https_in
    bind *:443
    mode tcp
    option tcplog

    # Inspect TLS handshake for SNI
    tcp-request inspect-delay 5s
    tcp-request content accept if { req_ssl_hello_type 1 }

    # Use backend based on requested SNI
    use_backend %[req.ssl_sni,lower]_tls if { req.ssl_sni -m found }

And the Proxmox backend:

backend da3.aurispetreus.net_tls
    mode tcp
    option tcp-check
    server da3.aurispetreus.net 192.168.3.144:8006 check
# Note: No need to bind HAProxy to port 8006 externally

Important detail: The assistant reminded me explicitly:
“No, you do not bind port 8006 on HAProxy, because HAProxy is meant to expose a public listener on the standard HTTPS port (443).”
This avoids exposing internal Proxmox ports directly and keeps the network cleaner.

---

5. Initial Testing

I tested the TLS handshake locally:

curl -vk https://192.168.3.144:8006/

Output confirmed that TLSv1.3 was being used, and the correct certificate was presented.

Externally, trying to access 188.21.68.172:8006 failed because the ISP router blocked the direct connection, which was expected.

---

6. Debugging “Wrong Version Number” Error

When testing public access via HAProxy:

curl -vk https://da3.aurispetreus.net:443

I got:

error:0A00010B:SSL routines::wrong version number

Analysis:

• The HAProxy frontend for port 443 was in HTTP mode.
• TLS traffic to Proxmox requires TCP passthrough, not HTTP mode.

Solution: Change the frontend to TCP mode. After doing so, the handshake worked correctly, and the Proxmox Android app could connect.