Add support for `ClusterAPI IPAM Provider`

[Archisman-Mridha]

/kind feature

Describe the solution you'd like

I would like ClusterAPI Provider Hetzner to support ClusterAPI IPAM Provider.

Other bare-metal related infrastructure providers like ClusterAPI Provider Metal3 and ClusterAPI Provider VSphere have it.

I need it to assign private IPs to the Hetzner Bare Metal servers.

Anything else you would like to add:

We initially tried :

• running Kea DHCP server in an HCloud server. And try using it to allocate private IPs to the Hetzner Bare Metal servers belonging to the same cluster.

But the problem is : the Kea DHCP server is running in a separate subnet. And Hetzner Bare Metal servers are running in a separate subnet : the VSwitch subnet. And I cannot put HCloud servers in the VSwitch subnet.

A solution is : to run a DHCP relay server in the first Hetzner Bare Metal server in the VSwitch subnet. But we didn't want to go that route and rather add support for ClusterAPI IPAM Provider.

The plan is to :

• Create an IPAddressClaim resource in the preparing phase of the Hetzner Bare Metal host.

• Wait for a private IP address to be assigned.

• Then, we read that private IPv4 address and put stuff in the cloud-init data, so that the private IP address binds to the NIC of the server.

  This is the part I've to look into. Since, the cloud-init data is getting generated by the Kubeadm bootstrap provider AFAIK.
  Maybe I need to parse and add the extra data for binding the private IP. Or any better approach in your mind?
  You can see my commit here.

[guettli]

Maybe you can help yourself by using:

KubeadmConfig.PreKubeadmCommands - The Cluster API Book

This way you can run arbirtary commands before kubeadm gets executed.

Does this help you?

[Archisman-Mridha]

Yes, we’re already using the preKubeadm hook, for other reasons.

But in this case, the issue is : we cannot get the privateIP in the preKubeadm section.

I mean we can, but first we need to bind the private IP to the primary Network Interface. And then, we can get the private IP in the preKubeadm section, by running some Linux commands.

I’m stuck with how to do that binding.

[guettli]

@Archisman-Mridha have you used the private network feature between hcloud and robot before?

Please search the web. We do not recommend it to use that. Public IPs are fine.

[Archisman-Mridha]

Yes, I have used the private network feature before : but had to setup manually. But haven't used it in scale.

@guettli are you referring to this : https://www.reddit.com/r/hetzner/comments/1mdxa44/do_you_use_private_networks_of_hetzner/.

[guettli]

I have never used the private network feature, but I heard complaints. Some customers with a lot of experience with Hetzner switched to us (so that we manage the management Kubernetes cluster for them), and they said the same: avoid it, just use the public network.

[Archisman-Mridha]

If you guys are not using the Hetzner Network feature, then I have a question :

do you keep the Kubernetes API server off the public internet? Or is port 6443 of the control-plane nodes exposed to the public internet via an HCloud LoadBalancer?

In case of a Hetzner hybrid cluster : where we have Hetzner Bare Metal nodes, they need to be able to reach the Kubernetes API server. This forces to keep the Kubernetes API server exposed to the public internet.

[Archisman-Mridha]

@guettli I was wondering, should I still work on this issue, so we can use it when Hetzner's Network feature becomes stable?

[guettli]

We wrote our down what we think: https://syself.com/docs/hetzner/apalla/concepts/zero-trust

I was wondering, should I still work on this issue, so we can use it when Hetzner's Network feature becomes stable?

I think there are more interesting things do. Imagine the issue is solved in two years, then the foundation (capi, caph) might have changed a lot. I would not do it now.