GKEHubFeatureMembership with the private GitHub repo

[fforloff]

@mathieu-benoit , setting up configSync via the GKEHubFeatureMembership works great if the GitHub repo is public, however I can't figure out how it can be done for the private repo. Setting secretType to ssh expects the git-creds secret in the config-management-system namespace of the newly provisioned cluster. Isn't it a chicken/egg situation?

[mathieu-benoit]

Hi @fforloff, thanks for reaching out! Good point and good question!

You are right, it's a chicken/egg situation :)

In this page, that's what I would add to accomplish this:

# Before running the following commands, the 2 resources GKEHubFeature and GKEHubMembership need to be commited in the TENANT_PROJECT_DIR_NAME dir/repo in order to have the following Connect Gateway working.
cd ${WORK_DIR}
gh repo create $GKE_CONFIGS_DIR_NAME --private --clone --template https://github.com/mathieu-benoit/config-sync-template-repo
cd ${WORK_DIR}$GKE_CONFIGS_DIR_NAME
git pull
git checkout main
#GKE_CONFIGS_REPO_URL=$(gh repo view --json url --jq .url)
GKE_CONFIGS_REPO_URL=$(gh repo view --json sshUrl --jq .sshUrl)
GKE_CONFIGS_REPO_NAME_WITH_OWNER=$(gh repo view --json nameWithOwner --jq .nameWithOwner)

mkdir tmp
ssh-keygen -t rsa -b 4096 \
    -C "${GKE_CONFIGS_REPO_NAME_WITH_OWNER}@github" \
    -N '' \
    -f ./tmp/github-gke-configs-repo
gcloud container fleet memberships get-credentials ${GKE_NAME}
kubectl create secret generic git-creds \
    -n config-management-system \
    --from-file ssh=./tmp/github-gke-configs-repo
gh repo deploy-key add ./tmp/github-gke-configs-repo.pub
rm -r tmp

# And then change the following GKEHubFeatureMembership with secretType as ssh.

I haven't tested this as-is yet, but that's pretty much close to what will work. If you have any feedback, please do let me know.

Note: using the Connect Gateway Fleet feature (gcloud container fleet memberships get-credentials) which supports the fact that the GKE cluster in tenant project is private. In this case, as Platform Admin, I need to leverage that feature to avoid the chicken/egg situation.

I will soon update this workshop with that, and actually I want to move all the 6 GitHub repos to private (tracking this here), in order to continue showing security best practices.

Thanks for the feedback/question, keep them coming! :)

[fforloff]

Thanks @mathieu-benoit. I'm trying to avoid as much manual steps as possible. So I think another option is to have the Cloud Source <-> GitHub sync happening, so that I can use the IAM GSA/KSA to access the Cloud Source copy of the code instead.

[fforloff]

So that worked beautifully

I made the config controller SA a source.admin:

gcloud projects add-iam-policy-binding ${HOST_PROJECT_ID} \
    --member="serviceAccount:${CONFIG_CONTROLLER_SA}" \
    --role='roles/source.admin'    

Then I added to the equivalent of your host repo

kind: IAMPolicyMember
metadata:
  name: <tenant>-iam-source-adm
  namespace: config-control
spec:
  memberFrom:
    serviceAccountRef:
      name: <sa_name>
  role: roles/source.admin
  resourceRef:
    kind: SourceRepoRepository
    external: projects/<host_project>/repos/<github_copy_in_the_cloud_source>

Then in the equivalent of the tenant repo:

apiVersion: iam.cnrm.cloud.google.com/v1beta1
kind: IAMPartialPolicy
metadata:
  name: <tenant_sa>-wi
  namespace: <tenant_namespace>
spec:
  resourceRef:
    name: <gke_sa>
    kind: IAMServiceAccount
  bindings:
    - role: roles/iam.workloadIdentityUser
      members:
        - member: serviceAccount:<tenant_project_name>.svc.id.goog[config-management-system/root-reconciler]

and

apiVersion: iam.cnrm.cloud.google.com/v1beta1
kind: IAMPolicyMember
metadata:
 name: <gke_sa>-sa-source-reader
spec:
 member: serviceAccount:<gke_sa>@<tenant_project_name>.iam.gserviceaccount.com
 role: roles/source.reader
 resourceRef:
   kind: SourceRepoRepository
   external: projects/<host_project>/repos/<github_copy_in_the_cloud_source>

[fforloff]

BTW, thanks for the Connect Gateway hint - it helped me with my other issue, where I could not access the cluster from the coronet, since the endpoint was an IP address, not the FQDN.

[mathieu-benoit]

Yep, SourceRepRepository works great with Workload Identity. Another alternative could be to convert your Git repo to either OCI artifact registry or Helm chart registry with Artifact Registry and Workload Identity. In the future, that's where I want to convert this workshop.

And yes, Connect Gateway is not very well known, but very powerful! :)