Half II: A Platform Integration Instance
In Half I, we explored the structure of the CRI-O credential supplier and walked by way of a handbook setup. On this half, we’ll see how platforms like OpenShift and its upstream open-source mission OKD combine the credential supplier natively, making deployment less complicated. OpenShift consists of the credential supplier beginning with model 4.21, with totally different integration ranges throughout variations displaying the evolution towards extra native platform assist.
Utilizing the credential supplier with OpenShift 4.21
OpenShift 4.21 ships the crio-credential-provider RPM package deal together with CRI-O v1.34, which is the minimal model required for the credential supplier assist. Earlier CRI-O variations don’t assist namespace-scoped auth recordsdata. Since there isn’t any Customized Useful resource Definition or API for managing the credential supplier configuration in OpenShift 4.21, customers should manually create a MachineConfig useful resource to deploy the configuration recordsdata. By overriding the present ECR credential supplier configuration file, the kubelet mechanically makes use of the CRI-O credential supplier with out requiring further configuration. This method works on all OpenShift installations whatever the underlying cloud supplier.
Allow function gate
Allow the KubeletServiceAccountTokenForCredentialProviders function gate:
kubectl patch FeatureGate cluster --type merge --patch '{"spec":{"featureSet":"CustomNoUpgrade","customNoUpgrade":{"enabled":["KubeletServiceAccountTokenForCredentialProviders"]}}}'Create Ignition Config
Create a file named machine-config.bu with the next Ignition Config. This configuration creates each the credential supplier configuration and the registry mirror configuration on employee nodes. Notice that it will overwrite each /and many others/kubernetes/credential-providers/ecr-credential-provider.yaml and /and many others/containers/registries.conf. Test what credential suppliers and registry mirrors are at the moment configured in your nodes earlier than making use of this to keep away from breaking present setups.
Notice: The configuration file is deliberately named ecr-credential-provider.yaml to override OpenShift’s present ECR credential supplier configuration. Whereas the KubeletConfig API helps configuring kubelet settings, the imageCredentialProviderConfigFile subject is handed as a command-line flag to the kubelet and isn’t at the moment configurable by way of the KubeletConfig useful resource. By reusing the present file path that OpenShift’s kubelet is already configured to make use of, we keep away from needing to switch kubelet flags or systemd items. This method simplifies deployment and works throughout all OpenShift installations.
variant: openshift
model: 4.20.0
metadata:
labels:
machineconfiguration.openshift.io/function: employee
identify: 99-worker-crio-credential-provider-config
storage:
recordsdata:
- path: /and many others/kubernetes/credential-providers/ecr-credential-provider.yaml
mode: 0644
overwrite: true
contents:
inline: |
apiVersion: kubelet.config.k8s.io/v1
sort: CredentialProviderConfig
suppliers:
- identify: crio-credential-provider
matchImages:
- docker.io
defaultCacheDuration: "1s"
apiVersion: credentialprovider.kubelet.k8s.io/v1
tokenAttributes:
serviceAccountTokenAudience:
cacheType: "Token"
requireServiceAccount: false
- path: /and many others/containers/registries.conf
mode: 0644
overwrite: true
contents:
inline: |
unqualified-search-registries = ["docker.io", "registry.access.redhat.com"]
[[registry]]
location = "docker.io"
[[registry.mirror]]
location = "localhost:5000"
insecure = true
Compile to MachineConfig
The config must be compiled from the Ignition Config format to MachineConfig utilizing Butane in order that the Machine Config Operator is ready to apply the configuration:
podman run -it -v $(pwd):/w -w /w quay.io/coreos/butane:launch machine-config.bu -o machine-config.ymlThis could end in:
# Generated by Butane; don't edit
apiVersion: machineconfiguration.openshift.io/v1
sort: MachineConfig
metadata:
labels:
machineconfiguration.openshift.io/function: employee
identify: 99-worker-crio-credential-provider-config
spec:
config:
ignition:
model: 3.5.0
storage:
recordsdata:
- contents:
compression: gzip
supply: information:;base64,H4sIAAAAAAAC/1SPMW+EMAyFd36FxU5Ot1XZTnTpVqmn7jnHHFYgoY6D1H9fEUFps8V+z+97buFPkswpWgjlQROpwRQHfprwkg2ny3ptAkdvoRfyFJXd9C5pZU/SV2Gz7N9sG4AOopvJAgqnDn8t3SFqAABmpzi+ze5J1bO9DnzCQGI41YmnwZVJe4cjvRZxWhHba27r+i/3mXKEmKPK2WEzaQoUb6rCj6JndCZZGemGmErUexUVzxSRLIyqS7aXy3ZRIills6OZvOJ+ATfK+/dCFtrqb/eF0FdhoY9/CRYGN2VqfgIAAP//tTtvdHwBAAA=
mode: 420
overwrite: true
path: /and many others/kubernetes/credential-providers/ecr-credential-provider.yaml
- contents:
compression: gzip
supply: information:;base64,H4sIAAAAAAAC/1zMsaoDIRCF4d6nGKzvyja3CfgkYiHjJA5xHTKjRd4+BBJY0h34P84aj1U6X5nqZlQU26Z0Y5vKZBAh+Sp4Jw0s/g/8pz1DQSSzoFRbmQHl8Nm5lL49Z9cFy2QZEOH0cUbhYFXRH/vevYnNy/++797xMMKlBBGmLnKvAAAA//92BbzesgAAAA==
mode: 420
overwrite: true
path: /and many others/containers/registries.conf
Notice: The base64-encoded information within the generated MachineConfig will mirror the precise file paths out of your Ignition Config. The instance above is for reference solely.
Apply MachineConfig
Apply the MachineConfig to deploy the configuration recordsdata to employee nodes:
kubectl apply -f machine-config.ymlLook forward to the Machine Config Operator to roll out these modifications to employee nodes. This course of entails node reboots. You possibly can monitor the rollout standing:
oc get machineconfigpool employee -wAs soon as all nodes present UPDATED=True and UPDATING=False, the MachineConfig rollout is full and the kubelet is configured to make use of the CRI-O credential supplier.
Put together a particular node for testing
Now that the MachineConfig has been utilized, put together a employee node for testing. You might want to begin the native registry mirror on the goal node and apply the cluster-level RBAC configuration for every node, just like the Configure RBAC part within the earlier instance:
# Choose any employee node for the demo
export NODE_NAME=$(kubectl get node -l node-role.kubernetes.io/employee --output jsonpath="{.items[0].metadata.name}")
# Enter the node
oc debug "node/$NODE_NAME"
Now on the node itself, execute the next instructions:
# Entry the host filesystem
chroot /host
# Confirm that the crio-credential-provider is out there on the node
/usr/libexec/kubelet-image-credential-provider-plugins/crio-credential-provider --version
# Clone the repository
git clone --depth=1 ~/crio-credential-provider
# Begin the native take a look at registry
~/crio-credential-provider/take a look at/registry/begin
Apply RBAC
Transfer again to the host, and apply the required RBAC to the cluster:
# Clone the repository
git clone --depth=1
cd crio-credential-provider
# Replace RBAC to make use of the precise node identify
sed -i 's;system:node:127.0.0.1;system:node:'"$NODE_NAME"';g' take a look at/cluster/rbac.yml
# Apply RBAC and secret to the cluster
kubectl apply -f take a look at/cluster/rbac.yml -f take a look at/cluster/secret.yml
Check credential supplier
Check the credential supplier through the use of a node selector:
# Label the node for testing
kubectl label nodes "$NODE_NAME" app=take a look at
# Add node selector to pod spec
sed -i "s;spec:;spec:n nodeSelector:n app: test;g" take a look at/cluster/pod.yml
# Deploy the take a look at pod
kubectl apply -f take a look at/cluster/pod.yml
View logs
Examine the credential supplier logs utilizing journald on the node:
journalctl _COMM=crio-credential
… crio-credential[…]: app.go:33: Working credential supplier
… crio-credential[…]: app.go:45: Studying from stdin
… crio-credential[…]: app.go:62: Parsed credential supplier request for picture "docker.io/library/nginx"
… crio-credential[…]: app.go:64: Parsing namespace from request
… crio-credential[…]: app.go:71: Matching mirrors for registry config: /and many others/containers/registries.conf
… crio-credential[…]: app.go:84: Obtained mirror(s) for "docker.io/library/nginx": "localhost:5000"
… crio-credential[…]: app.go:86: Getting secrets and techniques from namespace: default
… crio-credential[…]: k8s.go:119: Utilizing API server host: api-int.ci-ln-62qi4bb-76ef8.aws-4.ci.openshift.org:6443
… crio-credential[…]: app.go:101: Obtained 1 secret(s)
… crio-credential[…]: auth.go:87: Parsing secret: my-secret
… crio-credential[…]: auth.go:97: Discovered docker config JSON auth in secret "my-secret" for "
… crio-credential[…]: auth.go:112: Checking if mirror "localhost:5000" matches registry "localhost:5000"
… crio-credential[…]: auth.go:115: Utilizing mirror auth "localhost:5000" for registry from secret "localhost:5000"
… crio-credential[…]: auth.go:48: Wrote auth file to /and many others/crio/auth/default-7e59ad64326bc321517fb6fc6586de5ee149178394d9edfa2a877176cdf6fad5.json with 8 variety of entries
… crio-credential[…]: app.go:108: Auth file path: /and many others/crio/auth/default-7e59ad64326bc321517fb6fc6586de5ee149178394d9edfa2a877176cdf6fad5.json
Additionally, podman logs registry ought to point out that the picture has been pulled utilizing the native mirror.
OpenShift 4.22+: CRIOCredentialProviderConfig API
OpenShift 4.22 is deliberate to introduce a CRIOCredentialProviderConfig Customized Useful resource Definition (config.openshift.io/v1alpha1) that can present a declarative API for managing credential supplier configuration. The Machine Config Operator will deal with all of the configuration particulars mechanically, eliminating the necessity for handbook MachineConfig sources.
Instance utilization:
apiVersion: config.openshift.io/v1alpha1
sort: CRIOCredentialProviderConfig
metadata:
identify: cluster
spec:
matchImages:
- "docker.io"
The Machine Config Operator will validate the configuration, generate the credential supplier config recordsdata on all nodes, configure the kubelet, and carry out rolling restarts mechanically.
Conclusion
All through this two-part collection, we’ve explored how the CRI-O credential supplier addresses an actual operational downside: sustaining namespace-scoped safety boundaries whereas utilizing personal registry mirrors in Kubernetes. The implementation leverages present Kubernetes primitives (service account tokens, secrets and techniques, and RBAC) to offer an answer that integrates naturally with cluster safety insurance policies.
The kubelet credential supplier API is steady in Kubernetes 1.26, and the service account token function is out there in Kubernetes 1.33 with a function gate. The CRI-O credential supplier is transport in OpenShift 4.21, with declarative API assist coming in 4.22.
This allows organizations working multi-tenant platforms to make use of authenticated registry mirrors and pull-through caches with out breaking namespace isolation. Groups can handle their very own registry credentials as normal Kubernetes Secrets and techniques, rotate them independently, and keep strict safety boundaries between initiatives. For air-gapped environments and organizations with compliance necessities, this implies registry mirrors can be utilized safely with out exposing credentials globally on the node degree. The answer leverages present Kubernetes RBAC for entry management, making it pure to combine into present safety insurance policies.
The mission is actively maintained as a part of the CRI-O ecosystem. Supply code, documentation, and end-to-end assessments can be found at github.com/cri-o/crio-credential-provider. For OpenShift customers, the evolution from handbook configuration in 4.21 to the declarative CRIOCredentialProviderConfig API in 4.22 reveals the trail towards absolutely automated, operator-managed setup.
Thanks for studying! We’d love to listen to your ideas on this function and the way it would possibly assist together with your registry authentication challenges. Be happy to open points or contribute to the mission on GitHub, or be a part of the dialogue within the #cri-o channel on Kubernetes Slack.
