Standardize Resource Labels by cognifloyd · Pull Request #351

Standardize Resource Labels by cognifloyd · Pull Request #351 · StackStorm/stackstorm-k8s

OK. Thank you for testing the migration script @mamercad! Seeing that output was very helpful. From the output, it looks like your ConfigMaps and Secrets did not get the app.kubernetes.io/name label. Would you please look at the labels in your test cluster and make sure everything (managed by this helm chart) has that label?

I tested the migration script with my test cluster and fixed a couple of issues with it:

improve the output so progress is clearer

handle the st2sensorcontainer deployments which can have generated app names

handle the extra-helm-hook jobs which also have generated app names

Note that re-running the script is safe, but ugly. If you re-run it you will see a bunch of errors you can ignore like: 'app.kubernetes.io/name' already has a value (st2workflowengine), and --overwrite is false The script should continue past such errors, and we don't want to overwrite anything that is already set, so that's fine.

No worries, I blew everything away and started with master, then fetched and checked out your branch:

Output

❯ helm ls -n stackstorm
NAME         	NAMESPACE 	REVISION	UPDATED                             	STATUS  	CHART                	APP VERSION
stackstorm-ha	stackstorm	1       	2023-01-27 19:23:30.599787 -0500 EST	deployed	stackstorm-ha-0.100.0	3.8
stackstorm-k8s  ≡

❯ git checkout cognifloyd/std-labels
Note: switching to 'cognifloyd/std-labels'.
...
HEAD is now at 658afdf Simplify migration script: the ServiceAccount token does not have labels.

❯ RELEASE_NAME=stackstorm-ha NAMESPACE=stackstorm migrations/standardize-labels.sh
Adding label app.kubernetes.io/instance=stackstorm-ha (which will replace release=stackstorm-ha) ...
configmap/stackstorm-ha-redis labeled
configmap/stackstorm-ha-redis-health labeled
configmap/stackstorm-ha-redis-scripts labeled
configmap/stackstorm-ha-st2-config labeled
configmap/stackstorm-ha-st2-pack-configs labeled
configmap/stackstorm-ha-st2-urls labeled
configmap/stackstorm-ha-st2actionrunner-post-start-script labeled
configmap/stackstorm-ha-st2client-post-start-script labeled
configmap/stackstorm-ha-st2tests labeled
secret/stackstorm-ha-rabbitmq-definitions labeled
secret/stackstorm-ha-st2-apikeys labeled
secret/stackstorm-ha-st2-auth labeled
secret/stackstorm-ha-st2-datastore-crypto-key labeled
secret/stackstorm-ha-st2-kv labeled
secret/stackstorm-ha-st2-ssh labeled
service/stackstorm-ha-redis labeled
service/stackstorm-ha-redis-headless labeled
service/stackstorm-ha-st2api labeled
service/stackstorm-ha-st2auth labeled
service/stackstorm-ha-st2stream labeled
service/stackstorm-ha-st2web labeled
serviceaccount/stackstorm-ha labeled
deployment.apps/stackstorm-ha-st2actionrunner labeled
deployment.apps/stackstorm-ha-st2api labeled
deployment.apps/stackstorm-ha-st2auth labeled
deployment.apps/stackstorm-ha-st2client labeled
deployment.apps/stackstorm-ha-st2garbagecollector labeled
deployment.apps/stackstorm-ha-st2notifier labeled
deployment.apps/stackstorm-ha-st2rulesengine labeled
deployment.apps/stackstorm-ha-st2scheduler labeled
deployment.apps/stackstorm-ha-st2sensorcontainer labeled
deployment.apps/stackstorm-ha-st2stream labeled
deployment.apps/stackstorm-ha-st2timersengine labeled
deployment.apps/stackstorm-ha-st2web labeled
deployment.apps/stackstorm-ha-st2workflowengine labeled
replicaset.apps/stackstorm-ha-st2actionrunner-6989dd88f4 labeled
replicaset.apps/stackstorm-ha-st2api-849bdff8c7 labeled
replicaset.apps/stackstorm-ha-st2auth-7f9494c759 labeled
replicaset.apps/stackstorm-ha-st2client-69f976747b labeled
replicaset.apps/stackstorm-ha-st2garbagecollector-5f5964574d labeled
replicaset.apps/stackstorm-ha-st2notifier-5dc79694f7 labeled
replicaset.apps/stackstorm-ha-st2rulesengine-79d568ff49 labeled
replicaset.apps/stackstorm-ha-st2scheduler-6696f84b6b labeled
replicaset.apps/stackstorm-ha-st2sensorcontainer-7fc9f89769 labeled
replicaset.apps/stackstorm-ha-st2stream-5bbd79dcd8 labeled
replicaset.apps/stackstorm-ha-st2timersengine-79cd5b5c69 labeled
replicaset.apps/stackstorm-ha-st2web-7c96c574ff labeled
replicaset.apps/stackstorm-ha-st2workflowengine-7b77749884 labeled
pod/stackstorm-ha-job-st2-apikey-load-9wdtz labeled
pod/stackstorm-ha-job-st2-key-load-96w6s labeled
pod/stackstorm-ha-job-st2-register-content-xn42q labeled
pod/stackstorm-ha-redis-node-0 labeled
pod/stackstorm-ha-redis-node-1 labeled
pod/stackstorm-ha-redis-node-2 labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-h7fwb labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-kplx2 labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-lzl2n labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-ntz6v labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-vf9h8 labeled
pod/stackstorm-ha-st2api-849bdff8c7-2grqq labeled
pod/stackstorm-ha-st2api-849bdff8c7-4g8bb labeled
pod/stackstorm-ha-st2auth-7f9494c759-5dwmv labeled
pod/stackstorm-ha-st2auth-7f9494c759-brg4r labeled
pod/stackstorm-ha-st2client-69f976747b-hfz8r labeled
pod/stackstorm-ha-st2garbagecollector-5f5964574d-w9mbx labeled
pod/stackstorm-ha-st2notifier-5dc79694f7-4qqjz labeled
pod/stackstorm-ha-st2notifier-5dc79694f7-ds5r2 labeled
pod/stackstorm-ha-st2rulesengine-79d568ff49-hwjr9 labeled
pod/stackstorm-ha-st2rulesengine-79d568ff49-nxhxt labeled
pod/stackstorm-ha-st2scheduler-6696f84b6b-qkslr labeled
pod/stackstorm-ha-st2scheduler-6696f84b6b-z6th8 labeled
pod/stackstorm-ha-st2sensorcontainer-7fc9f89769-78lvn labeled
pod/stackstorm-ha-st2stream-5bbd79dcd8-dhs7j labeled
pod/stackstorm-ha-st2stream-5bbd79dcd8-fhhpv labeled
pod/stackstorm-ha-st2timersengine-79cd5b5c69-xn5hc labeled
pod/stackstorm-ha-st2web-7c96c574ff-f62j2 labeled
pod/stackstorm-ha-st2web-7c96c574ff-w67gr labeled
pod/stackstorm-ha-st2workflowengine-7b77749884-jrcz8 labeled
pod/stackstorm-ha-st2workflowengine-7b77749884-t7zzk labeled
job.batch/stackstorm-ha-job-st2-apikey-load labeled
job.batch/stackstorm-ha-job-st2-key-load labeled
job.batch/stackstorm-ha-job-st2-register-content labeled

Adding label app.kubernetes.io/name=<app> (which will replace app=<app>) ...
configmap/stackstorm-ha-st2-config labeled
configmap/stackstorm-ha-st2-pack-configs labeled
configmap/stackstorm-ha-st2-urls labeled
configmap/stackstorm-ha-st2actionrunner-post-start-script labeled
configmap/stackstorm-ha-st2client-post-start-script labeled
secret/stackstorm-ha-rabbitmq-definitions labeled
secret/stackstorm-ha-st2-apikeys labeled
secret/stackstorm-ha-st2-auth labeled
secret/stackstorm-ha-st2-datastore-crypto-key labeled
secret/stackstorm-ha-st2-kv labeled
secret/stackstorm-ha-st2-ssh labeled
serviceaccount/stackstorm-ha labeled
Deployment app=st2actionrunner ...
deployment.apps/stackstorm-ha-st2actionrunner labeled
replicaset.apps/stackstorm-ha-st2actionrunner-6989dd88f4 labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-h7fwb labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-kplx2 labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-lzl2n labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-ntz6v labeled
pod/stackstorm-ha-st2actionrunner-6989dd88f4-vf9h8 labeled
Deployment app=st2api ...
deployment.apps/stackstorm-ha-st2api labeled
replicaset.apps/stackstorm-ha-st2api-849bdff8c7 labeled
pod/stackstorm-ha-st2api-849bdff8c7-2grqq labeled
pod/stackstorm-ha-st2api-849bdff8c7-4g8bb labeled
Deployment app=st2auth ...
deployment.apps/stackstorm-ha-st2auth labeled
replicaset.apps/stackstorm-ha-st2auth-7f9494c759 labeled
pod/stackstorm-ha-st2auth-7f9494c759-5dwmv labeled
pod/stackstorm-ha-st2auth-7f9494c759-brg4r labeled
Deployment app=st2chatops ...
Deployment app=st2client ...
deployment.apps/stackstorm-ha-st2client labeled
replicaset.apps/stackstorm-ha-st2client-69f976747b labeled
pod/stackstorm-ha-st2client-69f976747b-hfz8r labeled
Deployment app=st2garbagecollector ...
deployment.apps/stackstorm-ha-st2garbagecollector labeled
replicaset.apps/stackstorm-ha-st2garbagecollector-5f5964574d labeled
pod/stackstorm-ha-st2garbagecollector-5f5964574d-w9mbx labeled
Deployment app=st2notifier ...
deployment.apps/stackstorm-ha-st2notifier labeled
replicaset.apps/stackstorm-ha-st2notifier-5dc79694f7 labeled
pod/stackstorm-ha-st2notifier-5dc79694f7-4qqjz labeled
pod/stackstorm-ha-st2notifier-5dc79694f7-ds5r2 labeled
Deployment app=st2rulesengine ...
deployment.apps/stackstorm-ha-st2rulesengine labeled
replicaset.apps/stackstorm-ha-st2rulesengine-79d568ff49 labeled
pod/stackstorm-ha-st2rulesengine-79d568ff49-hwjr9 labeled
pod/stackstorm-ha-st2rulesengine-79d568ff49-nxhxt labeled
Deployment app=st2scheduler ...
deployment.apps/stackstorm-ha-st2scheduler labeled
replicaset.apps/stackstorm-ha-st2scheduler-6696f84b6b labeled
pod/stackstorm-ha-st2scheduler-6696f84b6b-qkslr labeled
pod/stackstorm-ha-st2scheduler-6696f84b6b-z6th8 labeled
Deployment app=st2sensorcontainer ...
deployment.apps/stackstorm-ha-st2sensorcontainer labeled
replicaset.apps/stackstorm-ha-st2sensorcontainer-7fc9f89769 labeled
pod/stackstorm-ha-st2sensorcontainer-7fc9f89769-78lvn labeled
Deployment app=st2stream ...
deployment.apps/stackstorm-ha-st2stream labeled
replicaset.apps/stackstorm-ha-st2stream-5bbd79dcd8 labeled
pod/stackstorm-ha-st2stream-5bbd79dcd8-dhs7j labeled
pod/stackstorm-ha-st2stream-5bbd79dcd8-fhhpv labeled
Deployment app=st2timersengine ...
deployment.apps/stackstorm-ha-st2timersengine labeled
replicaset.apps/stackstorm-ha-st2timersengine-79cd5b5c69 labeled
pod/stackstorm-ha-st2timersengine-79cd5b5c69-xn5hc labeled
Deployment app=st2web ...
deployment.apps/stackstorm-ha-st2web labeled
replicaset.apps/stackstorm-ha-st2web-7c96c574ff labeled
pod/stackstorm-ha-st2web-7c96c574ff-f62j2 labeled
pod/stackstorm-ha-st2web-7c96c574ff-w67gr labeled
Deployment app=st2workflowengine ...
deployment.apps/stackstorm-ha-st2workflowengine labeled
replicaset.apps/stackstorm-ha-st2workflowengine-7b77749884 labeled
pod/stackstorm-ha-st2workflowengine-7b77749884-jrcz8 labeled
pod/stackstorm-ha-st2workflowengine-7b77749884-t7zzk labeled
Service app=st2api ...
service/stackstorm-ha-st2api labeled
Service app=st2auth ...
service/stackstorm-ha-st2auth labeled
Service app=st2chatops ...
Service app=st2stream ...
service/stackstorm-ha-st2stream labeled
Service app=st2web ...
service/stackstorm-ha-st2web labeled
Job app=st2 ...
job.batch/stackstorm-ha-job-st2-apikey-load labeled
job.batch/stackstorm-ha-job-st2-key-load labeled
pod/stackstorm-ha-job-st2-apikey-load-9wdtz labeled
pod/stackstorm-ha-job-st2-key-load-96w6s labeled
Job app=st2-apply-rbac-definitions ...
Job app=st2-register-content ...
job.batch/stackstorm-ha-job-st2-register-content labeled
pod/stackstorm-ha-job-st2-register-content-xn42q labeled
configmap/stackstorm-ha-st2tests labeled

❯ helm ls -n stackstorm
NAME         	NAMESPACE 	REVISION	UPDATED                             	STATUS  	CHART                	APP VERSION
stackstorm-ha	stackstorm	1       	2023-01-27 19:23:30.599787 -0500 EST	deployed	stackstorm-ha-0.100.0	3.8

❯ k -n stackstorm get deployment/stackstorm-ha-st2actionrunner -o json | jq .metadata.labels
{
  "app": "st2actionrunner",
  "app.kubernetes.io/instance": "stackstorm-ha",
  "app.kubernetes.io/managed-by": "Helm",
  "app.kubernetes.io/name": "st2actionrunner",
  "chart": "stackstorm-ha-0.100.0",
  "heritage": "Helm",
  "release": "stackstorm-ha",
  "tier": "backend",
  "vendor": "stackstorm"
}

❯ k -n stackstorm get cm/stackstorm-ha-mongodb-scripts -o json | jq .metadata.labels
{
  "app.kubernetes.io/component": "mongodb",
  "app.kubernetes.io/instance": "stackstorm-ha",
  "app.kubernetes.io/managed-by": "Helm",
  "app.kubernetes.io/name": "mongodb",
  "helm.sh/chart": "mongodb-10.0.1"
}

❯ k -n stackstorm get secrets stackstorm-ha-mongodb -o json | jq -r .metadata.labels
{
  "app.kubernetes.io/component": "mongodb",
  "app.kubernetes.io/instance": "stackstorm-ha",
  "app.kubernetes.io/managed-by": "Helm",
  "app.kubernetes.io/name": "mongodb",
  "helm.sh/chart": "mongodb-10.0.1"
}

Let me know if there's anything else you want to see.