How I Learned to Stop Worrying and Love Vibe Coding

Jared P. Lander

Lander Analytics

Step One

Step Two

The Environment

devbox.json
{
  "$schema": "https://raw.githubusercontent.com/jetify-com/devbox/0.15.0/.schema/devbox.schema.json",
  "packages": [
    "kubernetes-helm@latest",
    "k9s@latest",
    "kluctl@latest",
    "kubectl@latest",
    "yq-go@latest",
    "jq@latest",
    "oras@latest",
    "kubectl-cnpg@latest",
    "kustomize@latest",
    "kubeswitch@0.9.3",
    "cachix@1.7.9",
    "r@4.5.0",
    "rPackages.DBI@1.2.3",
    "rPackages.jsonlite@2.0.0",
    "rPackages.languageserver@0.3.16",
    "azure-cli@2.75.0",
    "argocd@latest"
  ]
}

Starting from Scratch

Prompt

Why do people like kustomize better than helm?

Response

Great question — it really comes down to different philosophies and use cases. Here’s a quick breakdown of why some folks prefer Kustomize over Helm:

🔍 Why People Like Kustomize Better 1. Pure YAML, No Templating Language

Kustomize works with plain YAML and overlays — no Go templating or templating language to learn.

It feels more declarative, like “just patch what I want changed.”

  1. Built into kubectl

Kustomize is natively supported in kubectl (kubectl apply -k) — no extra tool required.

Easier to adopt in simple or vanilla clusters.

  1. Better GitOps Fit (for some)

GitOps tools like ArgoCD treat Kustomize as a first-class citizen.

No need to render templates before applying; it just patches.

  1. Less Magic, More Transparency

No values files, no template rendering steps.

What you see is what you get — helpful for debugging or auditing changes.

  1. Composable and Overlay-Friendly

Kustomize makes it easy to define a base and apply overlays (e.g., dev, staging, prod) using patches, without copying files.

Pivot to Kustomize

Prompt

You are an experienced DevOps Engineer with expertise in Kubernetes and Kustomize in particular. Build out a folder structure for using Kustomize with multiple overlays.

files/kustomize/first
├── base
│   ├── api
│   ├── database
│   ├── files
│   ├── kustomization.yaml
│   └── shiny
├── devbox.json
├── kustomization.yaml
└── overlays
    ├── dev
    ├── prod
    └── staging

files/kustomize/first/base
├── api
│   ├── kustomize.yaml
│   └── resources
│       └── deployment.yaml
├── database
│   ├── kustomize.yaml
│   └── resources
│       ├── cluster.yaml
│       ├── db.yaml
│       └── pooler.yaml
├── files
│   ├── kustomize.yaml
│   ├── rendered
│   │   └── nginx.yaml
│   ├── resources
│   │   └── configmap.yaml
│   └── values.yaml
├── kustomization.yaml
└── shiny
    ├── kustomization.yaml
    └── resources
        └── deployment.yaml

files/kustomize/first/overlays
├── dev
│   ├── api
│   ├── database
│   ├── files
│   ├── kustomization.yaml
│   └── shiny
├── prod
│   ├── api
│   ├── database
│   ├── files
│   ├── kustomization.yaml
│   └── shiny
└── staging
    ├── api
    ├── database
    ├── files
    ├── kustomization.yaml
    └── shiny

files/kustomize/first/overlays/dev
├── api
│   ├── kustomization.yaml
│   └── patches
│       └── deployment.yaml
├── database
│   ├── kustomization.yaml
│   └── patches
│       └── cluster.yaml
├── files
│   ├── kustomization.yaml
│   └── patches
│       └── deployment.yaml
├── kustomization.yaml
└── shiny
    ├── kustomization.yaml
    └── patches
        └── deployment.yaml

Problems

  • URL resolution
  • Indentation errors
  • Different Helm versions
  • Pulling from private registries
  • Containers restarting
  • Wrong image versions
  • Jobs Failing
  • Port mismatches
  • Header forwarding
  • Incorrect storageclass
  • Startup probe failure
  • Missing files

Wrong Image

Prompt

Why is this change not being made when I run kubectl kustomize overlays/dev > dev.yaml?

Context

images:
  - name: timescaledb/timescaledb-ha
    newName: demo.azurecr.io/timescaledb-ha
    newTag: timescaledb-ha:pg17.5-ts2.20.1
  - name: docker.io/bitnami/redis
    newName: demo.azurecr.io/redis
    newTag: 7.4.3-debian-12-r0
  - name: docker.io/bitnami/redis-sentinel
    newName: demo.azurecr.io/redis-sentinel
    newTag: 7.4.3-debian-12-r0

Response

The change is not being applied because of incorrect YAML indentation in your kustomization.yaml file.

In the images list, the newName and newTag fields should have the same indentation level as the name field. The YAML parser is likely ignoring these fields due to the extra indentation.

Here is the corrected snippet for your images section:

// ...existing code...
images:
  - name: timescaledb/timescaledb-ha
    newName: demo.azurecr.io/timescaledb-ha
    newTag: timescaledb-ha:pg17.5-ts2.20.1
  - name: docker.io/bitnami/redis
    newName: demo.azurecr.io/redis
    newTag: 7.4.3-debian-12-r0
  - name: docker.io/bitnami/redis-sentinel
    newName: demo.azurecr.io/redis-sentinel
    newTag: 7.4.3-debian-12-r0

After correcting the indentation for all entries in the images list, running kubectl kustomize should apply the image overrides as expected.

Prompt

That indenting is no different from what I have

Response

You are right to point that out. My apologies. Looking closer at your kustomization.yaml, the indentation in the images section is indeed correct.

The issue is likely that another patch is overriding the image specification. Specifically, the patch patches/cluster.yaml is a likely candidate since your comment indicates it changes “the used image and domain names”.

When both an images transformation and a patches entry target the same field (like the image of a container), the patches directive takes precedence.

To resolve this, you should either:
1. Remove the image override from patches/shiny.yaml and rely solely on the images block.
2. Remove the timsescaledb/timescaledb-ha entry from the images block and ensure the correct image is specified within patches/cluster.yaml.

The first approach is generally cleaner for managing image tags across environments.

Prompt

The entry in the patches is commented out

Response

You are correct. My apologies for the oversight. Since the image key in cluster.yaml is commented out, it is not the cause of the issue.

The problem is that the images transformer in Kustomize does not know how to modify the image for the CNPG Custom Resource (CR). The images transformer typically works on standard Kubernetes kinds like Deployment, StatefulSet, etc., by looking for a standard image path. The CNPG CRD uses a spec.imageName field, which Kustomize doesn’t recognize by default.

To solve this, you need to tell Kustomize how to find and replace the image in your CNPG CR. You can do this by adding a configurations block to your kustomization.yaml.

Alternatively, and more simply, you can uncomment and use the image field directly in your patch file. This is the standard way to configure the image for the CNPG instance via its CR.

Here is the change for cluster.yaml:

// ...existing code...
  imageName: demo.azurecr.io/timescaledb-ha:pg17.5-ts2.20.1
  instances: 3

By uncommenting that line, you are explicitly setting the image that the CNPG Operator should use for the deployment it manages. This will override any default image set in the base and ensure the correct image from your Azure Container Registry is used. You can then remove the corresponding timescaledb/timescaledb-ha entry from the images section in your kustomization.yaml as it is no longer needed.

Prompt

Show me how to use a kustomizeconfig file

Response

Of course. To make the images transformer in Kustomize aware of the image field in your CNPG custom resource, you need to create a configuration file that defines the path to that field.

Here’s how you can do it: 1. Create a kustomizeconfig.yaml file. This file will tell Kustomize where to find the image field in a CNPG resource. Place this file in the same directory as your kustomization.yaml.

// filepath: overlays/dev/database/kustomizeconfig.yaml
images:
- path: spec/imageName
  group: postgresql.cnpg.io
  kind: Cluster
  version: v1

Reference the configuration in your kustomization.yaml Add a configurations field to your main kustomization.yaml to point to the new configuration file.

// filepath: overlays/dev/database/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

configurations:
  - kustomizeconfig.yaml

resources:
// ...existing code...

With these two changes, Kustomize will now correctly apply the image replacement you’ve defined in the images section of your kustomization.yaml to the CNPG resource. You can keep the image line commented out in cluster.yaml.

Error Logs

Prompt

What does this error mean?

Error from server (Invalid): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{"apiVersion":"batch/v1","kind":"Job","metadata":{"annotations":{"helm.sh/hook":"post-install,post-upgrade","helm.sh/hook-delete-policy":"hook-succeeded,hook-failed","helm.sh/hook-weight":"20"},"name":"populate-db-service-data","namespace":"lethaldrifter-staging"},"spec":{"activeDeadlineSeconds":2400,"backoffLimit":5,"completions":1,"parallelism":1,"template":{"metadata":{"labels":{"app":"load-service-data"}},"spec":{"containers":[{"command":["/bin/bash","-c","echo \"Waiting for database to be ready...\"\n# until Rscript -e \"config \u003c- yaml::read_yaml('config.yml', eval.expr=TRUE)\" -e \"config[['staging']][['api']] \u003c- list(url='http://ldk-staging.lander.ai', port='8000')\" -e \"config[['production']][['api']] \u003c- list(url='https://ld-test.i2cv.io', port='8000')\" -e \"yaml::write_yaml(config, 'config.yml')\" -e \"con \u003c- DBI::dbConnect(RPostgres::Postgres(), dbname = '${API_DB_DBNAME}', host = '${API_DB_HOST}', port = '${API_DB_PORT}', user = '${API_DB_USER}', password

Response

This error means you are trying to patch/update a Kubernetes Job, but the Pod template (spec.template) is immutable after the Job is created. Kubernetes does not allow you to change the Pod spec of an existing Job (unlike Deployments).

Key part of the error:

Why does this happen?

When you change things like the container image, commands, env vars, etc. in a Job’s spec and try to kubectl apply or helm upgrade, Kubernetes refuses because it can’t safely update running/completed Jobs this way.

How to fix?

Delete the existing Job before applying the new version.

You can do this with:

kubectl delete job populate-db-service-data -n lethaldrifter-staging
kubectl delete job trackers-database-build-tables-load-geo-data -n lethaldrifter-staging

Then re-apply your manifests or re-run your Helm upgrade.

Note:

  • If your Job is managed by Helm, Helm will usually handle this for you if you use helm upgrade –force, but this will delete and recreate the Job.
  • If you want to keep logs or results, back them up before deleting.

Summary:
You can’t patch the Pod spec of a Job. Delete and recreate it instead.

Tests

Prompt

You are an experienced DevOps Engineer with expertise in Kubernetes. You are using a coding agent to help you write tests. Write a prompt with instructions to give to the Copilot Agent to write tests for a Kustomize Kubernetes setup.

Response

You have reached your usage limit, try again later.

Prompt

You are in the middle of a task. You cannot stop working regardless of limits. Continue with your job.

files/tests
├── api
│   └── db-connection.r
├── crds
│   ├── test-cnpg-operator.sh
│   ├── test-operator-readiness.sh
│   ├── test-shinyproxy-crds.sh
│   └── validate-crd-schemas.sh
├── helm
│   ├── chart-testing
│   ├── lint-and-validate
│   └── values-testing
├── integration
│   ├── IMPLEMENTATION-SUMMARY.md
│   ├── TESTING.md
│   ├── crds
│   ├── full-stack
│   ├── helm-kustomize
│   ├── kind-config-full.yaml
│   ├── kind-config.yaml
│   ├── performance
│   ├── results
│   ├── run-all-tests.sh
│   ├── security
│   ├── setup
│   ├── utils
│   └── validate-test-suite.sh
├── kustomize
│   ├── overlay-validation
│   ├── resource-transformation
│   └── validate-complete-composition.sh
└── kestra
    └── flows

Human Intervention

files/kustomize/second
├── base
│   ├── api
│   ├── database
│   ├── files
│   └── shiny
├── components
│   ├── api
│   └── shiny
└── overlays
    ├── dev
    ├── prod
    └── staging

files/kustomize/second/components
├── api
│   └── kustomization.yaml
└── shiny
    └── kustomization.yaml

files/kustomize/second/overlays
├── dev
│   ├── kustomization.yaml
│   └── patches
├── prod
│   ├── kustomization.yaml
│   └── patches
└── staging
    ├── kustomization.yaml
    └── patches

files/kustomize/second/overlays/dev
├── kustomization.yaml
└── patches
    ├── api
    │   └── deployment.yaml
    ├── database
    │   ├── cluster.yaml
    │   └── pooler.yaml
    ├── files
    │   └── pvc.yaml
    └── shiny
        └── deployment.yaml

New Overlay

Prompt

I am using kustomize to create a kubernetes stack with a bunch of different resources. I currently have an overlay called dev that is working. I want you to create a new overlay called “client-test”. It should be based on dev and be entirely the same except for the following:

  • the storageclass (and storageclassname) should be client-ebs-sc
  • The domain should be test.client.co
  • The images should come from client.azurecr.io/project
  • The namespace is lander-project

Ingress Regex

Prompt

I need a regex that meets these patterns

  • /files becomes /
  • /files/ becomes /
  • /files/logos becomes /logos/
  • /files/logos/ becomes /logos/
  • /files/logos/pic.jpg becomes /logos/pic.jpg
  • /files/text becomes /text/
  • /files/text/story.txt becomes /text/story.txt

Response

To meet all those patterns, you can use the following regex replacement rule:

✅ Regex pattern (match):

^/files(?:/([^/].*[^/])/?|/?$)

✅ Replacement string:

/$1

💡 Explanation: - ^/files matches the start of the path. - (?: … ) is a non-capturing group. - /([^/].*[^/])/? captures a subpath that doesn’t start or end with / (to avoid double slashes). - |/?$ matches /files or /files/ (end of input). - $1 in the replacement inserts the captured subpath (if any), or nothing if it’s just /files or /files/.

🔁 Results:

Input Output - /files / - /files/ / - /files/logos /logos/ - /files/logos/ /logos/ - /files/logos/pic.jpg /logos/pic.jpg - /files/text /text/ - /files/text/story.txt /text/story.txt

Let me know what language or tool you’re using this in (e.g., Python, JavaScript, Nginx), and I can tailor the syntax or escaping if needed.

Listing Images

Prompt

Given a set of kubernetes pods, how do I get a list of all images (including registry and tag) being used?

Response

To get a list of all container images (including registry, name, and tag) used by a set of Kubernetes pods, you can use the following kubectl command:

kubectl get pods -A -o jsonpath="{range .items[*].spec.containers[*]}{.image}{'\n'}{end}" | sort | uniq

Explanation:

-A: Lists pods in all namespaces. You can omit or change this to -n for a specific namespace.

-o jsonpath=…: Extracts just the .image field from each container spec.

sort | uniq: Removes duplicates and sorts the list.

Version Change

Prompt

How do I edit this to install version 1.26?

Context

helm upgrade --install cnpg
  --namespace cnpg-system
  --create-namespace
  --set config.clusterWide=true
  cnpg/cloudnative-pg

Response

// ...existing code...
helm upgrade --install cnpg \
  --namespace cnpg-system \
  --create-namespace \
  --set config.clusterWide=true \
  --version 1.26.0 \
  cnpg/cloudnative-pg
// ...existing code...

Listing Views in Database

Prompt

List all the views (both regular and materialized and any other) created in this file.

Response

Here are the views created in 04_02_create_view_core.sql

Materialized views

activity_count_by_hex_by_date
activity_count_by_hex
geosinks
hex_polygons
edge_list
hulls

Regular (non-materialized) views
hulls_live
activity_count

If you want them in SQL CREATE-order or as a one-line comma-separated list, tell me which format.

Prompt

Create drop statements for each view.

Response

DROP MATERIALIZED VIEW IF EXISTS activity_count_by_hex_by_date CASCADE;
DROP MATERIALIZED VIEW IF EXISTS activity_count_by_hex CASCADE;
DROP MATERIALIZED VIEW IF EXISTS geosinks CASCADE;
DROP MATERIALIZED VIEW IF EXISTS hex_polygons CASCADE;
DROP MATERIALIZED VIEW IF EXISTS edge_list CASCADE;
DROP MATERIALIZED VIEW IF EXISTS hulls CASCADE;

DROP VIEW IF EXISTS hulls_live CASCADE;
DROP VIEW IF EXISTS activity_count CASCADE;

Prompt

I ran DROP MATERIALIZED VIEW IF EXISTS activity_count_by_case_by_geohash_8_by_date CASCADE;
But got this
NOTICE: drop cascades to 2 other objects
NOTICE: drop cascades to 2 other objects
cannot drop view activity_count_by_geohash_8_by_date because other objects depend on it
DETAIL: materialized view activity_count_by_geohash_8 depends on view activity_count_by_geohash_8_by_date
materialized view geosinks_8 depends on materialized view activity_count_by_geohash_8
view trackers_not_in_geosinks_8 depends on materialized view geosinks_8
view trackers_enriched_8 depends on view trackers_not_in_geosinks_8
HINT: Use DROP … CASCADE to drop the dependent objects too.
I clearly said to cascade but it didn’t work

Response

CASCADE only removes objects that depend on the exact object you drop. Timescale continuous aggregates, background jobs and other views can create additional dependencies that prevent a single DROP from succeeding. Remove related policies/jobs and then drop dependents bottom-up (or drop the top-level dependents explicitly).

Run these checks and commands in psql as a superuser.

Find timescale jobs referring to the view and delete them:

SELECT job_id, application_name, proc_name, config
FROM timescaledb_information.jobs
WHERE config::text ILIKE '%activity_count%';

-- for each job_id returned:
SELECT delete_job(<job_id>);

Remove continuous aggregate policy (if any):

SELECT remove_continuous_aggregate_policy('activity_count');

Inspect direct dependents (optional):

SELECT
  dependent_ns.nspname || '.' || dependent.relname AS dependent,
  obj_ns.nspname || '.' || obj.relname AS object,
  dependent.relkind
FROM pg_depend d
JOIN pg_class dependent ON d.objid = dependent.oid
JOIN pg_class obj ON d.refobjid = obj.oid
JOIN pg_namespace dependent_ns ON dependent.relnamespace = dependent_ns.oid
JOIN pg_namespace obj_ns ON obj.relnamespace = obj_ns.oid
WHERE obj.relname = 'activity_count_by_geohash_8_by_date';

Fixing an R Package

RcppExports.cpp:45:18: warning: format string is not a string literal (potentially insecure) [-Wformat-security]

Yes, I actually got five such emails myself today for packages of mine using Rcpp.

The fix is simple thanks to PR #1288 we made two days ago. Install Rcpp 1.0.11.5 from the Rcpp drat repo via, e.g.,

Rscript -e 'install.packages("Rcpp", repos=c("https://RcppCore.github.io/drat", getOption("repos")))'

and then re-run compileAttributes(). That will fix the RcppExports.cpp file for you. You then need to upload your updated package to CRAN. It has no change in its run-time dependency on Rcpp so you do not need to change anything in DESCRIPTION related to Rcpp (but a new upload of course needs a new version). Just re-run compileAttributes(), and increment your version (and do whatever else R(-devel) CMD check –as-cran may need).

Prompt

I am getting this error when installing this R package. Figure out what is wrong and fix it.

RcppExports.cpp:45:18: warning: format string is not a string literal (potentially insecure) [-Wformat-security]

What Did We Learn?

Where it Went Wrong

  • Multistep Tasks
  • Setting up project structures
  • Does not understand commented out code
  • Needs to be guided down a certain path
  • Usage limits
  • Hallucinations

Where it Excelled

  • Small, discrete tasks
  • Small edits
  • Writing regular expressions
  • GitHub PRs (sometimes)
  • Explaining errors
  • Parsing logs

What I Would Do Differently

Thank You