During my first experience working on CI/CD pipelines I had to wait a lot. Some jobs took forever to finish and silly me just need to verify or test one job adjustment and off-course the job would fail due to misconfiguration or other stupid mistakes.

Well that meant I had a lot of time to figure out how to be productive while working on new pipeline jobs or maintenance.

Introduction

At first, I tried to comment out all unnecessary jobs or stages in the pipeline. That saved me some time during testing, but it came with a cost: I had to manually clean everything up afterwards. A bit of a hassle, but I got things done — a little faster.

Still, I was spending too much time waiting. So, more time to tinker!

Faster feedback loops

To speed things up, I realized I needed faster feedback. Running jobs locally seemed like the way to go.

Installing a runner or agent on my machine was an option, but felt overly complex for most of my use cases. Luckily, since my jobs mostly consisted of running Bash or Python scripts, I could easily execute them locally.

Here’s what I did:

my_hello_world_job:
	variable:
		MY_NAME:Sjors
	script: scripts/say_hello.sh

I moved my scripts into a scripts/ directory. Each script contains metadata like a short description and a list of requirements. I also added checks for environment variables, so it’s obvious when something is missing:

#!/bin/bash
# This is a simple script which outputs: hello to MY_NAME

if [[ -z "${MY_NAME}" ]]; then  
  echo "Environment variable MY_NAME is not set!"
  exit 1
fi

echo "Hello $MY_NAME!"

And boom! It prints Hello Sjors! — no need to commit and push to the pipeline just to test this.

Matching Environments

The only downside: you need to have the same dependencies locally as the runner.

This can be solved by using container images that include all required tools and versions. That way, your local environment mirrors the CI runner, ensuring consistent results.

The End (Or Beginning?)

This experience taught me a valuable lesson: keep your scripts external. Don’t write all your logic directly in your gitlab-ci.yml file — it becomes unreproducible locally and wastes both your time and your sanity.

For example, this might look simple:

my_hello_world_job:
	variable:
		MY_NAME:Sjors
	script: |
		echo "Hello $MY_NAME"

But it’s much harder to test locally than this:

./scripts/say_hello.sh

Trust me — your future self will thank you.

Bonus Tip

Want even more reproducibility? Wrap your script execution in a container using your CI image:

podman run --rm -e MY_NAME=Sjors -v $(pwd):/app my-ci-image bash /app/scripts/say_hello.sh

That way, you’re testing in the exact same environment as your CI/CD pipeline.

Note: bash is not installed by default in a Alpine image!

Conclusion

CI/CD pipelines don’t need to slow you down. A few simple tweaks — like extracting scripts, validating variables, and using containers — can drastically improve your feedback loop and productivity. Happy coding! 🚀