Building a Firmware CI/CD Pipeline from Scratch — Phase 1: The Foundation

Why Firmware Pipelines Are Stuck in the Past

Ask an embedded engineer how firmware gets tested before it ships. Nine times out of ten, the answer involves some version of: someone builds it, someone flashes it, someone checks if the LED blinks. Maybe they wrote down what they checked. Maybe they didn't.

This isn't because embedded engineers are bad at their jobs. It's because the tooling that made CI/CD standard practice in web and mobile development took a long time to reach the hardware world. Containerized builds, automated test runners, reproducible environments. All of that is matured in software-only contexts where the test environment was just another process on a server. When your test environment is a physical microcontroller that needs to be flashed, things get complicated.

The result is that most embedded teams are still running manual QA workflows that wouldn't pass muster in any other engineering discipline. And "it works on my machine" hits different when "my machine" is a specific dev board on a specific engineer's desk with a specific version of a toolchain they installed three years ago and never documented.

This project exists to fix that, for one codebase, built in public, fully documented. By the time this series wraps, there will be a working firmware CI/CD pipeline with hardware-in-the-loop testing, signed firmware images, and staged OTA rollout with automatic rollback. This is Phase 1. We're laying the foundation.

What Phase 1 Actually Delivers

Before getting into how, here's exactly what exists at the end of this phase:

• A reproducible firmware project using PlatformIO, buildable on any machine without manual environment setup
• Automated firmware builds running in GitHub Actions on every push
• Python-based hardware test scripts that validate real board behavior locally
• A repository structure designed to scale cleanly through the remaining phases

That last point matters more than it looks. A pipeline you build on a shaky foundation will fight you at every phase. The repo structure you pick now is the one you're living with when the hardware-in-the-loop testing, the self-hosted runner, and the OTA rollout all come online. Getting it right at the start is cheaper than refactoring it later.

The Tools and Why They Were Chosen

PlatformIO is the build system. If you've spent time in the Arduino ecosystem you've probably seen it. It's the tool that adds professional-grade project management, dependency handling, and multi-platform build support to embedded development without requiring you to fight CMake or hand-roll Makefiles. The critical capability here is that PlatformIO build environments are reproducible and portable. When GitHub Actions runs a build, it runs the same build you run locally, with the same toolchain version, targeting the same platform definitions. No "works on my machine" situations.

GitHub Actions is the CI platform. It's free for public repositories, it has a massive library of community actions, and it supports self-hosted runners, which becomes important in a later phase when we need to physically attach hardware to the pipeline. Using it from Phase 1 means the workflow config files are already in place and working by the time we need to extend them.

Python for the hardware test scripts. Not because it's exotic, but because it's practical. Python's pyserial library handles serial communication with microcontrollers cleanly, the test scripts are readable by anyone on the team regardless of their primary language, and the same scripts that run locally will run on the self-hosted runner in Phase 3 without modification.

The Repository Structure

firmware-ci-demo/
├── .github/
│ └── workflows/
│ └── firmware.yml ← GitHub Actions pipeline definition
├── .pio/ ← PlatformIO internal files (gitignored)
├── .vscode/ ← VS Code config for PlatformIO IDE integration
├── docs/ ← Documentation, architecture notes
├── firmware/ ← Firmware source (in progress)
├── hardware/
│ ├── flash.py ← Script to flash firmware to a connected board
│ └── serial_test.py ← Hardware validation test script
├── include/ ← Shared header files
├── lib/ ← Project-local libraries
├── scripts/ ← Build utility scripts
├── src/
│ └── main.cpp ← Application firmware source
├── test/ ← PlatformIO unit tests
├── .gitignore
├── platformio.ini ← Project config, platform and board targets
└── README.md

The separation between src/ (firmware), hardware/ (host-side test tooling), and .github/workflows/ (CI config) is intentional. Each layer of the pipeline has its own home. When Phase 3 adds a self-hosted runner config and Phase 4 adds OTA tooling, they slot into this structure without displacing anything that already exists.

platformio.ini is the single source of truth for what platforms and boards this project targets. Right now it defines three environments:

[platformio]
default_envs = esp32dev, bluepill_f103c8, native

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino

[env:bluepill_f103c8]
platform = ststm32
board = bluepill_f103c8
framework = arduino

[env:native]
platform = native

Three targets, three very different environments. The ESP32 is the primary dev board for this project — the one that will eventually be physically wired into the CI system. The STM32 Blue Pill is there to prove the build system handles multi-platform firmware correctly. The native target is for running unit tests without hardware, which becomes critical in Phase 2.

The Build Results (and Why Two Failures Are Actually Fine)

Looking at the build output, esp32dev passes. bluepill_f103c8 and native both fail. At Phase 1, this is expected and intentional, it's information, not a problem.

The STM32 failure is a platform dependency issue that gets resolved as the firmware code matures. The native failure is essentially a placeholder — native builds require the unit test framework to be properly configured, which is Phase 2 work. Both failures are tracked, both have known resolution paths, and neither of them blocks the ESP32 build that's actually being actively developed.

This is one of the important distinctions between CI pipelines that are useful and CI pipelines that are theater. A failing build in CI isn't always a fire drill. Sometimes it's a build configuration that's intentionally not complete yet. The pipeline's job is to report accurate state, not to always be green. A team that only merges when everything is green has actually hidden a lot of information about what's in progress.

The GitHub Actions Workflow

name: Firmware CI

on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]

jobs:
build:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3

- name: Cache PlatformIO
uses: actions/cache@v3
with:
path: ~/.platformio
key: ${{ runner.os }}-pio

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'

- name: Install PlatformIO
run: pip install platformio

- name: Build firmware (all environments)
run: pio run

Every push to main or develop and every pull request triggers a full build across all configured environments. The PlatformIO cache step is important — without it, CI would download and compile the full ESP32 and STM32 toolchains on every single run, which adds several minutes of pure toolchain-install time to every build. With caching, after the first run the toolchain is already there and builds run significantly faster.

The workflow is intentionally minimal right now. No test execution, no firmware upload, no artifact publishing. Those come in Phases 2 and 3. What exists here is a working CI gate that proves the firmware compiles on a clean machine every time code is pushed. That alone catches a category of bugs that manual builds miss constantly: environment-specific issues, missing headers, accidental platform-specific code that only compiles on the dev machine.

The Hardware Test Scripts: Running Locally Before They Run in CI

serial_test.py is the first real hardware validation script. It connects to the ESP32 over USB serial, reads the output, and validates that the board is behaving correctly. Right now it's checking a blink pattern — LED ON/OFF cycles and confirming they match expected timing and sequence.

The output shows exactly what's happening: a series of LED ON and LED OFF messages, a BOOT OK confirmation, and a final PASS. When this test fails, the output shows FAIL and the point where the expected behavior diverged from actual behavior.

import serial
import time

SERIAL_PORT = '/dev/tty.usbserial-0001'
BAUD_RATE = 115200
TIMEOUT = 10

def run_test():
with serial.Serial(SERIAL_PORT, BAUD_RATE, timeout=TIMEOUT) as ser:
time.sleep(2) # Wait for board reset
output = []

for _ in range(30):
line = ser.readline().decode('utf-8').strip()
if line:
output.append(line)
print(line)

# Validate boot
if 'BOOT OK' not in output:
print('FAIL — no boot confirmation received')
return False

# Validate blink pattern
blink_lines = [l for l in output if l.startswith('LED')]
if len(blink_lines) < 10:
print(f'FAIL — expected at least 10 LED state changes, got {len(blink_lines)}')
return False

print('PASS')
return True

if __name__ == '__main__':
result = run_test()
exit(0 if result else 1)

Exit codes matter here. exit(0) for pass, exit(1) for fail. When this script runs in CI later, the exit code is what tells the pipeline whether the hardware test succeeded or failed. Getting that right from the start means zero changes to the script when it moves from local development to the runner.

This is the test living locally for now. In Phase 3, when the self-hosted runner has the ESP32 physically attached via USB, this same script runs automatically on every push, against real hardware, not a simulator.

What's Not Here Yet (and Why That's Deliberate)

Phase 1 doesn't include:

Unit tests. PlatformIO has a native unit test framework and the native build target is already configured for it. But unit tests against firmware code are only useful once there's enough firmware logic to test. Writing tests for a blink demo is a waste of time. Phase 2 adds meaningful firmware functionality and the unit test suite alongside it.

A self-hosted runner. GitHub's hosted ubuntu-latest runner can't physically connect to a microcontroller. That hardware-in-the-loop capability requires a self-hosted runner, a machine that has the ESP32 wired to it, with a custom runner configuration. That's Phase 3. Building it before the test scripts are solid would be assembling the factory before finishing the product.

Firmware signing. Signing firmware images for secure boot verification is the right answer for any production deployment. It's also something that needs to come after the basic build and test pipeline is stable, because the signing toolchain adds complexity to the build step and the verification has to match the target platform's secure boot implementation. That's Phase 4 territory.

The phases are sequenced so that each one produces something real and working before the next one adds complexity. A pipeline that's 40% built but fully functional at 40% is more useful than one that has all the pieces in place but nothing working end-to-end.

What This Looks Like to a Hiring Manager or Technical Lead

If you're reviewing this project from a hiring or technical leadership perspective, here's the signal this phase is sending.

The toolchain choice shows platform awareness. PlatformIO is the right tool for a project that needs to build for multiple MCU architectures from a single codebase, and knowing why matters more than just knowing how to use it.

The repo structure shows pipeline thinking. The separation of concerns between firmware source, host-side tooling, and CI configuration isn't just tidiness, it's the structure you need to add phases without breaking what already exists.

The failing builds being tracked and documented rather than hidden shows engineering maturity. A candidate who only shows you green builds hasn't built a real project. Real projects have known failures in flight. Knowing the difference between "failing and untracked" and "failing and intentional" is a judgment call that separates senior engineers from junior ones.

And the hardware test scripts existing from Phase 1. Before there's a runner to run them on, shows planning. The scripts are already written to exit with the right codes and produce the right output format. Plugging them into the CI pipeline in Phase 3 is a configuration change, not a rewrite.

What's Coming in Phase 2

Phase 2 is where the firmware gets real. The blink demo becomes a more complete application with actual logic to test, PlatformIO's native unit test framework comes online, and the GitHub Actions workflow gets extended to run those tests on every push.

By the end of Phase 2 the pipeline will be catching logic errors in CI before they ever make it to hardware. That's when the investment in Phase 1's structure starts paying obvious dividends.

Follow the series to see it get built.

Phase 1 Deliverables: Quick Reference