bsnel.com

  1. Prologue · Union Bay, Seattle

    Brian Snelgrove

    Software Developer · DevOps Engineer · Architect

    June 2016: a dinghy, a pair of oars, and a dock by the University of Washington. I didn’t know how to sail yet.

    Kubernetes is Greek for helmsman. This is the voyage version of my career; the pipeline view has every detail.

  2. Act I · Helpdesk Cay

    Learning to sail

    Every ticket that washed ashore taught me something about the place: every lab, every person, every machine, and every way they break.

    Ashore

    Helpdesk Technician

    Full chapter

  3. Act I · Helpdesk Cay

    Catching the wind

    Rowing one ticket at a time doesn’t scale, so I raised a sail: scripts that did the work for me, until one command onboarded a new employee end to end.

    Ashore

    Automating myself out of tickets

    Tools aboard

    Full chapter

  4. Act II · Boatyard Island

    Terraform builds the fleet

    Nothing leaves this yard until Terraform has drawn it: 26 blueprints for the whole estate, every hull cast from a Packer mold on our own vSphere timber. Then AWX blows the whistle, and Ansible’s riggers run Kubespray to hoist the same flag on every ship, so they sail as one fleet.

    Ashore

    Terraform: the whole estate as code

    state roots
    26
    managed resources
    ~600
    providers
    24

    Tools aboard

    Full chapter

  5. Act II · Boatyard Island

    Taking the wheel

    Kubernetes is Greek for helmsman. She slid down the slipway and I took the wheel: four clusters, 46 nodes, all on hardware we already own.

    Ashore

    Kubernetes on hardware we own

    clusters
    4
    Kubernetes nodes
    46
    managed-cluster fees
    $0

    Tools aboard

    Full chapter

  6. Act III · Muster Cove

    Enlisting a crew

    A fleet needs a crew. Developers sign on with a few lines of YAML and become fleet-sworn crew. Deckhands appear when there’s work and vanish when it’s done, at $0 a minute.

    Ashore

    On-prem CI: builds that cost $0 a minute

    per runner-minute
    $0
    reusable workflows
    21
    composite actions
    7

    Tools aboard

    Full chapter

  7. Act III · Muster Cove

    The supply line

    Flux ferries every new image from the GitHub harbor to the fleet. Nobody climbs aboard to rearrange the cargo; the manifest in Git decides.

    Ashore

    FluxCD: the bridge between GitHub and Kubernetes

    apps deployed by Flux
    ~53
    automated image policies
    102

    Tools aboard

    Full chapter

  8. Act III · Open water

    The whirlpool

    Every release sails past the whirlpool, and none of them goes down: before a ship moves, its databases are backed up and a buoy is dropped in S3.

    Ashore

    A safety net under every release

    MongoDB recovery point
    10 min
    of backup history
    ~1 yr
    to restore a database
    $0.004

    Tools aboard

    Full chapter

  9. Act III · Open water

    Cannon fire

    Raiders never stop firing. Dependabot patches the hull before most shots land, and Trivy keeps watch on three decks.

    Ashore

    Patched before it’s a problem

    layers scanned by Trivy
    3
    ecosystems kept current by Dependabot
    8
    node2shell: triage to verified fix
    < 3 h

    Tools aboard

    Full chapter

  10. Act III · Lighthouse Rock

    The lighthouse

    If a disk, a node or a database starts going bad, the lighthouse sees it first and rings the right bell.

    Ashore

    Watching the whole fleet

    custom alert rules
    72
    database alerts
    25
    HPC hardware alerts
    24

    Tools aboard

    Full chapter

  11. Act IV · Night crossing

    The asteroid

    Then AI hit the water. I built the breakwater first: a Claude Code plugin marketplace with three rules no wave gets past.

    Ashore

    Claude Code skills, delivered to every scientist

    skills in the CI/CD plugin
    14
    secrets on disk
    0
    read-only cluster MCP servers
    2

    Tools aboard

    Full chapter

  12. Act IV · Night crossing

    The wake

    The wake brought a flotilla of apps from people who had never shipped before, and the fleet didn’t rock. Ten years ago, I was the one in the dinghy.

    Ashore

    From idea to production in one afternoon

    1. 14:56Plugin installed The BRI CI/CD plugin, from the marketplace, in Claude.
    2. 15:48Deploy PR opened Container, pipeline, secrets and GitOps manifests, generated and reviewed.
    3. 16:14Live on staging Built once, scanned, rolled out and verified by the pipeline.
    4. 16:34Live in production Backed up, promoted by retag, rollout verified.
    pull request → production
    46 min
    plugin install → production
    1h 38m
    apps shipped by accounting
    2

    Tools aboard

    Full chapter

  13. Act IV · Lab Island

    Lab coats aboard

    Scientists joined the fleet with workspaces of their own. The $25K-a-year license stayed on the dock.

    Ashore

    Coder: $25K a year in licenses, now $0

    a year in IDE licenses
    $25K → $0
    workspaces
    ~52
    cores · GB per workspace
    48 · 360

    Tools aboard

    Full chapter

  14. Act IV · The depths

    Cheaper, deeper water

    Most sequencing data goes cold, so we let it sink to cheaper water: the same data, one recall away.

    Ashore

    Data at scale on a nonprofit budget

    1. Day 0$1,210 / month 50 TiB lands in the frequent-access tier, cached on-prem for speed.
    2. Day 90$217 / month Untouched objects slide into the archive tier automatically.
    3. Day 180$83 / month Deep archive: 93% cheaper than day one, still one recall away.
    cheaper after 180 days
    93%
    files on JuiceFS
    ~3.8M
    primary storage reclaimed
    ~76 TB

    Tools aboard

    Full chapter

  15. Act V · Ships I built

    The purser’s ship

    Those 2019 onboarding scripts grew into Humani, the ship that keeps everyone’s pay and papers in sync between ports. I own all of it.

    Ashore

    Humani: HR & payroll integrations

    lines of TypeScript
    ~40K
    on PostgreSQL 16, Oct 2026
    v5.0
    golden-file differences
    0

    Tools aboard

    Full chapter

  16. Act V · Ships I built

    The cartographer

    Before anything is deleted, dater charts every chest on five islands, hashes every byte on both copies, and waits for a second signature.

    Ashore

    dater: a catalog and mover for genomics data

    lines of Go + TypeScript
    ~100K
    Go test files
    110
    of duplicates found
    41 TiB

    Tools aboard

    Full chapter

  17. Act VI · Treasure Island

    X marks the spot

    A nonprofit’s dollars belong to the science. Add it all up and the chest holds the savings, every year.

    Ashore

    The savings ledger

    Full chapter

  18. Act VI · Treasure Island

    Dressing the ship

    If it has a power cable, a login or a deploy button, I’ve probably had a hand in it. One flag for each.

    Ashore

    Fingers in many pies

    1. 01Networking Load balancers, VPN, guest Wi-Fi, DNS
    2. 02Storage & DR Filers, JuiceFS, S3, warm standby
    3. 03Research computing HPC nodes, Coder, pipelines
    4. 04Identity Single sign-on, key vaults, per-person cluster identities
    5. 05Security Dependabot, Trivy at every layer, CVE fixes in hours, patient-data guardrails
    6. 06HR & finance Humani, employee lifecycle, payroll files
    7. 07Web The public website on Kubernetes
    8. 08Research apps ~53 apps through one pipeline
    9. 09Observability Prometheus, Grafana, Loki and Alertmanager → Slack
    10. 10AI Claude Code plugins, Claude in Coder
    Full chapter

Logos and names are trademarks of their respective owners, shown only to identify the tools I use; no endorsement is implied.