Introduction

I didn’t set out to write this book. I set out to teach a module on software engineering, something I’d done many times before, and discovered that the familiar words didn’t mean what I thought they meant.

I’m a software engineer by training and by instinct. I spent years building systems, debugging production incidents, and thinking in terms of deterministic inputs and outputs. When I moved into teaching, I brought that same mindset to software engineering degree apprenticeships: version control, testing, architecture, deployment. Familiar territory.

Then I was invited to teach a software engineering module on a data science degree apprenticeship. On paper, the fit was obvious. The learners needed to write reproducible Python, work with pandas and scikit-learn, structure their code properly, and test it. I could do all of that.

What I couldn’t do, not at first, was understand the world their code was operating in.

The moment it clicked was during a session on unit testing. I was explaining how to write tests for data pipeline code: assert that a function returns the expected output, check edge cases, verify that transformations are correct. Standard practice. But the learners kept asking questions about testing their models. Was the accuracy high enough? Were the predictions reliable? Were the results “correct”? We were using the same word, testing, and meaning entirely different things.

That gap turned out to be everywhere. Validation to a software engineer means checking inputs against a schema. To a data scientist, it means holding out data to evaluate a model’s generalisability. A model in software engineering is an abstraction of a domain: a class diagram, an entity relationship. In data science, it’s a learned function that makes predictions from data. Accuracy in engineering is binary: the system does what the spec says, or it doesn’t. In data science, accuracy is a metric on a continuum, and sometimes it’s not even the right metric to use.

I wasn’t wrong about these terms, and neither were my learners. We were working in overlapping vocabularies with different meanings, and neither side had a bridge between them.

Building bridges is something engineers do. Not just in the structural sense, but as a daily practice. You translate a customer’s problem into a technical solution. You turn technical detail into narrative so that stakeholders can act on it. The gap between software engineering and data science was the same kind of problem: two disciplines that share tools and vocabulary but think differently about fundamental questions.

So I started reading statistics textbooks, working through probability theory, and implementing the algorithms I was teaching around. The computational side came quickly; writing Python to fit a model or transform a dataset sits comfortably in a software engineer’s hands. The conceptual side was harder. Probability distributions feel like they should be analogous to type systems, but they aren’t. Hypothesis testing has the surface structure of debugging, right down to the null hypothesis that plays the role of “nothing is wrong,” but the logic runs differently. And Bayesian inference requires you to state what you believe before you see the data, which felt like the opposite of empiricism.

The more I learned, the more I noticed a pattern. The concepts that tripped me up weren’t difficult in the mathematical sense. They were difficult because they required a different mode of thinking. Software engineering trains you to eliminate uncertainty: pin your dependencies, make your builds reproducible, write assertions that pass or fail. Data science asks you to quantify uncertainty, model it, and make decisions in its presence. The shift isn’t from ignorance to knowledge. It’s from one kind of rigour to another.

This book is the bridge I built. Every chapter starts from something a software engineer already understands and works toward the statistical concept that extends or challenges it. Where an analogy between the two domains holds, I push it as far as it goes. Where it breaks down, I say so, because a misleading bridge is worse than no bridge at all.

The title, Thinking in Uncertainty, reflects the core skill this book teaches. Not any single algorithm or technique, but a way of reasoning about problems where the answer is never exact and the data is never complete. That’s uncomfortable for engineers. It was uncomfortable for me. But once you learn to work with uncertainty rather than against it, you make better decisions, and not just about data.

If you’re a software engineer who needs to work with data, evaluate models, or collaborate with data scientists, this book is where I’d start you off. If you’re just curious about what’s on the other side of the fence, it’s a good place to start too.

Matthew Gibbons
North Cornwall, UK
February 2026

--- number-sections: false --- # Introduction {.unnumbered} I didn't set out to write this book. I set out to teach a module on software engineering, something I'd done many times before, and discovered that the familiar words didn't mean what I thought they meant. I'm a software engineer by training and by instinct. I spent years building systems, debugging production incidents, and thinking in terms of deterministic inputs and outputs. When I moved into teaching, I brought that same mindset to software engineering degree apprenticeships: version control, testing, architecture, deployment. Familiar territory. Then I was invited to teach a software engineering module on a data science degree apprenticeship. On paper, the fit was obvious. The learners needed to write reproducible Python, work with `pandas` and `scikit-learn`, structure their code properly, and test it. I could do all of that. What I couldn't do, not at first, was understand the world their code was operating in. The moment it clicked was during a session on unit testing. I was explaining how to write tests for data pipeline code: assert that a function returns the expected output, check edge cases, verify that transformations are correct. Standard practice. But the learners kept asking questions about testing their *models*. Was the accuracy high enough? Were the predictions reliable? Were the results "correct"? We were using the same word, *testing*, and meaning entirely different things. That gap turned out to be everywhere. *Validation* to a software engineer means checking inputs against a schema. To a data scientist, it means holding out data to evaluate a model's generalisability. A *model* in software engineering is an abstraction of a domain: a class diagram, an entity relationship. In data science, it's a learned function that makes predictions from data. *Accuracy* in engineering is binary: the system does what the spec says, or it doesn't. In data science, accuracy is a metric on a continuum, and sometimes it's not even the right metric to use. I wasn't wrong about these terms, and neither were my learners. We were working in overlapping vocabularies with different meanings, and neither side had a bridge between them. Building bridges is something engineers do. Not just in the structural sense, but as a daily practice. You translate a customer's problem into a technical solution. You turn technical detail into narrative so that stakeholders can act on it. The gap between software engineering and data science was the same kind of problem: two disciplines that share tools and vocabulary but think differently about fundamental questions. So I started reading statistics textbooks, working through probability theory, and implementing the algorithms I was teaching around. The computational side came quickly; writing Python to fit a model or transform a dataset sits comfortably in a software engineer's hands. The conceptual side was harder. Probability distributions feel like they should be analogous to type systems, but they aren't. Hypothesis testing has the surface structure of debugging, right down to the null hypothesis that plays the role of "nothing is wrong," but the logic runs differently. And Bayesian inference requires you to state what you believe *before* you see the data, which felt like the opposite of empiricism. The more I learned, the more I noticed a pattern. The concepts that tripped me up weren't difficult in the mathematical sense. They were difficult because they required a different mode of thinking. Software engineering trains you to eliminate uncertainty: pin your dependencies, make your builds reproducible, write assertions that pass or fail. Data science asks you to *quantify* uncertainty, *model* it, and make decisions in its presence. The shift isn't from ignorance to knowledge. It's from one kind of rigour to another. This book is the bridge I built. Every chapter starts from something a software engineer already understands and works toward the statistical concept that extends or challenges it. Where an analogy between the two domains holds, I push it as far as it goes. Where it breaks down, I say so, because a misleading bridge is worse than no bridge at all. The title, *Thinking in Uncertainty*, reflects the core skill this book teaches. Not any single algorithm or technique, but a way of reasoning about problems where the answer is never exact and the data is never complete. That's uncomfortable for engineers. It was uncomfortable for me. But once you learn to work with uncertainty rather than against it, you make better decisions, and not just about data. If you're a software engineer who needs to work with data, evaluate models, or collaborate with data scientists, this book is where I'd start you off. If you're just curious about what's on the other side of the fence, it's a good place to start too. ::: {.text-end} Matthew Gibbons\ North Cornwall, UK\ February 2026 :::