Software systems rule our world. My regular newsletter explores the human factors that make software engineering so unique, so difficult, so important and all consuming.
The Human Software 285 - When does AI help, and when does it hinder?
|
Last couple of weeks, I've been rebuilding some Windows base images in order to comply with corporate patching policies. The new images are CIS hardened which means they follows guidelines set out by the Center for Internet Security benchmark. This ultimately means that the images are restricted in what they can do, what they can access, what is installed upon them by default. These security measures work in opposition to the automation we already have in place for our customers. This is the natural tension of security that we see in any form of software. Users need software to be useful, but adding unnecessary features and complexity increases the likelihood of exploitation. As an experiment, I asked Github Copilot to do the work for me to fix the problems with the Windows images. Copilot itself uses a variety of different LLM models (ChatGPT, Claude and others). None of the models could really help fix any of the weirdness that we saw happening. Essentially, the code suggestions it made cost me more time and effort in the long run and it disappeared down rabbit holes of code complexity which were actually dangerous in themselves. Eventually, I took to LinkedIn to vent my frustration. Full post below but here's the closing remark. Config management is where you need to be forensically accurate and focus on repeatability and a complete lack of surprise. Unsurprisingly this is where LLMs really show their limitations. Despite LLMs sucking for certain tasks, it definitely excels at quickly and effortlessly spotting patterns and explaining things to you that should be obvious. For example, as we near the end of the year I've been organising my thoughts. I asked ChatGPT to analyse my archive of blog posts and provide me with an an overview of what I like to talk about. It not only did that and it also drew me a picture to illustrate the themes I like to write about. This is not bad. It's not perfect, but I do like to talk about a lot of these things. Alongside what I write about, I also wanted to know more about how I could more effectively reach a wider audience. I'm experimenting with social media - for example I post quite a lot on LinkedIn and increasingly on BlueSky and now Instagram too. So likewise when I asked ChatGPT what kind of a newsletter this is it told me a few home truths about the kind of writer I am and what I should do more of. You are not a “5 tips to improve your sprint planning” type of writer. You’re a reflective, human-centered, philosophical, narrative-driven writer. So trying to force yourself into “value delivery content” is like asking a poet to publish “10 Productivity Hacks.” ChatGPT understands me. Should I be worried? Until next time. p.s. to celebrate Black Friday, I’ve introduced a special limited-time price across all retailers. If you've not picked up your copy of "Human Software" yet, be sure to grab a bargain! When to Say Stop to Copilot’s Unhelpful HallucinationsPublished on November 14, 2025 It’s quite a regular requirement in my day job that we update the Linux and Windows images associated with our EC2 instances in AWS. There’s a platform team dedicated to rolling out company-specific versions of our base images and recently it was, once again, time to update. So I’ve spent a week or so using… Read More »When to Say Stop to Copilot’s Unhelpful Hallucinations
|
The Human Software
Software systems rule our world. My regular newsletter explores the human factors that make software engineering so unique, so difficult, so important and all consuming.
Aside from being a writer, I'm also an avid film and TV watcher. This week we were blessed by the arrival of Pluribus on AppleTV. It's a speculative science fiction series written by Vince Gilligan, co-creator of Breaking Bad and Better Call Saul. He's someone who knows what they are doing when it comes to writing, producing and directing these incredibly successful series. More than that though, he picks the best people and directs them with ease. Here's a bit on when Vince Gilligan knew...
Greg Wilson of Software Carpentry fame knows how to write a headline. His recent talk Cocaine and Conway's Law is a mine of brilliant ideas and books to add to your reading list. They talk invested me immediately through his excoriation of Mark Andreessen's "Techno Optimist Manifesto" as a part of the Peter Thiel/Elon Musk narrative - work harder and longer, fix all problems via tech. Conway's Law, for those who are unfamiliar is the implicit link between social organisation of a company and...
The Horizon Post Office Scandal is one of the biggest IT failures in recent times, directly responsible for thirteen of the wrongly accused taking their own lives after prosecutions were brought against them. There is a highly technical deep dive into the findings made by Computer Weekly in this incredible Corecursive podcast episode. It is worth a detailed listen if you want to understand how this could come about and what systemic failings caused it to be covered up for so long. Software...