Software Engineering Is About More Than Just the Code

It’s heartening to see the positivity coming from people learning to code at the bootcamps I follow, people who had previously considered coding to be reserved for the domain of ‘experts’, seeing the fruits of their efforts come to life is so encouraging. Just about anyone can learn to code, at least anyone who wants to learn. Diversity in software engineering teams is so very important. Obviously diversity is important everywhere but that’s too big a subject for this post, so I’m just focusing on building software. The importance of the work these bootcamps do to open up coding to people who may have followed a completely different career path cannot be overstated.

Thinking about what these bootcamp graduates have in front of them led me to remember that I was once asked to explain the IT industry. I remember that at the time, that question stopped me in my tracks, it’s such a huge, complex machine, how could anyone even attempt to start explaining it? I had a go at it anyway and it seemed to be marginally successful, at least the people I was explaining it to went away a little happier.

If anyone reading this hasn’t watched the film Hidden Figures (Film Wiki Page here) then I urge you to do so. Whilst it opens up the importance of the role that people like Katherine Johnson played in the history of science and technology to a wider audience (I think the women depicted in this film are up there with Rosa Parks), it also shows something about the origins of commercially available computers and the people that wrote the code for those first machines. Apart from anything else, it is just a really good film.

The way I see the IT Industry is a little like living cells splitting and replicating. The very first job in IT would have been programmers, like the ones depicted in the film. At some point, further down the line, some of those programmers would have started specialising in testing the code which would have started the discipline of software Quality Engineering. There would have been people that started specialising in running the computers, operating them, keeping them maintained, so was born the roles like System Administrators and Operational Support. As so it would have gone on, each of these jobs would have kept splitting into more specialist disciplines; Security, Database Administrators, Network Engineers, Business Analysts etc etc. Probably one of the more recent roles to split off and become its own thing is Platform Engineering, depending on the context often referred to as Site Reliability Engineering (SRE) or DevOps Engineering and I have no doubt as time goes on, more specialisms will appear. I think that just about all of the roles available within the IT industry today can probably trace their origins back to the people depicted in the Hidden Figures film.

One thing all these jobs have in common though is that they all contribute to the Software Value Chain. The idea of the Value Chain in this context is all the activities a business needs to carry out, to take a product or change to a product, from inception to delivery. Most businesses would like to see the Value Chain for their software delivery to be as efficient as possible and this drive for efficiency has in part driven the adoption of ‘the Cloud’. Probably the other main driver for the adoption of Cloud type resources is the flexibility and only paying for what is used.

So, goings back to the new Software Engineers launching themselves into their new careers, one of the first things they will need to understand is how what they do fits into the big picture and how the value chain works in their new company.

First of all, the days of the Software Engineer disappearing into a corner by themselves to work on their Software and then when it’s finished going ‘Taa Daa’ there you go, are pretty much done with. From a business point of view this way of working is full of risk. When will it be finished, how can we plan for other things that depend on it? What happens if the programmer is off sick or if they misunderstood the requirements? Basically the business will have no visibility of what the programmer is doing until the code is finished which is probably too late to fix any problems.

Most Software Engineers now work within teams or squads but most people coming out of the bootcamps know about working like this, they have already been pairing, writing tests and all that good stuff. One thing most Software Engineers are having to get more involved in, and I’m not sure the bootcamps cover this, is getting their software into production. Manually copying executables on to a production server, or running production software from someones laptop obviously isn’t going to be allowed within just about any company with any understanding of service or security. Deploying the software is now nearly always done through automated pipelines and Infrastructure as Code (IaC).

Whilst building automation around things like pipelines and IaC are typically regarded as the domain of those Platform Engineers (PEs), Software Engineers will often find themselves at least collaborating with the PEs doing the work and more often than not, there won’t be PE available and the Software Engineers will be doing this work themselves. The ability to build IaC and automation pipelines are becoming important tools in the Software Engineers toolbox.

IaC could be as simple as bash scripts although bash is not the best tool for this. Pipelines are usually built with specialist tools, like Jenkins, TeamCity, GoCD, and Concourse (and there are many more such tools). Then there are the tools like Ansible, Terraform, Puppet and Chef which are more typically used for the IaC and creating deployments, there are plenty more of these too. Whilst the IaC that gets written to work with these tools is usually just a series of configuration files, that configuration gets complicated enough for it to be close to coding, with things like conditionals and loop iteration often being part of the configuration syntax. Cloud providers will also often provide their own tooling to work with their infrastructure which makes the landscape even more cluttered, Cloud Formation from AWS and Bicep from Microsoft Azure are examples of this. Many organisations try to steer away from these provider specific tools though as they lock you in that one specific provider which isn’t always a good idea.

The concept of IaC encompasses both the configuration of pipelines and the deployments, and any other automation code or configuration (automated testing for example) that goes into getting an application into, and supporting it in production. IaC means as it says, treat the infrastructure as if it were application code. Don’t manually configure or build anything, write your IaC and put it in a repository. Test it, track changes, deploy it, built it using the same level of control as you would for the applications themselves, because it is just as important, just as critical.

CI/CD

This is a term I hear banded about from people who clearly don’t actually know what it means. They seem to lump it in with anything vaguely related to automation and it just isn’t that. To start with CI and CD are different things, CI means Continuous Integration. CD itself can mean two different things depending on the context, Continuous Delivery or Continuous Deployment.

Continuous Integration

Martin Fowler explains CI here much better than I can, but very briefly it means everyone committing their code to a common, shared code repository branch at least once a day, as opposed to everyone committing their code to a feature specific branch, only merging it with the integrated code branch when the feature is complete. It’s not saying don’t ever use feature branches, it just means they’re the exception rather than the rule. The idea is that with everyone working on the same branch, integration and merging becomes a non-event because everyone is committing/pushing often enough that conflicts are picked up before they get big. It also removes the overhead of having to mange all the feature branches. The definition of a branch isn’t straightforward either. What I consider a branch may be different to someone else’s definition. As well as the term branch meaning different things to different people, it can also mean different things within different tools. Again, Martin Fowler’s blog about the permutations around managing branches here, is another good explanation.

Continuous Delivery Vs Continuous Deployment

Continuous Delivery basically means keeping the software in a releasable state, so that it is ready to be released to production at any time.

This can’t really be done effectively without the use of automation; automated builds, automated tests, pushing each build to production like environments to ensure that the software will work as it is supposed to. The main difference between Continuous Delivery and Continuous Deployment is that Delivery is being ready to deploy, but not necessarily doing it, Continuous Deployment is where every successful build is actually deployed, with potentially several deployments happening each day.

Not many organisation do (or could do) Continuous Deployment, it requires mature, comprehensive, trusted automation and monitoring otherwise it’s like walking through a minefield with your fingers in your ears.

There is obviously a lot to all of this, all I’ve done here really is to say it’s there and throw some names and terms about. So for all of those new Software Engineers launching their careers, I’m just opening the door to the world beyond the functional application code. As well as the IaC and automation, there is code to support things like licensing, monitoring, that code that automatically downloads and installs updates to the app, managing configuration, and everyones favourite, the documentation. There is plenty more, I haven’t even mentioned things like choices around Serverless and Containerisation. There are so many things that are peripheral to the business logic code, but are still needed to make your app enterprise ready.

Don’t let this put you off, it’s all there to be learnt, and none of it is especially complicated or difficult in isolation. The complexity is in the amount of different aspects that there are. Your learning will never stop, if it does, you’re doing something wrong, lots of people know a lot, but nobody knows it all. The Software Engineering landscape is constantly changing and evolving and whilst some of that change disappears just as quickly as it arrives, just as much of that change stays the course and often fundamentally changes one of those many aspects I mentioned. Embrace the changes, challenge what you do and how you do it but make sure the change isn’t just for the sake of it, make sure there is value in doing it.

So, make sure you enjoy what you’re doing, don’t be afraid to move on if you’re bored (although even the best jobs will have a bit you won’t like), the opportunities are always there, do even the most mundane job to the best of your ability, be interested, and never stop learning.