Producing work or value?

I’ve been working as an “IT architect” (with apologies to Architects who bear this title as part of a regulated profession, something for another post, so excuse the digression) for a long time. In this field we’re working with various abstraction levels, fancy language that talks of “Business and IT alignment”, viewpoints, current and target states, solutions, systems, software, data, information and what not. All that work produced lots and lots of documentation, which was valuable in its own right in that it provided a track record and history of how an enterprise had grown its systems over time. It captured what was significant in terms of investment and what were meaningful differentiating factors that resonated with the enterprise. A reflection of the culture and mindset at which it operated in a larger society. The documents were of utmost importance, to demonstrate real thought had gone into consequential decision making to support boards and decision makers justify their coarse of action.

Becoming an architect myself

I’ve grown up in the field of software engineering, starting late-last-century (yes, that long ago) building software in various settings (mostly financial services). During those times architects were the senior engineers on the projects. This used to be somewhat of a mystery but seeing them at work I rapidly understood that you had to have affinity with the bigger picture of the systems you’re developing beyond pure technical expertise. By brining this vast amount of experience to decision making in novel terrain, it proved valuable way of navigating uncertainty at the time. By building my experience, over time I got respected for my views and somehow I got in the drivers seat for a lot of the technical decision making. Not a job change, just something you do. I’ve always seen that as the best position to be in, architect and engineer, close to the subject matter, preferably hands on. This makes decision making faster and keeps you well-connected to the problem domain.

Growing systems

If the product you’re working on is successful, the architecture needs to scale from product, to system, to ecosystem and along the way needs to survive the “internal transformation of the organization” that is delivering the product/service as well. Products will be optimized for delivering margins at scale, by removing distinctive yet costly features, and focus on the common core, the commodity, attempting to disallow disruption and allowing creation of higher order products to emerge. This is basically the journey from start-up to enterprise, we know this from Wardely mapping.

Ineffective or inefficient?

Probably both! My personal experience in working in very large enterprises is that we’re not that good in managing the current and target state of the systems we’re documenting. So besides tech-debt we’ll see a lot of architecture-debt. This shows up as :

• Outdated decisions that limit progress
• “Design-by-committee architectures” that inhibit actual progress and innovation to be delivered
• Reference to technologies that have been retired long ago but still drive consequences for new initiatives
• Principles that seemed to work when the Internet was not yet public
• Complete and utter gibberish that has lost all meaning without the context it maybe once was valid for

This maybe a good demonstration of “The road to hell is paved in good intentions”, where at a previous point in time, we worked diligently to document solutions, and were proud on the result, but currently are constraining the options negatively. We actively create our own bumpy road ahead and are the source of our own headaches. That is both ineffective and inefficient.

Should that limit us?

In larger companies architecture is often treated as “the law of the land” and together with standards and guidelines provides a harness that guides current decision making. That harness is the result of thousands-and-thousands of decisions that have been built up over time. These are not just a document of nostalgia, but also a source of inertia. That nicely captures the reason start-ups can move so much faster, they don’t care as much about prior decisions. ( Another point here is that decisions are tied to people that are proud to be called “the Architects”. Some of these people tend to be sensitive about their names being tied to these decisions, which leads me to believe there is a bit too much ego at play in these matters )

Challenging the status-quo

So lately, I’ve been reflecting on these prior observations and have been thinking on how to go about this from now on. We’re on the advent of the AI age, a place in time where we can “at will” generate the architecture documentation at very low cost and with better quality than most of us would be able to produce. This also applies to the fact that changing systems has become much cheaper and faster through the use of AI. This implies that: “What we used to document thoroughly because it was considered consequential” may be “less consequential” right now.Leading to the question: Then why do we still act the same?

What must change

So, this is in my opinion where the profession of IT architecture needs to change. We cannot continue our old ways, we otherwise will never be able to catch up to the reality of where companies need the input of the IT architect. BTW That is one thing I do not doubt, that this job of IT architect will still be relevant many years from now.

The shortlist of things to change:

• Adopt a process that allows “Architecture in the small”, use ADRs use documents like the Amazon 5-pager, provide agility
• Build a loosely coupled network of decisions, compose these into an acyclic directed graph so you can provide multiple perspectives/viewpoints on the same subject matter
• Document artifacts using open standards to provide interoperability and longevity of artifacts
• Implement continuous architecture to evolve current state along actual development and set better goals for target state
• Every architecture artifact needs to have an expiry date, once expired it needs to be archived and may no longer be referenced, unless the expiry is renewed and the artifact reinstated
• Every architecture artifact that points to an expired artifact needs to be either expired as well, tagged for renewal or updated automatically.
• Every IT architect needs to (learn and) use AI

These improvements are providing the basis of what I’d call “High Fidelity Architecture”, for short “HiFi Architecture”. It will be more relevant, a better representation of where you are and thus providing a better frame of reference (a map!) for deciding where to go next. This is also where IT architecture originally promised to take you, but never really could deliver that at scale.

On being vulnerable

I wrote that “Every IT architect needs to (learn and) use AI” and given the personal experience I have this is a must. I know that this may instill a feeling of inadequacy in some of us, where we may think: “Is the machine really better than me?”. I’m telling you, you should ignore that feeling because the answer for certain skills is “Yes!”. But that doesn’t mean that you as an individual have become useless, no, to the contrary. You have become more valuable, because by mastering these tools, you can express your thoughts and reasoning faster, with more impact. Especially when there is a need to build consensus among stakeholders, having the ability to quickly draw up alternative solutions and adjust/navigate complex or political issues, you as architect are providing the fine-judgement that makes that happen. This literally makes the time between ideation and value delivery shorter.

Use AI on what its good at

The way to scale yourself is to use AI as a tool that creates leverage and relevant output based on large sets of input. AI can digest more information than you, provide more consistency and is relentless in polishing and shaping stories. It is not creative in the way that you see things, it doesn’t understand connections you may understand, but it can help you shape and fit solutions to problems in ways that would take you days or months if you had to bring all the details together. That’s where it excels and that is what you should aim to master.

Scrutinize your own work, bring a “growth mindset”

The area where you really need to get comfortable with the use of AI is when you ask it to review your work (preferably from different perspectives). For example check logical consistency, writing consistency, or prioritization of decisions, alignment with standards and applicable law, and even the way that you’ve framed a problem. When it comes back with a mediocre score, that may feel very uncomfortable at first but bringing a “growth mindset” can actually help you here and improve the quality and fidelity of the work you’re presenting. I remember my first dabblings with this approach and it was enlightening and a bit humbling, but also necessary for me to double down and keep improving my own writing and thinking.

Value-added work

AI is really good at producing output given the right instructions. This is also where I think all “current state architecture” should be a function of directly generated output. We as architects should not be documenting things that we’ve built already, it should be generated as a consequence of building a target state and deploying it. That’s how we connect direction setting from where we are today, to meet the next set of ambitions of our companies which is the true value of architecture.

Wrapping up

There are still many (IT Architects) that doubt AI for the presence of hallucinations and things going “off the rails”, which to a degree is true. However if we want to advance our field and meet the acceleration AI provides in software engineering and product development, we then need to take a good look in the mirror and recognize “what we’ve been producing was not always the greatest work either”. So, we have the opportunity and it’s a matter of learning, and start chasing that dot on the horizon.

As always, let me know what you think!