Architecting Architecture: Makers and Takers

Step 2 out of 10: RFP

The step has been made. Not sure where to, but for sure from the point of no return. Keep calm and keep walking. It is about time to look around and understand the smelly and slippery route before you. And what are those noisy creatures swarming around our fishy “innovative” design we called Mandelbrot blueprint? You don't get a buzzing noise like that, just buzzing and buzzing, without its meaning something.

A part of the noise is produced by your potential customers. They rustle RFPs, walk like goose showing off golden eggs, poke in calendars and always smile. They always smile. Above the piles of goose feathers and fish scale swarm lawyers, consultants, marketers and analysts on catalysts. Here is where the smell comes from. Once you approach the customer and try to look at the structure from the inside, it instantly becomes clear that nothing is clear. Especially the age and titles of the people around you. In startups, everyone is at their 20^th, experts and managers around the age of 40 - straight forward hierarchy picture without rank patches. But in a corporation, you will be greeted by fresh graduate Tigress managing a department filled with Masters Shifu. Brutally looking Panda Po is polishing windows accompanied by an enormous vacuum cleaner. A Mantis in a suit and a tie staring at a multimonitor desktop without ever blinking an eye can easily be a security officer or an accountant. Monkey will look at you and close a door keeping his secret. Shiny Viper stuck somewhere in between the age of 40 and infinity with a devil touch in the eyes and silver touch in the hair will lead you to a large video conference room. Who is he and what is his role - that is a question you will be solving the whole meeting.

Let’s get back to our honey pot. Big business is big. Size does matter. It can grow in different ways and this greatly affects the structure. Somewhere it was merged, somewhere absorption, and somewhere cooperation. This will result in the number of heads and legs spread all over the world. But there is always a common feature - layering. Matrix or hierarchy is not that important. The main thing is to understand that any contact with the outside world triggers a process of internal communication upward. And moving up – is moving against the flow. From top to bottom, everything flows down very quickly. By the way, this is the answer to why it is so slippery. But upward - only under heavy pressure.

Einstein clearly came to his theories for a reason. Time in an enterprise is relative. Project presentation exactly 2 hours? An hourglass will be placed in front of you, and you will be kindly asked to pack a bit earlier in order not to delay next presenter. You have sent an urgent request to clarify the details? Someday they will answer you. The pilot stage of the project is exactly a week and "for sure it will only in this environment on those 2 machines"? It will be examined, and monkey tested for at least a month in 20 different places. Yes, it is just a demo, and you are in different time zone and it is 5 a.m. now, but anyway there will be an urgent call with “a button is not grey enough when disabled”. As you probably understood: your time and theirs are two different entities. Had no time to express an idea or ask a question – talk to the wall. If a question is raised somewhere at the top of the rank pyramid, it will fall on you instantly and precisely. With these constraints one of the main tasks is to find key figures. Inside this hive you need to find experienced and technical people. And they are usually not invited to the meetings. But with every question you ask, you will notice that smiling people look for the phone or email and fetch information from somewhere else. You need those who put this information there. We need a holy grail to summon the knights of this kingdom. Usually, the grail is to be found in DR. A couple of questions about the behavior in unforeseen circumstances and some expert will take the stage. You stick to the expert and fetch as much information as you can and with some luck also contact details. If the person who has entered a room hands you over a business card, then most likely he is another smiley, useless to our case. Because any expert will probably have a rage face. Taken away from his desk for the sake of some clown, who obviously accidentally combined the words he heard somewhere into a summoning spell.  A good engineer does not like context switches. Therefore, he does not smoke and drinks tea from a huge mug.

The higher the pyramid, the further the heads from the brains. In huge companies, everyone has their own narrow corridor without doors and windows. The one who accepts the application for the tender and the one who is evaluating it can be in competing departments. The person who makes the requirements will most likely never touch the system. And probably has zero expertise on subject. But now will subject yours. For example, the director of the IT department will conduct a tender for mobile devices for counting goods in a tomato warehouse. Why exactly IT? Well, because the vice president of the sales department is above him in the food chain. And VP has reports that the profits fall due to an incorrect assessment of the inventory and stock. VP decides to do something exciting today and search his salesfalse-coprosocial network for magic keywords: "innovation" and "cloud" (and we all know that nothing happens nowadays without a cloud). He ends up with IT. It does not matter that IT simply stores pictures from corporate events in the cloud or maybe handles user administration in some “Office 256”. Therefore, IT directors’ primary goals are taking no responsibility and understanding at least something. As this is a very popular scenario, my advice to you: most important arrows in the diagrams at the first stages - communication. What port to open and what will be the deployment model for hardware - this is what he knows and wants to see. He is also responsible for security. You guessed it right, it does not mean asymmetric encryption or non-colliding hashes. Port 443, a couple of certificates and a database with many backups. Serve him something like a stone age painting. Primitive, but the plot is clear.

Fear and loathing in Enterprise, without them a frame of the architecture will not be complete:

• There is no single contact to get more details. You should start creating an address book. Only in a couple of projects did I manage to work with appropriate client departments from the beginning. An architect against/with an architect. In common case, you need to collect data for integrations and interfaces piece by piece.

• Time is money. If you missed the deadline, goodbye. Better without details and partially but served in time. If it was worth something, they will contact you. Therefore, dive as little as possible. The lack of details is not only acceleration, but also the freedom to choose them further on.

• There will be changes. Always. Development cycle of the system can be a couple of years. Another rule of a thumb – expect regulations and laws to change every 3 years. And these are never logical and never expected. Like the Spanish Inquisition. You cannot be prepared so do not try to.

• Size matters. If your system requires installation on client machines, then keep in mind that there may be thousands of them. Modern and old ones. Yes, even with CRT display and running Windows XP or even CE in some handhelds of gov. structures. In this case you declared some futuristic TLS 1.3 it will result in handwritten encryption and all sorts of termination proxies.

• Get a presentation designer.  Does not matter how good your architecture is, does matter how it looks. Make those lines and blocks look fancy.

• Get a UI designer. UI is the most important part. It is what these managers see, skipping all your technical details and documentation. The evaluation committee will not use system, but they pay for one. It just has to look better than one of competitors. 

• Anything you say can be used against you. Minutes of conversations and meeting summaries in conversations with lawyers. If you mentioned real-time just for a pitch, and then in the final product there will be some fixed delay – time to discuss the fines. Fear and surprise. And Lawyers. I had a pleasure of discussing a cluster failure in the presence of lawyers from both sides. And one "neighboring" project got first a notification letter for non-compliance with the SLA in performance that may result in project termination. Team has not managed to improve performance after a couple of additional letters and wave of layoffs wiped out a floor. And we talk about 1% delay above expected response time. The client just wanted to break up and found a legitimate reason. So, no commitments and no premature fine tuning. You do not know exactly whether one server can take it or you will need 2 - indicate in the application that you need 3, but it may vary based on… and depends on ...

• Vendor lock – 50 shades of hate. The Enterprise does not tolerate this. As many industrial standards and open protocols as possible.

• Vendor lock - they don't like it. Really. There will be separate tenders for supply, service, support, etc. You can win in several or in one. Often you supply software, someone else hardware, someone else will install and maintain it. The best option is when you have a black box for on-prem - your company does not sell software separately. But anyway, the support contract will be handled to someone else. Do not rely on a specific version of the software and that there will be no competition for resources. Be stateless wherever possible.

• Vendor lock is one of the most important. Even open-source technologies can be vendor-specific. Take Google for example. Can become an issue in China.

My path of experience lays through the Western Europe, and therefore whether to use it or not is your personal responsibility. The scenario is usually simple and inevitable. Marketing and sales find a potential client, collect documents and set meeting to decide whether to throw a harpoon. We covered that back in previous article. Now it is time to show yourself! No, if you are a tough specialist, then, probably, you can lure in the shadows and wait. Let the sales managers figure it out on their own. And they will. They always do. After all, their job is to go through steps to get customer signature. And then they get a prize, a raise and final credits. Move to next customer- they don't need to develop or deliver anything! They don't even always understand what the tender is about. Therefore, they will promise and bargain according to the strategy of the greatest personal potential gain. As much as possible features, in the shortest possible time and as expensive as it can be (it is “expensive” and not on “what” and “when” their bonus depends on). At first CEO sets a plan for the sales department, and then, when it is met and overfulfilled - it turns out that 9 programmers cannot give birth in a month, because apparently, they are all of the same sex. Next presale team is accompanied by business analyst and/or architect. Don’t underestimate the risk of Doing Nothing.

In general, in the corporations, the connection between business and architecture is blurred for the outside observer. But from inside everything is simple. The analyst is not good at code. Unless he migrated from engineers to a warm place where he will have less responsibility. Product and business departments are like an elephants’ graveyard where architects go for retirement. But you are not there yet. Now you are responsible for everything. Since in the world of semiconductors, you are the only conductor between the customer's desire and the final product. And the code. Well! I've often seen a code without an architect, but an architect without a code!

The complexity of being a conductor usually lies in the fact that your expectation is to be a conductor of electricity and signals that are continuously flowing to both ends. Or conductor of the orchestra even if it sometimes sounds like dialing tones of old modem. The reality is that you are a conductor on the train and carry tea from the first car to the last one, passing through the restaurant car and completely losing contact with the previous car as soon as sliding door closes. Everyone understands that without tea it is worse and making it yourself is not always quick and tasty, just do not expect a warm greeting.

• The driver in the lead car thinks all the time that his train pulls too much. He sees and hears what is happening in the nearest carriages, but that long tail is clearly a ballast. Just detach it and to-da-moon! He does not think that in this train is kind of reversed and the cars are not pulled but pushed by loco-motive. Therefore, he likes to add carriages to the head of the train- they carry useful payload. He sees it personally.

• The restaurant car is close to the head and tea with sugar is carried from there in all directions, though loaded only at short stops. Since the best stops are not on schedule, their main concern is to protect the existing stock. Then you proceed to management sections.

• Sales are communists – teas for everyone based on their needs! For some reason you don’t support them.

• Infrastructures and support love classic ceramic cups served with strong black tea and sugar – ready to drink right away. And all these tea bags in varieties, several types of sugar and artificial sweeteners in different packages and everything is so humidity sensitive at storage. And all this disposable stuff distributed all over the train has to be disposed of. In short, tea used to be tastier, and the trainman knew his place.

• In the Product carriage you are only allowed to server a boiling water. And they mix up tea leaves in every sip because they suddenly may have a desire for different sort of things. And you dare to come here and demand pre-ordering! And unable to make cold-hot coffee-tea with unsweetened sugar and transparent milk.

• Dev managers love tea. But they do not appreciate your advice on how to drink and when. And do not criticize the fact that tea is spilled if it is poured into a pocket, drunk without a glass, from thimbles, standing on the head. And you always disagree with them. You say no to pouring leftovers into one glass, and no it is not a new tea type. And eating sugar is not the same as drinking. And yesterday’s tea served next week is not of the same taste.

• Then the carriage of senior developers. Esthetic stack overflow.  They must brew and mix everything by themselves and do not like transparent glasses that make things visible.

• You are really wanted only in the caboose, where the juniors sit. But they need to be served one by one with a spoon. And, of course, you don't have your own car.

At this point one may ask how all of this is related to architecture? As I have already noted, architecture is a response to business requirements. But in addition to the necessary functionality, there is also an environment where it this should fit into. I did not have a chance to work in large outsourcing companies and it means that I will look at everything from a huge product company. With history, market, products and inertia. In the lead carriage, they understand that the rails are not as they used to be, but it is difficult to switch them or make a turn. Therefore, they always try to change their shoes on the go. This means that you are not working from tabula rasa, but with a variety of already existing products collapsing internally to a set of black holes. Or you are trying to squeeze into a new segment of competing business. But it is inevitably placed in the same galaxy of your black holes. The difference between a big customer and a regular customer is in a million of green reasons. The name of your company and its history are the entrance ticket to the big market.

What you may not have known, if you were not allowed to the league, but must be considered during design:

• The average customer gets vanilla. The Enterprise will get chocolate and nuts inside.
  The functionality and appearance can be very different. Your task is not to go the easy way and create a brunch per client or wrap every line with IF/ELSE. Your company will not want to spend tons of money on extra testing and modifications, fragile code and endless support. Try to stay with as few client specific parts as possible, it would not be bad to use replaceable modules, enrichment, event drive, plugins, no-code.

• An ordinary client just takes an ice cream, and a cool one will tell you who will serve it.
  The client can actually go through the list of employees and choose who will code and manage the development. The common reasons for rejecting a candidate are experience and conflicts of interest. Either not qualified enough or involvement in projects of competitors. When choosing technologies, do not rely on specific people - they can be blacklisted.

• An ordinary client integrates to your system, but with a big client you will integrate to his legacies. You will not be able to change the contract or direction of the communication. Prepare for interpreters and adapters in advance. Especially if you've been on microservices for a long time. Distribution tax will increase dramatically.

• An ordinary client has one business, and big one has many and everywhere.
  This means that you need to take into account geography, availability of communication and regulations. It may be technically unrealistic to distribute pseudo-vanilla around the world with minimal separate deployments. Even with clouds it can be impossible to spread your usual business model over multiple time zones, currencies, cultures and inconsistent laws (and they almost always do not fit within a range of your off-shelves products). And the same cloud providers have a different set of tools and prices in different regions.

• The average customer eats when given. A client with a great appetite chooses when.
  So, if you are used to the fact that once a quarter your functionality is updated and everyone switches to new version. This may not be the case at all. First, customer may require rearranging the backlog and deliver something faster. And not in planned release of the version, but before a certain date. Besides maintaining production as usual you will probably need to have several parallel versions in every deployment. A huge organization won't update quickly. You used to work with 2-3 versions in production, but here there can be a dozen with requirement for full backward compatibility. Code maintenance and testability are becoming a top priority.

• The average client will be happy when he gets a nut, and the fat one has a strict diet.
  You will be told what fixes and features to include in the release and what will be left to some future patch/release. And keep in mind previous point. Infrastructure is often harder to push into a release. The client is afraid of big changes. Try to do everything step by step and from end to beginning - forward compatibility. Well, decoupling, decoupling, decoupling.

• Remember Vendor lock?
  Don't expect a system to be completely in your hands. If it is on-prem, then there can be no access at all. Keep in mind that it will be necessary to debug "by logs", and support will operate with a huge delay. You will receive a partial and belated description of the problem. Multilevel logs, informative error messages, system of warnings and health check services. Things that often named “production readiness” should be part of the architecture.

• DDD for Product and DDD for architects have different meanings.
  For you it should be Deferred Decision Design. A number of irreversible or expensive decisions you can delay - defines how good your architecture is.

TLDR: you apply for a tender and begin the first communication. Everything on time and as few obligations as possible to the customer. Within your organization, you will have to take responsibility for the decisions that others make. Therefore, it is necessary to "help" them to make the right strategic choices. Plan how to please client but with minimal risk and as many deferred decisions as possible.