Saturday, January 14, 2012

Top Down Design/Bottom up implementation

In my previous blog i made a case for management as a service profession. And that is something I really believe.

Like any good idea, however, it can be taken to bad extremes. In this post I thought I'd address that.

In traditional procedural programming, we have a concept of "top down design/bottom up implementation." What this means is that you start at the general goal and break it done into sub-tasks. You then break each of those sub-tasks up, and keep going until you hit a level that "feels right" for implementation. (There really is no hard and fast rule for what this is, knowing it when you hit it is more art then science.) Then we start building the software starting at that bottom level and working our way back up to the goal.

This results in a software design that is well focused on the goal, but also is built in clear and separable layers. Top down design is a matter of designing software functionality and interfaces to that functionality. In today's object oriented world it can be seen as a form of "encapsulation' but it is a very specific form that leads to software with some very good properties. Should any individual layer prove to have an issue it can be fixed or replaced with no impact on the layers above it and below it. Higher layers can be stripped away without sacrificing lower level functionality. Finally, each layer can be created by people with the best understanding of that layers' functionality.

An engineering project can be, and I will argue, should be designed in the same way. At the top are those defining the business goals. Beneath them on the tree are one or more layers of architect who take the output of the management layer above them and decompose it into goals for the layer below them until the decomposed parts of the design finally arrive at the implementing teams.

Such layered project design has all the advantages of layered software design. Failures on part of the process do not directly impact other parts of the process below them and impact above them only so far as the architected interfaces need to change. It also focuses each participant's efforts on the area where they add the most value to the project.

In my previous blog I talked about the management side of an engineering manager's job, but many people managing engineers are also engineers or architects themselves. And this calls for a different set of skills and approaches then the management side.

Non-engineering management can often get confused and think that "empowering the workers" means asking every engineer's opinion on everything. Or worse, asking everyone's opinion on every functional part of the process from engineering to art to game design.

This devalues the very expertise you probably hired these people for to begin with and creates a whole lot of confusing noise for upper management. In such an environment, decisions become highly politicized and those who can "sell"(1) what they want in terms that make the most sense to those who often know the least-- upper management-- prevail in defining the entire project.

Part of empowerment is authority. And if you give everyone equal authority over everything, then noone has authority over anything. A football team where everyone is trying to play quarterback doesn't get very far down the field.

So, empower your people... but empower them to do *their* jobs, not everyone else's. Trust that you have hired the right people for the right positions and let them play them without undo interference from others with other functions. That is true empowerment.

(1) Engineers have a technical term for the act of sales. We call it lying. I have *no* tolerance for engineers who deliberately color the information they are passing up in order to get a desired reaction and, if i am allowed to, will fire anyone I catch doing it. That is the flip side of giving some one authority and your trust. If they violate it, they are gone. The kind of management i do calls for honest and open communication, not political positioning.

How to effectively manage professionals

Looking around for my next gig has made me really think about how I manage professionals and why. If you haven't seen this TED talk, and you manage professionals, you should.

On top of Dan's excellent anaylsis, let me add a few observations from a good deal of experience on both sides of the management fence:

(1) If you have a system, and its getting lousy compliance, the problem is *the system* not the employees.

(2) If you have a problem employee, 90% of the time what you really have is a mis-match of employee to situation. It is far less wasteful and generally more productive to try to fix the situation then to try to "fix" the employee.

Both one and two above are based on a simple and true premise. Its one Dan talks about in his work. Its that people are intrinsically motivated. Most people really want to do a good job. Rather then the typical management view of "if someone isn't doing a good job then they don't care enough", I propose that 90% of the time if someone isn't doing a good job its because something is in their way and they are at least as frustrated by it as you are.

This leads to my third observation:

(3) Self-organization is the most powerful solution to (2) in your arsenal.

I have seen this over and over. If I try to over-structure a team, I make mistakes. People struggle with the roles I have assigned them. Some do well, some not so well. This is not surprising. After all, all I really know about them is a resume and some short conversations.

HOWEVER they get to know each other very well very quickly. They spend 8 hours a day problem solving together. Left to their own devices they inevitably come up with better solutions as to who is suited to what then I could. Furthermore, their solutions are fluid and can change to meet changing requirements.

Which leads me to my most important point....

(4) The single most detrimental thing to professional productivity, is too much management.

Its wasteful in that it requires a lot of management time, and its wasteful in that generally all you are doing is getting in the way of your people. When I have a team spun up and functioning, that team takes half an hour a day to manage, and (depending on its seniority) 5 to 10 hours a week to coach. Thats it.

If you think every team needs its own manager, you are doing way too much management. 9 women cannot make a baby in 1 month, and 9 managers can't make a baby at all! In fact, put even one "manager" in the room full time directing and grading a couple trying to make a baby and its the rare couple that can manage to achieve the goal.

The old school of management suggested that the best motivators were fear and greed. As Dan points out, that works great for assembly workers, but not for professionals.

The best motivators for professionals are autonomy and trust. The *first* job of a manager of professionals is not to manage the workers, but to manage their environment such that they can work effectively.

This isn't to say that a team should not be directed, but the most effective direction is in the form of engineering goals to accomplish not ways to accomplish them. A team that feels empowered will come to you, in most cases, if they are unsure of how to proceed. And such "event driven" management is far more efficient for both sides then "polling."

For the cases where they don't realize they are headed into trouble, thats what monitoring is for... but that monitoring should always be a light touch. A peek over the shoulder of a mentor, not the drumbeat of a task master.

In my case, I use a white-board based scrum burn-down chart and it takes 10 min or less at the end of the day to discuss and update it. The goal is not to grade them on productivity, but simply to know what is getting in their way and to adjust estimates as we understand more about the problem at hand.

The modern manager needs to serve the company's greater goals by serving his employees needs for both direction and an environment in which they can do their best work. And a manager with time on his or her hands is a manager who has succeeded in those goals. A constantly busy manager, on the other hand, is one who is floundering.

In short, modern effective management is not a position of mastery, but of service. And true service means doing just what is helpful, and then getting out of the way.

My wife, the theologian, would say someone told us this 2000 years ago. Its time we listened.