I often get phone calls about ‘full-time’ jobs where the initial task will take about 5 to 6 weeks. The scenario goes like this: we have finished phase 1, but we are concerned about performance; or we have a Spring application running but we need someone who is an expert in Spring to do some ‘fine tuning’.

What they mean is: We completed the project, within budget and on schedule. Although the system looks OK, it continually breaks. To fix it will place us behind schedule, and we need someone who really knows the framework to ‘fix’ it immediately.

It can also be stated as: We used the cheapest labor possible to do want was most visible. The hard stuff such as design or error handling was ignored because it would hinder progress. Now we need to do that as fast as possible within changing what is ‘complete’.

Remember the Tower of Pisa. Lovely building. Looks great. Pity that no-one ever spent enough time on the foundations.

This is a problem common in poorly managed agile systems. To show progress, what gets done first is what is most visible. If only the simple stuff is done, it is not a product, it is a prototype without a proof of concept.

Leonardo da Vinci’s Last Supper has major damage. Even when da Vinci was painting the wall, the wall was deteriorating. He finished it ( sort of ), but the foundation was bad. He got 80% of the job done. The missing 20% was stabilizing or finding a suitable wall as a foundation for the painting; that he ignored.

Michelangelo was meticulous in preparing the ceiling of the Sistine Chapel, and employed assistants who knew how to prep a wall ( Michelangelo had never done a fresco before ). He spent a lot of time in finding the best people to do the 80% of the work up front. The 80% of the work was the wall preparation. Once the wall was prepared, he could paint it. The prep is why the ceiling survived.

So, if you want a completed project, it is essential that you spend the 80% of the time up front to be ready for the visible work that follows. No matter how good the result looks, if the prep is bad, it will crumble, or lean.

The problem is that managers and clients want to see progress; they would rather see you working on the visible 80% than the foundation 20%; they want to see measurable progress; this is a customer management issue.

The issue is determining how must time to spend up front. The infrastructure has to be done up front without urgency as opposed to at the end of the project, when all that happens is a poor restoration job.

No matter how much work is done on da Vinci’s last super, if the basic structure is poor, the painting will never survive.

Interestingly, the most succesful method now used to restore frescos is to actually physically remove the fresco from the wall i.e. to separate it from any underlying problems, or rebuilding the foundation of the work.

RUP has four phases: Inception, Elaboration, Construction and Transition. Inception gets the stakeholder’s ( owner’s ) buy-off up front. Elaboration defines the scope of the project and the architectural overview. Construction is the process of building the project, and Transition is deployment and support.

 What I propose is the use of RUP’s Inception and Elaboration phases only. The Construction phase gets rellegated to Agile.

 What we gain from RUP is a customer buy-off, an evaluation of the feasibility of the project, a comprehensive system architecture.

By adding Waterfall’s Prototypes, we gain an evaluation of the finished product very early in the process ( and customer feedback ). By adding Waterfalls Proof of Concept, we ensure that any show stoppers are found early in the project. ( Agile tends to relegate the more complex tasks to late in the project ).

By using Agile methodologies in development, we regularly present the product to the customer, gaining frequent feedback from the stakeholders. Agile places a customer representative as part of the development team so that requirements can be easily evaluated. Agile also has some major coding productivity gains, most notably refactoring and pair-programming.

Using this approach, we use Waterfall, RUP and Agile in the areas where they have maximum benefit.

So, how does this tie into art scaling.

The artist produces a series of rough sketches of the project which are presented to the client for ‘visual’ and budgetary approval ( RUP ).   Any potential risks are evaluated, and if needs be, models or detialed sketches made ( Waterfall ). When the project is approved and the budget assigned ( RUP ), the process of construction commences. The work proceeds with the customer frequently evaluating the work ( Agile ). The work is complete when the product fulfils the initial concept ( something not possible in Agile development, where a totally defined final goal never exists ).

• I have seen more successful waterfall projects than agile projects
• Waterfall includes two critical but overlooked processes: proof of concepts and prototypes. This is interesting since a lot of agile projects that I see really only develop prototypes. The lack of a proof of concept is the reason that agile projects fail.

In architecture, the client is often presented with a model of the completed structure. This is the grand up front design criticised in Waterfall methodology. The client looks at the model and decides if it is really what they want. Once the model ( the prototype ) is approved, the plans are drawn up. There is not just one model. Individual components, such as a chapel or staircase may have a model created of them also. Any component of the design is modeled. This is done to evaluate any potential risk to the project; it is a proof of concept.

What distinguishes waterfall is that the client has to make a committment to a product up front ( Rational Unified Process also has a similar buy-off ). This means that the customer must decide exactly what they want.

Agile development lacks this up front committment. Most Agile projects are done below the radar – they are below some budgetary constraint that requires a signoff by senior management. Once they reach a certain stage of completion ( most likely deployment ), some high level signoff is required.

At this point, a committment must be made, and the true cost of project completion is appraised. The cost has been a series of hidden costs, and finally senior management now discovers how much they paid for something that they never approved. They are also being asked for money put this same unapproved object into production. This is when the project may die. Unless the ‘prototype’ is very successful, the management buyoff which was never achieved at the start of the project is now required – it is pretty hard to convince someone to trust you when you have been making an end-run around their restrictions. At this point the project dies.

In sumary, up front ( waterfall ) design may have its problems, but it really is a more honest way to develop software. It has an up front committtment and budget, and gets a senior management buy-in before any money is spent.

After an initial definition of the system requirements, the programmers start to implement the system. Once there is something that a customer can use ( or view ), the system is given to the customer to use ( evaluate ). Based on the feedback from the customer, the system is corrected, modified or enhanced. What makes Agile unique is that the process is adaptive in that the system will change to meet the customers requirements or changing needs / priorities during development. That is, it does not involve some up-front grand design, but involves continual feedback.

Now let’s look at Abstract Expressionism. This is an art movement of the ’50s personified by Jackson Pollock and Willem deKooning.

The essence of abstract expressionism is that there is an initial concept of a painting, it is not preplanned; there is no master design. The painter applies paint based on his initial concept. He then steps back from the work to evaluate the changes, and then enhances the work based on what he feels is needed to move the work towards completion. There is no customer involved in the feeedback loop, it is the artist who is the client and evaluates each addition, and changes the work according. In this respect it is an agile method of creation; no grand design, frequent feedback and adaptive to change.

So, applying an agile approach to art, Pollock achieves: Hardly what was initially intended.
The point is that if you really wanted this: then you don’t want an abstract Expressionist painting. If you wanted the Mona Lisa, then you needed a grand design, or waterfall methodology.

This highlights the issue with agile software development. There is no grand design, and therefore the result can sometimes be totally incompatible with the original intent. Therefore if you know what you wanted in the beginning, agile development is a bad choice, and if the person providing feedback during development is not “the customer”, then the result is not what was wanted.

As in art, the choice of style oftens dictates the end result.

Waterfall is to

as Agile is to

When a full size model was created, it was copied to each individual face of the marble using a machine. Holes were drilled into the marble block to the correct depth, and then the marble was chipped away to remove the marble between the holes.
But even for these full-size models, sculptors do not start at this scale. There is a series of works which progressively scale up.

This practice has continued into the 20th Century with such sculptors as Henry Moore.

Scaling up traditionally starts with a rough sketch. Models are then created at inceasing scales: 1/4″ to 1′, 1″ to 1′ and 1/4 scale. Painters work similarly. The Sackler Gallery in Washington D.C., even has Egyptian samples of scale models used by sculptors. So, it is method in use for a very long time.
I have seen many computer systems created full scale and optimized. These systems often fail.
The critical step in the scaling process in art is that each enlargement is critically reviewed, and detail appropriate to that scale is added. Software is an evolutionary process, and creating a system full size makes any revision to the design impossible as development progresses. Therefore scaling of computer systems seen more as an evolutionary process, when an initial conceptis evaluated, and each stage of scaling reviewed to ensure that the details added are appropriate to the original concept and appropriate to the current with the current scale.

As in art, computer system scaling should be an evolutionary process and the final stage of developemnt, as opposed to the initial step in the design of the system. To develop software otherwise is synonomous with the much aligned waterfall methdology.

In Florence, Italy, there are two painting of the Last Supper that are almost identical. These are painting by the artist Ghirlandaio in the churches of San Marco and Ognissanti. What is unusual is that these pieces are almost identical, as opposed to just similar.

Most people know of da Vinci’s last supper, but what they don’t realize is that there are many last suppers. It is a painting commonly painted on the walls of the dining rooms of monasteries. The purpose of religious art is to recreate a past event from the scriptures so it can bring that event into the present, hence a last supper painting in a dining room.

There is a belief that creativity involves individuality, but in religious painting, there are very strict rules about the subject matter. The subject matter and arrangement of elements is very much dictated by the church. But, there is room for variation. For example, the Ghirlandaio Last Suppers have Judas seated on the opposite side of the table. Da Vinci’s has them all seated on the same side of the table. This makes it unique.

So how does his relate to programming?

In programming, there are a set of standard rules ( called Design Patterns ). These are the standard way of doing things.

Although we reuse patterns, we really are very poor at reusing code.

There is open source, which gives a framework, but not really a solution.

Look at some examples of reuse in art.

The Architect Frank Lloyd Wright designed a house for Ayn Rand. When Ayn Rand decided not to build the house, he had no issues with ‘selling’ the design to another client.

In painting, a major portion of the creative time is consumes with setting up the composing the subject matter and drawing the subject. The artist Degas would make a drawing on a non-absorbent surface and press paper onto it, creating a monoprint ( literally unique one-time print ) . He would then do another. Because the process basically involves transferring the image to paper, the image eventually is removed from the original surface. So, Degas created a series of prints. He took the sheets of paper with images that we too faded to be sold as prints, and colored these with pastel. This gave him a set of drwaings. Hence one drawing could become about 5 works or art. This is a classic example of reuse.

So, how can we do this in programming?

I believe that the proverbial walking skeleton is the solution. Whereas Frameworks provide low level reuse, a walking Skeleton provides a reusable backbone for code. Its major advantage is that since it provides a computer system that is complete, skeletons can be reused. This is because a skeleton is application independent. It’s sort of like Degas’s monorpints in that it reuses the structure.

Also low level code can be reused. On every project I write an indexOf method which accepts a String and an array of Strings. It returns the index of the String in the Array ( -1 if not found ). This means that I can execute a case statement based on an input string. The method ignores case. I also create duplicates of lot of Java methods that are case insensitive versions. This saves me a massive amount of coding. Most projects with unfamiliar technology could benefit if the first thing developed on the project were a set of common utilities. To ease the delay in developing these common components, the initial version merely needs to be stubs returning dummy data. On a recent project, I made the switch between dummy and real data switchable. When the license to one of the external services expired, we could consider developing using the dummy data.

Open Source is touted as a great solution, but I am becoming less enamored with Open Source the more I use it. There is a philosophy that you can change the source code if required. How many people change Open Source source, and how many customers would let you do it? The biggest joke is that on a recent project, a custom coding solution was rejected in favor of open source; it was suggested that the custom code be uploaded to source forge and then reimported to the project. My issue with Open Source solutions, is that the one thing that is missing is adequate documentation. If a customer would not accept a project without documentation, why would ( or should ) they accept a project that is totally dependent on an undocumented product.

In summary, reuse in art has been used to reduce the choices made about mundane issues. In programming, we need to think about reuse as a means of reducing the mundane repetitive coding tasks that are present on every project. We need to treat programing like a creative process with different stages and priorities at each stage of the process. If we can minimize the time spent on the mundane, we can spend more time on the enjoyable creative elements. Look at some paintings of the Last Supper and see the creative variations that exist in each of the works. The creativity is in the variations, not the mundane.