This article is the first of two that examine in depth the origins of the Model-View-Controller (MVC) design pattern and its misapplication to servlet framework architectures. The purpose of this first piece is threefold. First, it attempts to provide an accurate description of the problems brought about by MVC in servlet middleware. Second, it suggests techniques and strategies for coming up with a new design, one better suited to the needs of servlet infrastructure developers. Third, it offers an example of a completely new, nonderivative pattern we might use moving forward. The second article backs up my assertions by introducing and exploring a reference implementation of the new design.

Ultimately, my goal with these two articles is to convince the servlet middleware community once and for all to put the dark days of MVC behind us and to lay the groundwork for completely new, nonderivative servlet middleware architectures that better address our common needs. Please bear with me; I wouldn't write a piece about MVC in the servlet tier unless I believed it was a funeral dirge.

Part One: History of the MVC Pattern
We begin with a historical account of MVC's origins in the early 1970s and its subsequent use as a pattern for thick-client UI component frameworks.

Tracing the historical origins of the Model-View-Controller pattern is not a difficult thing to do. The man who brought us MVC has published historical documents that date back to 1973, when object-oriented programming was still being conceived and scientists were only beginning to explore the concepts of "distributed, communicating components." (See Reenskaug, Trygve, "The History of MVC.")

Following the paper trail back to the early days of object modeling is instructional in two ways. First, it serves as a reminder that the problems we face today in distributed computing are by and large the same ones faced by computer scientists from three decades ago — in other words, the proverbial wheel is still in dire need of reinvention. Second, it shows us how much we still have to learn about simplifying complex systems.

Oslo Norway, 1973
In August of 1973, Dr. Trygve Reenskaug, while working for the Central Institute for Industrial Research in Oslo Norway, wrote a paper called "Administrative Control in the Shipyard," which he presented at the first International Conference on Computing Applications in Shipbuilding that autumn. In his paper, Reenskaug analyzed the modern shipyard as an information system. In this exercise, he identified many of the technical and social complexities inherent in shipyards and suggested several techniques that could be used while describing the system. His goal was to reduce the overall complexity so the shipyard could be more easily modeled in a computer application.

If the goal of a component framework is to reduce complexity for developers modeling an information system, what strategies can the framework designer use to make this possible? Early on, Reenskaug touches upon the idea of "distributed, communicating components." His strategy is that the "total system will be decomposed into a number of components" (emphasis his; see Reenskaug, Trygve, "Administrative Control in the Shipyard"). Decomposition of a large or complex system into modular components played a central role in his strategy. By now, most of us understand this concept very well. We call it the "Separation of Concerns" of architectural components.

Following this strategy, Reenskaug begins to describe "a general framework" whose "main purpose is to enable us to mold a new information system to fit an existing way of organizing things, the resulting system being sufficiently flexible to be easily adapted to changes in the organization." (ibid.; emphasis mine.)

He then outlined several "major requirements [of] systems developed within the framework." (ibid) The following behaviors are those that the framework should seek to enable:

We want to build man-machine systems. Manual override on automatically produced results must be a routine matter, and the systems must be able to compute the consequences of such overrides.

Responsibility, authority, and competence will be spread out among groups of people in the shipyard. Our total data processing system must be divided into subsystems that are attached to the various areas of responsibility. The subsystems shall be controlled by these groups and further developed by them.

A subsystem must be transparent to its responsibility group, so that its owners may fully understand their system's behavior.

All subsystems must be open-ended, so that they may be attached to other subsystems belonging to the same or other responsibility groups.

The total system must be capable of continuous growth; it must be possible to add new subsystems or change old ones without disturbing the total system. It must be possible to include "foreign" systems developed by other people without upsetting the total system. The painful transition from one "generation" to the next can no longer be tolerated. (ibid.)

A condensed list of these requirements would read:

Simplified manual override.
Divisible into modular subsystems.
Behavioral transparency of subsystems.
Subsystem modules are pluggable.
The total system is capable of continuous growth.
Despite the solid foundation that Dr. Reenskaug had laid out for implementing his framework, the lack of an object-oriented programming language stalled progress on what was to become MVC. It wasn't until several years later, in California, that he got access to the world's "first personal computer, the ALTO, with its powerful direct manipulation interfaces." It was at this time that he began experimenting "with tools for end users manipulating their part of a total plan." (see "The History of MVC")

Xerox Palo Alto Research Center, 1978
This experimentation took place at the Xerox Palo Alto Research Center (PARC). Access to the ALTO helped Dr. Reenskaug identify the end user of a distributed system. Dr. Reenskaug and his colleagues devised "MVC as a solution to controlling a large and complex data set." Their task was similar to the task he'd defined for the component framework in 1973 — to reduce the complexity inherent in presenting an interface to the system. This time, the "subsystem" was actually the user interface.

Describing a new framework for accomplishing this task was not easy — it never is. "The hardest part was to hit upon good names for the different architectural components," Reenskaug writes. "Model-View-Editor was the first set," In his "Thing-Model-View-Editor" paper (see Reenskaug, Trygve, "Thing-Model-View-Editor"), Dr. Reenskaug defines the following terms:

Thing: "Something that is of interest to the user." (A real-world concept.)
Model: "An active representation of an abstraction in the form of data in a computing system." (Abstraction of the Thing.)
View: "One or more pictorial representations of the model."
Editor: "An interface between a user and one or more Views." (Later called "Controller" and "Tool.")
Out of these early prototypes, MVC Triads emerged in SmallTalk-80. The model was an abstraction of the real-world concept, the view was its visual representation, and the controller was the buttons and slider bars that allowed the user to interact with it (thereby "controlling" the view). All pieces in the triad were interconnected and could communicate with the other two pieces, so there was no layering or abstraction involved. Since then, Reenskaug has "preferred to use the term Tool rather then Controller." According to his notes, these are the terms he has used in later implementations (see "The History of MVC")

Shortly thereafter, Dr. Reenskaug left the PARC laboratories and his work was carried on by Jim Althoff and Dan Ingalls. Because of this early work, UI Development in ST-80 was greatly simplified, and the MVC pattern continues to this day to be an effective and appropriate solution for the design of user interface components.

MVC Pattern Language
It is important while we are studying and using MVC that we bear in mind the original purpose of the pattern: to reduce the complexity of user interfaces for a large and complex information system. It was designed for a specific purpose and evolved to meet a specific need.

Dr. Reenskaug's work in MVC has not been abandoned. Earlier this year, he began publishing material on his new MVC pattern language.

Nowhere, ever, in any of his papers was it suggested that the MVC pattern could be used to address the needs of n-tiered workflow processing infrastructures. But here we are, 24 years later, trying to pound in nails with a screwdriver. Despite the careful effort of scientists from Reenskaug's generation, it seems that we've not learned a thing.

 

Part Two: The Servlet Framework Antipattern
At this point, we're going to try to break through the recent buzz surrounding the MVC pattern and illustrate how, despite its success in traditional UI frameworks, MVC fails to address the needs of servlet framework architectures and the framework development community.

But before we go any further, I'd like to define the terms "pattern" and "antipattern," so that the reader understands the context in which I'm using them throughout the remainder of the article. I borrow my definitions from the book AntiPatterns and Patterns in Software Configuration Management by Brown, McCormack, and Thomas.

"A Pattern is a form of solution intended to address a problem. The problem for Pattern readers has always been to identify when and how a Pattern should be used."

An AntiPattern is defined as "a commonly occurring solution to a problem that generates decidedly negative consequences;" in other words, an AntiPattern is generally a commonly repeated practice in industry that is flawed in some manner and causes symptoms that are more undesirable than whatever positive results were sought.

Design patterns can become dangerous in a development community when their popularity eclipses their ability to address the evolving needs of a development effort. By implementing patterns for their own sake, without a critical evaluation of their ability to describe the practical concerns of an information system, we contribute to the spread of antipatterns in our development community.

Some antipatterns are easy to identify and with a little effort can be easily avoided. However, some are not. At first blush, many antipatterns are indistinguishable from beneficial design patterns. They spread by giving us a sense of security and confidence in their use. By reducing them to advertising buzzwords, we re-enforce the perception that they are widely accepted, address a common problem, or confer some predictable advantage, outside of the context of their original use. There are no "one-size-fits-all" solutions when it comes to information system design. A pattern might confer advantages within a given context, but outside of that context, all bets are off.

To make matters worse, technical, economic, and social circumstances can also contribute to the "silver bullet" mentality surrounding the antipattern, further contributing to its popularity in the community. The inevitable resulting misapplication of the antipattern can introduce new complexities into our software and conflict into our community.

Pressure (whether real or imagined) to conform to the antipattern is often greater than the technical or economic need to abandon the practice — at least in the short term. Time lends a different perspective to matters, however, as what Reenskaug described as "the painful transition from one 'generation' to the next" becomes incrementally more difficult to avoid.

Unwieldy, Inflexible Architectures
Although the Model-View-Controller design has proven itself throughout the years to be an appropriate and beneficial pattern for modeling user-interface components, its use in designing distributed servlet infrastructures has led to very unfortunate results. Among these are the advancement of unwieldy, inflexible architectures that are incapable of being quickly altered to support the ever-changing demands of the technological and economic environment.

On October 19th, 2003, on the Struts development list, a user suggested the addition of a feature his business had found lacking from the framework. His suggestions were given a fair but brief consideration and never brought up again. This is not because the Struts community is apathetic to the needs of its users — nothing could be further from the truth. These people write software that they give to the world for free.

The most likely reason they didn't implement his change is because it would have required rewriting several core classes in their infrastructure and would probably have introduced unwanted complexity to the application. In addition, it is questionable whether or not this particular feature would enjoy a broad enough use to warrant inclusion into the default feature platform for Struts. As a result, the novice developer and his company are forced to rewrite portions of Struts to add crosscutting, system-wide behavior. There is no ability to plug in crosscutting system logic or to easily extend the feature offering of the platform.

In this way, Struts has failed to pass Reenskaug's second, fourth, and fifth requirements, that "the total system must be capable of continuous growth, it must be possible to add new subsystems or change old ones without disturbing the total system. It must be possible to include 'foreign' systems developed by other people without upsetting the total system." He stresses the point twice, and for good reason — the only guarantee we have in technology and business is that our needs will change. Systems that are incapable of changing in a fluid manner will inevitably fail to meet our needs, and we will find ourselves facing the difficult challenge of migration to the next generation.

Fragmented Development Community
Another unfortunate side effect the MVC antipattern has had on our community is the fragmentation and polarization of its members, who have in recent years become entrenched around dozens of incompatible technologies. Business leaders are forced to choose between a myriad of nonstandard options, whose feature offerings largely mimic each other.

Earlier this year, I began exploring WebWork as an alternative to Struts. While Struts had been the first framework to separate the command URI from the underlying logical system (thus separating the concerns of the end user typing in a URL from the developer chaining together actions in the servlet tier) and to introduce a rough workflow processor, WebWork was beginning to introduce the concept of interceptor chaining and a dynamic value store that used an expression language for parsing through the contents.

But at the time, the WebWork advocates were focused on the differences between Rickard Öberg's implementation of MVC and the Struts implementation. The documentation has improved quite a bit since then, and the reader might get a sense that MVC is no longer the selling point it once was. At least a half-dozen MVC derivatives have been introduced into the community, but none have met with great applause. A good friend of mine swears he went to a presentation by Craig McClanahan where he brought up a diagram of MVC and there were four boxes. Model, View, Control and ... ? There have been so many half-hearted explanations about how "actions are really Controller components and ..." ad nauseum. I'm sure you've heard it all a hundred thousand times.

The point is that no matter how you cut it, spin it, twist it, melt it down, push, pull, or try to explain it, no matter how many books you write on the topic, nobody has ever been able to adequately explain how an n-tiered servlet component infrastructure is supposed to implement the Model-View-Controller pattern.

And as a result of our insane quest after this philosopher's stone, we've managed to splinter what could otherwise be an extremely large and potent section of the development community.

Rising, Ongoing Opportunity Costs
But worst of all, aside from the obvious conditions brought about by our stubborn adherence to this antipattern, we must also factor in the ongoing opportunity cost of our failure to build a common, extensible, flexible servlet component infrastructure that meets the continuously changing demands of the market and is positioned to take advantage of the technological progress that is happening all around us. In the meantime, the main competitors to J2EE are capitalizing on the unreasonable complexity involved with bridging the web-persistence gap.

 

Part Three: Refactoring the MVC Pattern
In this third and final part of the article, we retrace the footsteps used by Trygve Reenskaug to isolate the MVC design pattern and apply his modeling techniques to the servlet framework tier. We also introduce a new design pattern that specifically addresses the needs of servlet framework developers in the J2EE sphere.

Fortunately, we're not bound to the MVC pattern. If we're willing to do some legwork, the techniques developed by Dr. Reenskaug to arrive at an appropriate design for the problem of servlet framework architectures are at our disposal.

While we're designing this new pattern, we must keep in mind the five requirements listed above for a component framework. We know that we must create a system that can be decomposed into modular components, that the behavior of each subsystem component should be transparent to the larger system, that components should be able to plug in to the larger system, and that the system should therefore be capable of continuous growth without adding complexity or management chokepoints to the development process. Finally, the inner workings of the system should be subject to some form of manual override, wherein a manager can redefine the system behavior at runtime without taking it offline.

Concerns of Architectural Components
We have to start by considering the different roles that different objects play in our system. There are only a few higher patterns of behavior for all of the different architectural components in a servlet framework. If we are able to isolate those behaviors and capture them into interfaces, then our system will become more flexible and modular.

By considering the nature of the different components in a servlet framework and employing basic separation of concerns, we arrive at three different roles for objects — three higher patterns of behavior that can be observed in all existing systems:

DataSource: In the most abstract sense of the term, this is a place from which data can be gathered and stored. It could be a message bean, a request object, or a map, and would have a set of very simple method signatures similar to a map.
Action: An action is the basic logical building block of a business system. Actions are all executed and needn't necessarily know the order of their execution.
Workflow: The core concern of the servlet framework is to manage the execution of a sequence of Action objects using one or more DataSource objects. The internal logic of the framework that determines the flow of control over the request can be isolated in a Workflow interface.
And what about presentation components? First, bear in mind that the user interface for a system needn't necessarily be visible. It could be audible or thermal or even olfactory, so "View" components is an inappropriate and limiting term. Second, considering the requirements, it follows reasonably that our framework should easily plug into any type of presentation technology, several of which are becoming extremely advanced and needn't be replicated.

Trying Out the New Design
How do we know this would work? Let's try applying this new design concept to a simple application like a card catalog, and if it doesn't work, out we'll refactor. The data written on the cards should probably be kept in some kind of DataSource objects so it can be stored, retrieved, examined, and altered. There are many different types of data object solutions. EJBs, POJOs, and XML are just a few — the list goes on for miles.

Next, we'll want to operate on the data in the cards. Pulling them out of storage, putting them back in, shuffling them, erasing their contents, using them as coasters, etc. This logic can be encapsulated in actions (preferably as fine-grained as we can get them). If we break down the actions into small enough pieces, we can use them in different combinations to get different effects.

The decisions governing the order in which these action combinations are executed represents the application's workflow. To encapsulate this logic, we can write a series of workflow controllers whose job is to execute actions in a certain order. This workflow could be hard-coded into the program, but it would probably be better if we incorporated a dynamic, rules-based system like Drools, so that the workflow logic could be controlled and altered at runtime to meet the changing needs of the business.

Of course, all of the different, dynamic rules approaches require some kind of external data for directing traffic. Do we need to refactor the design and add another category for components? Not at all — workflow descriptors and user input are just two more types of DataSources.

The actual human interface for the web application can take place in another tier, and its developers can use whatever solution they like for building their components. They can share access to the same DataSource components, and can implement their own presentation components and tools for manipulating the data however they want. If they're smart, they'll probably use MVC or some variant in their component design, because UI is where MVC shines.

Abstracting the Feature Platform
If we were able to break down system in this interface-heavy manner, we would be able to abstract the actual features provided by the platform. Things like form validation, value caching, dynamic EJB bridges, security integration, and other actions could all be abstracted into a platform that actually rests on top of the underlying workflow system (or plugs into it — I say potato ...). At that point, developing feature platforms of useful and common actions could be a cooperative community effort. Businesses could develop their own libraries to implement application-specific, or even system-level, actions without having to alter the underlying workflow system or even know how it works, and development teams could combine their efforts to build libraries of mission-critical action components for common use.

The interface-heavy design could also potentially allow us to swap out entire portions of the workflow or action subsystems at runtime and replace them with custom, more specialized subsystems that are tailored to our individual needs. Imagine: exchangeable parts for servlet frameworks. With Maven, we could dynamically assemble the framework and feature platforms at build time from specialized component subsets.

But it doesn't have to stop there. The social benefits of such a flexible, modular, and extensible system should be obvious. The different factions of the development community could reassemble around implementing commonly needed features instead of warring over architectural minutiae. Businesses could develop strategic, proprietary feature platforms that are fine-tuned to the needs of their companies, or could collaborate on common component libraries without having to get their changes accepted by the team developing the workflow core.

And perhaps the best news of all is that the investments we've already made into the feature platforms of different frameworks could easily be ported to this new infrastructure. Features from Struts could be rewritten as action platforms and plugged into the framework without a hitch. It might even be possible to write compatibility layers for existing action implementations, so that the end users don't suffer unnecessarily in making the transition forward.

Results and Conclusion
We all have this task in common: to implement a framework that will reduce the complexity of bridging the web interface to the application server, and to do it in such a way that it is flexible, extensible, and capable of constant mutation throughout the coming years.

As open source middleware developers, we also share a common desire to improve ourselves, our businesses, and our communities by collaborating to build better tools. The time is long overdue for each of us to reconsider our assumptions about the MVC pattern, to acknowledge the negative effect it has had on our community, and to work together to address the problem. We need to reevaluate our common technical and social needs, overcome our stubbornness, and completely abandon the MVC design as the cornerstone of the servlet framework architecture. A new solution is waiting that addresses those needs.

In the next piece in this series, I'll back up my assertions with actual code and introduce the Shocks Servlet Framework, a full implementation of the design I've outlined here. I'll go over the architecture of the framework, discuss how it evolved conceptually, and show how it addresses each of the five challenges for frameworks Dr. Reenskaug laid out over thirty years ago.

This article is the first of two that examine in depth the origins of the Model-View-Controller (MVC) design pattern and its misapplication to servlet framework architectures. The purpose of this first piece is threefold. First, it attempts to provide an accurate description of the problems brought about by MVC in servlet middleware. Second, it suggests techniques and strategies for coming up with a new design, one better suited to the needs of servlet infrastructure developers. Third, it offers an example of a completely new, nonderivative pattern we might use moving forward. The second article backs up my assertions by introducing and exploring a reference implementation of the new design.

Ultimately, my goal with these two articles is to convince the servlet middleware community once and for all to put the dark days of MVC behind us and to lay the groundwork for completely new, nonderivative servlet middleware architectures that better address our common needs. Please bear with me; I wouldn't write a piece about MVC in the servlet tier unless I believed it was a funeral dirge.

Part One: History of the MVC Pattern
We begin with a historical account of MVC's origins in the early 1970s and its subsequent use as a pattern for thick-client UI component frameworks.

Tracing the historical origins of the Model-View-Controller pattern is not a difficult thing to do. The man who brought us MVC has published historical documents that date back to 1973, when object-oriented programming was still being conceived and scientists were only beginning to explore the concepts of "distributed, communicating components." (See Reenskaug, Trygve, "The History of MVC.")

Following the paper trail back to the early days of object modeling is instructional in two ways. First, it serves as a reminder that the problems we face today in distributed computing are by and large the same ones faced by computer scientists from three decades ago — in other words, the proverbial wheel is still in dire need of reinvention. Second, it shows us how much we still have to learn about simplifying complex systems.

Oslo Norway, 1973
In August of 1973, Dr. Trygve Reenskaug, while working for the Central Institute for Industrial Research in Oslo Norway, wrote a paper called "Administrative Control in the Shipyard," which he presented at the first International Conference on Computing Applications in Shipbuilding that autumn. In his paper, Reenskaug analyzed the modern shipyard as an information system. In this exercise, he identified many of the technical and social complexities inherent in shipyards and suggested several techniques that could be used while describing the system. His goal was to reduce the overall complexity so the shipyard could be more easily modeled in a computer application.

If the goal of a component framework is to reduce complexity for developers modeling an information system, what strategies can the framework designer use to make this possible? Early on, Reenskaug touches upon the idea of "distributed, communicating components." His strategy is that the "total system will be decomposed into a number of components" (emphasis his; see Reenskaug, Trygve, "Administrative Control in the Shipyard"). Decomposition of a large or complex system into modular components played a central role in his strategy. By now, most of us understand this concept very well. We call it the "Separation of Concerns" of architectural components.

Following this strategy, Reenskaug begins to describe "a general framework" whose "main purpose is to enable us to mold a new information system to fit an existing way of organizing things, the resulting system being sufficiently flexible to be easily adapted to changes in the organization." (ibid.; emphasis mine.)

He then outlined several "major requirements [of] systems developed within the framework." (ibid) The following behaviors are those that the framework should seek to enable:

We want to build man-machine systems. Manual override on automatically produced results must be a routine matter, and the systems must be able to compute the consequences of such overrides.

Responsibility, authority, and competence will be spread out among groups of people in the shipyard. Our total data processing system must be divided into subsystems that are attached to the various areas of responsibility. The subsystems shall be controlled by these groups and further developed by them.

A subsystem must be transparent to its responsibility group, so that its owners may fully understand their system's behavior.

All subsystems must be open-ended, so that they may be attached to other subsystems belonging to the same or other responsibility groups.

The total system must be capable of continuous growth; it must be possible to add new subsystems or change old ones without disturbing the total system. It must be possible to include "foreign" systems developed by other people without upsetting the total system. The painful transition from one "generation" to the next can no longer be tolerated. (ibid.)

A condensed list of these requirements would read:

Simplified manual override.
Divisible into modular subsystems.
Behavioral transparency of subsystems.
Subsystem modules are pluggable.
The total system is capable of continuous growth.
Despite the solid foundation that Dr. Reenskaug had laid out for implementing his framework, the lack of an object-oriented programming language stalled progress on what was to become MVC. It wasn't until several years later, in California, that he got access to the world's "first personal computer, the ALTO, with its powerful direct manipulation interfaces." It was at this time that he began experimenting "with tools for end users manipulating their part of a total plan." (see "The History of MVC")

Xerox Palo Alto Research Center, 1978
This experimentation took place at the Xerox Palo Alto Research Center (PARC). Access to the ALTO helped Dr. Reenskaug identify the end user of a distributed system. Dr. Reenskaug and his colleagues devised "MVC as a solution to controlling a large and complex data set." Their task was similar to the task he'd defined for the component framework in 1973 — to reduce the complexity inherent in presenting an interface to the system. This time, the "subsystem" was actually the user interface.

Describing a new framework for accomplishing this task was not easy — it never is. "The hardest part was to hit upon good names for the different architectural components," Reenskaug writes. "Model-View-Editor was the first set," In his "Thing-Model-View-Editor" paper (see Reenskaug, Trygve, "Thing-Model-View-Editor"), Dr. Reenskaug defines the following terms:

Thing: "Something that is of interest to the user." (A real-world concept.)
Model: "An active representation of an abstraction in the form of data in a computing system." (Abstraction of the Thing.)
View: "One or more pictorial representations of the model."
Editor: "An interface between a user and one or more Views." (Later called "Controller" and "Tool.")
Out of these early prototypes, MVC Triads emerged in SmallTalk-80. The model was an abstraction of the real-world concept, the view was its visual representation, and the controller was the buttons and slider bars that allowed the user to interact with it (thereby "controlling" the view). All pieces in the triad were interconnected and could communicate with the other two pieces, so there was no layering or abstraction involved. Since then, Reenskaug has "preferred to use the term Tool rather then Controller." According to his notes, these are the terms he has used in later implementations (see "The History of MVC")

Shortly thereafter, Dr. Reenskaug left the PARC laboratories and his work was carried on by Jim Althoff and Dan Ingalls. Because of this early work, UI Development in ST-80 was greatly simplified, and the MVC pattern continues to this day to be an effective and appropriate solution for the design of user interface components.

MVC Pattern Language
It is important while we are studying and using MVC that we bear in mind the original purpose of the pattern: to reduce the complexity of user interfaces for a large and complex information system. It was designed for a specific purpose and evolved to meet a specific need.

Dr. Reenskaug's work in MVC has not been abandoned. Earlier this year, he began publishing material on his new MVC pattern language.

Nowhere, ever, in any of his papers was it suggested that the MVC pattern could be used to address the needs of n-tiered workflow processing infrastructures. But here we are, 24 years later, trying to pound in nails with a screwdriver. Despite the careful effort of scientists from Reenskaug's generation, it seems that we've not learned a thing.

 

Part Two: The Servlet Framework Antipattern
At this point, we're going to try to break through the recent buzz surrounding the MVC pattern and illustrate how, despite its success in traditional UI frameworks, MVC fails to address the needs of servlet framework architectures and the framework development community.

But before we go any further, I'd like to define the terms "pattern" and "antipattern," so that the reader understands the context in which I'm using them throughout the remainder of the article. I borrow my definitions from the book AntiPatterns and Patterns in Software Configuration Management by Brown, McCormack, and Thomas.

"A Pattern is a form of solution intended to address a problem. The problem for Pattern readers has always been to identify when and how a Pattern should be used."

An AntiPattern is defined as "a commonly occurring solution to a problem that generates decidedly negative consequences;" in other words, an AntiPattern is generally a commonly repeated practice in industry that is flawed in some manner and causes symptoms that are more undesirable than whatever positive results were sought.

Design patterns can become dangerous in a development community when their popularity eclipses their ability to address the evolving needs of a development effort. By implementing patterns for their own sake, without a critical evaluation of their ability to describe the practical concerns of an information system, we contribute to the spread of antipatterns in our development community.

Some antipatterns are easy to identify and with a little effort can be easily avoided. However, some are not. At first blush, many antipatterns are indistinguishable from beneficial design patterns. They spread by giving us a sense of security and confidence in their use. By reducing them to advertising buzzwords, we re-enforce the perception that they are widely accepted, address a common problem, or confer some predictable advantage, outside of the context of their original use. There are no "one-size-fits-all" solutions when it comes to information system design. A pattern might confer advantages within a given context, but outside of that context, all bets are off.

To make matters worse, technical, economic, and social circumstances can also contribute to the "silver bullet" mentality surrounding the antipattern, further contributing to its popularity in the community. The inevitable resulting misapplication of the antipattern can introduce new complexities into our software and conflict into our community.

Pressure (whether real or imagined) to conform to the antipattern is often greater than the technical or economic need to abandon the practice — at least in the short term. Time lends a different perspective to matters, however, as what Reenskaug described as "the painful transition from one 'generation' to the next" becomes incrementally more difficult to avoid.

Unwieldy, Inflexible Architectures
Although the Model-View-Controller design has proven itself throughout the years to be an appropriate and beneficial pattern for modeling user-interface components, its use in designing distributed servlet infrastructures has led to very unfortunate results. Among these are the advancement of unwieldy, inflexible architectures that are incapable of being quickly altered to support the ever-changing demands of the technological and economic environment.

On October 19th, 2003, on the Struts development list, a user suggested the addition of a feature his business had found lacking from the framework. His suggestions were given a fair but brief consideration and never brought up again. This is not because the Struts community is apathetic to the needs of its users — nothing could be further from the truth. These people write software that they give to the world for free.

The most likely reason they didn't implement his change is because it would have required rewriting several core classes in their infrastructure and would probably have introduced unwanted complexity to the application. In addition, it is questionable whether or not this particular feature would enjoy a broad enough use to warrant inclusion into the default feature platform for Struts. As a result, the novice developer and his company are forced to rewrite portions of Struts to add crosscutting, system-wide behavior. There is no ability to plug in crosscutting system logic or to easily extend the feature offering of the platform.

In this way, Struts has failed to pass Reenskaug's second, fourth, and fifth requirements, that "the total system must be capable of continuous growth, it must be possible to add new subsystems or change old ones without disturbing the total system. It must be possible to include 'foreign' systems developed by other people without upsetting the total system." He stresses the point twice, and for good reason — the only guarantee we have in technology and business is that our needs will change. Systems that are incapable of changing in a fluid manner will inevitably fail to meet our needs, and we will find ourselves facing the difficult challenge of migration to the next generation.

Fragmented Development Community
Another unfortunate side effect the MVC antipattern has had on our community is the fragmentation and polarization of its members, who have in recent years become entrenched around dozens of incompatible technologies. Business leaders are forced to choose between a myriad of nonstandard options, whose feature offerings largely mimic each other.

Earlier this year, I began exploring WebWork as an alternative to Struts. While Struts had been the first framework to separate the command URI from the underlying logical system (thus separating the concerns of the end user typing in a URL from the developer chaining together actions in the servlet tier) and to introduce a rough workflow processor, WebWork was beginning to introduce the concept of interceptor chaining and a dynamic value store that used an expression language for parsing through the contents.

But at the time, the WebWork advocates were focused on the differences between Rickard Öberg's implementation of MVC and the Struts implementation. The documentation has improved quite a bit since then, and the reader might get a sense that MVC is no longer the selling point it once was. At least a half-dozen MVC derivatives have been introduced into the community, but none have met with great applause. A good friend of mine swears he went to a presentation by Craig McClanahan where he brought up a diagram of MVC and there were four boxes. Model, View, Control and ... ? There have been so many half-hearted explanations about how "actions are really Controller components and ..." ad nauseum. I'm sure you've heard it all a hundred thousand times.

The point is that no matter how you cut it, spin it, twist it, melt it down, push, pull, or try to explain it, no matter how many books you write on the topic, nobody has ever been able to adequately explain how an n-tiered servlet component infrastructure is supposed to implement the Model-View-Controller pattern.

And as a result of our insane quest after this philosopher's stone, we've managed to splinter what could otherwise be an extremely large and potent section of the development community.

Rising, Ongoing Opportunity Costs
But worst of all, aside from the obvious conditions brought about by our stubborn adherence to this antipattern, we must also factor in the ongoing opportunity cost of our failure to build a common, extensible, flexible servlet component infrastructure that meets the continuously changing demands of the market and is positioned to take advantage of the technological progress that is happening all around us. In the meantime, the main competitors to J2EE are capitalizing on the unreasonable complexity involved with bridging the web-persistence gap.

 

Part Three: Refactoring the MVC Pattern
In this third and final part of the article, we retrace the footsteps used by Trygve Reenskaug to isolate the MVC design pattern and apply his modeling techniques to the servlet framework tier. We also introduce a new design pattern that specifically addresses the needs of servlet framework developers in the J2EE sphere.

Fortunately, we're not bound to the MVC pattern. If we're willing to do some legwork, the techniques developed by Dr. Reenskaug to arrive at an appropriate design for the problem of servlet framework architectures are at our disposal.

While we're designing this new pattern, we must keep in mind the five requirements listed above for a component framework. We know that we must create a system that can be decomposed into modular components, that the behavior of each subsystem component should be transparent to the larger system, that components should be able to plug in to the larger system, and that the system should therefore be capable of continuous growth without adding complexity or management chokepoints to the development process. Finally, the inner workings of the system should be subject to some form of manual override, wherein a manager can redefine the system behavior at runtime without taking it offline.

Concerns of Architectural Components
We have to start by considering the different roles that different objects play in our system. There are only a few higher patterns of behavior for all of the different architectural components in a servlet framework. If we are able to isolate those behaviors and capture them into interfaces, then our system will become more flexible and modular.

By considering the nature of the different components in a servlet framework and employing basic separation of concerns, we arrive at three different roles for objects — three higher patterns of behavior that can be observed in all existing systems:

DataSource: In the most abstract sense of the term, this is a place from which data can be gathered and stored. It could be a message bean, a request object, or a map, and would have a set of very simple method signatures similar to a map.
Action: An action is the basic logical building block of a business system. Actions are all executed and needn't necessarily know the order of their execution.
Workflow: The core concern of the servlet framework is to manage the execution of a sequence of Action objects using one or more DataSource objects. The internal logic of the framework that determines the flow of control over the request can be isolated in a Workflow interface.
And what about presentation components? First, bear in mind that the user interface for a system needn't necessarily be visible. It could be audible or thermal or even olfactory, so "View" components is an inappropriate and limiting term. Second, considering the requirements, it follows reasonably that our framework should easily plug into any type of presentation technology, several of which are becoming extremely advanced and needn't be replicated.

Trying Out the New Design
How do we know this would work? Let's try applying this new design concept to a simple application like a card catalog, and if it doesn't work, out we'll refactor. The data written on the cards should probably be kept in some kind of DataSource objects so it can be stored, retrieved, examined, and altered. There are many different types of data object solutions. EJBs, POJOs, and XML are just a few — the list goes on for miles.

Next, we'll want to operate on the data in the cards. Pulling them out of storage, putting them back in, shuffling them, erasing their contents, using them as coasters, etc. This logic can be encapsulated in actions (preferably as fine-grained as we can get them). If we break down the actions into small enough pieces, we can use them in different combinations to get different effects.

The decisions governing the order in which these action combinations are executed represents the application's workflow. To encapsulate this logic, we can write a series of workflow controllers whose job is to execute actions in a certain order. This workflow could be hard-coded into the program, but it would probably be better if we incorporated a dynamic, rules-based system like Drools, so that the workflow logic could be controlled and altered at runtime to meet the changing needs of the business.

Of course, all of the different, dynamic rules approaches require some kind of external data for directing traffic. Do we need to refactor the design and add another category for components? Not at all — workflow descriptors and user input are just two more types of DataSources.

The actual human interface for the web application can take place in another tier, and its developers can use whatever solution they like for building their components. They can share access to the same DataSource components, and can implement their own presentation components and tools for manipulating the data however they want. If they're smart, they'll probably use MVC or some variant in their component design, because UI is where MVC shines.

Abstracting the Feature Platform
If we were able to break down system in this interface-heavy manner, we would be able to abstract the actual features provided by the platform. Things like form validation, value caching, dynamic EJB bridges, security integration, and other actions could all be abstracted into a platform that actually rests on top of the underlying workflow system (or plugs into it — I say potato ...). At that point, developing feature platforms of useful and common actions could be a cooperative community effort. Businesses could develop their own libraries to implement application-specific, or even system-level, actions without having to alter the underlying workflow system or even know how it works, and development teams could combine their efforts to build libraries of mission-critical action components for common use.

The interface-heavy design could also potentially allow us to swap out entire portions of the workflow or action subsystems at runtime and replace them with custom, more specialized subsystems that are tailored to our individual needs. Imagine: exchangeable parts for servlet frameworks. With Maven, we could dynamically assemble the framework and feature platforms at build time from specialized component subsets.

But it doesn't have to stop there. The social benefits of such a flexible, modular, and extensible system should be obvious. The different factions of the development community could reassemble around implementing commonly needed features instead of warring over architectural minutiae. Businesses could develop strategic, proprietary feature platforms that are fine-tuned to the needs of their companies, or could collaborate on common component libraries without having to get their changes accepted by the team developing the workflow core.

And perhaps the best news of all is that the investments we've already made into the feature platforms of different frameworks could easily be ported to this new infrastructure. Features from Struts could be rewritten as action platforms and plugged into the framework without a hitch. It might even be possible to write compatibility layers for existing action implementations, so that the end users don't suffer unnecessarily in making the transition forward.

Results and Conclusion
We all have this task in common: to implement a framework that will reduce the complexity of bridging the web interface to the application server, and to do it in such a way that it is flexible, extensible, and capable of constant mutation throughout the coming years.

As open source middleware developers, we also share a common desire to improve ourselves, our businesses, and our communities by collaborating to build better tools. The time is long overdue for each of us to reconsider our assumptions about the MVC pattern, to acknowledge the negative effect it has had on our community, and to work together to address the problem. We need to reevaluate our common technical and social needs, overcome our stubbornness, and completely abandon the MVC design as the cornerstone of the servlet framework architecture. A new solution is waiting that addresses those needs.

In the next piece in this series, I'll back up my assertions with actual code and introduce the Shocks Servlet Framework, a full implementation of the design I've outlined here. I'll go over the architecture of the framework, discuss how it evolved conceptually, and show how it addresses each of the five challenges for frameworks Dr. Reenskaug laid out over thirty years ago.

]]>