As geeks, we love to dive head first into technical challenges, and yet, we are often less interested in learning the business domain and engrossing ourselves in solving the little peoples’ pains, you know, the users.  Focusing on them can have a number of negative effects on us, like having to write code that doesn’t challenge us, or even worse, “What if it doesn’t add anything new to my resume?”

So, we put on our senior engineer hat and plow to the front of our geek kingdom, asserting that we must develop significant long-term infrastructure in the system because it will save the company thousands, if not millions, of dollars down the road.  The infrastructure is not needed to satisfy the agreed upon use cases for this release, but it’s so important that the team would be crazy to omit the work we are proposing from this iteration.  We are so convinced of our own superior planning and genius, that we are unconcerned if no one has recognized how much pain we are saving them from yet, or that it has put the project behind schedule.  They will see some day…

Throughout the industry services are often closely identified with Service Oriented Architecture (SOA).  There is a wealth of quality architectural thinking within the SOA world.  However, there are some fundamental issues that have propagated throughout large parts of the software development industry because of the underlying philosophies found in SOA.

A central theme of the SOA development paradigm is the focus on reuse.  Software developers with even the shoddiest training have been taught to place enormous importance on reuse, where logic is modularized in a single place and used over-and-over.  So, there is nothing unusual about the SOA assertion that services should be loosely coupled and designed to support numerous clients.

A major challenge is that services in a Service Oriented architecture must also be built to support clients that have not yet been formally defined through a requirements process, or even yet conceived.  In seemingly countless instances, this has led to teams completely over emphasizing the reuse of the services being built, while immediate functional requirements are often missed, not completely supported, or in the worst cases blatantly ignored.

Pragmatic software architects recognize the considerable pain the industry has experienced from projects that over-accentuate one software tenet (reuse), while ignoring others.  Overly generic services frequently lead to extended development schedules, and in some cases they are fatal to the entire project. The blame for this trend is not SOA itself of course, but software engineers taking a good concept and overusing it until it becomes a liability.

There are absolutely cases where reuse is the chief priority for the service implementers.  A common example is creating a true, public API, such as a travel booking service developing integration endpoints for their present or future partners.  Or, for an internal IT example, consider the case where the system holding the source of customer records needs to expose the data to multiple systems throughout the organization, and a reusable service is required.  These cases are limited though.  In particular, they are most rare in IT shops, where the insanity of completely over-emphasizing reuse can be at its highest.

Quality architects avoid this trap by working with the following understanding:

• Risks should be identified.
• The business domain should be captured through the requirements process and implemented in its own layer.
• The implementation should only include things that address specific project risks that have been identified or satisfy the functional requirements of the system.

Thus, the system should be built for the immediate needs, unless there is a priority placed on the future in the business requirements, or the project’s owners specifically dictate this as a priority.

Reality check: It is not your money! Building a software system should be focused on the urgent needs of the organization, unless the team is clearly directed to focus on other priorities, which is rare.

I am guessing I read this somewhere at some point, but my current project has motivated me to think about tight coupling and its impact on productivity.

I am a big believer in the idea of “flow,” where developers find their rhythm and begin to move through things quickly.  Over 10 years in the software development industry, I have encountered two times where the projects I was on never reached a state of “flow” in any way.  Usually, no matter what the challenges, I find the team and myself reach this in some form.  So, it is actually kind of scary to be on a project where we haven’t found this state at all.

The first time I experienced this it was because the needs of an internal customer were so ambiguous and bogged down in politics.  So, we were never able to get a clear picture of what we were to build.  This instance ended with our entire team being laid off as part of large reduction in the work force through out the company.

In the current case, the requirements are quite straightforward and clear.  There is plenty of weirdness in them, but the fundamental function of the system is fairly simple.  From a functionality standpoint, it is probably the simplest system I have ever worked on, outside of dinky web site contact forms and the like.  So, what’s the hang up to finding the “flow?”

I believe it is the high coupling through out the system.  We are working with an internal development team, and our customer has mandated a fully data driven approach and implementation that does little to separate the meta-data from the data.  In most cases, the database tables are being propagated up to the presentation layer, which has created heavy dependencies throughout the system between all tiers of the system.

In many ways, I have always thought the industry was a bit to extreme with the desire to decouple every single little thing, and found many of the driving forces for doing the decoupling to be a bit silly.  However, I am seeing now how much total coupling limits team productivity.  It brings team productivity down to the level of the least talented- least motivated developer, as everyone is always waiting on everyone else.

I think there is a very natural level of decoupling in systems.  We don’t need to go crazy on this issue, but when the natural needs for decoupling are ignored there are many side effects, including killing the team productivity.

At QCon San Francisco 2008, I went to a session with Eric Evans about working in the Core Domain.  It was one of the best software talks I have ever heard, and I am looking forward to InfoQ posting the video so I can share it with a number of friends in the industry.

He basically said to be sure and contribute where it matters, don’t spend your career cleaning up after hackers, allow consequences to happen, the core domain is where the value is, and rewriting legacy systems is generally not a good thing (unless they are just too expensive to operate).

Another interesting thing he pointed out is that his writing on ubiquitous language is about a team, not about an entire organization.  He stated that single monolithic models do not work and differences are ok.  Groups just have to agree on how to translate things that are shared…

The most interesting point to me is the idea of allowing consequences to happen.  I find that most software developers are passionate (including myself).  It is hard for us to allow bad things to happen when we can see them coming a long way off, but I think Mr. Evans’ advice is dead on here.  We have to do our best to make recommendations in hopes that our organizations will avoid costly mistakes, but as my boss tells me often ‘we cannot save them from themselves.’

So, here is to focusing on the core domain, and making ourselves as valuable as possible…

I was chatting at the office last week about code quality, and thought I would jot down some of the things we talked about.  I think it is fair to say that it is difficult to know when code is “good” and when it is “bad.”  There are of course obvious times when the coder was incompetent, or cut corners with little explanation.

Ignoring those types of issues, judging code is difficult because we do not understand the reality of when the code was written.  Was the code written with an extreme deadline?  Was it built with a clear expectation of being prototype code and then put into production by some aggressive manager?  Was the architecture changed multiple times without time to rewrite / refactor?

In addition to not understanding the actual circumstances around the time of original development, developing perfect code is expensive and there is a point where we all need to ship our imperfect code and move on.  Only a really rich, somewhat misguided, client would pay for the real amount of time to develop perfect code.  And of course, we have to understand that much of our feelings toward code come back to style and preference, and not everyone has the same approach to these items.

So, let’s all try and buck the tradition of bashing the last guy’s code, and when there are real issues that need to be addressed then please approach them gracefully.  Remember, someone will be following you soon too!

Corner-Case-Driven-Development is an exciting version of development where all extreme corner cases are highlighted each step of the way.  It is especially important to interject a corner-case once someone starts to talk about the fundamental features of the system, as getting distracted by the core things, like what a user wants to accomplish, is a no-no in the corner-case methodology.  When embracing corner-case-driven-development, a team can expect a number of exciting things to happen, including: difficulty developing and maintaining the code, un-met deadlines, and generally low team morale.

Read about more exciting methodologies here: http://www.scottberkun.com/blog/2007/asshole-driven-development/.

James Ward and I launched our new video podcast tonight, DrunkOnSoftware.com.  The concept is a bit silly, and of course tongue in cheek as we aren’t drunks, but one of the main things we have in common is the ability to not take ourselves too seriously.  We do love talking about software and the occasional drink.  So, we figure why not make that our niche… everybody needs a niche.  We are planning on talking about all things software, but will probably focus more on Adobe Flex than other things.

In the first episode, we talk about the Adobe Flash Player 10.  James shares about the main updates of interest to developers.

Please contact us if you are in the Denver area and would like to appear on the Videocast.  We would love to have you!

I have been working on a project with a very complex domain for the last year and a half.  Our team has been doing our best to make use of agile principles, but I am seeing a few places where Agile breaks down.  The greatest challenge for our project is that the stakeholders do not have a complete or clear understanding of the domain.  This isn’t necessarily their fault, as we are replacing an old main frame government system based on 30 years of convoluted and conflicting legislation.  Within those realities agile makes a few implied assumptions that limits its usefulness on our project:

1) That a clear understanding of the domain exists somewhere, and you must work closely with those individuals to extract that information.

2) That there is something for the stakeholders / users to interact with as the project iterates (i.e. a user interface).

In addition to the ongoing new domain revelations that we seem to continually be discovering along with our stakeholders, the guts of the application we are building computes complex tax calculations that run in a batch process like fashion, using attributes across seemingly countless domain objects.  This means that often the only artifacts to deliver to the stakeholders at the end of an iteration are reports – reports that have to be near complete in order for the stakeholders to make sense of them and be able to compare with the previous systems results (main frame / Excel).  This makes it extremely difficult to quickly iterate and get the necessary feedback to finalize the systems most difficult features.

The reality is that there isn’t a methodology that can fully address issue number one, but it is important to realize that there are issues that methodology cannot solve.  Another example of this, is expecting Agile to compensate or gloss over poor performers.  This is a common one and of course never works, and it only serves to set the methodology up for the blame when things fail. 

For our project, it is important that we iterate in hopes of getting feedback as early as possible, but that doesn’t necessarily mean that our stakeholders ongoing ‘new discoveries’ will come any earlier in the process.  I do like Agile and think it can solve a number of problems, but I often feel that it is over emphasized.  It is just one part of a software project.  To really succeed in building quality software highly competent people are required – methodology should only be enhancing their efforts.  I hope that is a fairly common thought.  I guess the new realization for me personally is that you also need highly informed (in both the domain and methodology) and motivated stakeholders too…

I have been found of calling Agile, “only a vocabulary” for quite sometime. As most of my colleagues on the projects I have work on for the last couple of years seem to use an impressive blend of Agile terms, but somehow we never seem to be doing actual Agile development. I know there is a lot of nuance out there between the different Agile methodologies. I am not trying to get caught in that trap. I am taking the simplest broadest definition of Agile I have. Which to me basically comes down to, feature based iterative development, where new working-fully-tested (shippable) code is ready for release at the each iteration.

Yet, I continue to work on countless iterations that don’t even cycle through QA. Or the funniest case to me, ‘sprints’, that are dedicated to requirements gathering for the whole development team. As far as I am concerned, if you are not producing releasable code every single iteration, then you aren’t doing Agile development. I have no idea how that fits with my other post on Agile this week, but I will say that I am addressing two different extremes of the problem. In this case, I am talking about teams with no barriers to doing actual Agile development, but for some reason cannot seem to understand what Agile actually looks like.

I realize in one way it doesn’t matter, but for some reason I just take issue with our tendency to adapt our current habits to new vocabulary to allow us to keep making the same old mistakes…

Well, I haven’t had much time to play with my new favorite technology lately (Adobe Flex). So, I figured I would blog about one of the other software topics that peaks my curiosity.

For a few weeks, I have been thinking through a simple but odd idea, as it would seem to be an oxymoron, “Agile without the implementation.” Those of you that embrace Agile methodologies will immediately want to smack me down, as that statement would seem to be a complete contradiction.

There is however, no matter how much the Agile advocates want to argue the point, a mismatch between Agile and a number of the environments that software is developed within. The most obvious example being the government, as all Agile methodologies are based in an anti-bureaucracy mentality, and our government agencies are anything but.

In the last couple of months, the great state of Colorado, which I call home, started clamping down on state agency software projects. Our new governor, Bill Ritter, issued an executive order that all software projects over $10,000 are subject to oversight and management from a single state office (http://www.rockymountainnews.com/drmn/local/article/0,1299,DRMN_15_5641110,00.html). Experienced software engineers immediately understand this will only exacerbate the difficulty of building quality software, rather than help to minimize the failures that different technical groups within our state government have experienced.

This is obviously (or maybe ‘hopefully’) the extreme of government bureaucracy, but even within the most relaxed government agencies there is typically the requirement to do a big-up-front analysis/design to even get funding. As we all know, a main tenant of virtually all Agile methodologies is the aim to eliminate this very practice, as it almost always leads right into a waterfall like implementation methodology.

So, how are government software managers to succeed in circumstances that are clearly not conducive to building great software? As crazy as it sounds, I think the answer is ‘Agile without the implementation.’ The concept is based on being able to validate and execute requirements, without implementing them. Thus, a team is still able to produce tangible artifacts that are validated, while not getting stuck with the time or cost of implementation details.

Most must be asking, “How would one truly validate and execute requirements?” The company I work for has developed a product for executing UML 2.0 models, the Gorilla Execution Engine (GXE). This allows a small team of developers/analysts to create a true code ready specification, which has been validated by the end user through a functional user interface generated from the UML.

Here is how I would see the process of Agile without the implementation working. To be true to the concept of Agile, features would still be digested in small chunks iteratively. Stakeholders and users would still get a functioning user interface through each iteration that allows them to validate that the requirements have been captured correctly. The ‘domain model’ that is being executed acts as a code ready specification that would be implemented down the road in separate implementation iterations.

This would bring a number of benefits to government agencies. First off, they would be able to reap real benefits from the up-front requirements time that they are typically required to provide, as they would have validated requirements. Second, it allows them to still borrow from proven Agile techniques while still satisfying the bureaucratic requirements. Third, it would allow governments groups to more accurately estimate the cost of implementing the system, as they would be working from validated requirements.

As I work through this idea on paper, it seems fairly obvious that developing software like this could help to close the gap that governments have between there innate bureaucracy and Agile methodologies. Taking it a step further, I am starting to really embrace one of my company’s core philosophies behind the GXE, which basically says we should avoid implementation details when trying to digest requirements. Tools like the GXE give us the abstraction to be able to do this. At first I question this as incompatible with Agile, but the reality is that when done as part of feature based iterations, then it can only serve to enhance an Agile project.

To take one more leap from this thought process, this idea is really just a stop gap. With all the work currently being done in the declarative programming space, soon modeling will just become the core implementation. Thus, making this whole idea a moot point anyway as the requirements and implementation will simply be a model.