Hanging out with XPers at the agilemodeling news group makes me realize how large the disconnect is between the XP people and me, and if I were to boil it down to one thing it would be they utterly fail to understand the need for large scale software development.
Their arguments invariably come back to, and I paraphrase, 'but its easier to do small scale development'. Of course its easier to do small scale development! But organizations don't large scale development for fun. Its hard! The risk of failure is high! And it costs A LOT more $$$. So why do large scale development?
We need to explain why large scale development is sometimes (and I would argue frequently) the only solution. Given our current state of understanding of software development.
I believe it results from the domain being unitary and I'd like to run some stuff I'm writing by the people here.
The domain is unitary
I refer to the business domain as being unitary. Dictionary.com is not that helpful and gives me the following definition: Having the nature of a unit; whole.
What I mean by unitary, is that a business domain is always a coherent whole, because what any organization does is always a coherent whole (even though it may not seem that way to people working in it). Or maybe what I am saying is that the domain is always sufficiently coherent to achieve the organizations objectives. If it wasn't then the organization would disappear because it couldn't achieve what it exists to do, or at least a commercial company would. Government is a different discussion.
It follows that a domain model that accurately describes the domain must also be a coherent whole. And if we base a software application on a domain model as FDD does and I fully agree we should, then the software application must also be a coherent whole.
This is the bit I have trouble explaining.
If the domain is a coherent whole, then any division of it must be arbitrary, unless we can organize our requirements such that we produce a clear division of the domain (and consequently can develop the domain model incrementally).
Anyone who has done as many requirements as I have (and understands the concept of a domain) knows that this is not possible. That is, there is no breakdown of requirements that results in a breakdown of the domain (and incidentally this is why scope is an intractable problem). If there were, we could dispense with large scale software development entirely and all be happy XP hackers. Although happy XP hackers doing incremental domain modeling.
I think I might have just invented a new agile method â Extreme Domain Modeling. 
Now wheres my publisher's telephone number?
phil
proper scale
Here's why XPers talk about small projects -- inside of every large project, there is a small project that is being fed too much...
Johnson's comparison of two SACWIS systems. SACWIS is a child welfare system that all the states in the US must implement. He stated that Florida began its SACWIS system in 1990 with an original cost estimate of $32 million for delivery in 1998 with a development team of around a 100 people. When they last looked the new cost was $170 million due to ship in 2005. He pointed out the state of Minnesota has to build the same capability with pretty much the same demographic issues as Florida - leading to very much the same broad system requirements. Minnesota started work in 1999, finished in 2000 with eight people costing $1.1 million.
http://homepage.mac.com/keithray/blog/2003/09/30/
Proper scale
I agree that large projects are often bigger than they should be and government seems particularly prone to this. However, it doesn't follow that all systems can be developed by small teams. And breaking down large projects into multiple small systems without a sophisticated engineering solution is just not a viable option.
The reality is that large enterprises spend the bulk of their development $ on large projects. They do this because they believe its the only way to get broad scope integrated applications. They pay the $ because its worth it to them.
I know Martin Fowler is using XP on a largish project (not large by my standards) and is substantially modifying XP in the process. So maybe we will get some variant of XP that is (more) scaleable.
phil
Poor science
inside of every large project, there is a small project that is being fed too much...
Every large project? I think not.
(I wonder what number = large and what number = small?)
Anyway... I know of a two programmer project. All it had to do was deliver 15 reports in a browser over an existing database. It was due in August. It's now December and not only has the first interim deliverable still not been met, the project manager has no confidence in the latest revised schedule.
Projects go awry. This is not news. Is it only because of size? I think not.
Jeff
Your SACWIS example is not accurate
Minnesota Total System Costs: 38M Annual Maintenance Costs 12M
Minnesota went operational in 1996.
At 12M Annual Maintenance it is 3-4 times the average of many of other completed SACWIS projects.
The Minnessota System does not include full SACWIS functionality it Excludes Interstate Compact Functionality
You may want to read the August 2002 report http://lakids.dss.state.la.us/Documents/General/Analysis%20of%20Other%
20SACWIS%20Projects.pdf
You will see it is not the least expensive total system cost.
So I dont think anyone can draw a conclusion, forget the facts are not 100%.
Replacing Legacy Systems must be delivered Holistically
Phil,
Rather than "unitary" I developed the use of the term "holistic" for the problem you described.
I would like to posture a very simple example of why not all development can be incremental or iterative even though it started that way - replacement of legacy systems.
A legacy system say - hospital management - might have been built incrementally or iteratively over 20 years. However, the technology on which it is based has reached its limit of growth and the whole system must be replaced in order to take it to the next level. This might be more than just recoding for the web or a pretty GUI - it could involve replacing the database with a semantic network tool for improved searching and inferencing (as an example).
The new system MUST have all the functionality of the old system and the two cannot be run together side-by-side - for all sorts of operational reasons.
This problem is common.
The result is a new project which is huge - replacing 20 years of development - and must be delivered holistically.
These types of problems push the envelope of agile methods and require subtle thought on the problem of grouping features for internally partial delivery at the end of each increment or iteration. In my experience, this is world few XPers understand.
David
--
David J. Anderson
author of "Agile Management for Software Engineering"
http://www.agilemanagement.net/
Holistic and unitary
David
Holistic is a bit too 'new age' for my taste
But I know what you are getting at. This is what I and others call a 'whole product'
What I was getting at was somewhat different (although related), which is the problem space contains things that collectively constitute a more-or-less indivisible whole. Think of this in model terms. Can you take a domain model and subdivide it without losing semantics? In my experience the answer is always 'no'. Thats what I mean by unitary.
Considering small pieces of the problem space in isolation results in large amounts of lost semantics. Only when you consider the pieces in relation to each other does this lost semantics reappear.
The domain is unitary
Could it be that another way of thinking about what is a domain, is that it is the set of requirements (or features, if you like) that could not exist in any other domain?
My thinking here is that if a requirement could also exist in any other domain, then this forms a non-arbitary division.
Large scale development is most likely a collection of small-scale developments that must co-exist in harmony.
--
Derek
Response to Derek
Derek
If I understand you correctly, you are saying that a domain can be divided into managably small development projects that then cooperate to achieve a large-scale system.
To an extent I agree with you and we made considerable progress towards this realizing this approach when I was at a large internet company. Yes, you can divide the domain for implementation purposes. Which seems to contradict my original statement.
I should have clarified what I meant by requirements, which is the kind of requirements that Cockburn refers to user goal use cases and you could characterize as actor-system interactions incorporating domain feature compositions.
My point was that starting with these kinds of requirements you can not organize them such that you achieve a division of the domain permiting a series of independent small scale developments.
What you can do, is once you have a unitary domain model across the scope of all your requirements you can then sub-divide the domain for implementation purposes, and I like this approach, although its a scheduling and organizational challenge.
You can also take a similar approach without requirements and relying on some definition of scope to tell you where the model boundaries are.
And BTW, I still consider a domain implemented in parts as unitary.
phil