I have a general question on implementation in the seperation of concerns model.
Having read the semi-finished paper on seperation of concerns in one of the other threads, I'm a little confused on some aspects of its implementation.
For example in my example simple PD, I have say these classes:
Customer <>--- Account <>--- Transaction
I have to deal with these "external" systems: a database, a message queue that will get me legacy information on all 3 objects.
I also have a "service locater" (or factory or whatever, you get what i mean :P) that will link my specific subsystem implementations to my problem domain interfaces.
It is here I have a bit of diffculty understanding what goes on. I think I'm right to assume for the message queue that I have something like a:
pd.ICustomerAccountManagementSystem with the factory linking this to my implementation si.
If this is correct, how is this then applied to databases? It seems weird to have just one monolithic IPersistenceSystem, and it seems weird to have ICustomerPersistenceService, IAccountPersistenceService and ITransactionPersistenceService. To me it seems obvious that these interfaces mean little to the client and wouldnt be included as such in the domain model and are expected as part of a developed feature, but come development time how do people typically organise this aspect?
What kind of persistence?
Hi Mark,
are we talking about persistence to a relational database that's "owned" by the application? Or is it just reading things from a "remote" database (i.e. one owned by another system or application that we won't/can't modify)?
If it's a database that's "owned" by the app we usually don't use the SI layer to interface to that. Most often we buy instead of build the persistence layer, e.g. using an object-relational mapping tool such as Hibernate or TOPLink etc. You could use anything, even DAO's here. In other words this is a plain old persistence issue, and doesn't require any SI involvement.
If it is an "external" data source, and it just happens to be a database, and you want to connect up to it via the SI layer, then you have two things going on: the first is the usual PD-SI layer separation issues, and then you have the SI implementation that *may* also use the same technoologies.
Some more clues about which of these we're talking about would be helpful.
Thanks, Paul.
Hi Paul,Thanks for answer
Hi Paul,
Thanks for answering :)
I don't think I was clear enough when I wrote that first comment so I'll try again.
First of all, the database is owned by the application and the question is totally around PD(-DM) and not PD-SI (the si example i gave was to draw the similarites in potential approaches).
::Begin quote::
Most often we buy instead of build the persistence layer, e.g. using an object-relational mapping tool such as Hibernate or TOPLink etc. You could use anything, even DAO's here. In other words this is a plain old persistence issue, and doesn't require any SI involvement.
::End quote::
This is the area I need more detail on. When I am building the persistence layer using hibernate, jdo or DAO's I typically have an interface that allows me to control the implementation using a properties file (e.g. pd.IPersistenceService=dm.JDOPersistenceServiceImpl). My question surrounds the granularity of this interface. On the larger side, I could have this IPersistenceService (which was meant to be analagous to the one si subsystem that i was interacting with in my orginial post: for example if i built a credit card system and i only dealt with one external subsystem it would be something like: ICreditCardSystem). On the small side I could have a persistence interface for each class that requires it in the domain layer (e.g. If IPersistenceService would have all the methods required by my underlying database interaction, then ICustomerPersistenceService would only have database interaction methods specifically that deal with the customer object from the problem domain). These granularities just seem too polarised.
To further focus my question, I'm specifically asking that when I am using an object-relation mapping tool (or DAO's), by what mechanism is it gelled with the problem domain?
can you abstract dm and si?
I thought I submitted a reply to this but must have clicked preview instead of submit :o
After rereading my above example I'm not surprised you would need more info. Anyway, I'll try and explain it again.
I wanted to know that when you are using an object relational mapping tool of some kind, what technique to you use to link the problem domain to the tool?
I was thinking in one of two ways:
1. As an analogy, when using another system you would typically create something like this in the problem domain: ICreditCardSystem which would have an implementor in the si layer. Could you then say, for a database that is "owned" by the system being built, an interface called IPersistenceSystem would be created, which would have an implementor in the dm layer, would would handle all calls to a database? While the client would find value in the credit card system, they wouldnt what the persistence system is (unless it is an existing database of another system!). Not only that but the interface would have HEAPS of methods and the implementor would be MASSIVE.
2. This is the way i usually do it, but i think it is too granular, and harder to associate to the theory im thinking about (see below).
For each class (or package or class hierachy) create an interface to talk to a database for it. This way you can hide whatever implementation e.g. for the Customer class there is an ICustomerPersistenceService and in this case a JDOCustomerPersistenceServiceImpl, which is dynamically associated at runtime by a factory).
I should explain why I'm asking this. I had discussions with a friend of mine who was suggesting that you could isolate the problem domain entirely and create a service layer around it that depended on it (visually to me, it looked like combining the si and dm layers into one). This way, whether the system needed to talk to a database, a queue to another system, flat files, xml etc, it would always just be a well defined problem domain interface implemented. I thought it had some merit so i was investigating how it would be done. I could bundle everything into the si layer quite easily except for the DM layer. For me it is an interesting excercise to see if you could abstract the two layers to be just one as my friend suggested, or if it will never be possible to do practically (never is a dirty word!).
DM Implementation
I'm still lost. What do you mean by this:
"what technique to you use to link the problem domain to the tool?"
I certainly don't do anything like the persistence interface thing you mentioned.
More clues please. Perhaps give me a concrete example?
Paul.
Lets say we are using JDO (co
Lets say we are using JDO (coz its the one I know best).
After process 1, you have a domain model. The model then typically is spewed into code by a tool (e.g. together).
Just say I have an Account class, a Customer class and a Transaction class and they are arranged in some graph. I give this model to a tool which will then generate me a minimal JDO xml that will be used for the transparent persistence. After the enhancement is done, any instance in this specific object graph can be persisted/unpersisted. Cool. Now I need to write the code that will actually do the persisting. Just where this goes is the issue for me. I like the idea of having an interface there so i can insert a mocked implementation if i desire. So using the above example, my Customer class would have an ICustomerPersistenceService and that would have an implementation say written for JDO or hibernate. The methods it would have would be like static finders etc. (therefore a call to public static Customer findByCustomerNumber(String customerNumber) would make a call to the public Customer findByCustomerNumber(String customerNumber) in the CustomerPersistenceService interface which would be dynamically linked to the JDO or mock implementation.
The only thing thats not right with doing this is transaction management, which would reside in the UI or SI tier (e.g. to persist my customer object graph and say another object that is unrelated in the same transaction, the persistence/transaction manager is usually called in the UI and something like persistenceManager.persist(customer);persistenceManager.persist(myUnrelatedToCustomerObject);
)
If know that the SI layer typically has an interface in the PD (like ICreditCardSystem). As an analogy do we expect similar interfaces in the PD layer (just not exposed to the client) for some database related stuff? Or is it all just bundled into whereever the transaction orginates (the UI or SI)?
My 2 cents.
I think I understand what you are looking for. The answer to your question was also already given in another thread but was scoped in another context but you need to grasp the underlying concepts in practice:
http://www.featuredrivendevelopment.com/node/721#comment-779
When using and ORM framework/tool most of the work necessary to abstract the Database from the PD is already done. You just need to use the services (objects and methods) provided by the tool do store and retrieve "pure" objects. Usually such a tool has a master object called Broker that provide all the services necessary. You just need to state the mapping rules from objects to tables and records in some XML file or similar format.
When not using such a tool, the approach to build the DM layer may vary according to the technical context and the non functional aspects of the project (performance, etc etc). Currently we are migrating all content from Vignette CMS to a homegrown CMS system built by an IT department of a Finantial Group (Bank, Insurance and Investement Companies). The new CMS system is implemented in .NET and the old system is implemented in Vignette (TCL and a couple of ASP pages). To migrate we are actually going directely to the Vignette database (not using the Vignette API's to get the content to migrate as it is implemented in TCL, a language not compatible with .NET in the scope of Vignette). We started by designing the PD model. This is a model that maps Vignette concepts and the surrounding customizations to the new CMS concepts (objects). This took us a week. We also "cracked" the way Vignette organizes its data store. We are not using a ORM.
So now we are building the migration tool. As an example let's focus on three concepts:
VGNProject - Vignette organizes all content within projects. Projects can have child projects and parent projects. So we have a hierachy of projects. All inform regarding the structure of Vignette Projects are stored in a specific table.
CustomCategory - Custom category is a customization made by the Insurance company over Vignette CMS. A CustomCategory may or not have an associated VGNProject. When its the case of having an associated and associated a Vignette Project the aCustomCategory dynamically inherits the its data. In other words, the collaboration between a VGNProject and a Custom category follows the Actor - Role pattern (read Streamlined Object Modeling). The tables that have the CustomCategory data are actually two.
CMSSection - Some categories are mapped to sections. A CMSSection is a concept of the new CMS.
CMSContext - Some categories are mapped to contexts. A CMSContext is a concept of the new CMS.
So a CustomCategory can be mapped to a CMSSection, a CMSContext or both.
What we are doing is the following:
1) For each class DAO, say DAOVGNProject for VGNProject. All DAO's are singleton classes. This classes are responsible for fetching data from the databases, and the "build" the domain objects accordingly. In the case of data concerining a VGNProject and a CustomCategory they are all loaded in the begining of the migration process in to Hashtables appropriatly. This was done becouse all content is actually grouped by this concepts, and becouse these concepts are hierachical. It's much easier and faster to search hirarchies data in memory the in the database.
2) For each group of objects we created a OAM (Object Access Manager). As it would facilitate implementation we decided that a VGNProject and a CustomCategory implement an ICategory interface. So we created a OAMCategory class that provide object level services. This class is a sigleton class that orchestrates the relevant DAO classes to get categories from the legacy system. We have planned to to implement another OAM class called OAMContent that will retrieve any peace of content in the legacy system. You can call an OAM a service, according to fowler. Business services such as mapToVGNProject, mapToSection, mapToContext are actually performed by the business objects.
In conclusion:
DAO - provides the interface to talk to the database and works as a factory of PD Objects. It implements fast finder methods etc etc. This can be mapped to 2.
OAM - orchestrates DAO objects to retrieve the relevant objects. Let's say it hides the complexty of DAO's from the migration tool. Works. There is no mapping for this within your email. But you can easily argue that an OAM is not needed as the migration tool collaborated directly with PD objects.
So as you can see we are working with an external systems from the the point of view of the Migration tool still we use DAO's (The migration tool does not own any database). This DAO's are not in the SI but in the DM layer of the migration tool. Wether it should be in the SI layer its debateble. In the end of the day you decide.
Sorry for this lengthy post, but probably now you can understand this things better. Not everything is black and white. Well in theory might be.
Best regards,
Nuno Lopes
How do you abstract what is not common? (rhetorical)
What do they have in common? DM - as in persistence, is a fairly well understood problem with several well understood solutions. What do they have in commmon with the SI layer? SI layer is about how to encapsulate the way we access and map domain concepts to other systems. It's not about flattening and storing and reconstituting object graphs in an effective way at all. It's about how to semantically map domain concepts between systems (scoped implementations of domains) and how to handle conversation and protocol specific technologies for getting to the other system.
The pd-si pattern in the referenced thread is wholly unsuitable for persistence because the pattern was created to solve an entirely different kind of problem.
JeffI agree!
Hi Jeff,
Jeff - "The pd-si pattern in the referenced thread is wholly unsuitable for persistence because the pattern was created to solve an entirely different kind of problem."
I fully agree. In the case of the migration tool I mentioned we don't need persistance. So the DM in my case is all about data access and reconstructing objects from that, not persistance. If I understand DM layer is all about accessing the database to store and restrive/find PD objects (store and retrive the related object data). SI is all about accessing external systems as you mentioned. A system is external if the current system does not mandate either data , business logic or protocols regarding that system. In the case of the migration tool the current system does not own the data or the business logic of either the Vignette CMS or the new homegrown CMS, but still we decided to access directly the Vignette CMS database due to technical constraints, that is why I mentioned it is not black and white what one can classify in this case as DM "features" and what one can classify as SI "features".
mangrish - "I could bundle everything into the si layer quite easily except for the DM layer. For me it is an interesting excercise to see if you could abstract the two layers to be just one as my friend suggested, or if it will never be possible to do practically (never is a dirty word!)."
No you can't 99% of the times, but even if you could but that exercise would be futile. Becouse in the end of the day talking the both layers will return PD objects that encapsulate both data and business logic. Wether that business logic regarding a specific object is processed locally or by an external system should mostly irrelevant for the clients of such object.
Jeff - "What do they have in commmon with the SI layer?"
Very close to zero. Only in such cases such as data Migration systems bypassing external systems API's and protocols to get data things get blurry.
To solve this I decided that whatever code calls the a database and creates objects go to the DM layer, whatever code calls external API's goes to the SI layer. There is no benefit in putting everything in the same umbrella (or layer).
Nuno Lopes
They aren't systems
A database and a message queue aren't external systems.
I presume a message queue is how you get to some external system? For example, assuming a pd CreditCard class with a getBalance() method on it that is provided by Cardpac or VisionPlus on a mainframe, then the implementation of the (si) CreditCardSystem.getBalance() method is the start point for dealing with how you get to/from a message queue. Thus the pd-si pattern described in the paper applies as-is.
A database? Explain please? Another systems data you don't want to access directly (i.e. via direct db access such as jdbc). You want code that provides an interface to it. And thus it becomes just another pd-si concern as in the referenced paper.
Jeff
My bad, but here is another question!
Yep, sorry I didn't explain that more clearly, hopefully the reply I gave Paul above makes more sense ;)
Just on the last point though, I have a question (it is quite general but pretend the project is all in control except this!)
Say you had another system A (say written in C using a DB2 database), and the system you were building B (say java and an oracle database) had to interact with it, but the existing functionality written in the system A didnt deliver what you required, would you
a) as a part of the programme being delivered, would you get some C guys in to give you the functionality you needed from system A? Then get your java SI guys to build to the newly defined contract between the systems?
b) directly deal with the DB2 database of system A, and if so in the DM layer or the SI layer?
a)
a)
Jeff
Don't talk directly to the database
To borrow a phrase from Martin Fowler, sell your favourite grandmother in preference to doing (b).
Consider the database private. You aren't allowed to access it directly. The whole point of the system is to abstract away from that. The storage may change, but any external interface should remain the same. Encapsulation.
Paul.
Separation of the model from the model's implementation
"how is this then applied to databases? It seems weird to have just one monolithic IPersistenceSystem, and it seems weird to have ICustomerPersistenceService, IAccountPersistenceService and ITransactionPersistenceService."
This is a conceptual answer that relates back to my comments in the other thread you started. The domain exists independant of any actual (or possible) implementation of the domain becuase the business process exists independant of a specific implementation of the business process. The domain is stable and is rarely affected by changes to domain implementations ('never is a long time' as my mother used to say). Persistence and SI relate to (and are the interface to) specific implementations and therefore are relatively unstable (as they should be). They are the interface between the stable domain and the unstable implementations that implement the domain. I hope it is obvious why it is important to separate the stable domain from the relatively unstable implementations that support it (i.e. provide a specific implementation).
Maybe an example would make this clearer. I have a business process - recieve order. There are many possible ways I can implement this; by telephone, filling in a form in person, email, website, salesrep (and potentially others, telepathy, secret signs painted on the sidewalk). The important point is that the order is the same irespective of how I received it and I can say there is a recieve order process independent of how the order is received. The domain model represents the order and the recieve order process irrespective of how it is implemented.
So to answer your implied question. Ideally the perfect interface between the domain and the implementations that support it is .IJustDoIt(). I.e. maximum separation is the optimum solution.