I frequently encounter a situation where I need to write a program which reads data-objects from a file or system, does a transformation on the data, convert the data to another datamodel and then push the data in another file, or system. Do you know of a object-model which facilitates these type of programs in such a way that it is pluggable and scalable?
1
-
It sounds like you are talking about a pipe and filter architecture?Macneil– Macneil2010-11-27 18:36:02 +00:00Commented Nov 27, 2010 at 18:36
Add a comment
|
2 Answers
here's a link to an article on an alleged Extract/Transform/Load (ETL) design pattern but it sounds like what you want is a framework
-
Thanks for the link, but I'm more looking for a GoF like design pattern, inlcuding Objects, relations etc.Edward Smit– Edward Smit2010-11-27 15:21:00 +00:00Commented Nov 27, 2010 at 15:21
-
@Edward: if you find one be sure to edit your question and publish it; ETL is not a GoF design pattern. There are ETL frameworks however. good luck!Steven A. Lowe– Steven A. Lowe2010-11-27 15:23:09 +00:00Commented Nov 27, 2010 at 15:23
Read Enterprise Application Integration Patterns.
There is a website related to it here that contains a wealth of inter-application processing design patterns for you to browse. In particular, I'd draw your attention to these patterns:
Messaging How can I integrate multiple applications so that they work together and can exchange information?
Message Channel How does one application communicate with another using messaging?
Message Router How can you decouple individual processing steps so that messages can be passed to different filters depending on a set of conditions?
I've got a copy of the book myself and it's packed with detail about inter-application and inter-system design patterns.
-
Thanks Gary, I have the book myself :) I do appreciate this book for SOA based design, but I'm looking for a little bit more lowlevel, and need a detailed Object-design.Edward Smit– Edward Smit2010-11-28 08:18:34 +00:00Commented Nov 28, 2010 at 8:18