Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > Selecting Technology for a Data Filtering and Manipulation Project

Reply
Thread Tools

Selecting Technology for a Data Filtering and Manipulation Project

 
 
sengsational
Guest
Posts: n/a
 
      08-05-2008
New Project: SAP to i2 using Oracle (< sorry to do that to you in a
Java forum).

Last week I played with Oracle's JDeveloper a little bit. This week
I've been playing with Eclipse DTP with Oracle's plug in. I'm sure
I've only scratched the surface of possibilities, so thought I'd see
if there were any ideas from the community (that's you!).

The project is to manage and manipulate data between SAP's Sales and
Distribution (for demand data), various i2 supply chain apps, and back
to SAP Product Planning. Master data and transactional data. For
those coders not into manufacturing, all of that doesn't really
matter. The point is that between these commercial applications will
be periodic batch jobs that involve large amounts of data which needs
heavy manipulation sometimes, needs a significant amount of
persistence (Oracle will be used), and will need many batch jobs to
run in a coordinated way.

One solution would be to write a bunch of PL/SQL procedures and drive
those from an external batch scheduling system. But of course that
produces spaghetti in the way of cursors, temp tables, sub-sub-sub
selects that are impossible to diagnose, and most of all, doesn't
employ Java!

So the question is: What tool-set should be deployed to handle this
project so that it can be treated like a real application, and not as
a heap of steaming "stuff"? Is there a "clean" way to do this? There
are so many projects running on Eclipse, I don't know if any actually
fit this kind of project. Can an OR mapping idea be employed? BPEL
maybe? Which tools should I look at?

--Dale--
 
Reply With Quote
 
 
 
 
Abhijat Vatsyayan
Guest
Posts: n/a
 
      08-05-2008
sengsational wrote:
> New Project: SAP to i2 using Oracle (< sorry to do that to you in a
> Java forum).
>
> Last week I played with Oracle's JDeveloper a little bit. This week
> I've been playing with Eclipse DTP with Oracle's plug in. I'm sure
> I've only scratched the surface of possibilities, so thought I'd see
> if there were any ideas from the community (that's you!).
>
> The project is to manage and manipulate data between SAP's Sales and
> Distribution (for demand data), various i2 supply chain apps, and back
> to SAP Product Planning. Master data and transactional data. For
> those coders not into manufacturing, all of that doesn't really
> matter. The point is that between these commercial applications will
> be periodic batch jobs that involve large amounts of data which needs
> heavy manipulation sometimes, needs a significant amount of
> persistence (Oracle will be used), and will need many batch jobs to
> run in a coordinated way.
>
> One solution would be to write a bunch of PL/SQL procedures and drive
> those from an external batch scheduling system. But of course that
> produces spaghetti in the way of cursors, temp tables, sub-sub-sub
> selects that are impossible to diagnose, and most of all, doesn't
> employ Java!
>
> So the question is: What tool-set should be deployed to handle this
> project so that it can be treated like a real application, and not as
> a heap of steaming "stuff"? Is there a "clean" way to do this? There
> are so many projects running on Eclipse, I don't know if any actually
> fit this kind of project. Can an OR mapping idea be employed? BPEL
> maybe? Which tools should I look at?
>
> --Dale--

Sounds like an data integration/ETL (extraction, transformation and
loading) job for which something like Informatica or its competitors
should be a good choice. There are open source java based tools which I
haven't used but look into clover ETL and Pentaho data integration. And
search for (open source?) data integration tools.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Is selecting column, filtering , sorting, paging is done in database or memory when use linq to sql? Ryan Liu ASP .Net Web Controls 0 12-12-2008 06:40 PM
Creating a new data structure while filtering its data origin. Python 5 03-30-2007 02:37 AM
Layer 2 Filtering Project...... abhils44@gmail.com Cisco 1 09-15-2006 11:02 AM
"L" lens technology or "DO" technology?? ..Jan Plexy.. Digital Photography 4 10-04-2005 12:52 PM
Is technology making mankind a threatened species? Thoughts on technology, the future, and somwhere, ethics. Waylon Kenning NZ Computing 31 10-17-2004 09:44 AM



Advertisments