Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Java > Distributed Java Programming Model

Reply
Thread Tools

Distributed Java Programming Model

 
 
howa
Guest
Posts: n/a
 
      12-10-2006
Hi,

Are there any recommended Java Programming Model or Framework that is
suitable for writing something like that:

1. A master program control many workers, each workers will perform
some cpu intensive computation

2. Therefore, it is perferred these workers run on remote PC

3. The master take care of monitoring, logging, and could restart the
worker if any problem, e.g. worker's process being killed

4. Finally, all the workers send result to master, and ask for other
jobs

 
Reply With Quote
 
 
 
 
Marc E
Guest
Posts: n/a
 
      12-10-2006
I do a lot of this, but not #3 with restarting. I use RMI.

so a farm of 10 servers are the workers, and each has RMI started, waiting
to accept requests. the master then sends requests via RMI and gets the
result. if the RMI invocation fails because the server isn't accepting
connections, it notifies people so they can check out that worker.

I use the spring framework for doing all this which makes the RMI stuff as
trivial as calling a method on the local machine. The only thing you have to
do is make sure the result that the RMI workers send back is a serializable
object, i.e. just add "implements Serializable" to your result object and
it's all good.




"howa" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed) ups.com...
> Hi,
>
> Are there any recommended Java Programming Model or Framework that is
> suitable for writing something like that:
>
> 1. A master program control many workers, each workers will perform
> some cpu intensive computation
>
> 2. Therefore, it is perferred these workers run on remote PC
>
> 3. The master take care of monitoring, logging, and could restart the
> worker if any problem, e.g. worker's process being killed
>
> 4. Finally, all the workers send result to master, and ask for other
> jobs
>



 
Reply With Quote
 
 
 
 
Chris Smith
Guest
Posts: n/a
 
      12-10-2006
howa <(E-Mail Removed)> wrote:
> Are there any recommended Java Programming Model or Framework that is
> suitable for writing something like that:


Are you looking for something that already exists? As Marc mentioned,
RMI and serialization provides a way to move the work around without a
lot of low-level network details. The rest of it, though, is not part
of RMI. Building the rest of the system is an interesting idea, but I
haven't seen it yet.

Incidentally, despite Marc's comment, I don't think Spring buys you
anything at all here; it looks like Marc is attributing to Spring
features of the basic RMI setup. I wouldn't add a framework like Spring
unless you have a real need for its features, and it's generally not
designed for this kind of endeavor.

In any case, if you were to make yourself familiar with RMI, building
something to handle the logging, monitoring, and coordination could be a
nice weekend task; it shouldn't be prohibitive.

--
Chris Smith
 
Reply With Quote
 
Lew
Guest
Posts: n/a
 
      12-11-2006
Marc E wrote:
> The only thing you have to
> do is make sure the result that the RMI workers send back is a serializable
> object, i.e. just add "implements Serializable" to your result object and
> it's all good.


After reading Joshua Bloch's _Effective Java_, wherein he deeply attacks the
myth of "just add 'implements Serializable'", I am skeptical. Making a class
Serializable for the long haul (he does say the simple method works for
throwaway classes) requires careful design.

The book explains it better than I can, but you need a serialVersionUid static
member, object serialization and deserialization routines, invariant
guarantors and more.

None of this matters if different versions of the class need not be
compatible, which makes it valid to "just add 'implements Serializable'" after
all. In that circumstance.

- Lew
 
Reply With Quote
 
Marc E
Guest
Posts: n/a
 
      12-11-2006
Very true, Chris. Never having worked with RMI before, and having the
project already using Spring, it was nice to just add a few lines of XML to
the context file and have the RMI plumbing handled for me

"Chris Smith" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed).. .
> howa <(E-Mail Removed)> wrote:
>> Are there any recommended Java Programming Model or Framework that is
>> suitable for writing something like that:

>
> Are you looking for something that already exists? As Marc mentioned,
> RMI and serialization provides a way to move the work around without a
> lot of low-level network details. The rest of it, though, is not part
> of RMI. Building the rest of the system is an interesting idea, but I
> haven't seen it yet.
>
> Incidentally, despite Marc's comment, I don't think Spring buys you
> anything at all here; it looks like Marc is attributing to Spring
> features of the basic RMI setup. I wouldn't add a framework like Spring
> unless you have a real need for its features, and it's generally not
> designed for this kind of endeavor.
>
> In any case, if you were to make yourself familiar with RMI, building
> something to handle the logging, monitoring, and coordination could be a
> nice weekend task; it shouldn't be prohibitive.
>
> --
> Chris Smith



 
Reply With Quote
 
howa
Guest
Posts: n/a
 
      12-11-2006

Chris Smith ¼g¹D¡G

> howa <(E-Mail Removed)> wrote:
> > Are there any recommended Java Programming Model or Framework that is
> > suitable for writing something like that:

>
> Are you looking for something that already exists? As Marc mentioned,
> RMI and serialization provides a way to move the work around without a
> lot of low-level network details. The rest of it, though, is not part
> of RMI. Building the rest of the system is an interesting idea, but I
> haven't seen it yet.
>
> Incidentally, despite Marc's comment, I don't think Spring buys you
> anything at all here; it looks like Marc is attributing to Spring
> features of the basic RMI setup. I wouldn't add a framework like Spring
> unless you have a real need for its features, and it's generally not
> designed for this kind of endeavor.
>
> In any case, if you were to make yourself familiar with RMI, building
> something to handle the logging, monitoring, and coordination could be a
> nice weekend task; it shouldn't be prohibitive.
>
> --
> Chris Smith


I am looking for some library or framework, such as IBM's DPPEJ.

I want to expore more in direction, since the underlying detail of
monitoring, load balacing can be hidden from the application layer.

 
Reply With Quote
 
Chris Uppal
Guest
Posts: n/a
 
      12-11-2006
howa wrote:

> 1. A master program control many workers, each workers will perform
> some cpu intensive computation
>
> 2. Therefore, it is perferred these workers run on remote PC
>
> 3. The master take care of monitoring, logging, and could restart the
> worker if any problem, e.g. worker's process being killed
>
> 4. Finally, all the workers send result to master, and ask for other
> jobs


Have you looked into JavaSpaces ?

(Most, perhaps all, of the material on the Web about JavaSpaces is laden with
idiotic "enterprisey" manager-oriented double-speak, if you can ignore that
then you should find that the underlying reality is quite simple, and /may/
provide the features you are looking for.)

-- chris


 
Reply With Quote
 
Patrick May
Guest
Posts: n/a
 
      12-13-2006
"howa" <(E-Mail Removed)> writes:
> Are there any recommended Java Programming Model or Framework that is
> suitable for writing something like that:
>
> 1. A master program control many workers, each workers will perform
> some cpu intensive computation
>
> 2. Therefore, it is perferred these workers run on remote PC
>
> 3. The master take care of monitoring, logging, and could restart the
> worker if any problem, e.g. worker's process being killed
>
> 4. Finally, all the workers send result to master, and ask for other
> jobs


As Chris Uppal has already noted, this is an ideal application
for a JavaSpace. The master-worker pattern is at the heart of many
uses of JavaSpaces. Here's an example:

http://www.gigaspaces.com/wiki/display/GS/Compute+Grid

Regards,

Patrick

------------------------------------------------------------------------
S P Engineering, Inc. | Large scale, mission-critical, distributed OO
| systems design and implementation.
http://www.velocityreviews.com/forums/(E-Mail Removed) | (C++, Java, Common Lisp, Jini, middleware, SOA)
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
High performance distributed programming on Java howa Java 0 02-15-2008 03:50 PM
Convert Java Model to Java Model without XML erinbot@gmail.com Java 1 10-06-2006 09:00 PM
Survey on distributed pair programming Riad Djemili Java 0 07-02-2006 08:14 PM
concurrent distributed programming online resources? manu.diaz.gomez@gmail.com C++ 2 02-03-2006 07:34 PM
ANN: YAMI 2.1 lib for distributed programming available Maciej Sobczak Python 0 07-08-2003 07:36 AM



Advertisments