Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C++ > Trouble developing API for easy parallel/multithreaded programming

Reply
Thread Tools

Trouble developing API for easy parallel/multithreaded programming

 
 
k04jg02@gmail.com
Guest
Posts: n/a
 
      02-18-2008
I have an idea for an easy to use API that would allow some degree of
parallelism without the programmer thinking too hard about it. I'm
having some trouble figuring out how to implement it though. The idea
is you have tasks, and some tasks depend on other tasks having been
completed. These dependencies form a tree, e.g.:

R
/ \
A B
/ \ / \
C D E

So in this case, tasks A and B depend on R having finished, C depends
on A and R being finished, D depends on R, A, and B being finished,
etc. The programmer would explicitly build this tree, writing
something like this:

A.depends(R);
B.depends(R);
D.depends(A);
D.depends(B);

etc. After the user constructs the tree, they should just be able to
tell it to run: R.start(). This is where the parallelism comes in.
When R finishes, it should create two threads, one that runs A and one
that runs B. When A finishes, it should run C and somehow register to
D that it is ready. When B is finished it should run E and somehow
indicate to D that it is ready. When A and B are both done, a thread
running D should start.

What I've described so far isn't too difficult -- the only tricky part
is who creates the thread that runs D. Using either a mutex guarding a
pointer to D's thread or boost::call_once, I can make sure that the
thread running D only gets created once.

Where I'm stuck is for how these tasks should pass data to one
another. The idea is that child tasks depend on their parent being
completed because their parent is going to generate some data that
they will use. Rather than relying on the user to avoid accessing data
too early, I'd like to try and setup for this to be checked at compile
time. Basically, I need to somehow indicate that say A produces output
X, which should be input to task's C and D. C and D should know the
type of input they expect, and A should know the type of output it
produces, and that these match should be typechecked. My intuition is
that some template magic is the solution here, but the best solutions
I've come up with so far involve runtime checks.

Ideas?
 
Reply With Quote
 
 
 
 
Jim Langston
Guest
Posts: n/a
 
      02-18-2008
wrote:
> I have an idea for an easy to use API that would allow some degree of
> parallelism without the programmer thinking too hard about it. I'm
> having some trouble figuring out how to implement it though. The idea
> is you have tasks, and some tasks depend on other tasks having been
> completed. These dependencies form a tree, e.g.:
>
> R
> / \
> A B
> / \ / \
> C D E
>
> So in this case, tasks A and B depend on R having finished, C depends
> on A and R being finished, D depends on R, A, and B being finished,
> etc. The programmer would explicitly build this tree, writing
> something like this:
>
> A.depends(R);
> B.depends(R);
> D.depends(A);
> D.depends(B);
>
> etc. After the user constructs the tree, they should just be able to
> tell it to run: R.start(). This is where the parallelism comes in.
> When R finishes, it should create two threads, one that runs A and one
> that runs B. When A finishes, it should run C and somehow register to
> D that it is ready. When B is finished it should run E and somehow
> indicate to D that it is ready. When A and B are both done, a thread
> running D should start.
>
> What I've described so far isn't too difficult -- the only tricky part
> is who creates the thread that runs D. Using either a mutex guarding a
> pointer to D's thread or boost::call_once, I can make sure that the
> thread running D only gets created once.
>
> Where I'm stuck is for how these tasks should pass data to one
> another. The idea is that child tasks depend on their parent being
> completed because their parent is going to generate some data that
> they will use. Rather than relying on the user to avoid accessing data
> too early, I'd like to try and setup for this to be checked at compile
> time. Basically, I need to somehow indicate that say A produces output
> X, which should be input to task's C and D. C and D should know the
> type of input they expect, and A should know the type of output it
> produces, and that these match should be typechecked. My intuition is
> that some template magic is the solution here, but the best solutions
> I've come up with so far involve runtime checks.
>
> Ideas?


This question is a combination of comp.programming.threads and
comp.programming. We usually don't deal so much with algorithms here but
more specific C++ issues.

Now, there are a few ways to say
A depends on R at compile time.
class A: public R
{
};

class A
{
R inst;
};

But your input is at run time, not compile time.

If A and R were both derived from some common class I could see a
constructor requiring the creation of the other.

class A: public Base
{
public:
A( /* something */ ) { /* instantize Base* depending on something */ };
private:
Base* Required;
};

something could be a character such as 'R', or a type if used as a template.

You really haven't said what A, B, C etc.. are so can only guess.

I don't know, even this portion might be better served in comp.programming.

--
Jim Langston



 
Reply With Quote
 
 
 
 
k04jg02@gmail.com
Guest
Posts: n/a
 
      02-18-2008
Sorry, I was unclear. The graph is _not_ meant to be any sort of
inheritance relationship between classes. The idea is each letter, R,
A, B, C, D, E would be objects of some sort of Task class. The Task
class would have a method, call it Run(), that would cause the task to
do some work, generating data. When this work was finished, that data
would somehow be passed on to tasks dependent on the current task, and
if the dependent tasks had no other tasks to wait for they would
Run().

This is a C++ question rather than an algorithms/threading question in
the sense that I'm trying to figure out how to do the passing of data
from parent tasks to child tasks in a type safe way. I could just have
a bunch of globals representing each tasks output, but I'm going for
something generic and that will make sure children don't have access
to their parents data until it's done "cooking" (the parent task has
finished).

On Feb 18, 4:36 pm, "Jim Langston" <tazmas...@rocketmail.com> wrote:
> k04j...@gmail.com wrote:
> > I have an idea for an easy to use API that would allow some degree of
> > parallelism without the programmer thinking too hard about it. I'm
> > having some trouble figuring out how to implement it though. The idea
> > is you have tasks, and some tasks depend on other tasks having been
> > completed. These dependencies form a tree, e.g.:

>
> > R
> > / \
> > A B
> > / \ / \
> > C D E

>
> > So in this case, tasks A and B depend on R having finished, C depends
> > on A and R being finished, D depends on R, A, and B being finished,
> > etc. The programmer would explicitly build this tree, writing
> > something like this:

>
> > A.depends(R);
> > B.depends(R);
> > D.depends(A);
> > D.depends(B);

>
> > etc. After the user constructs the tree, they should just be able to
> > tell it to run: R.start(). This is where the parallelism comes in.
> > When R finishes, it should create two threads, one that runs A and one
> > that runs B. When A finishes, it should run C and somehow register to
> > D that it is ready. When B is finished it should run E and somehow
> > indicate to D that it is ready. When A and B are both done, a thread
> > running D should start.

>
> > What I've described so far isn't too difficult -- the only tricky part
> > is who creates the thread that runs D. Using either a mutex guarding a
> > pointer to D's thread or boost::call_once, I can make sure that the
> > thread running D only gets created once.

>
> > Where I'm stuck is for how these tasks should pass data to one
> > another. The idea is that child tasks depend on their parent being
> > completed because their parent is going to generate some data that
> > they will use. Rather than relying on the user to avoid accessing data
> > too early, I'd like to try and setup for this to be checked at compile
> > time. Basically, I need to somehow indicate that say A produces output
> > X, which should be input to task's C and D. C and D should know the
> > type of input they expect, and A should know the type of output it
> > produces, and that these match should be typechecked. My intuition is
> > that some template magic is the solution here, but the best solutions
> > I've come up with so far involve runtime checks.

>
> > Ideas?

>
> This question is a combination of comp.programming.threads and
> comp.programming. We usually don't deal so much with algorithms here but
> more specific C++ issues.
>
> Now, there are a few ways to say
> A depends on R at compile time.
> class A: public R
> {
>
> };
>
> class A
> {
> R inst;
>
> };
>
> But your input is at run time, not compile time.
>
> If A and R were both derived from some common class I could see a
> constructor requiring the creation of the other.
>
> class A: public Base
> {
> public:
> A( /* something */ ) { /* instantize Base* depending on something */ };
> private:
> Base* Required;
>
> };
>
> something could be a character such as 'R', or a type if used as a template.
>
> You really haven't said what A, B, C etc.. are so can only guess.
>
> I don't know, even this portion might be better served in comp.programming.
>
> --
> Jim Langston
> tazmas...@rocketmail.com


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Where to start for developing a service REST API? Markus Fischer Ruby 2 09-10-2010 03:02 PM
Developing Authentication and Roles thinking about API Giammarco ASP .Net 2 08-25-2005 02:10 PM
Trouble developing on remote web server. Terry Olsen ASP .Net 1 05-12-2005 05:22 PM
What easy technique shall i use in developing a java program that will output to screen or to a file changes/updates made inside a specific directory ...? bronby Java 6 04-22-2005 12:05 PM
easy to look at and easy to maintain web page menuing system. Hazzard ASP .Net 2 04-06-2004 03:51 AM



Advertisments
 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57