Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C++ > Producer consumer architecture

Reply
Thread Tools

Producer consumer architecture

 
 
hg@x-formation.com
Guest
Posts: n/a
 
      10-19-2007
As a part of a TCP/IP server I want to split up the incoming socket
requests from the work queue.
To do this I have one thread which figures out which sockets are
pending to be queried and another set of workerthreads which actually
does this work.

So far I've used std::list as a means of pushing a request from the
socket thread to the worker threads.
Each worker thread would then look every 10 ms to see if there is a
job in the queue.

Now I'm facing a problem though as there can be more requests of the
same job coming from the socket thread.
This has lead to concurrency problems when closing sockets (e.g. one
worker thread detects the socket is done communicating and next one
believes there is still action to perform).

Therefore I want to remove duplicates when adding jobs to the work
queue.
So I'm looking for a replacement container which will tell me rather
fast if a key element already exists and if preferedly still have this
FIFO behavior of std::list.
It's not unusual that there can be several thousands (e.g. up to
10000) objects in the worker queue.

Should I aim for anything better than std::list? The only thing list
won't give me is fast duplicate handling.

Thanks.

-- Henrik

 
Reply With Quote
 
 
 
 
Victor Bazarov
Guest
Posts: n/a
 
      10-19-2007
wrote:
> As a part of a TCP/IP server I want to split up the incoming socket
> requests from the work queue.
> To do this I have one thread which figures out which sockets are
> pending to be queried and another set of workerthreads which actually
> does this work.
>
> So far I've used std::list as a means of pushing a request from the
> socket thread to the worker threads.
> Each worker thread would then look every 10 ms to see if there is a
> job in the queue.
>
> Now I'm facing a problem though as there can be more requests of the
> same job coming from the socket thread.
> This has lead to concurrency problems when closing sockets (e.g. one
> worker thread detects the socket is done communicating and next one
> believes there is still action to perform).
>
> Therefore I want to remove duplicates when adding jobs to the work
> queue.
> So I'm looking for a replacement container which will tell me rather
> fast if a key element already exists and if preferedly still have this
> FIFO behavior of std::list.
> It's not unusual that there can be several thousands (e.g. up to
> 10000) objects in the worker queue.
>
> Should I aim for anything better than std::list? The only thing list
> won't give me is fast duplicate handling.


What if you split the job of keeping the order and preventing duplicates
between, say, std::list and std::set? Let your std::set store the
iterators to the std::list of objects. Make sure the comparator for
the std::set is implemented properly (it can be tricky since the functor
isn't returning the [in]equality but rather the less-than relationship).

V
--
Please remove capital 'A's when replying by e-mail
I do not respond to top-posted replies, please don't ask


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Thread Pool (Producer / Consumer) matt_kennelly@hotmail.com Java 0 04-22-2006 08:14 PM
producer/consumer remove problem Jeff Java 4 10-22-2004 03:31 AM
Producer - Consumer Threads why use "while" loop instead of "if"? Usenet Poster!!! Java 4 09-30-2004 03:48 PM
Simple Producer/Consumer Thread Question Buck Turgidson Java 5 02-21-2004 03:58 AM
Speed comparison of JAI to Producer/Consumer Mark McKay Java 0 12-09-2003 06:00 PM



Advertisments
 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57