«

I don't even really know how to describe this problem. I know what I
want the end result to be, but I've run into a dead end trying to figure
out possible solutions.

I've got a script which looks for files of a given type in a directory,
processes them, then moves the result elsewhere and removes the sources.

Currently, if I add another file to the directory while the script is
running, it will not be noticed (I'm using a simple directory read).

I would like to be able to add file while the script is busy and have it
append the new filename to its list. I've considered inotify type
monitoring but I don't know much about it and if it could be made to fit
this single-threaded script.

I've also thought about re-reading the directory after each file
operation is complete, and adding the previously unknown filenames to
the list.

I'm just not sure which way I should begin looking, if there's a good
solution for this already out there, or...what.

Thanks for some input

Matt Harrison <> wrote:
> I would like to be able to add file while the script is busy and
> have it append the new filename to its list. I've considered inotify
> type monitoring but I don't know much about it and if it could be
> made to fit this single-threaded script.

Inotify descriptors have an internal queue[1] so events can be
backlogged while your script is doing something else.

The sleepy_penguin[2] RubyGem provides inotify bindings that are
subclasses of Ruby IO objects, so you can IO.select on them in
an event loop (sleepy_penguin also provides an epoll wrapper).

sleepy_penguin is Linux-only and a fairly "raw" interface to the
underlying system calls. Cool.io (libev wrapper)[3] and EventMachine[4]
also support inotify and they may be more portable to systems without
inotify (using kqueue or stat() loops).

> I've also thought about re-reading the directory after each file
> operation is complete, and adding the previously unknown filenames
> to the list.

I had to do this once in a daemon since it ran on an NFS directory
since inotify doesn't work on NFS :<


[1] The inotify(7) manpage is a great resource for learning about
how it works.
[2] http://bogomips.org/sleepy_penguin (disclaimer: I'm the author)
[3] http://coolio.github.com/
[4] http://rubyeventmachine.com/
--
Eric Wong

On 16/06/2011 00:10, Eric Wong wrote:
> Matt Harrison<> wrote:
>> I would like to be able to add file while the script is busy and
>> have it append the new filename to its list. I've considered inotify
>> type monitoring but I don't know much about it and if it could be
>> made to fit this single-threaded script.
>
> Inotify descriptors have an internal queue[1] so events can be
> backlogged while your script is doing something else.
>
> The sleepy_penguin[2] RubyGem provides inotify bindings that are
> subclasses of Ruby IO objects, so you can IO.select on them in
> an event loop (sleepy_penguin also provides an epoll wrapper).
>
> sleepy_penguin is Linux-only and a fairly "raw" interface to the
> underlying system calls. Cool.io (libev wrapper)[3] and EventMachine[4]
> also support inotify and they may be more portable to systems without
> inotify (using kqueue or stat() loops).
>
>> I've also thought about re-reading the directory after each file
>> operation is complete, and adding the previously unknown filenames
>> to the list.
>
> I had to do this once in a daemon since it ran on an NFS directory
> since inotify doesn't work on NFS :<
>
>
> [1] The inotify(7) manpage is a great resource for learning about
> how it works.
> [2] http://bogomips.org/sleepy_penguin (disclaimer: I'm the author)
> [3] http://coolio.github.com/
> [4] http://rubyeventmachine.com/

Thanks Eric,

I'll read up on inotify when I get some time. I'm not quite
understanding how it would work with a single-threaded process. I mean,
if it's busy processing data, I don't see how it could receive events.

I'm sure it'll become more clear as I read up.

Thanks

[Note: parts of this message were removed to make it a legal post.]

Hi!!!

I know what is the problem, PROBABLY.

I've used inotify with node to do the same, the reason why it isn't working
it's because the "inotify script" is probably waiting to a certain event
happen(like a file moved to the directory) and that isn't happening.

When a file is moved to that directory, a chain of events is sent by inotify
like "directory X is open, X file have been moved here, directory X is
closed".

I don't remember correctly, but there's events that:
opens a file to read
opens a file to write
and so on...

So that script is probably waiting for a event that NEVER happens.

Instead of "moving" the file to the directory, try to create a new one, or
PASTE the file and see if it works...

Inotify is simple, if you have the script waste a little of your time
checking what is going on and change the script into your needs.

I hope my answer were helpful to you!

2011/6/15 Matt Harrison <>

> I don't even really know how to describe this problem. I know what I want
> the end result to be, but I've run into a dead end trying to figure out
> possible solutions.
>
> I've got a script which looks for files of a given type in a directory,
> processes them, then moves the result elsewhere and removes the sources.
>
> Currently, if I add another file to the directory while the script is
> running, it will not be noticed (I'm using a simple directory read).
>
> I would like to be able to add file while the script is busy and have it
> append the new filename to its list. I've considered inotify type monitoring
> but I don't know much about it and if it could be made to fit this
> single-threaded script.
>
> I've also thought about re-reading the directory after each file operation
> is complete, and adding the previously unknown filenames to the list.
>
> I'm just not sure which way I should begin looking, if there's a good
> solution for this already out there, or...what.
>
> Thanks for some input
>
>
>


--
Thiago Fernandes Massa
11 83979414

On Wed, Jun 15, 2011 at 11:58 PM, Matt Harrison
<> wrote:
> I don't even really know how to describe this problem. I know what I want
> the end result to be, but I've run into a dead end trying to figure out
> possible solutions.
>
> I've got a script which looks for files of a given type in a directory,
> processes them, then moves the result elsewhere and removes the sources.
>
> Currently, if I add another file to the directory while the script is
> running, it will not be noticed (I'm using a simple directory read).
>
> I would like to be able to add file while the script is busy and have it
> append the new filename to its list. I've considered inotify type monitoring
> but I don't know much about it and if it could be made to fit this
> single-threaded script.
>
> I've also thought about re-reading the directory after each file operation
> is complete, and adding the previously unknown filenames to the list.

In this case you would need to keep track of all files seen already so
you do not add files multiple times.

> I'm just not sure which way I should begin looking, if there's a good
> solution for this already out there, or...what.
>
> Thanks for some input

Since your script is single threaded anyway and if you do not want to
change too much you could simply wrap your whole script in a loop
which is terminated if the reading of the directory does not find any
matching files any more, e.g.

files = nil

until (files = find_files).empty?
files.each do |f|
process f
move v
end
end

Kind regards

robert

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

On 16/06/2011 08:07, Robert Klemme wrote:
> On Wed, Jun 15, 2011 at 11:58 PM, Matt Harrison
> <> wrote:
>> I don't even really know how to describe this problem. I know what I want
>> the end result to be, but I've run into a dead end trying to figure out
>> possible solutions.
>>
>> I've got a script which looks for files of a given type in a directory,
>> processes them, then moves the result elsewhere and removes the sources.
>>
>> Currently, if I add another file to the directory while the script is
>> running, it will not be noticed (I'm using a simple directory read).
>>
>> I would like to be able to add file while the script is busy and have it
>> append the new filename to its list. I've considered inotify type monitoring
>> but I don't know much about it and if it could be made to fit this
>> single-threaded script.
>>
>> I've also thought about re-reading the directory after each file operation
>> is complete, and adding the previously unknown filenames to the list.
>
> In this case you would need to keep track of all files seen already so
> you do not add files multiple times.
>
>> I'm just not sure which way I should begin looking, if there's a good
>> solution for this already out there, or...what.
>>
>> Thanks for some input
>
> Since your script is single threaded anyway and if you do not want to
> change too much you could simply wrap your whole script in a loop
> which is terminated if the reading of the directory does not find any
> matching files any more, e.g.
>
> files = nil
>
> until (files = find_files).empty?
> files.each do |f|
> process f
> move v
> end
> end
>
> Kind regards
>
> robert
>

Thanks Robert,

It's such a simple solution but it's always the easy ones that I tend to
overlook.