Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > run perl program when there is new file?

Reply
Thread Tools

run perl program when there is new file?

 
 
Jie
Guest
Posts: n/a
 
      10-01-2007

Hi,

I have a folder where new files will be generated when some other
programs finished working.

Now, I need to write a perl program to store some text output files
generated from the other programs into a MySQL database. i think that
i could just write a perl code to generate a sql command file and
insert text into database.

The challenge is that this perl program needs to know what files have
already been processed, while only process on NEW files.. Maybe in
the mysql database, i could create a field to record the time of the
last run, then compare that time with the timestamp of all files to
find what files are new and therefore needs to be processed. if this
is the way to go, can you please let me know how to get the timestamp
of files? Also, to retrieve the time recorded in mySQL, the idea in my
mind now is to use something like below. There might be a better way,
i believe...
the_times = `mysql -u me - p the_password; select last_time from
dummp_table`

Thanks!!

Jie

 
Reply With Quote
 
 
 
 
Benoit Lefebvre
Guest
Posts: n/a
 
      10-01-2007
On Oct 1, 1:26 pm, Jie <(E-Mail Removed)> wrote:
> Hi,
>
> I have a folder where new files will be generated when some other
> programs finished working.
>
> Now, I need to write a perl program to store some text output files
> generated from the other programs into a MySQL database. i think that
> i could just write a perl code to generate a sql command file and
> insert text into database.
>
> The challenge is that this perl program needs to know what files have
> already been processed, while only process on NEW files.. Maybe in
> the mysql database, i could create a field to record the time of the
> last run, then compare that time with the timestamp of all files to
> find what files are new and therefore needs to be processed. if this
> is the way to go, can you please let me know how to get the timestamp
> of files? Also, to retrieve the time recorded in mySQL, the idea in my
> mind now is to use something like below. There might be a better way,
> i believe...
> the_times = `mysql -u me - p the_password; select last_time from
> dummp_table`
>
> Thanks!!
>
> Jie


Can you move the files once they have been processed by Perl ?

For the perl script, maybe you can add an entry to the crontab to
check every minutes if there is new files in the specified directory
and if there is new files, put the content in the MySQL database and
move the files somewhere else.

Or you can create a table of files and if the files listed in the
directory are not in the table, put it in the file list table and add
their content to the table where it should be.

If you can't play with the crontab, maybe you can add the SQL stuff at
the end of your other program.. or if you don't have the source you
can create a wrapper for this application that will run the
application and once it's done, add the output to the database

I'm pretty sure there is MANY other ways to do the job too.

--Ben

 
Reply With Quote
 
 
 
 
Ben Morrow
Guest
Posts: n/a
 
      10-01-2007

Quoth Jie <(E-Mail Removed)>:
>
> I have a folder where new files will be generated when some other
> programs finished working.
>
> Now, I need to write a perl program to store some text output files
> generated from the other programs into a MySQL database. i think that
> i could just write a perl code to generate a sql command file and
> insert text into database.


Better would be to use

perldoc DBI
perldoc DBD::mysql

If you don't have DBI installed, start with

perldoc CPAN

basically, you need to run

perl -MCPAN -e'install DBI'

and answer all the prompts. Defaults are usually good if you don't
understand the questions .

> The challenge is that this perl program needs to know what files have
> already been processed, while only process on NEW files.. Maybe in
> the mysql database, i could create a field to record the time of the
> last run, then compare that time with the timestamp of all files to
> find what files are new and therefore needs to be processed. if this
> is the way to go, can you please let me know how to get the timestamp
> of files?


perldoc -f stat

and also I find

perldoc File::stat

very helpful.


> Also, to retrieve the time recorded in mySQL, the idea in my
> mind now is to use something like below. There might be a better way,
> i believe...


DBI again. The answer to almost any database-related question in Perl
involves DBI .

Ben

 
Reply With Quote
 
Dr.Ruud
Guest
Posts: n/a
 
      10-02-2007
Jie schreef:

> The challenge is that this perl program needs to know what files have
> already been processed, while only process on NEW files.


Mimic how maildir-type delivery works (new. tmp, cur).

--
Affijn, Ruud

"Gewoon is een tijger."
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Can parellelized program run slower than single process program? Yesterday Paid Python 1 06-21-2012 05:58 AM
Program will not run, used to run, have reinstalled numerous times,no luck Boppy NZ Computing 15 01-24-2011 10:30 AM
How to get inputs for a python program that run from another python program pradeep nair Python 2 06-12-2007 01:20 PM
Can Java program evoke and run a FORTRAN program directly? Shawn Java 2 12-06-2006 07:08 PM
[python] Is there a python written fax program out there? David Stockwell Python 2 06-08-2004 08:28 PM



Advertisments