Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > Newbie Question: Best way to Extract Post Hashes?

Reply
Thread Tools

Newbie Question: Best way to Extract Post Hashes?

 
 
Andrew
Guest
Posts: n/a
 
      11-21-2003
Hello,

I am a Perl newbie and I was wondering if anyone could tell me the
best way to extract expected post keys and values from websites? is
there an easy way to extract the expected post/value from a web site
with the intention of using the information to write an agent or
spider script for? Does anyone know of way to automatically gather it
and post dynamically?


Thanks!
 
Reply With Quote
 
 
 
 
Tad McClellan
Guest
Posts: n/a
 
      11-21-2003
Andrew <(E-Mail Removed)> wrote:

> I am a Perl newbie



You are expected to check the Perl FAQs before posting to
the Perl newsgroup.


> and I was wondering if anyone could tell me the
> best way to extract expected post keys and values from websites?



Use the CGI module for that:

perldoc CGI


> is
> there an easy way to extract the expected post/value from a web site
> with the intention of using the information to write an agent or
> spider script for?



I'm afraid I cannot make sense of your question there. Sorry.


> Does anyone know of way to automatically gather it
> and post dynamically?



perldoc -q CGI

What is the correct form of response from a CGI script?

How do I decode a CGI form?


perldoc -q HTML

How do I fetch an HTML file?

How do I automate an HTML form submission?



--
Tad McClellan SGML consulting
http://www.velocityreviews.com/forums/(E-Mail Removed) Perl programming
Fort Worth, Texas
 
Reply With Quote
 
 
 
 
Ben Liddicott
Guest
Posts: n/a
 
      11-22-2003
See the CGI group of modules.

# perldoc CGI

In general, the documentation which comes with a distribution is very good, and is accessible through the perldoc command.

For more information:

# perldoc perldoc

--
Cheers,
Ben Liddicott

"Andrew" <(E-Mail Removed)> wrote in message news:(E-Mail Removed) om...
> Hello,
>
> I am a Perl newbie and I was wondering if anyone could tell me the
> best way to extract expected post keys and values from websites?

 
Reply With Quote
 
James Willmore
Guest
Posts: n/a
 
      11-22-2003
[THIS is top posting - please DON'T do this]

On Sat, 22 Nov 2003 10:34:27 +0000 (UTC)
"Ben Liddicott" <(E-Mail Removed)> wrote:
> See the CGI group of modules.
>
> # perldoc CGI


This is a *single* module - not a "group". There are other modules
that have the prefix "CGI", but this command will provide
documentation for a *single* module.

<snip>

--
Jim

Copyright notice: all code written by the author in this post is
released under the GPL. http://www.gnu.org/licenses/gpl.txt
for more information.

a fortune quote ...
Those of you who think you know everything are very annoying to
those of us who do.
 
Reply With Quote
 
Eric J. Roode
Guest
Posts: n/a
 
      11-22-2003
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

(E-Mail Removed) (Andrew) wrote in news:9b7dcecd.0311211429.3c7a7735
@posting.google.com:

> Hello,
>
> I am a Perl newbie and I was wondering if anyone could tell me the
> best way to extract expected post keys and values from websites? is
> there an easy way to extract the expected post/value from a web site
> with the intention of using the information to write an agent or
> spider script for? Does anyone know of way to automatically gather it
> and post dynamically?


I have been a web programmer for eight years, and I have no clue what you
mean by "expected post keys and values". Can you give some more details
about what you are trying to do?

- --
Eric
$_ = reverse sort $ /. r , qw p ekca lre uJ reh
ts p , map $ _. $ " , qw e p h tona e and print

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

iQA/AwUBP79Wl2PeouIeTNHoEQK1hACfX+Z6OXkItewefV39bPOaMI BAAj4AoMP+
iKKbBa3dkinz64meSFD/DG6c
=Xn84
-----END PGP SIGNATURE-----
 
Reply With Quote
 
Juha Laiho
Guest
Posts: n/a
 
      11-23-2003
"Eric J. Roode" <(E-Mail Removed)> said:
>(E-Mail Removed) (Andrew) wrote in news:(E-Mail Removed) om:
>> I am a Perl newbie and I was wondering if anyone could tell me the
>> best way to extract expected post keys and values from websites?

[...]
>I have been a web programmer for eight years, and I have no clue what you
>mean by "expected post keys and values". Can you give some more details
>about what you are trying to do?


For some reason his wording made slight problems for me as well, but
what he apparently wants to write is software that:
- downloads a HTML form
- parses the form to determine which fill-in elements it contains
- somehow generate data for those elements
- submits the filled-in form appropriately

Now, the hard parts here are
- to determine what data is requested by the form
- what are the allowable values for free-text fields
- what all can be filled in to the form, if it happens to be something
that is partially generated by client-side technologies while the user
is filling it
--
Wolf a.k.a. Juha Laiho Espoo, Finland
(GC 3.0) GIT d- s+: a C++ ULSH++++$ P++@ L+++ E- W+$@ N++ !K w !O !M V
PS(+) PE Y+ PGP(+) t- 5 !X R !tv b+ !DI D G e+ h---- r+++ y++++
"...cancel my subscription to the resurrection!" (Jim Morrison)
 
Reply With Quote
 
Andrew
Guest
Posts: n/a
 
      11-24-2003
Juha Laiho <(E-Mail Removed)> wrote in message news:<bpr0ri$k4v$(E-Mail Removed)-int>...
> "Eric J. Roode" <(E-Mail Removed)> said:
> >(E-Mail Removed) (Andrew) wrote in news:(E-Mail Removed) om:
> >> I am a Perl newbie and I was wondering if anyone could tell me the
> >> best way to extract expected post keys and values from websites?

> [...]
> >I have been a web programmer for eight years, and I have no clue what you
> >mean by "expected post keys and values". Can you give some more details
> >about what you are trying to do?

>
> For some reason his wording made slight problems for me as well, but
> what he apparently wants to write is software that:
> - downloads a HTML form
> - parses the form to determine which fill-in elements it contains
> - somehow generate data for those elements
> - submits the filled-in form appropriately
>
> Now, the hard parts here are
> - to determine what data is requested by the form
> - what are the allowable values for free-text fields
> - what all can be filled in to the form, if it happens to be something
> that is partially generated by client-side technologies while the user
> is filling it


Hello All,

Sorry about the confusion in my wording. What I was looking for tips
on how to automatically detect all necessary post values the form is
expecting. For example, When I look at the source code for
my.yahoo.com, there are several hidden attributes that it is
requesting aside form the obvious username and password fields. I was
wondering if anyone knew of a module that could help identify these
hidden values.

Thanks for the idea of checking out the CGI module. I havent used
that module yet and I am unaware of what it contains. I will check
out the perldocs on that.
 
Reply With Quote
 
Eric J. Roode
Guest
Posts: n/a
 
      11-24-2003
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

(E-Mail Removed) (Andrew) wrote in
news:(E-Mail Removed) om:

> Sorry about the confusion in my wording. What I was looking for tips
> on how to automatically detect all necessary post values the form is
> expecting. For example, When I look at the source code for
> my.yahoo.com, there are several hidden attributes that it is
> requesting aside form the obvious username and password fields. I was
> wondering if anyone knew of a module that could help identify these
> hidden values.


Ah. Sounds like you need a module for parsing HTML, such as
HTML::TokeParser.

- --
Eric
$_ = reverse sort $ /. r , qw p ekca lre uJ reh
ts p , map $ _. $ " , qw e p h tona e and print

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 7.0.3 for non-commercial use <http://www.pgp.com>

iQA/AwUBP8H3sGPeouIeTNHoEQIJpQCg2aK/tSBPzeK0N8+uPrMaaElrPdkAoK+b
wWeoC/ZiZYUbNottDhOoNuli
=Eqi2
-----END PGP SIGNATURE-----
 
Reply With Quote
 
Clayton L. Scott
Guest
Posts: n/a
 
      11-26-2003
Eric J. Roode <(E-Mail Removed)> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1


> (E-Mail Removed) (Andrew) wrote in
> news:(E-Mail Removed) om:


>> Sorry about the confusion in my wording. What I was looking for tips
>> on how to automatically detect all necessary post values the form is
>> expecting. For example, When I look at the source code for
>> my.yahoo.com, there are several hidden attributes that it is
>> requesting aside form the obvious username and password fields. I was
>> wondering if anyone knew of a module that could help identify these
>> hidden values.


> Ah. Sounds like you need a module for parsing HTML, such as
> HTML::TokeParser.


Or a way to submit forms WWW::Mechanise

CLayton
 
Reply With Quote
 
kokit
Guest
Posts: n/a
 
      01-16-2004
Juha Laiho wrote:
> "Eric J. Roode" <(E-Mail Removed)> said:
>
>>(E-Mail Removed) (Andrew) wrote in news:(E-Mail Removed) om:
>>
>>>I am a Perl newbie and I was wondering if anyone could tell me the
>>>best way to extract expected post keys and values from websites?

>
> [...]
>
>>I have been a web programmer for eight years, and I have no clue what you
>>mean by "expected post keys and values". Can you give some more details
>>about what you are trying to do?

>
>
> For some reason his wording made slight problems for me as well, but
> what he apparently wants to write is software that:
> - downloads a HTML form
> - parses the form to determine which fill-in elements it contains
> - somehow generate data for those elements
> - submits the filled-in form appropriately
>
> Now, the hard parts here are
> - to determine what data is requested by the form
> - what are the allowable values for free-text fields
> - what all can be filled in to the form, if it happens to be something
> that is partially generated by client-side technologies while the user
> is filling it


use CGI.pm module

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Best way to extract from regex in if statement bwgoudey Python 5 04-04-2009 02:26 PM
How do i extract vidios when winrar wont extract them??? help plzzzzzzzz smuttdog@sc.rr.com Computer Support 2 12-23-2007 07:03 AM
Re: Best way to extract an item from a set of len 1 Fredrik Lundh Python 2 01-26-2006 05:13 PM
Best way to extract an item from a set of len 1 Tim Chase Python 8 01-26-2006 08:11 AM
best way to extract the data from several tables into a file mike hengins Java 11 08-19-2005 05:18 AM



Advertisments