Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   C Programming (http://www.velocityreviews.com/forums/f42-c-programming.html)
-   -   What am I doing wrong? curl (http://www.velocityreviews.com/forums/t439308-what-am-i-doing-wrong-curl.html)

Shutdownrunner 09-05-2005 07:18 PM

What am I doing wrong? curl
 
I want to store result of curl in a variable, which means to store a
webpage in a variable in order to parse it later and get our some useful
information. But unfortunately I'm not too experienced in C and I'm
making some stupid mistake. Could someone help me solve it?
#include <string.h>
#include <curl/curl.h>


size_t write_data(const char *buffer, size_t size, size_t nmemb, char
*userp)
{
char *string = userp;
size_t len;
len = size * nmemb;
strncat(string, buffer,len); // I get segfault here. Why????
return len;
}



int main(int argc, char **argv)
{
char *tablica="";
CURL *curl;
CURLcode success;
curl = curl_easy_init();
if(curl) {
curl_easy_setopt(curl, CURLOPT_URL, "http://debian.org/");//just a
sample url.
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, tablica);
success = curl_easy_perform(curl);
curl_easy_cleanup(curl);
}
return 0;
}

Thanks in advance for your help.

Barry Schwarz 09-05-2005 09:05 PM

Re: What am I doing wrong? curl
 
On Mon, 05 Sep 2005 21:18:29 +0200, Shutdownrunner
<"shutdownrunner[NOSPAM]"@o2.pl> wrote:

>I want to store result of curl in a variable, which means to store a
>webpage in a variable in order to parse it later and get our some useful
>information. But unfortunately I'm not too experienced in C and I'm
>making some stupid mistake. Could someone help me solve it?
>#include <string.h>
>#include <curl/curl.h>


And we know what is in this header because ...

>
>
>size_t write_data(const char *buffer, size_t size, size_t nmemb, char
>*userp)
>{
> char *string = userp;
> size_t len;
> len = size * nmemb;
> strncat(string, buffer,len); // I get segfault here. Why????


write_data is called only through curl_easy_setopt. We have no idea
no idea how that is accomplished. What do buffer and userp point to?

> return len;
>}
>
>
>
>int main(int argc, char **argv)
> {
> char *tablica="";
> CURL *curl;
> CURLcode success;
> curl = curl_easy_init();
> if(curl) {
> curl_easy_setopt(curl, CURLOPT_URL, "http://debian.org/");//just a
>sample url.


curl_easy_setopt is a function taking three arguments. The third one
has a type consistent with pointer to char. It could be pointer to
void or const qualified.

> curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);


Here you call curl_easy_setopt and the third argument is a function
pointer. This is completely inconsistent unless curl_easy_setopt is a
variadic function. Is it?

> curl_easy_setopt(curl, CURLOPT_WRITEDATA, tablica);
> success = curl_easy_perform(curl);
> curl_easy_cleanup(curl);
> }
> return 0;
>}
>
>Thanks in advance for your help.



<<Remove the del for email>>

Shutdownrunner 09-05-2005 09:41 PM

Re: What am I doing wrong? curl
 
Barry Schwarz napisał(a):
>> curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);

>
>
> Here you call curl_easy_setopt and the third argument is a function
> pointer. This is completely inconsistent unless curl_easy_setopt is a
> variadic function. Is it?

I don't know if I understand you correctly, because I haven't heard
about variadic functions before. There's a paragraph in libcurl which says:
"Let's assume for a while that you want to receive data as the URL
identifies a remote resource you want to get here. Since you write a
sort of application that needs this transfer, I assume that you would
like to get the data passed to you directly instead of simply getting it
passed to stdout. So, you write your own function that matches this
prototype:

size_t write_data(void *buffer, size_t size, size_t nmemb, void *userp);

You tell libcurl to pass all data to this function by issuing a function
similar to this:

curl_easy_setopt(easyhandle, CURLOPT_WRITEFUNCTION, write_data); "

Userp can be FILE, but I wanted it to be a variable.

>> curl_easy_setopt(curl, CURLOPT_WRITEDATA, tablica);


I'm passing tablica to write_data, because I want all that is downloaded
by curl to be stored in this variable. I still have a lot to read about
local and global variables, so it might be not that necessary as I
think. Please tell me if there's any better solution to do what I want
to do.

I'm not getting any errors anymore. Unfortunately I only found a
temporary solution.
char tablica[100000];
tablica[0] = '\0';
I'm so used to PHP and not being forced to worry about memory
management:( Is it possible to change variables size at runtime? Because
I never know how big a webpage will be. E.g. I declare an empty
variable, read the buffer, see that it is currently e.g. 1000bytes long,
so I make my variable 1000bytes long, then I read buffer again and I see
than now it's 2000bytes long, so I add another 2000 bytes to my variable
(to make it 3000 bytes long), and so on and so forth. How do I do that?

Best regards,
Jacek

elzacho 09-05-2005 10:16 PM

Re: What am I doing wrong? curl
 

> I'm not getting any errors anymore. Unfortunately I only found a
> temporary solution.
> char tablica[100000];
> tablica[0] = '\0';
> I'm so used to PHP and not being forced to worry about memory
> management:( Is it possible to change variables size at runtime? Because
> I never know how big a webpage will be. E.g. I declare an empty
> variable, read the buffer, see that it is currently e.g. 1000bytes long,
> so I make my variable 1000bytes long, then I read buffer again and I see
> than now it's 2000bytes long, so I add another 2000 bytes to my variable
> (to make it 3000 bytes long), and so on and so forth. How do I do that?


So this is not too hard to do if you can tell how big the buffer needs
to be before you copy it to tablica. You can manage memory on the fly
by using malloc calls like so...

char *tablica;
....
tablica = (char *) malloc(buffersize * sizeof(char));
....
/* To make the buffer bigger: */
/* Free old memory */
free(tablica);
/* Allocate new memory */
tablica = (char *) malloc(newbuffersize * sizeof(char));

Ofcourse this loses all the stored values in tablica. To preserve this
data you would need to create a new buffer pointed to by a temp
varialbe and copy the memory over, then free the old buffer and have
tablica point to the new buffer.

Zach


Barry Schwarz 09-06-2005 02:45 AM

Re: What am I doing wrong? curl
 
On 5 Sep 2005 15:16:48 -0700, "elzacho" <elzacho@gmail.com> wrote:

>
>> I'm not getting any errors anymore. Unfortunately I only found a
>> temporary solution.
>> char tablica[100000];
>> tablica[0] = '\0';
>> I'm so used to PHP and not being forced to worry about memory
>> management:( Is it possible to change variables size at runtime? Because
>> I never know how big a webpage will be. E.g. I declare an empty
>> variable, read the buffer, see that it is currently e.g. 1000bytes long,
>> so I make my variable 1000bytes long, then I read buffer again and I see
>> than now it's 2000bytes long, so I add another 2000 bytes to my variable
>> (to make it 3000 bytes long), and so on and so forth. How do I do that?

>
>So this is not too hard to do if you can tell how big the buffer needs
>to be before you copy it to tablica. You can manage memory on the fly
>by using malloc calls like so...
>
>char *tablica;
>...
>tablica = (char *) malloc(buffersize * sizeof(char));


Casting the return from malloc only serves to hide real errors.

>...
>/* To make the buffer bigger: */
>/* Free old memory */
>free(tablica);
>/* Allocate new memory */
>tablica = (char *) malloc(newbuffersize * sizeof(char));
>
>Ofcourse this loses all the stored values in tablica. To preserve this
>data you would need to create a new buffer pointed to by a temp
>varialbe and copy the memory over, then free the old buffer and have
>tablica point to the new buffer.


Consider the realloc() function.



<<Remove the del for email>>

Barry Schwarz 09-06-2005 02:45 AM

Re: What am I doing wrong? curl
 
On Mon, 05 Sep 2005 23:41:28 +0200, Shutdownrunner
<"shutdownrunner[NOSPAM]"@o2.pl> wrote:

>Barry Schwarz napisał(a):
>>> curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);

>>
>>
>> Here you call curl_easy_setopt and the third argument is a function
>> pointer. This is completely inconsistent unless curl_easy_setopt is a
>> variadic function. Is it?

>I don't know if I understand you correctly, because I haven't heard
>about variadic functions before. There's a paragraph in libcurl which says:
>"Let's assume for a while that you want to receive data as the URL
>identifies a remote resource you want to get here. Since you write a
>sort of application that needs this transfer, I assume that you would
>like to get the data passed to you directly instead of simply getting it
>passed to stdout. So, you write your own function that matches this
>prototype:
>

snip

You need to discuss this in a newsgroup where libcurl is on-topic.


<<Remove the del for email>>

Keith Thompson 09-06-2005 03:47 AM

Re: What am I doing wrong? curl
 
"elzacho" <elzacho@gmail.com> writes:
>> I'm not getting any errors anymore. Unfortunately I only found a
>> temporary solution.
>> char tablica[100000];
>> tablica[0] = '\0';
>> I'm so used to PHP and not being forced to worry about memory
>> management:( Is it possible to change variables size at runtime? Because
>> I never know how big a webpage will be. E.g. I declare an empty
>> variable, read the buffer, see that it is currently e.g. 1000bytes long,
>> so I make my variable 1000bytes long, then I read buffer again and I see
>> than now it's 2000bytes long, so I add another 2000 bytes to my variable
>> (to make it 3000 bytes long), and so on and so forth. How do I do that?

>
> So this is not too hard to do if you can tell how big the buffer needs
> to be before you copy it to tablica. You can manage memory on the fly
> by using malloc calls like so...
>
> char *tablica;
> ...
> tablica = (char *) malloc(buffersize * sizeof(char));


Better:

tablica = malloc(buffersize);

> /* To make the buffer bigger: */
> /* Free old memory */
> free(tablica);
> /* Allocate new memory */
> tablica = (char *) malloc(newbuffersize * sizeof(char));
>
> Ofcourse this loses all the stored values in tablica. To preserve this
> data you would need to create a new buffer pointed to by a temp
> varialbe and copy the memory over, then free the old buffer and have
> tablica point to the new buffer.


Use realloc() for this.

One thing to watch out for is that realloc(), like malloc() can fail.
If you want to be able to recover from such an error, you need to
assign the result of realloc() to a separate variable; otherwise you
could lose your original data.

--
Keith Thompson (The_Other_Keith) kst-u@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.


All times are GMT. The time now is 10:47 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.