Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Long strings as function parameters

Reply
Thread Tools

Long strings as function parameters

 
 
onlyonemc@gmail.com
Guest
Posts: n/a
 
      01-09-2005
I would like to have functions that operate on long strings, 10-100 MB.
In C I would of course pass a pointer to the string for a quick
function call. What is an efficient way to do this in python?
Cheers,
-mark

 
Reply With Quote
 
 
 
 
Irmen de Jong
Guest
Posts: n/a
 
      01-09-2005
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:
> I would like to have functions that operate on long strings, 10-100 MB.
> In C I would of course pass a pointer to the string for a quick
> function call. What is an efficient way to do this in python?


Err, just pass the string to the function?
In Python, all function arguments are passed by (object)reference.
So if you are afraid that Python passes your 50Mb string object
/by value/ (thus creating a copy): it doesn't.

--Irmen
 
Reply With Quote
 
 
 
 
Dan Bishop
Guest
Posts: n/a
 
      01-09-2005
(E-Mail Removed) wrote:
> I would like to have functions that operate on long strings, 10-100

MB.
> In C I would of course pass a pointer to the string for a quick
> function call. What is an efficient way to do this in python?
> Cheers,


In Python, *every* expression is a pointer. This fact is clearest when
you look a C-implemented Python functions (which have parameter types
and return types of PyObject*), or when using mutable types.

>>> class Spam:

.... def __init__(self):
.... self.foo = 0
....
>>> x = Spam()
>>> y = x
>>> y.foo = 17
>>> x.foo

17

Compare this to the C code:

typedef struct {int foo;} Spam;
....
Spam *x, *y;
....
y = x;
y->foo = 17;
printf("%d\n", x->foo);

 
Reply With Quote
 
Jeremy Bowers
Guest
Posts: n/a
 
      01-10-2005
On Sun, 09 Jan 2005 08:36:04 -0800, onlyonemc wrote:

> I would like to have functions that operate on long strings, 10-100 MB. In
> C I would of course pass a pointer to the string for a quick function
> call. What is an efficient way to do this in python? Cheers,
> -mark



Others have pointed out that there is no particular inefficient way to
pass a string.

However that only solves the immediate problem of passing. Once inside the
function you should be aware that all string operations that change the
string will create a new string, including slicing and such.

This may not be a problem if you are examining it in small chunks, but if
you routinely taking distinct multi-megabyte hunks out of it with slices,
you will be constructing and destructing a lot of strings.

I had thought there was an obvious class in the standard library to assist
with this, but I must have been wrong. I think you can load it as an array
from the array module (as long as it is an array of bytes and not an
encoding like UTF-8 or something), or you might be able to use the mmap
module if the string comes from a file. Both of those techniques can also
load the info directly from a file so there is only one copy needed.
(Wasn't there a "buffer" kind of class at one point, that you could slice
into and get something back that didn't make any copies of strings?)

Again, this is only an issue depending on usage, and you should probably
prototype it while ignoring these issues and see if it is fast enough. But
if it isn't, there are options.
 
Reply With Quote
 
Terry Reedy
Guest
Posts: n/a
 
      01-10-2005

"Dan Bishop" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed) oups.com...
> In Python, *every* expression is a pointer.


Minor but to me important nit: this should start "In the CPython
implementation of Python" ...

Humans can understand and execute Python code without knowing about
computer memory addresses (pointers). Other computer languages can also do
something slightly different from CPython, as I believe is the case with
Jython.

The Python language is defined by sematics, not by mechanisms, especially
low-level interpreter-specific mechanisms. This is a large part of what
makes it readable.

To the OP: a Python function call binds argument objects to either local
parameter names or slots within a named catchall list or dict. Like other
binding (assignment) operations, there is no copying.

Terry J. Reedy



 
Reply With Quote
 
Nick Coghlan
Guest
Posts: n/a
 
      01-10-2005
Jeremy Bowers wrote:
> I had thought there was an obvious class in the standard library to assist
> with this, but I must have been wrong.


buffer is the closest current contender, but I believe it's on the outer due to
some problems with its implementation.

I think the intention is to eventually have a 'bytes' type, but I don't recall
if that was going to be mutable or immutable.

Cheers,
Nick.

--
Nick Coghlan | (E-Mail Removed) | Brisbane, Australia
---------------------------------------------------------------
http://boredomandlaziness.skystorm.net
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Having compilation error: no match for call to (const __gnu_cxx::hash<long long int>) (const long long int&) veryhotsausage C++ 1 07-04-2008 05:41 PM
long long and long Mathieu Dutour C Programming 4 07-24-2007 11:15 AM
Strings, Strings and Damned Strings Ben C Programming 14 06-24-2006 05:09 AM
unsigned long long int to long double Daniel Rudy C Programming 5 09-20-2005 02:37 AM
Assigning unsigned long to unsigned long long George Marsaglia C Programming 1 07-08-2003 05:16 PM



Advertisments