Easy way to make Numarray Bool use 1 bit?
I need to be able to store thousands of one-million-length bit arrays.
The first operations use the & operator. But eventually I must perform
an inner product with a one-million-length integer array (summing at
all the indices of the integer operand where the boolean operand has a
If I use the Boolean type, I end up using 1-byte, per needed bit, which
is not efficient enough (only 1,000 arrays per gigabyte of memory).
I'm limited by 32-bit windows to use only 2 GB total.
Anybody know of a fast way to use only 125 kilobytes of memory per
array and yet be able to inner product with an integer array quickly?
Don't tell me I have to write C/C++ ! (ugh!)
Re: Easy way to make Numarray Bool use 1 bit?
Ahh, I figured it out. The key is to set up the shapes so they
broadcast like an outerproduct, but using bitwise_and as the operation
and the second operand as the bit filter. So we temporarily expand
each bit to a byte for numarray to be able to use it for dot products.
If you use a temporary Boolean variable to store the bitwise_and ufunc
output, the answer is automatically converted to boolean.
Set up the shapes for an innerproduct-like operation to go from byte
format back to bit compact Uint8 format. Go numarray!
|All times are GMT. The time now is 08:19 PM.|
Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.