Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Perl > Perl Misc > skipping blank array items

Reply
Thread Tools

skipping blank array items

 
 
ccc31807
Guest
Posts: n/a
 
      09-06-2013
I have a csv file with 10K items. The header looks something like this:
CustNo,Name,FirstPaidItem,FirstOrderedItem,FirstSh ippedItem,Items

The first five fields are single valued. The last field (Items) has many items. Lines may look like this:

1,Joe,a,a,a,a,b,c,d,e,f,g,h,i
2,Jane,a,a,b,a,,,,,,a,b,c,d,e
3,Jim,b,a,a,a,b,c,d,e,f,g,h,i
4,Jill,b,b,b,b,,,,,c,,b,,e,f,g,h

I parse the file like this, putting the first five values in scalars and the remainder in the @items array:
foreach line
my ($custno,$name,$paid,$ordered,$shipped,@items) = split on the commas

The objective is to take the first item in the @items array and match it toone (or more) of $paid, $ordered, or $shipped. I just want to grab the first item in the @items array that has a value, IOW, skip all the blank items(as in line 2 above) that precede the first actual value.

Is there a quick and dirty way to do this without having to do this>

my $item = '';
foreach my $ele (@items)
{
$item = $ele if $ele =~ /\w/;
last;
}

Thanks, CC.
 
Reply With Quote
 
 
 
 
Rainer Weikusat
Guest
Posts: n/a
 
      09-06-2013
ccc31807 <(E-Mail Removed)> writes:
> I have a csv file with 10K items. The header looks something like this:
> CustNo,Name,FirstPaidItem,FirstOrderedItem,FirstSh ippedItem,Items
>
> The first five fields are single valued. The last field (Items) has many items. Lines may look like this:
>
> 1,Joe,a,a,a,a,b,c,d,e,f,g,h,i
> 2,Jane,a,a,b,a,,,,,,a,b,c,d,e
> 3,Jim,b,a,a,a,b,c,d,e,f,g,h,i
> 4,Jill,b,b,b,b,,,,,c,,b,,e,f,g,h
>
> I parse the file like this, putting the first five values in scalars and the remainder in the @items array:
> foreach line
> my ($custno,$name,$paid,$ordered,$shipped,@items) = split on the commas
>
> The objective is to take the first item in the @items array and match it to one (or more) of $paid, $ordered, or $shipped. I just want to grab the first item in the @items array that has a value, IOW, skip all the blank items (as in line 2 above) that precede the first actual value.
>
> Is there a quick and dirty way to do this without having to do this>
>
> my $item = '';
> foreach my $ele (@items)
> {
> $item = $ele if $ele =~ /\w/;
> last;
> }


use List::Util qw(first);

$item = first { /\w/ } @items;
$item = first { length() } @items;

But this is also trivial without using any module:

length() and $item = $_, last for @items;
(/\w/) and $item = $_, last for @items;



 
Reply With Quote
 
 
 
 
J. Gleixner
Guest
Posts: n/a
 
      09-06-2013
On 09/06/13 13:56, ccc31807 wrote:
> I have a csv file with 10K items. The header looks something like this:
> CustNo,Name,FirstPaidItem,FirstOrderedItem,FirstSh ippedItem,Items
>
> The first five fields are single valued. The last field (Items) has many items. Lines may look like this:
>
> 1,Joe,a,a,a,a,b,c,d,e,f,g,h,i
> 2,Jane,a,a,b,a,,,,,,a,b,c,d,e
> 3,Jim,b,a,a,a,b,c,d,e,f,g,h,i
> 4,Jill,b,b,b,b,,,,,c,,b,,e,f,g,h
>
> I parse the file like this, putting the first five values in scalars and the remainder in the @items array:
> foreach line
> my ($custno,$name,$paid,$ordered,$shipped,@items) = split on the commas


If items doesn't need to be an array...

my ($custno,$name,$paid,$ordered,$shipped,$items) = split(/,/,$line,6);
my ( $item ) = $items =~ /(\w)/;


>
> The objective is to take the first item in the @items array and match it to one (or more) of $paid, $ordered, or $shipped. I just want to grab the first item in the @items array that has a value, IOW, skip all the blank items (as in line 2 above) that precede the first actual value.
>
> Is there a quick and dirty way to do this without having to do this>
>
> my $item = '';
> foreach my $ele (@items)
> {
> $item = $ele if $ele =~ /\w/;
> last;
> }
>
> Thanks, CC.


 
Reply With Quote
 
ccc31807
Guest
Posts: n/a
 
      09-06-2013
On Friday, September 6, 2013 3:45:42 PM UTC-4, Rainer Weikusat wrote:
> use List::Util qw(first);
>
>
>
> $item = first { /\w/ } @items;
> $item = first { length() } @items;
>
>
>
> But this is also trivial without using any module:
>
>
>
> length() and $item = $_, last for @items;
> (/\w/) and $item = $_, last for @items;


FOR iterates through the array. I have not looked at List::Util but I will tomorrow. I wanted to avoid having to iterate through the array, which may have 1000 items, and the first item with an actual value may be 500 places down.

Thanks, CC.

 
Reply With Quote
 
ccc31807
Guest
Posts: n/a
 
      09-06-2013
On Friday, September 6, 2013 3:47:08 PM UTC-4, J. Gleixner wrote:
> If items doesn't need to be an array...


@items does not need to be an array, it could be a string. The database I mainly use is UniData, a multi-valued database where fields do not contain scalar values. Over the years, as a general rule, I find that using arrays rather than strings gives more consistent results overall, but there is no technical reason for using an array rather than a string.

> my ($custno,$name,$paid,$ordered,$shipped,$items) = split(/,/,$line,6);
> my ( $item ) = $items =~ /(\w)/;


Actually, all my 'items' consist of two digits, a forward slash, a character and another digit, like this: 13/T1. So the regex would be something likethis:
$items =~ m!,(\d\d/\w\d),!;
$item = $1;

Just off hand, is using a regex much faster than iterating through arrays? My script takes about 30 seconds to run and speed is not critical (I have all day to run it). What offended me was having to iterate through an array with potentially 1,000 elements just to grab the first one with a value.

Thanks, CC.
 
Reply With Quote
 
Rainer Weikusat
Guest
Posts: n/a
 
      09-06-2013
ccc31807 <(E-Mail Removed)> writes:

> On Friday, September 6, 2013 3:45:42 PM UTC-4, Rainer Weikusat wrote:
>> use List::Util qw(first);
>>
>>
>>
>> $item = first { /\w/ } @items;
>> $item = first { length() } @items;
>>
>>
>>
>> But this is also trivial without using any module:
>>
>>
>>
>> length() and $item = $_, last for @items;
>> (/\w/) and $item = $_, last for @items;

>
> FOR iterates through the array. I have not looked at List::Util but
> I will tomorrow. I wanted to avoid having to iterate through the
> array, which may have 1000 items, and the first item with an actual
> value may be 500 places down.


Eh ... provided you have an array with a number of empty leading
elements you want to skip but you don't know how many, how do you
propose to do that without 'looping through the array'?


 
Reply With Quote
 
Rainer Weikusat
Guest
Posts: n/a
 
      09-06-2013
Rainer Weikusat <(E-Mail Removed)> writes:
> ccc31807 <(E-Mail Removed)> writes:
>> On Friday, September 6, 2013 3:45:42 PM UTC-4, Rainer Weikusat wrote:


[...]

>>> length() and $item = $_, last for @items;
>>> (/\w/) and $item = $_, last for @items;

>>
>> FOR iterates through the array. I have not looked at List::Util but
>> I will tomorrow. I wanted to avoid having to iterate through the
>> array, which may have 1000 items, and the first item with an actual
>> value may be 500 places down.

>
> Eh ... provided you have an array with a number of empty leading
> elements you want to skip but you don't know how many, how do you
> propose to do that without 'looping through the array'?


Byzantinely recursive implementation:

sub fne { return (shift or &fne); }

NB: This will also skip over zeroes.
 
Reply With Quote
 
George Mpouras
Guest
Posts: n/a
 
      09-06-2013
Στις 6/9/2013 23:06, ο/η ccc31807 *γραψε:
> On Friday, September 6, 2013 3:45:42 PM UTC-4, Rainer Weikusat wrote:
>> use List::Util qw(first);
>>
>>
>>
>> $item = first { /\w/ } @items;
>> $item = first { length() } @items;
>>
>>
>>
>> But this is also trivial without using any module:
>>
>>
>>
>> length() and $item = $_, last for @items;
>> (/\w/) and $item = $_, last for @items;

>
> FOR iterates through the array. I have not looked at List::Util but I will tomorrow. I wanted to avoid having to iterate through the array, which may have 1000 items, and the first item with an actual value may be 500 places down.
>
> Thanks, CC.
>



what you can do is to run the two scripts:
a) one that building a trie index from your csv file
b) a second that give you the answers instantly without "fors" from the
index of of the a) script
but since you want only one execution per day I do not if this make
sense, also it is not quick and dirty
 
Reply With Quote
 
J. Gleixner
Guest
Posts: n/a
 
      09-06-2013
On 09/06/13 15:15, ccc31807 wrote:
> On Friday, September 6, 2013 3:47:08 PM UTC-4, J. Gleixner wrote:
>> If items doesn't need to be an array...

>
> @items does not need to be an array, it could be a string. [...]


>> my ($custno,$name,$paid,$ordered,$shipped,$items) = split(/,/,$line,6);
>> my ( $item ) = $items =~ /(\w)/;

>
> Actually, all my 'items' consist of two digits, a forward slash, a character and another digit, like this: 13/T1.

Oh, I guess my ESP isn't working again. Had you
provided that in your original post, you may have
received a more accurate solution. Regardless, we
don't really need to know your correct regex, you can
adjust it as needed without re-posting.

>So the regex would be something like this:
> $items =~ m!,(\d\d/\w\d),!;


Nope, the ',' in there avoid matching the first or last value. No need
to include the separator, in the regex, in this case.

> $item = $1;

Always test that the match was successful, never blindly assign $1.

dumb example...

$str = '123';
$str =~ /(\d+)/;
$val = $1;

$str =~ /(blah)/;
$val2 = $1; #oops


>
> Just off hand, is using a regex much faster than iterating through arrays? My script takes about 30 seconds to run and speed is not critical (I have all day to run it). What offended me was having to iterate through an array with potentially 1,000 elements just to grab the first one with a value.


You can answer that yourself using the Benchmark module.

 
Reply With Quote
 
ccc31807
Guest
Posts: n/a
 
      09-07-2013
On Friday, September 6, 2013 4:33:18 PM UTC-4, Rainer Weikusat wrote:
> sub fne { return (shift or &fne); }


I like that.

It reminds me of how I discovered to transpose a matrix in Common Lisp.

(defun transpose (matrix) (apply #'mapcar #'list matrix))

CC.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Each Array Method Skipping first Array position Chris R. Ruby 3 01-28-2011 04:00 PM
blank CD-R and blank DVD-R not recognized by Vista 64 Ultimate =?Utf-8?B?R3JlZyBLaXJrcGF0cmljaw==?= Windows 64bit 13 11-07-2007 12:23 PM
for loop skipping items Qubec C Programming 7 07-31-2004 12:36 AM
skipping an element in an array Ken C++ 9 05-24-2004 03:50 PM
Skipping content in Array within loop and go to the next (of NEXT function) Tad McClellan Perl Misc 3 05-13-2004 10:14 PM



Advertisments