«

On 2012-03-28, Ben Morrow <> wrote:
>
> Quoth Justin C <>:
>>
>> We have a database of thousands of clothing items. Some of the items are
>> almost identical apart from their size. Consequently we use the same
>> image in our web-shop to advertise items of the same style, design and
>> colour.
>>
>> In a program I have to get new images from the art guy's computer I end
>> up grepping the entire list of items $#(list-of-items) times, there must
>> be a better way. The file names are exactly the same as the style codes
>> apart from the size suffix being dropped. I'm using File::Find.
>>
>> Here's some code:
>>
>> find(\&set_flag, (keys %{ $stock_groups->{text2code} }));
>>
>> sub set_flag {
>> return unless (-f $_ );
>>
>> (my $item_code_part = $_) =~ s/\.jpg//;
>> $item_code_part = uc($item_code_part);
>> $item_code_part =~ s|_|/|g;
>>
>> my @matches = grep(/$item_code_part/, keys %{ $stock_items });
>
> Careful: you want \Q there, even if you think you're sure the filenames
> are all safe.

Done that, thank you.


>> foreach my $i (@matches) {
>> $stock_items->{$i}{got_image} = 1;
>> }
>> }
>
> I would probably turn this into a big pattern match. Something like
> this:
>
> use File::Find::Rule;
>
> my ($imgs) = map qr/$_/, join "|", map "\Q\U$_",
> map { (my ($x) = /(.*)\.jpg/) =~ s!_!/!g; $x }
> File::Find::Rule->file->in(keys %{...});
>
> while (my ($item, $entry) = each %$stock_items) {
> $item =~ $imgs and $entry->{got_image} = 1;
> }

That takes a bit of understanding, I'll have a read of the docs.


> If you're using 5.14 you can get rid of the ugly map block using s///r
> and tr///r:
>
> map tr!_!/!r, map s/\.jpg//r,

Still on 5.10, we follow Debian 'stable' and so won't be upgrading for a
while.

Thanks for the suggestions.

Justin.

--
Justin C, by the sea.

On 2012-03-29, J. Gleixner <glex_no-> wrote:
> On 03/28/12 10:24, Justin C wrote:
>> We have a database of thousands of clothing items. Some of the items are
>> almost identical apart from their size. Consequently we use the same
>> image in our web-shop to advertise items of the same style, design and
>> colour.
>>[...]
>> The bottle-neck, as I see it, is running grep 20k times, once for each
>> image found. Can anyone suggest a better way?
>
> Since you already have data in a DB, I'd suggest looking at
> associating these files, to the items, in the database.
>
> Maybe store the path to the file, or possibly the image as a BLOB, a
> many to one relationship.

Unfortunately it's not a DB I'm authorised to write to. I can run
queries that extract data, but any changes to the data must be done
through the 3rd party supplied interface.


Justin.

--
Justin C, by the sea.

On 2012-03-28 17:24, Justin C wrote:

> We have a database of thousands of clothing items. Some of the items are
> almost identical apart from their size. Consequently we use the same
> image in our web-shop to advertise items of the same style, design and
> colour.

Are these (hard- or soft-) linked to a single file? If so, then you can
use that attribute.

--
Ruud

On 03/30/12 08:05, Justin C wrote:
> On 2012-03-29, J. Gleixner<glex_no-> wrote:
>> On 03/28/12 10:24, Justin C wrote:
>>> We have a database of thousands of clothing items. Some of the items are
>>> almost identical apart from their size. Consequently we use the same
>>> image in our web-shop to advertise items of the same style, design and
>>> colour.
>>> [...]
>>> The bottle-neck, as I see it, is running grep 20k times, once for each
>>> image found. Can anyone suggest a better way?
>>
>> Since you already have data in a DB, I'd suggest looking at
>> associating these files, to the items, in the database.
>>
>> Maybe store the path to the file, or possibly the image as a BLOB, a
>> many to one relationship.
>
> Unfortunately it's not a DB I'm authorised to write to. I can run
> queries that extract data, but any changes to the data must be done
> through the 3rd party supplied interface.

OK. Then put it in your own database. MySQL, Postgres, SQLite, even a
DBM file might work.

"J. Gleixner" <glex_no-> writes:
> On 03/30/12 08:05, Justin C wrote:
>> On 2012-03-29, J. Gleixner<glex_no-> wrote:
>>> On 03/28/12 10:24, Justin C wrote:
>>>> We have a database of thousands of clothing items. Some of the items are
>>>> almost identical apart from their size. Consequently we use the same
>>>> image in our web-shop to advertise items of the same style, design and
>>>> colour.
>>>> [...]
>>>> The bottle-neck, as I see it, is running grep 20k times, once for each
>>>> image found. Can anyone suggest a better way?
>>>
>>> Since you already have data in a DB, I'd suggest looking at
>>> associating these files, to the items, in the database.
>>>
>>> Maybe store the path to the file, or possibly the image as a BLOB, a
>>> many to one relationship.
>>
>> Unfortunately it's not a DB I'm authorised to write to. I can run
>> queries that extract data, but any changes to the data must be done
>> through the 3rd party supplied interface.
>
> OK. Then put it in your own database. MySQL, Postgres, SQLite, even a
> DBM file might work.

This is a 'simple' key <-> value mapping, only complicated by the fact
that keys can have multiple values associating with them. Provided
that the code for a particular stock item can be calculated, the
simple first attempt at a solution would still be to use a Perl hash
mapping codes to sets of stock item keys. This hash can be generated
once in advance and then be used for any number of 'fast' lookups. A
more elaborate design would be to use a flat-file hashed database ('DBM
file') to store the mappings, update that whenever the set of stock
items changes and use it to associate presently existing images with
stock items without recalculating the mappings. An even better idea
would be to use a persistent database mapping stock items to image
files and update this database whenever the set of image files
changes.