«

On Thu, 26 Jun 2008 22:28:24 -0400, "John B. Matthews"
<> wrote, quoted or indirectly quoted someone who
said :

>Or, as the third shift operator once told me: throw it into a database
>and let Codd sort it out

It sounds like a joke, but I suspect that is how most big sorts are
done. Back in the olden days external sorts were pretty well the
first software to go commercial. Nearly everything else you wrote
yourself. You just don't see ads for them like you used to.
Opt-Tech is still around, but very quiet.


--

Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com

In article <>,
Roedy Green <> wrote:

> On Thu, 26 Jun 2008 22:28:24 -0400, "John B. Matthews"
> <> wrote, quoted or indirectly quoted someone who
> said :
>
> >Or, as the third shift operator once told me: throw it into a database
> >and let Codd sort it out
>
> It sounds like a joke, but I suspect that is how most big sorts are
> done. Back in the olden days external sorts were pretty well the
> first software to go commercial. Nearly everything else you wrote
> yourself. You just don't see ads for them like you used to.
> Opt-Tech is still around, but very quiet.

Yes, a terrible pun on Codd and God, but you're right about the olden
days. At the same time, I wonder if a JDBC-enabled database may be a
useful abstraction for a generic, flat-file disk sort.

Yes, I know the project is already out of hand

--
John B. Matthews
trashgod at gmail dot com
home dot woh dot rr dot com slash jbmatthews

Roedy Green wrote:
> On Thu, 26 Jun 2008 22:28:24 -0400, "John B. Matthews"
> <> wrote, quoted or indirectly quoted someone who
> said :
>> Or, as the third shift operator once told me: throw it into a database
>> and let Codd sort it out
>
> It sounds like a joke, but I suspect that is how most big sorts are
> done. Back in the olden days external sorts were pretty well the
> first software to go commercial. Nearly everything else you wrote
> yourself. You just don't see ads for them like you used to.
> Opt-Tech is still around, but very quiet.

Data manipulation are done in databases or in memory today.

The use of large flat files for data is rare today, so little
demand for sort utilities.

Arne

On Thu, 26 Jun 2008 16:44:26 GMT, Roedy Green
<> wrote, quoted or indirectly quoted
someone who said :

>I was thinking how you might go about writing a sort that could handle
>more data than could fit in RAM.

I have written this up a student project

http://mindprod.com/jgloss/project/externalsort.html
--

Roedy Green Canadian Mind Products
The Java Glossary
http://mindprod.com

On Fri, 27 Jun 2008 14:47:06 -0400, John B. Matthews wrote:

> At the same time, I wonder if a JDBC-enabled database may be a
> useful abstraction for a generic, flat-file disk sort.
>
It would be interesting to see how tipping records into an indexed
table and then reading them out again compared for speed with the
traditional in-memory sorted pre-string phase followed by n-way merge
passes. I don't think I'd lay bets either way.

Of course, if your system has enough swap space you could just substitute
a TreeMap for JDBC+database.


--
martin@ | Martin Gregorie
gregorie. |
org | Zappa fan & glider pilot