Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Version Control Software

Reply
Thread Tools

Version Control Software

 
 
Chris Angelico
Guest
Posts: n/a
 
      06-16-2013
On Sun, Jun 16, 2013 at 3:20 PM, Steven D'Aprano
<(E-Mail Removed)> wrote:
> On Sun, 16 Jun 2013 14:13:13 +1000, Chris Angelico wrote:
>
>> I didn't think there would be that much difference, tbh. Mainly, I'm
>> just seeing cpython as not being 200MB of history, or so I'd thought.
>> Pike has ~30K commits (based on 'git log --oneline|wc -l'); CPython has
>> roughly 80K (based on 'hg log|grep changeset|wc -l' - there's likely an
>> easier way but I don't know Mercurial). So yeah, okay, it's been doing
>> more. But I still don't see 200MB in that. Seems a lot of content.

>
> If you're bringing in the *entire* CPython code base, as shown here:
>
> http://hg.python.org/
>
> keep in mind that it includes the equivalent of four independent
> implementations:
>
> - CPython 2.x
> - CPython 3.x
> - Stackless
> - Jython


Hrm. Why are there other Pythons in the cpython repository? Yes,
CPython 2.x and 3.x, but why the other two?

> Plus, no offence intended at Pike which I'm sure is an awesome language,
> but it may not be quite as much active development as Python... as you
> point out yourself, there are nearly three times as many commits to
> CPython as to Pike, which coincidentally (or not) corresponds to the
> CPython repo being nearly three times as large as the Pike repo.


Yeah. Actually, I suspect that what's going on here, and what led to
my confusion, is that Pike wasn't always done using git, so quite a
few of the earlier versions simply aren't here. So it's an error in my
perceptions rather than any real difference.

However, comparisons aside, 200MB is still a fair bit to fetch before
doing anything with Python. Does Mercurial have any equivalent of
git's shallow clone feature?

ChrisA
 
Reply With Quote
 
 
 
 
Terry Reedy
Guest
Posts: n/a
 
      06-16-2013
On 6/16/2013 1:29 AM, Chris Angelico wrote:
> On Sun, Jun 16, 2013 at 3:20 PM, Steven D'Aprano


>> If you're bringing in the *entire* CPython code base, as shown here:
>>
>> http://hg.python.org/


This is the python.org collection of repositories, not just cpython.

>> keep in mind that it includes the equivalent of four independent
>> implementations:
>>
>> - CPython 2.x
>> - CPython 3.x


>> - Stackless
>> - Jython

>
> Hrm. Why are there other Pythons in the cpython repository?


There are not. The cpython repository
http://hg.python.org/cpython/
only contains cpython. As I write, the last revision is 84110. Windows
says that my cpython clone has about 1400 folders, 15000 files, and 500
million bytes

--
Terry Jan Reedy

 
Reply With Quote
 
 
 
 
Chris Angelico
Guest
Posts: n/a
 
      06-16-2013
On Sun, Jun 16, 2013 at 7:15 PM, Terry Reedy <(E-Mail Removed)> wrote:
> On 6/16/2013 1:29 AM, Chris Angelico wrote:
>>
>> On Sun, Jun 16, 2013 at 3:20 PM, Steven D'Aprano
>>> keep in mind that it includes the equivalent of four independent
>>> implementations:
>>>
>>> - CPython 2.x
>>> - CPython 3.x

>
>
>>> - Stackless
>>> - Jython

>>
>>
>> Hrm. Why are there other Pythons in the cpython repository?

>
>
> There are not. The cpython repository
> http://hg.python.org/cpython/
> only contains cpython. As I write, the last revision is 84110. Windows says
> that my cpython clone has about 1400 folders, 15000 files, and 500 million
> bytes


Ah, well it's this one that I have. So it should have only CPython in it.

ChrisA
 
Reply With Quote
 
Chris “Kwpolska” Warrick
Guest
Posts: n/a
 
      06-16-2013
On Sun, Jun 16, 2013 at 1:14 AM, Chris Angelico <(E-Mail Removed)> wrote:
> Hmm. ~/cpython/.hg is 200MB+, but ~/pike/.git is only 86MB. Does
> Mercurial compress its content? A tar.gz of each comes down, but only
> to ~170MB and ~75MB respectively, so I'm guessing the bulk of it is
> already compressed. But 200MB for cpython seems like a lot.


Next time, do a more fair comparison.

I created an empty git and hg repository, and created a file promptly
named “file” with DIGIT ONE (0x31; UTF-8/ASCII–encoded) and commited
it with “c1” as the message, then I turned it into “12” and commited
as “c2” and did this one more time, making the file “123” at commit
named “c3”.

[kwpolska@kwpolska-lin .hg@default]% cat * */* */*/* 2>/dev/null | wc -c
1481
[kwpolska@kwpolska-lin .git@master]% cat * */* */*/* */*/*/* 2>/dev/null | wc -c
16860 ← WRONG!

There is just one problem with this: an empty git repository starts at
15216 bytes, due to some sample hooks. Let’s remove them and try
again:

[kwpolska@kwpolska-lin .git@master]% rm hooks/*
[kwpolska@kwpolska-lin .git@master]% cat * */* */*/* */*/*/* */*/*/*
2>/dev/null | wc -c
2499

which is a much more sane number. This includes a config file (in the
ini/configparser format) and such. According to my maths skils (or
rather zsh’s skills), new commits are responsible for 1644 bytes in
the git repo and 1391 bytes in the hg repo.

(I’m using wc -c to count the bytes in all files there are. du is
unaccurate with files smaller than 4096 bytes.)

--
Kwpolska <http://kwpolska.tk> | GPG KEY: 5EAAEA16
stop html mail | always bottom-post
http://asciiribbon.org | http://caliburn.nl/topposting.html
 
Reply With Quote
 
Roy Smith
Guest
Posts: n/a
 
      06-16-2013
In article <(E-Mail Removed)>,
Chris メKwpolskaモ Warrick <(E-Mail Removed)> wrote:

> (I’m using wc -c to count the bytes in all files there are. du is
> unaccurate with files smaller than 4096 bytes.)


It's not that du is not accurate, it's that it's measuring something
different. It's measuring how much disk space the file is using. For
most files, that's the number of characters in the file rounded up to a
full block. For large files, I believe it also includes the overhead of
indirect blocks or extent trees. And, finally, for sparse files, it
takes into account that some logical blocks in the file may not be
mapped to any physical storage.

So, whether you want to use "du" or "wc -c" depends on what you're
trying to measure. If you want to know how much disk space you're
using, du is the right tool. If you want to know how much data will be
transmitted if the file is serialized (i.e. packed in a tarball or sent
via a "{hg,git} clone" operation), then "wc-c" is what you want.

All that being said, for the vast majority of cases (and I would be
astonished if this was not true for any real-life vcs repo), the
difference between what wc and du tell you is not worth worrying about.
And du is going to be a heck of a lot faster.
 
Reply With Quote
 
Lele Gaifax
Guest
Posts: n/a
 
      06-16-2013
Roy Smith <(E-Mail Removed)> writes:

> In article <(E-Mail Removed)>,
> Chris Kwpolska Warrick <(E-Mail Removed)> wrote:
>
>> (Im using wc -c to count the bytes in all files there are. du is
>> unaccurate with files smaller than 4096 bytes.)

>
> It's not that du is not accurate, it's that it's measuring something
> different. It's measuring how much disk space the file is using. For
> most files, that's the number of characters in the file rounded up to a
> full block.


I think “du -c” emits a number very close to “wc -c”.


ciao, lele.
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
http://www.velocityreviews.com/forums/(E-Mail Removed) | -- Fortunato Depero, 1929.

 
Reply With Quote
 
Jason Swails
Guest
Posts: n/a
 
      06-16-2013
On Sun, Jun 16, 2013 at 9:30 AM, Chris “Kwpolska” Warrick <
(E-Mail Removed)> wrote:

> On Sun, Jun 16, 2013 at 1:14 AM, Chris Angelico <(E-Mail Removed)> wrote:
> > Hmm. ~/cpython/.hg is 200MB+, but ~/pike/.git is only 86MB. Does
> > Mercurial compress its content? A tar.gz of each comes down, but only
> > to ~170MB and ~75MB respectively, so I'm guessing the bulk of it is
> > already compressed. But 200MB for cpython seems like a lot.

>
> Next time, do a more fair comparison.
>
> I created an empty git and hg repository, and created a file promptly
> named “file” with DIGIT ONE (0x31; UTF-8/ASCII–encoded) and commited
> it with “c1” as the message, then I turned it into “12” and commited
> as “c2” and did this one more time, making the file “123” at commit
> named “c3”.
>
> [kwpolska@kwpolska-lin .hg@default]% cat * */* */*/* 2>/dev/null | wc -c
> 1481
> [kwpolska@kwpolska-lin .git@master]% cat * */* */*/* */*/*/* 2>/dev/null
> | wc -c
> 16860 ← WRONG!
>
> There is just one problem with this: an empty git repository starts at
> 15216 bytes, due to some sample hooks. Let’s remove them and try
> again:
>
> [kwpolska@kwpolska-lin .git@master]% rm hooks/*
> [kwpolska@kwpolska-lin .git@master]% cat * */* */*/* */*/*/* */*/*/*
> 2>/dev/null | wc -c
> 2499
>
> which is a much more sane number. This includes a config file (in the
> ini/configparser format) and such. According to my maths skils (or
> rather zsh’s skills), new commits are responsible for 1644 bytes in
> the git repo and 1391 bytes in the hg repo.
>


This is not a fair comparison, either. If we want to do a fair comparison
pertinent to this discussion, let's convert the cpython mercurial
repository into a git repository and allow the git repo to repack the diffs
the way it deems fit.

I'm using the git-remote-hg.py script [
https://github.com/felipec/git/blob/...-remote-hg.py]
to clone a mercurial repo into a native git repo. Then, in one of the rare
cases, using git gc --aggressive. [1]

The result:

Git:
cpython_git/.git $ du -h --max-depth=1
40K ./hooks
145M ./objects
20K ./logs
24K ./refs
24K ./info
146M .

Mercurial:
cpython/.hg $ du -h --max-depth=1
209M ./store
20K ./cache
209M .


And to help illustrate the equivalence of the two repositories:

Git:

cpython_git $ git log | head; git log | tail

commit 78f82bde04f8b3832f3cb6725c4bd9c8d705d13b
Author: Brett Cannon <(E-Mail Removed)>
Date: Sat Jun 15 23:24:11 2013 -0400

Make test_builtin work when executed directly

commit a7b16f8188a16905bbc1d49fe6fd940078dd1f3d
Merge: 346494a af14b7c
Author: Gregory P. Smith <(E-Mail Removed)>
Date: Sat Jun 15 18:14:56 2013 -0700
Author: Guido van Rossum <(E-Mail Removed)>
Date: Mon Sep 10 11:15:23 1990 +0000

Warning about incompleteness.

commit b5e5004ae8f54d7d5ddfa0688fc8385cafde0e63
Author: Guido van Rossum <(E-Mail Removed)>
Date: Thu Aug 9 14:25:15 1990 +0000

Initial revision

Mercurial:

cpython $ hg log | head; hg log | tail

changeset: 84163:5b90da280515
bookmark: master
tag: tip
user: Brett Cannon <(E-Mail Removed)>
date: Sat Jun 15 23:24:11 2013 -0400
summary: Make test_builtin work when executed directly

changeset: 84162:7dee56b6ff34
parent: 84159:5e8b377942f7
parent: 84161:7e06a99bb821
user: Guido van Rossum <(E-Mail Removed)>
date: Mon Sep 10 11:15:23 1990 +0000
summary: Warning about incompleteness.

changeset: 0:3cd033e6b530
branch: legacy-trunk
user: Guido van Rossum <(E-Mail Removed)>
date: Thu Aug 09 14:25:15 1990 +0000
summary: Initial revision

They both appear to have the same history. In this particular case, it
seems that git does a better job in terms of space management, probably due
to the fact that it doesn't store duplicate copies of identical source code
that appears in different files (it tracks content, not files).

That being said, from what I've read both git and mercurial have their
advantages, both in the performance arena and the features/usability arena
(I only know how to really use git). I'd certainly take a DVCS over a
centralized model any day.

All the best,
Jason

[1] I know I just posted in this thread about --aggressive being bad, but
the packing from the translation was horrible --> the translated git repo
was ~2 GB in size. An `aggressive' repacking was necessary to allow git to
decide how to pack the diffs.

 
Reply With Quote
 
Terry Reedy
Guest
Posts: n/a
 
      06-16-2013
On 6/16/2013 11:48 AM, Lele Gaifax wrote:
> Roy Smith <(E-Mail Removed)> writes:
>
>> In article <(E-Mail Removed)>,
>> Chris Kwpolska Warrick <(E-Mail Removed)> wrote:
>>
>>> (I��m using wc -c to count the bytes in all files there are. du is
>>> unaccurate with files smaller than 4096 bytes.)

>>
>> It's not that du is not accurate, it's that it's measuring something
>> different. It's measuring how much disk space the file is using. For
>> most files, that's the number of characters in the file rounded up to a
>> full block.

>
> I think “du -c” emits a number very close to “wc -c”.


In Windows Explorer, the Properties box displays both the Size and 'Size
on disk', in both (KB or MB) and bytes. The block size for the disk I am
looking at is 4KB, so the Size on disk in KB is a multiple of that.

--
Terry Jan Reedy


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: Where to get stand alone Dot Net Framework version 1.1, version2.0, version 3.0, version 3.5, version 2.0 SP1, version 3.0 SP1 ? MowGreen [MVP] ASP .Net 5 02-09-2008 01:55 AM
Re: Where to get stand alone Dot Net Framework version 1.1, version 2.0, version 3.0, version 3.5, version 2.0 SP1, version 3.0 SP1 ? PA Bear [MS MVP] ASP .Net 0 02-05-2008 03:28 AM
Re: Where to get stand alone Dot Net Framework version 1.1, version 2.0, version 3.0, version 3.5, version 2.0 SP1, version 3.0 SP1 ? V Green ASP .Net 0 02-05-2008 02:45 AM
looling for free version of software called something like software repair kit ... ms smiling bookworm Computer Support 2 09-29-2005 05:20 PM
Is Apache Axis Release software version 1.2alpha or other version support intermediary of soap1.2? pos Java 0 02-18-2004 08:45 AM



Advertisments