Statistical significance of score differences

Main

Date: 17 Aug 2007 13:52:16
From: Dave
Subject: Statistical significance of score differences - new release of ChessDB

I've made a new release of ChessDB, a chess database based on Scid from
Shane Hudson. There is a fork too of ChessDB from the lying
plagiarist Pascal Georges who passes of work of mine as his own, as I
have documented at:

http://groups.google.co.uk/group/rec.games.chess.computer/browse_frm/thread/658a58d9a17b9aaf/dcb9e3c5e4e7266a?lnk=st&q=scid+released&rnum=1&hl=en#dcb9e3c5e4e7266a

Anyway, the main reason for my post it to introduce a new feature in
ChessDB and I would be interested in comments from others about this.

Basically ChessDB has a tree window, like many databases (Scid,
ChessBase, Chess Assistant etc). But I've added code that will determine
if the difference in score between two moves is really real
('statistically significant') , or if it could be due to chance. (If you
toss a coin 20 times and it lands on heads 12 times and tails 8 times,
you can't deduce the coin is biased) - such a small difference can be
due to chance with only 20 tosses. In contrast, if it landed on the head
19 times and the tail only once, you be pretty sure it is biased.

It is assumed that the difference in score between two moves is not due
to chance if the probability of the observed (or any larger) difference
being due to chance, with no underlying reason, is less than 0.05.

See:

http://chessdb.sourceforge.net/tutorial/t_search_tree.php

Some interesting observations can be seen looking at my database of 3.5
million games

1) 1.d4 scores better than 1.e4 with a p-value of less than 0.01. In
other words, the chance of the observed or any large score difference
being due to chance is less than 1%.

2) In my database, the opening move with the highest score is 1.Na3.
Despite the score being a lot higher than 1.e4 or 1.d4, this is *not*
statistically significant. In other words, whilst we can't say it 1.Na3
is any better or worst than 1.e4 or 1.d4, we can say that that there is
a high probability that the observed difference is due to chance. As
such, we should pay very little attention to the relative scores.

3) In my database, 3.Nd2 (Tarrash variation) in the French (1.e4 e6 2.d4
d5) scores higher than 3.Nc3 (main line) and is statistically
significant at the 5% level, but not at the 1% level. In other words, we
can be 95% sure there is a real difference in score between 3.Nf3 and
3.Nd2 in my database, but we can't be 99% sure.

In contrast, the difference in scores of 3.Nf3 (or 3.Nd2) to the
exchange (3.exd5) or advance (3.e5) variations is statistically
significant at the p=0.01 level, so there is less than a 1% chance the
observed difference in score is due to chance and can be more than 99%
sure there is an underlying reason. (The reason can't be determined in
ChessDB, but one might strongly suspect the advance or exchange are
inferior for white than the main line (3.Nc3) or Tarrasch (3.Nd2).

(I personally have a much better success rate with the Tarrasch than the
advance too. I will not contemplate the exchange as it is too boring and
while it is drawish, it scores pretty low for white.)

Anyone with a reasonable knowledge of statistics might guess I am using
a chi-squared test, which is what I am doing. Chi-squared is calculated
then the p-value determined from that, using an algorithm good to 4
decimal places. I intend changing that to a more accurate approximation
soon.

Other changes in ChessDB include

* Native support for UCI engines (using some code from P. Georges, which
I fully acknowledge, unlike him when he uses my code).

* The facility to download a database of either 100,000 or 3.5 million
games. The database is split into multiple parts for easy downloading,
then reconstructed by ChessDB and an MD5 checksum used to verify the
database has not been corrupted in transmission.

* Quickly download games from the history of anyone on ICC or FICS.

* Numerous other changes documented at:
http://chessdb.sourceforge.net/Scid/

I'm interested in what others think of the idea of testing the
statistical significance in the difference of two moves. To the best of
my knowledge, no other chess database does this, yet it seems to me
quite logical.

Date: 23 Aug 2007 00:47:53
From:
Subject: Re: Proposition of the year [Re: Why Scid should stay alive]

On 23 ao=FBt, 00:08, Dave <[email protected] > wrote:

> > I am ready to discuss about the opportunity of chessDB or Scid namings,
> > but there are certainly some mid-terms that would avoid the massive
> > renamings someone did one day, don't you think? Just call the result
> > "Scid2 3.6.x" or what you want (we saw worst namings with Sun).
>
> You may not agree with the name change. But I believe my startup window
> make is *very* clear most of the code came from Shane.

I never said the opposite !
Shane was, is, and will always be rewarded for his great work by
everybody (users and developers) : look at Scid for Pocket : 95% of
the code is still exactly the same as what he wrote years ago. So the
only rewards I'd make for ScidPocket : Shane Hudson, and Evolane.

> http://chessdb.sourceforge.net/tutorial/t_intro_start.php
>
> It's hard to believe really that anyone would have any doubts, with the
> program printing each time it starts up that more than 90% of the code
> is from Scid and Shane was the main author of Scid. Every user should
> see that - not only those which read the source code, which is a
> minority of users.

That is sure. I never meant anything else.

> The README thanks you for some help for producing me the first windows
> binary. The THANKS window obviously thanks Shane, but you too. If I have
> omitted anyone, it is by accident rather than purpose.

*I* also may omit people. But that is not exactly the same as
forgetting to send a payment, even if omitting people is something
that people should avoid.

> Hence I don't feel guilty about it. I think it was the right thing to do.
>
> The name change is very common in software

Yes it is. Look at Borland, changed to Inprise then, ooops, go back to
Borland. My conviction is that the product and the preservation/
preparation of the future are more important.

> Fruit is no longer developed, but there is Gambit Fruit and there is
> Toga. I think it would be rather silly to have 3 different chess
> engines, all called Fruit. There was a fork of Scid prior to ours, which
> was to be called 'newscid', but in later dropped using Scid's code and
> became ChessX.

First let's hope the strongest challenger of Scid reach the 1.0
milestone soon. What they do at ChessX is interesting (and I love Qt).
Second you will notice they used "newscid" (the name Scid remained),
and I doubt (I did not check) that they renamed the internals (Scid
tokens).
Concerning Fruit, it has never been abandonned by Fabien Letouzey (too
many french people around there don't you think ?), and if I
understood well, Gambit Fruit, Toga, etc. were developed because
Fabien decided to close Fruit's code (and it appears that, after a
commercial period, Fruit 2.3.1 is again free).

> I'm quite happy with the name change to ChessDB.

Ok, but please consider I make a huge step toward you. I hope this
will suffice (if i make a bigger step, I will lose balance and fall at
once).
And *maybe* my arguments related to Scid's name in internal are not
*so* bad.

> I wish you luck with the Pocket PC. You will find several posts from me
> (not always my full name or correct email) about chess programs for
> Pocket PC. I've tried several - Pocket Grandmaster (the best I have
> tried to date), Pocket Fritz 2, Intellichess and several more.

In my opinion, with ScidPocket, there are only 2 challengers on the
PDA ket left for me :
- Palm Hiarcs (the choice of the heart, don't ask me why :-) );
- Pocket CT ART.

> Things I felt where missing from the Pocket PC world were:
>
> 1) A good database - an issue you are I believe addressing. Te best I
> found was the free CEboard, but that had nowhere near the fuctionality
> of Scid or ChessDB.

And moreover, Alain is a really nice guy (and it appears we have many
things in common : our first dedicated chess computer was the same : a
Chess Challenger 7), but I could not correctly use CEBoard on my Dell.
I know he is working on it (he even borrowed a x51v for this, because
this is the PDA I have : when I say he is nice !).

> I tend to carry my laptop with me now pretty much all the time, so there
> is less need for the PDA now, hence my interest in the pocket PC code is
> a little less than a year or so ago.

When I ported chess engines to PalmOpenChess, I felt that engines are
better on PDA because of the lack of CPU power. On a PC when an engine
tells you "score =3D 1.5 at ply 14", I am unable to see where the pawn
will be lost. But when two engines fight on a PDA, I can watch it and
understand what is going on, and have more fun at it.
Hiarcs is between 2400 and 2600 ELO FIDE on a PDA : far enough for
me ! And the Toga I ported (calculating at 20000-30000 nps) is
certainly close to those levels.

> On pocket PC, there i far less choice of programs from ICC and FICS.

This is why I changed the way ScidPocket communicates with the
engines : I used sockets but this leads to sync problem when the user
starts/stops his internet connection. So I switched to Message
Queueing, which leaves TCP/IP stack away.

> > Please, if there is 1% of chance for you to consider this, DO NOT
> > CONSIDER THIS AS A GIVE UP PROPOSAL FOR YOU : this is exactly the
> > opposite !
>
> I will consider any serious proposal - but not today.

This proposal is serious. I hope you don't doubt about it.

I am glad you intend to consider it with cool head, because all the
sterile and puerile mess here hardly permit deep and wise thoughts,
for the least.

Pascal

Date: 22 Aug 2007 09:13:52
From: Richard
Subject: Re: Opening traps - was Statistical significance of score differences - new release of ChessDB

On Aug 22, 9:16 am, David Richerby <[email protected] >
wrote:
> Dave (from the UK) <[email protected]> wrote:
>
> > The problem I have seen with many of these bad moves is that one
> > side sets a trap for their opponent to play a bad move. This usually
> > (but not always) means playing a slightty sub-optimal move. If your
> > opponent falls for the traps, you have a good chance of winnning the
> > game quickly. If they don't you will have an inferior position.
>
> Quite. Heisman calls this `hope chess' and repeatedly advises against
> it. Really, this whole approach is just a slightly more grown-up
> version of 1.e4 2.Bc4 3.Qf3/h5.
>
> Dave.
>

Those aren't the type of bad moves I was talking about in my earlier
posts. While there are traps that give you an inferior position if
your opponent notices the trap and doesn't fall for it, there are
plenty of solid moves that happen to have a trap to them if one side
doesn't realize it. While I've seen books on cheap traps and really
lousy opening moves (Pandolfini's Traps and Zaps, etc), I haven't seen
any books that warn against the reasonable seeming opening moves that
just don't work.

For instance, the Vienna Gambit: 1. e4 e5 2. Nc3 Nf6 3. f4. Many
players who don't know this position think they can treat it like a
King's Gambit with knights developed earlier, and they accept the
gambit pawn with 3. ... exf4. In this position, though, that's a
blunder due to 4. e5, forcing the f6 knight to retreat to its starting
square (4. ... Qe7 5. Qe2 doesn't help). But that's not what white's
hoping for when he plays 3. f4. He just wants more control over the
center and a chance to open the f file so his castled rook will attack
f7. That's not a cheap trap, just a reasonable opening move.

And 3. ... fxe4 seems like a reasonable move at first glance, to
someone who hasn't seen this position before. While this may be
discussed in a book on the Vienna, how many players who respond to 1.
e4 with e5 and don't play the Vienna as white are going to read a book
on the Vienna? This type of thing isn't covered in MCO or NCO or any
similar reference that someone might use just to get a basic feel for
openings they might run into but don't routinely play.

So that's the type of book I'd like to see - something that covers
these types of reasonable looking moves that don't work for a wide
variety of openings.

And you've got Heisman's definition of "hope chess" wrong. He uses
that term to describe playing a move without looking at your
opponent's possible responses, just hoping that they don't have a good
response. It's not about hoping your opponent doesn't see something.

--Fromper

Date: 22 Aug 2007 20:13:22
From: David Richerby
Subject: Re: Opening traps - was Statistical significance of score differences - new release of ChessDB

Richard <[email protected] > wrote:
> For instance, the Vienna Gambit: 1. e4 e5 2. Nc3 Nf6 3. f4. Many
> players who don't know this position think they can treat it like a
> King's Gambit with knights developed earlier, and they accept the
> gambit pawn with 3. ... exf4. In this position, though, that's a
> blunder due to 4. e5, forcing the f6 knight to retreat to its
> starting square (4. ... Qe7 5. Qe2 doesn't help). [...]
>
> And 3. ... fxe4 seems like a reasonable move at first glance, to
> someone who hasn't seen this position before.

1) Don't play the first move that comes into your head.
2) The opening principles that development and the centre are
important tell you that 4.e5 is strong here.

> And you've got Heisman's definition of "hope chess" wrong. He uses
> that term to describe playing a move without looking at your
> opponent's possible responses, just hoping that they don't have a
> good response.

Yes, you're right.

Dave.

--
David Richerby Gigantic Mexi-Composer (TM): it's like
www.chiark.greenend.org.uk/~davidr/ a pupil of Beethoven that comes from
Mexico but it's huge!

Date: 18 Aug 2007 16:57:01
From: Guy Macon
Subject: Re: Statistical significance of score differences - new release of ChessDB

Dave wrote:

>documented at:
>
>http://groups.google.co.uk/group/rec.games.chess.computer/browse_frm/thread/6
> 58a58d9a17b9aaf/dcb9e3c5e4e7266a?lnk=st&q=scid+released&rnum=1&hl=en#dcb9e3c
> 5e4e7266a

You might want to consider polishing up your Google Groups citing skills:

http://groups.google.co.uk/group/rec.games.chess.computer/msg/dcb9e3c5e4e7266a

--
Guy Macon
<http://www.guymacon.com/ >

Date: 18 Aug 2007 19:55:11
From: Guy Macon
Subject: Re: Statistical significance of score differences - new release of ChessDB

I have some comments as a neutral third party who is quite
familiar with this sort of issue.

PG=Pascal Georges
DK=David Kirkby

Everything else=Guy Macon

PG: "Scid is a chess database application (original version available
PG: at http://scid.sourceforge.net). As Scid's author has been unreachable
PG: for a very long time, I decided to continue his work. So the version
PG: of Scid found here is not a fork of the original project but a
PG: continuation of it. "

It isn't a continuation unless the original author says that it is. It's
a fork. I could choose to continue the work too. By using the correct
term (fork) the issue of who is the "real" continuation doesn't come up.

DK: "I started ChessDB, http://chessdb.sourceforge.net/ as a continuation
DK: /fork (you use what ever term you want)

No. You *can't* use any term you want. In particular, you can't
use the term "continuation" unless the original author says that
you are the new maintainer of his work.

DK: I'm NOT concerned about what he has taken from Scid (that is allowed
DK: under the GPL), but what he claims to have done, but has not. In
DK: particular, what code he took of mine, without acknowledgment.

Taking your code without acknowledgment is allowed under the GPL.
In particular, [ http://www.gnu.org/copyleft/gpl.html ] section 7
says that "requiring ... author attributions" and "prohibiting
misrepresentation of the origin of that material" and "requiring
that modified versions of such material be ked in reasonable
ways as different from the original version" are all additional
terms, not part of the GPL.

PG: "So I will switch naming of my releases from scid-pg to scid
PG: as I have the right to."

No. You do not have the right to the name "Scid" or "scid" unless the
original author says that you are the new maintainer of his work. The
original author doesn't seem to be around to sue you, but he could.
If you think that something being Open Source under the GPL means that
the name isn't a tradek, try forking off a distribution and calling
it "Red Hat Linux" or "Slackware."

DK: which has code taken from ChessDB but not acknowledged.

See above.

PG: As chessDB is a fork of Scid

Yes. As is scid-pg.

PG: and until anybody else is better placed than me to continue Scid,

You don't get to decide that.

PG: I request things to be strictly separated : given Kirkby's
PG: attitude, I hope he'll be honest enough to continue his fork
PG: in a strict separated way than mine

Unless DK has called ChessDB "scid-pg", he has kept his fork
as separate from yours as required.

PG: I deny him the right to use my own code

You *CAN'T* deny *ANYONE* the right to use code derived from a work
licensed under the GPL. The *ONLY* way you can deny anyone the right
to use your code is to take out every bit of GPL code. Then you
can use whatever license you prefer.

PG: and put your (C) on files where you did not change one byte.

So? He changed the name and made it clear that it is a derivative
work. Look at all the forks from Red Hat, Debian, and Slackware;
they have many unchanged files but the files are part of the new,
derived work with the new name and thus are properly copyrighted
with the name of the author of the fork.

DK: SO TO MAKE IT VERY CLEAR TO ANYONE. THE SOFTWARE PASCAL CALLS
DK: SCID IS A FORK FROM ChessDB. It does *NOT* start from the Scid
DK: sources, but uses code I wrote and code others wrote.

That's not how GPL software works. You act as if PG isn't allowed to
copy all the code from ChessDB and still still call it a fork of Scid.
Assuming that scid-pg has lots of Scid code in it and lots of ChessDB
code in it, he can call it a fork of either. It works the other way,
too. You could abandon all of your work, copy all of scid-pg, rename
it ChessDB and put your name in all of the copyrights and still call
it a fork of Scid. It is very common for a fork of a fork of a fork
to call itself a fork of the original (and usually best-known) program.
To require otherwise ignores one of the main advantages of GNU FOSS;
the practice of incorporating improvements from multiple sources with
no restrictions. It goes back to Stallman's "software should be free"
argument.

PG: "you first started renaming all Scid's occurences in source
PG: files (more than 1600) without any added value

Doing that is correct behavior. Look at CentOS; they renamed all
occurrences of the phrase Red Hat, yet CentOS is as near to a perfect
clone of Red Hat as possible. It is PG who is in the wrong when he
falsely call scid-pg "scid" without permission. That's a tradek
violation. GPL doesn't shield him, either; GPL is about copyrights,
not tradeks, service-ks or patents.

For a taste of some of the issues involved, See:
http://weblog.infoworld.com/openresource/archives/2007/01/why_im_tired_of.html
http://www.regdeveloper.co.uk/2007/07/30/osi_tiemann_responds/
http://www.nicholasgoodman.com/bt/blog/2006/11/27/compromise-attribution-rider-on-any-osi-license/
http://www.internetnews.com/dev-news/article.php/3694076
http://asay.blogspot.com/2007/01/my-very-last-attribution-post.html

I hope this helps.

--
Guy Macon
<http://www.guymacon.com/ >

Date: 20 Aug 2007 13:03:28
From: David Richerby
Subject: Re: Statistical significance of score differences - new release of ChessDB

Guy Macon <http://www.guymacon.com/ > wrote:
> If you think that something being Open Source under the GPL means
> that the name isn't a tradek, try forking off a distribution and
> calling it "Red Hat Linux" or "Slackware."

That's a whole different kettle of fish. You can't call your forked
distribution Red Hat Linux or Slackware precisely because those two
names *are* tradeked. But, to the best of my knowledge, the name
Scid is not a tradek.

> It is PG who is in the wrong when he falsely call scid-pg "scid"
> without permission. That's a tradek violation.

No it isn't because Scid isn't a tradek.

Dave.

--
David Richerby Hungry Cat (TM): it's like a cat but
www.chiark.greenend.org.uk/~davidr/ it'll eat you!

Date: 17 Aug 2007 19:15:43
From: Ralf Callenberg
Subject: Re: Statistical significance of score differences - new release of

17.08.2007 14:52, Dave:
> But I've added code that will determine
> if the difference in score between two moves is really real
> ('statistically significant') , or if it could be due to chance.

Two important factors are completely ignored in this calculation:
development over time and strength of the involved players. The first is
important because once a refuation or at least a very strong answer for
a move is found, its frequency drops. So, the old statistics of this
move stay unchanged over a long time - possibly with a favourable result
for this move, although it might be well known that this move should be
avoided.

The second factor is quite obvious: games of higher rated players tend
to be less erratic, so that those results are more expressive.
Therefore, if I look at numbers, I check the average Elo and the
performance.

Greetings,
Ralf

Date: 17 Aug 2007 18:41:12
From: M Winther
Subject: Re: Statistical significance of score differences - new release of ChessDB

Den 2007-08-17 14:52:16 skrev Dave <[email protected] >:

> I've made a new release of ChessDB, a chess database based on Scid from
> Shane Hudson. There is a fork too of ChessDB from the lying
> plagiarist Pascal Georges who passes of work of mine as his own, as I
> have documented at:..........

Befrore I download it I would like to know whether it handles transpositions, i.e., are
*unplayed* moves visible in the tree that lead to a played position?

Mats

Date: 17 Aug 2007 20:43:29
From: Dave
Subject: Re: Statistical significance of score differences - new release of

M Winther wrote:

> Befrore I download it I would like to know whether it handles
> transpositions, i.e., are
> *unplayed* moves visible in the tree that lead to a played position?
>
> Mats
>

Yes. It shown positions, not moves. Hence sometimes you will find there
are no games in the database at move 5, but by move 6 there are thousands.

Date: 17 Aug 2007 18:38:41
From: Pascal
Subject: Re: Statistical significance of score differences - new release of

At least other people around the world clearly got who is that guy !!

Pascal

http://prolinux.free.fr/alex_guestbook/
Dave a �crit :
> I've made a new release of ChessDB, a chess database based on Scid from
> Shane Hudson. There is a fork too of ChessDB from the lying
> plagiarist Pascal Georges who passes of work of mine as his own, as I
> have documented at:
>
> http://groups.google.co.uk/group/rec.games.chess.computer/browse_frm/thread/658a58d9a17b9aaf/dcb9e3c5e4e7266a?lnk=st&q=scid+released&rnum=1&hl=en#dcb9e3c5e4e7266a
>
>
> Anyway, the main reason for my post it to introduce a new feature in
> ChessDB and I would be interested in comments from others about this.
>
> Basically ChessDB has a tree window, like many databases (Scid,
> ChessBase, Chess Assistant etc). But I've added code that will determine
> if the difference in score between two moves is really real
> ('statistically significant') , or if it could be due to chance. (If you
> toss a coin 20 times and it lands on heads 12 times and tails 8 times,
> you can't deduce the coin is biased) - such a small difference can be
> due to chance with only 20 tosses. In contrast, if it landed on the head
> 19 times and the tail only once, you be pretty sure it is biased.
>
> It is assumed that the difference in score between two moves is not due
> to chance if the probability of the observed (or any larger) difference
> being due to chance, with no underlying reason, is less than 0.05.
>
> See:
>
> http://chessdb.sourceforge.net/tutorial/t_search_tree.php
>
> Some interesting observations can be seen looking at my database of 3.5
> million games
>
> 1) 1.d4 scores better than 1.e4 with a p-value of less than 0.01. In
> other words, the chance of the observed or any large score difference
> being due to chance is less than 1%.
>
> 2) In my database, the opening move with the highest score is 1.Na3.
> Despite the score being a lot higher than 1.e4 or 1.d4, this is *not*
> statistically significant. In other words, whilst we can't say it 1.Na3
> is any better or worst than 1.e4 or 1.d4, we can say that that there is
> a high probability that the observed difference is due to chance. As
> such, we should pay very little attention to the relative scores.
>
> 3) In my database, 3.Nd2 (Tarrash variation) in the French (1.e4 e6 2.d4
> d5) scores higher than 3.Nc3 (main line) and is statistically
> significant at the 5% level, but not at the 1% level. In other words, we
> can be 95% sure there is a real difference in score between 3.Nf3 and
> 3.Nd2 in my database, but we can't be 99% sure.
>
> In contrast, the difference in scores of 3.Nf3 (or 3.Nd2) to the
> exchange (3.exd5) or advance (3.e5) variations is statistically
> significant at the p=0.01 level, so there is less than a 1% chance the
> observed difference in score is due to chance and can be more than 99%
> sure there is an underlying reason. (The reason can't be determined in
> ChessDB, but one might strongly suspect the advance or exchange are
> inferior for white than the main line (3.Nc3) or Tarrasch (3.Nd2).
>
> (I personally have a much better success rate with the Tarrasch than the
> advance too. I will not contemplate the exchange as it is too boring and
> while it is drawish, it scores pretty low for white.)
>
> Anyone with a reasonable knowledge of statistics might guess I am using
> a chi-squared test, which is what I am doing. Chi-squared is calculated
> then the p-value determined from that, using an algorithm good to 4
> decimal places. I intend changing that to a more accurate approximation
> soon.
>
> Other changes in ChessDB include
>
> * Native support for UCI engines (using some code from P. Georges, which
> I fully acknowledge, unlike him when he uses my code).
>
> * The facility to download a database of either 100,000 or 3.5 million
> games. The database is split into multiple parts for easy downloading,
> then reconstructed by ChessDB and an MD5 checksum used to verify the
> database has not been corrupted in transmission.
>
> * Quickly download games from the history of anyone on ICC or FICS.
>
> * Numerous other changes documented at:
> http://chessdb.sourceforge.net/Scid/
>
>
>
> I'm interested in what others think of the idea of testing the
> statistical significance in the difference of two moves. To the best of
> my knowledge, no other chess database does this, yet it seems to me
> quite logical.

Date: 17 Aug 2007 09:10:48
From: Richard
Subject: Re: Statistical significance of score differences - new release of ChessDB

That is a very cool idea for a feature. I haven't used databases much
in general, so I don't know much about existing features, but I could
see how something like that could be very useful to a master trying to
refine their opening preparation. At my patzer level (1400ish USCF),
it's something that could be cool just for the sake of curiousity.

So is this a free program? As I said, I don't know much about chess
databases (which is ironic, since I specialize in SQL databases in my
profession). I'd like to get a database program and a large database
of master and GM games eventually, so I can see how better players
than me handle certain openings and the positions that result from
them. For now, I tend to just go to chesslab.com and look at games
there in the openings I play.

--Fromper

Date: 17 Aug 2007 20:41:22
From: Dave
Subject: Re: Statistical significance of score differences - new release of

Richard wrote:
> That is a very cool idea for a feature.

I'm glad you like it.

> I haven't used databases much
> in general, so I don't know much about existing features, but I could
> see how something like that could be very useful to a master trying to
> refine their opening preparation. At my patzer level (1400ish USCF),
> it's something that could be cool just for the sake of curiousity.
>
> So is this a free program?

Yes, its open source and free.

homepage
http://chessdb.sourceforge.net/

tutorial
http://chessdb.sourceforge.net/tutorial/

download page:
http://chessdb.sourceforge.net/downloads/

> As I said, I don't know much about chess
> databases (which is ironic, since I specialize in SQL databases in my
> profession). I'd like to get a database program and a large database
> of master and GM games eventually, so I can see how better players
> than me handle certain openings and the positions that result from
> them.

If you download it, then go to the Tools menu, Select "Download games
from" then select "3.5 million games site #1" it will download you a 3.5
million game database.

For now, I tend to just go to chesslab.com and look at games
> there in the openings I play.

Well with 3.5 million you have quite a few. ChessDB also has the
facility to download from The Week In Chess (TWIC), so you can update
the database every week (usually a Monday), when new games are added to
TWIC. See:

http://chessdb.sourceforge.net/tutorial/twic-retriveal.php

(The program has an http client to connect to the external sources of
data. There is also a telnet client which is used to download games from
FICS and ICC).

I do have a larger database, which I could make available, but as
databases get larger, the quality of the games goes down.

Date: 17 Aug 2007 18:18:31
From: Anonymous
Subject: Re: Statistical significance of score differences - new release of

Well, if you like chess databases with a lot of features you can grab
Scid at http://scid.sourceforge.net or http://prolinux.free.fr/scid (the
lattest with some training features, and the ability to play against
various engines).

Richard a �crit :
> That is a very cool idea for a feature. I haven't used databases much
> in general, so I don't know much about existing features, but I could
> see how something like that could be very useful to a master trying to
> refine their opening preparation. At my patzer level (1400ish USCF),
> it's something that could be cool just for the sake of curiousity.
>
> So is this a free program? As I said, I don't know much about chess
> databases (which is ironic, since I specialize in SQL databases in my
> profession). I'd like to get a database program and a large database
> of master and GM games eventually, so I can see how better players
> than me handle certain openings and the positions that result from
> them. For now, I tend to just go to chesslab.com and look at games
> there in the openings I play.
>
> --Fromper
>

Date: 17 Aug 2007 20:25:26
From: Dave
Subject: Re: Statistical significance of score differences - new release ofChessDB

Anonymous wrote:
> Well, if you like chess databases with a lot of features you can grab
> Scid at http://scid.sourceforge.net or http://prolinux.free.fr/scid (the
> lattest with some training features, and the ability to play against
> various engines).

And the latter of which has code taken from ChessDB but not acknowledged.

Date: 17 Aug 2007 23:01:47
From: Pascal
Subject: Re: Statistical significance of score differences - new release ofChessDB

Dave a �crit :
> Anonymous wrote:
>> Well, if you like chess databases with a lot of features you can grab
>> Scid at http://scid.sourceforge.net or http://prolinux.free.fr/scid
>> (the lattest with some training features, and the ability to play
>> against various engines).
>
> And the latter of which has code taken from ChessDB but not acknowledged.

Pure lies, but if you want you can put all Pocket PC code I wrote and
available on my site, change the name from PocketScid to DK_sucker, and
put your (C) on files where you did not change one byte.

You are used to it. This is what you did first with Scid, and this is
why I stayed few days tuned with you.

Date: 18 Aug 2007 09:01:41
From: Dave
Subject: Re: Statistical significance of score differences - new release ofChessDB

Pascal wrote:

>> And the latter of which has code taken from ChessDB but not acknowledged.
>
> Pure lies, but if you want you can put all Pocket PC code I wrote and

Rather than say "Pure lies" why not address the SPECIFIC I have raised
before?

I posted 9 SPECIFIC EXAMPLES before at:

http://groups.google.co.uk/group/rec.games.chess.computer/browse_thread/thread/658a58d9a17b9aaf/371410985ca374d4?lnk=st&q=scid+chess+pascal&rnum=1&hl=en#371410985ca374d4

But I'll ask you a few questions about the following 3 lines in your
very latest (3.6.18) plagiarised version of 'Scid' from the file
src/textbuf.h.

------------------------------------------
// void ClearTranslation (char ch) { Translation[ch] = NULL; }
// Changed ch to int, to avoid compiler warnings.
void ClearTranslation (int ch) { Translation[ch] = NULL; }
-----------------------------------------

I am being VERY SPECIFIC here - just restricting the example to 3 little
lines, with some VERY SPECIFIC questions.

1) Did you write those 3 lines Pascal?
2) If so, when?
3) If you did not write them, please indicate where they came from.

See I know those lines well. Shane wrote the first, but it was not
commented out (// is a comment in C++). The second, a comment about the
change to remove compiler warnings is what I personally wrote (yes me).
The 3rd is my changed line.

Now Sourceforge have this very nice facility on their CVS repository
that allows one to see differences between different versions of a file.

This link

http://chessdb.cvs.sourceforge.net/chessdb/chessdb/src/textbuf.h?r1=1.1&r2=1.2

shows in yellow the differences between version 1.1 of src/textbuf.h in
ChessDB (Tue Dec 26 19:37:44 2006 UTC) in and version 1.2 in ChessDB,
less than an hour later at Tue Dec 26 20:11:49 2006 UTC.

Funny how the lines in your code are EXACTLY the same as what I put on
Sourceforge. Your first release of scid-pg (later you renamed to Scid),
was not until AFTER that date.

So there is the challenge. Explain why the 3 lines are there? If you
find that too easy (which I doubt you will if you do it PROPERLY), then
try the others in the link on my post in Feb of this year.

SO TO MAKE IT VERY CLEAR TO ANYONE. THE SOFTWARE PASCAL CALLS SCID IS A
FORK FROM ChessDB. It does *NOT* start from tbe Scid sources, but uses
code I wrote and code others wrote.

Date: 18 Aug 2007 12:47:32
From: Pascal
Subject: Re: Statistical significance of score differences - new release ofChessDB

Could you please stop inventing false clues on senseless pityful stuff ?
You took patches on internet, as I did, you applied them to Scid 3.6.1
sources as I did : hence the common stuff on very few lines : ok,
congratulations ! And so what ? Don't you think it is better to reward
original authors of patches instead of you ?

Yes I tried to collaborate a few days with you, but I understood quickly
how insane you are : do you own copyright on mispellings, C++ comments
? Can't anybody write "//" without (c) D Kirkbye alongside ? Can't I
comment one line of code in Scid without checking if you did the same ?

I continue Scid development, you first started renaming all Scid's
occurences in source files (more than 1600) without any added value, and
putting (c) D Kirkby on files you did not change at all : that's the
truth, and you even did not deny your misbehaviour ! All you can do is
take others' work and act as if you were a genius.

What's the meaning of your endless childish arguing, that interests
*nobody* ?

Dave a �crit :
> Pascal wrote:
>
>>> And the latter of which has code taken from ChessDB but not
>>> acknowledged.
>>
>> Pure lies, but if you want you can put all Pocket PC code I wrote and
>
> Rather than say "Pure lies" why not address the SPECIFIC I have raised
> before?
>
> I posted 9 SPECIFIC EXAMPLES before at:
>
> http://groups.google.co.uk/group/rec.games.chess.computer/browse_thread/thread/658a58d9a17b9aaf/371410985ca374d4?lnk=st&q=scid+chess+pascal&rnum=1&hl=en#371410985ca374d4
>
>
> But I'll ask you a few questions about the following 3 lines in your
> very latest (3.6.18) plagiarised version of 'Scid' from the file
> src/textbuf.h.
>
> ------------------------------------------
> // void ClearTranslation (char ch) { Translation[ch] = NULL; }
> // Changed ch to int, to avoid compiler warnings.
> void ClearTranslation (int ch) { Translation[ch] = NULL; }
> -----------------------------------------
>
> I am being VERY SPECIFIC here - just restricting the example to 3 little
> lines, with some VERY SPECIFIC questions.
>
> 1) Did you write those 3 lines Pascal?
> 2) If so, when?
> 3) If you did not write them, please indicate where they came from.
>
> See I know those lines well. Shane wrote the first, but it was not
> commented out (// is a comment in C++). The second, a comment about the
> change to remove compiler warnings is what I personally wrote (yes me).
> The 3rd is my changed line.
>
> Now Sourceforge have this very nice facility on their CVS repository
> that allows one to see differences between different versions of a file.
>
> This link
>
> http://chessdb.cvs.sourceforge.net/chessdb/chessdb/src/textbuf.h?r1=1.1&r2=1.2
>
>
> shows in yellow the differences between version 1.1 of src/textbuf.h in
> ChessDB (Tue Dec 26 19:37:44 2006 UTC) in and version 1.2 in ChessDB,
> less than an hour later at Tue Dec 26 20:11:49 2006 UTC.
>
> Funny how the lines in your code are EXACTLY the same as what I put on
> Sourceforge. Your first release of scid-pg (later you renamed to Scid),
> was not until AFTER that date.
>
> So there is the challenge. Explain why the 3 lines are there? If you
> find that too easy (which I doubt you will if you do it PROPERLY), then
> try the others in the link on my post in Feb of this year.
>
>
>
> SO TO MAKE IT VERY CLEAR TO ANYONE. THE SOFTWARE PASCAL CALLS SCID IS A
> FORK FROM ChessDB. It does *NOT* start from tbe Scid sources, but uses
> code I wrote and code others wrote.
>
>
>
>
>
>
>
>
>
>
>

Date: 18 Aug 2007 12:08:50
From: Dave
Subject: Re: Statistical significance of score differences - new release ofChessDB

Pascal wrote:
> Could you please stop inventing false clues on senseless pityful stuff ?

There is no false or pitiful stuff.

> You took patches on internet,

Not the 3 lines I posted just posted (or a lot more, but I'll keep it
simple and restrict discussion it to just 3 lines).

That was *not* from the internet. I would ask you to show me *anywhere*
that patch can be found, or anyone that claims to have wrote it.

I wrote it - not you, not anyone else.

Date: 18 Aug 2007 14:13:26
From: Pascal
Subject: Re: Statistical significance of score differences - new release ofChessDB

If this was true, I would like to apologize and thank you so much Dave
for your contribution to Scid and for these 3 lines, and for all you did
to make Scid alive while its author's unavailability (due to long illness).

I thought you only wanted to kill Scid and bury it and take advantage of
other's work to cover your lack of skills, and I was certainly wrong.

Chess computer's world really need people like you.

Dave a �crit :

> I wrote it - not you, not anyone else.

Date: 18 Aug 2007 17:09:00
From: Dave
Subject: Re: Statistical significance of score differences - new release ofChessDB

Pascal wrote:
> If this was true, I would like to apologize and thank you so much Dave
> for your contribution to Scid and for these 3 lines, and for all you did
> to make Scid alive while its author's unavailability (due to long illness).

But it is not just 3 lines - and you know full well that it is not just
3 lines. I posted a longer list before:

http://groups.google.co.uk/group/rec.games.chess.computer/browse_thread/thread/658a58d9a17b9aaf/dcb9e3c5e4e7266a?lnk=st&q=scid+chessdb+This+is+a+little+more+complex+than+it+appears.+READ+ON+....+&rnum=1&hl=en#dcb9e3c5e4e7266a

which you never bothered responding to properly. Perhaps I will have
more luck with small pieces at a time, so here is a bit more - this time
12 lines, not 3.

http://chessdb.cvs.sourceforge.net/chessdb/chessdb/src/position.cpp?r1=1.1&r2=1.2

Want more?

Date: 18 Aug 2007 20:51:57
From: Pascal
Subject: Re: Statistical significance of score differences - new release ofChessDB

Dave a �crit :
> Pascal wrote:
>> If this was true, I would like to apologize and thank you so much Dave
>> for your contribution to Scid and for these 3 lines, and for all you
>> did to make Scid alive while its author's unavailability (due to long
>> illness).
>
> But it is not just 3 lines - and you know full well that it is not just
> 3 lines. I posted a longer list before:
>
> http://groups.google.co.uk/group/rec.games.chess.computer/browse_thread/thread/658a58d9a17b9aaf/dcb9e3c5e4e7266a?lnk=st&q=scid+chessdb+This+is+a+little+more+complex+than+it+appears.+READ+ON+....+&rnum=1&hl=en#dcb9e3c5e4e7266a
>
>
> which you never bothered responding to properly. Perhaps I will have
> more luck with small pieces at a time, so here is a bit more - this time
> 12 lines, not 3.
>
> http://chessdb.cvs.sourceforge.net/chessdb/chessdb/src/position.cpp?r1=1.1&r2=1.2
>
>
> Want more?
>

Yes ! We all want more !
What you say and your consideration for the past work of others is so
interesting !

Date: 18 Aug 2007 21:20:30
From: Dave
Subject: Re: Statistical significance of score differences - new release ofChessDB

Pascal wrote:

>> which you never bothered responding to properly. Perhaps I will have
>> more luck with small pieces at a time, so here is a bit more - this
>> time 12 lines, not 3.
>>
>> http://chessdb.cvs.sourceforge.net/chessdb/chessdb/src/position.cpp?r1=1.1&r2=1.2
>>
>>
>> Want more?
>>
>
> Yes ! We all want more !
> What you say and your consideration for the past work of others is so
> interesting !

What do you mean by 'others'? It is not others that wrote it, but me!!

I'll make it clearer, with the following statement::

The 12 changes between the Shane's source file src/position.cpp in the
real Scid (not your plagiarised version) and what appears in 'your'
first release of scid-pg and still exist in 'your' latest release of
scid (3.6.18) were *not* written by you. They were not written by any
others. They were written by me.

THAT IS FACT AS FAR AS I AM CONCERNED.

Do you claim to have made those changes in the file src/position.cpp?

If you do not claim to have made the changes yourself, would you mind
saying where you got them from?

Date: 19 Aug 2007 00:23:28
From: Pascal
Subject: Re: Statistical significance of score differences - new release ofChessDB

Dave a �crit :

> > Yes ! We all want more !
> > What you say and your consideration for the past work of others is so interesting !
> What do you mean by 'others'? It is not others that wrote it, but me!!

others = Shane Hudson, people contributing to Scid, etc.

> If you do not claim to have made the changes yourself, would you mind
> saying where you got them from?

Okay, Dave you got me. I am strictly unable to prove what I wrote : so I
make public apologizes to you :

- Scid 3.6.18 was written by you (except some code by Shane Hudson). I
never took Scid 3.6.1 sources and continued its development : all or
part of 3.6.18 (choose what pleases you the most) was made by you.
Thank you very much for continuing Scid, Dave. Thank you for having
tried to continue the great work on behalf of someone who is
unfortunately unable to do so ;

- the upcoming version of Pocket Scid is currently written by you : I
wait for its release with a great impatience.
Thank you very much for that, Dave.

Please, do you allow me to continue to host your work at my site ?

Pascal

Date: 17 Aug 2007 13:55:10
From: Dave
Subject: Re: Statistical significance of score differences - new release of

Dave wrote:
> I've made a new release of ChessDB, a chess database based on Scid from
> Shane Hudson.

I forgot to say, if you want to try ChessDB or use it to download a
large database, see:

http://chessdb.sourceforge.net/downloads/