| |
Main
Date: 13 Oct 2006 02:30:52
From: Dave (from the UK)
Subject: How to create opening book for Toga (fruit)
|
I've downloaded the sources for Toga II 1.2 (based on Fruit 2.1) which I
want to run on my UNIX box. Can anyone tell me how to create an opening
book?
Is there much to be gained by using a large opening book? I'm trying to
create one using polyglot (since I don't know how to do so in fruit),
but that seems to keep screwing up. (The PGN file has about 330,000
games in it.)
It keeps saying:
pgn_next_game(): malformed tag at line 2918540, column 22
Which is a bit odd, given I've cut the file down to less than that
number of lines:
teal /export/home/drkirkby/chess % wc -l games.pgn
2909992 games.pgn
I've even built a 64 bit binary (not really expecting it to make any
difference), but it does not help. I don't know whther there might be
some limit in the program that stops it allocating more than a fixed
amount of RAM. It keeps saying:
10000 games ...
allocating 48MB ...
<snip >
allocating 96MB ...
<snip >
allocating 192MB ..
.....
then:
allocating 384MB ...
It should not have any problems taking a few GB of RAM if it wants it,
so I can't see what the problem is.
But if there is not much point in having a large opening book, perhaps
I'll just cut it down to whatever size works.
--
Dave (from the UK)
Please note my email address changes periodically to avoid spam.
It is always of the form: [email protected]
Hitting reply will work for a few months only - later set it manually.
http://witm.sourceforge.net/ (Web based Mathematica front end)
|
|
| |
Date: 15 Oct 2006 23:26:09
From: M L
Subject: Re: How to create opening book for Toga (fruit)
|
Dave (from the UK) wrote:
> Is there much to be gained by using a large opening book? I'm trying to
> create one using polyglot (since I don't know how to do so in fruit),
> but that seems to keep screwing up. (The PGN file has about 330,000
> games in it.)
I have built books from larger files without problem.
>
> It keeps saying:
>
> pgn_next_game(): malformed tag at line 2918540, column 22
>
> Which is a bit odd, given I've cut the file down to less than that
> number of lines:
>
> teal /export/home/drkirkby/chess % wc -l games.pgn
> 2909992 games.pgn
It seems quite usual that a 330.000 games file is longer than 2.918.540
lines.
You already have 8 lines from the seven-tags header plus one empty line
before the moves for each game and a blank line after the moves before
next game so more than 10 lines for one game is common...
Maybe your cutting was in the middle of a game ...
If your PGN is malformed, polyglot won't accept it.
A good tool for "cleaning" your PGN is PGNextract by David Barnes ( a
google search will lead you to the homepage).
>
> I've even built a 64 bit binary (not really expecting it to make any
> difference), but it does not help.
Nonsense.
Polyglot requires _pure_ PGN files that are text files ...
I don't know whther there might be
> some limit in the program that stops it allocating more than a fixed
> amount of RAM. It keeps saying:
>
> 10000 games ...
> allocating 48MB ...
> <snip>
> allocating 96MB ...
> <snip>
> allocating 192MB ..
> .....
> then:
>
> allocating 384MB ...
>
> It should not have any problems taking a few GB of RAM if it wants it,
> so I can't see what the problem is.
I have worked with larger PGN files. Only the book-building process does
require a lot of RAM. Once built the book will be in the 1-5 MB range
(not GB !) for the largest ones.
>
> But if there is not much point in having a large opening book, perhaps
> I'll just cut it down to whatever size works.
Much more important are :
- careful selection of the games to be included
- careful selection of appropriated Polyglot switches when building the
book (see the doc!).
Most good-quality polyglot books are made through combining several
books for white and several books for black in one final book (see the doc).
By-the-way my home-made books that were included with Fruit 2.2.1
commercial release are available for free at
http://wbec-ridderkerk.nl/html/download.htm, and also at the official
Toga page at http://www.uciengines.de/UCI-Engines/TogaII/togaii.html
As far as I know they are still among the best "general purpose" books
for the Fruit-toga family of engines.
c Lacrosse
|
| | |
Date: 16 Oct 2006 00:46:53
From: Dave (from the UK)
Subject: Re: How to create opening book for Toga (fruit)
|
M L wrote:
> It seems quite usual that a 330.000 games file is longer than 2.918.540
> lines.
Yes, but I'd cut the file down in size, to see if the error disappeared
if the file was under 2918540 lines, but it does not.
> You already have 8 lines from the seven-tags header plus one empty line
> before the moves for each game and a blank line after the moves before
> next game so more than 10 lines for one game is common...
> Maybe your cutting was in the middle of a game ...
>
> If your PGN is malformed, polyglot won't accept it.
But the error message is not exactly helpful. Telling me there is a
problem on a line that does not exists.
> A good tool for "cleaning" your PGN is PGNextract by David Barnes ( a
> google search will lead you to the homepage).
I'll try that.
>>I've even built a 64 bit binary (not really expecting it to make any
>>difference), but it does not help.
>
>
> Nonsense.
> Polyglot requires _pure_ PGN files that are text files ...
No, its not nonsense.
The program polyglot is a binary (executable) file. I have built both
the normal 32-bit version and I tried a 64-bit version. I am well aware
PGN files are text files.
> I don't know whther there might be
>
>>some limit in the program that stops it allocating more than a fixed
>>amount of RAM. It keeps saying:
>>
>>10000 games ...
>>allocating 48MB ...
>><snip>
>>allocating 96MB ...
>><snip>
>>allocating 192MB ..
>>.....
>>then:
>>
>>allocating 384MB ...
>>
>>It should not have any problems taking a few GB of RAM if it wants it,
>>so I can't see what the problem is.
>
>
> I have worked with larger PGN files. Only the book-building process does
> require a lot of RAM. Once built the book will be in the 1-5 MB range
> (not GB !) for the largest ones.
My point being that there is plenty of RAM to create the opening books.
I am aware it needs quite a bit of RAM to create the books, but 4 GB
should be enough - it is more than most computers are sold with now.
>>But if there is not much point in having a large opening book, perhaps
>>I'll just cut it down to whatever size works.
>
>
> Much more important are :
> - careful selection of the games to be included
> - careful selection of appropriated Polyglot switches when building the
> book (see the doc!).
Where is the docs? There is virtually nothing with the C++ source code I
downloaded.
> Most good-quality polyglot books are made through combining several
> books for white and several books for black in one final book (see the doc).
>
> By-the-way my home-made books that were included with Fruit 2.2.1
> commercial release are available for free at
> http://wbec-ridderkerk.nl/html/download.htm, and also at the official
> Toga page at http://www.uciengines.de/UCI-Engines/TogaII/togaii.html
I'm not sure if these will work on my machine, since I assume these are
binary files. Since my processor is big-endian, rather than the little
endian of Intel and AMD CPUs, I suspect they will not work.
> As far as I know they are still among the best "general purpose" books
> for the Fruit-toga family of engines.
I'll bear that in mind, although I'd be surprised if they would work on
my machine, which has Sun UltraSPARC CPUs.
> c Lacrosse
>
>
--
Dave (from the UK)
Please note my email address changes periodically to avoid spam.
It is always of the form: [email protected]
Hitting reply will work for a few months only - later set it manually.
http://witm.sourceforge.net/ (Web based Mathematica front end)
|
|
