New DOSbox idea...

This is a forum section for ideas howto implement and running DOS games in ECC (using DOSBox), this forum wil be temporally here until the problem is solved.
User avatar
Vicman
PLATINUM member
Posts: 1233
Joined: 17 Jun 2007, 16:12
Location: Germany,Bochum

New DOSbox idea...

Unread post by Vicman » 11 Dec 2009, 20:54

First :
On Underground Gamers are some people, who are creating a complete Dos-Collection.
Yes, i know....there are many dos-collections out in the worldwideweb....
But they do there job very good, and they also rename the Games in that way, that TOSEC use.

Fact is : This could be the best DOS-Collection ! We should use this dumps for a DOS-Imagepack.

btw.
The Guy's of this DOS-Projekt are searching for a good Frontend (CRC32 based) , and I think, eCC can make it. :clapshout:

Here's what is written on WIKI-UG about the :
Complete PC MS-DOS Collection :
Spoiler: show
What is it?

We're giving go at collecting up all the PC games released for DOS (as in not MS-Windows) from the dawn of the PC through today. That's a pretty big task, but we've hit the ground running, and there is no stopping us now.

For the casual observer, please use the year table at the bottom of this page to see how we're progressing. We've already released a torrent of 2200 files from the DOS "golden years" of 1981-1989, with just a few missing bits remaining, and more is on the way. Stay with us for the long haul, it'll be worth it.

Aren't there already DOS collections out there?

Yes, and we will freely admit that a majority of the items in our collection were sourced from other collections. We're not trying to take credit for releasing the stuff, we just want the stuff that's out there to be cleaned up a little bit, named in a consistent manner, virus checked, and organized in a useful fashion. For a quick example of the inaccuracies of most torrents, take a look at any DOS collection torrent and see how many titles claimed to have been released for DOS in 1980. Then take a look at when the IBM PC was released to the public. We've also found there are LARGE amounts of games that have not yet been released in any torrent or on any abandonware site yet. This will hopefully remedy both of these problems.

What can you do to help?

The first thing you can do is see if you happen to have anything on the DOS collection Missing list These are the top priority of the collection, and we'd like them by any means necessary.
We need some researchers.
Please visit our unknown titles page and help us decide if these titles are windows or DOS, and what year they were released in. It's easy work, just a little mind numbing.
We need more eyes.
The more people who are looking at the titles we've named, and are familiar with the games we're working on, the better accuracy we'll have in the entire project. Please use the year links in the chart at the bottom of this page to display lists of all the games that go into those years. If you spot anything weird, typoed, incorrect, or anything else of note, please use the wiki talk pages (or forum) to let us know something is amiss.
If you really feel like dedicating some effort into this cause, please keep reading. There are lots of guidelines that we're trying to follow, as accuracy in this type of project is important.

Definition of "Complete"

So what exactly is meant by "complete"? How do you know it's complete?

Part of the problem with this project is that there is no known guide or list of all the DOS games that were ever released. You've seen the torrents, abandonware sites, and game sites out there-there are thousands of games, but no encyclopedias or guides have ever been created to list them all, so we decided that we'd first try and make one.

Enter the master list: master list

The idea behind the master list was to take every possible DOS game listing, from torrents, abandonware sites, personal collections, and BBS listings, merge them all together, and attempt to weed out the duplicates.

Once we knew what was out there, then it's a matter of collecting, testing, verifying and placing games into the proper year. The master list is a key to this collection. It is constantly being updated, de-duped, and appended to. As we properly place games in their respective years, the master list is updated to reflect its status. We have no illusions that this collection will ever be 100% complete, but it is shaping up to be an excellent baseline for something that could be complete, and it is by far the most complete and accurate DOS collection out there.

Naming Conventions

Properly naming a game is ESSENTIAL to this project. Spelling, capitalization, and subtitles must all accurately reflect the game being named.

This is the basic structure:

Title- subtitle version (language/country) [flags] (Year)(Publisher) [Genre]

The naming convention is based on TOSEC's convention, with a few additions for special DOS cases.

Examples:
Stardust (Sp) (1988)(Topo Soft) [Action]
Space Quest- The Sarien Encounter v1.1a [h1] (1986)(Sierra On-Line, Inc.) [Adventure]
Spider-Man v2.6 [a1][f1] (1985)(Load'N'Go Software) [Adventure]

Notice in space quest there, whenever there is a subtitle with a colon, the naming convention is to NOT put a space before the dash. Whenever the subtitle uses a dash, or no colon is used, then there is a space between the title, dash, and subtitle.


Version It is assumed that all games that do not display the version number in the game itself should be v1.0. Therefor, there is no need to add v1.0 to all games, including games that actually show a v1.0 in the game. In other words, only non v1.0 games should utilize this section of the naming convention.

(Language) Just like most game sets, the language field indicate the language/country of the game, for now :

No field means English

(Fr) means French

(De) means German

(Ru) means Russian

(Cn) means Chinese

(Jp) means Japanese

(Cz) means Czech

(Po) means Polish

(No) means Norwegian

(It) means Italian

(Sp) means Spanish

(Multi-X) means that the game offers choosing between X languages, with English being one of the selections.


Flags

[DC] (DOS conversion) This represents PC-Booter games that have been converted into DOS based games by people throughout the years. A good source for such Booters->DOS conversion is Retrograde Station

[Genre] The Genre field, utilized in the MobyGames database, was added to our top level filenames entirely because the data was already there, and was too good to pass up! ;)
While the genre is sometimes rather arbitrary, it can be useful information for finding new games to explore. It was also decided to be left at the end of the filename so that the information could be easily, bulk, stripped off of filenames with a decent file renamer.

[ax] = where x is 1-n = alternate version. Used whenever a new version of a title comes up, but the .exes or other files do not match known checksums. Sometimes this is due to cracks in the copy protection, sometimes it's due to the publisher re-releasing a different version, but no actual version number is included. It is sometimes possible to compare two different versions of the same .exe and actually reverse the copy protection changes that various crackers have done to the game. Creating/obtaining "clean" versions of games is a 2ndary goal to this project.

[fx] = where x is 1-n = fixed/patched (not copy protection but things like graphics, speed, not running from A:, etc) Note that a LOT of games were/are already fixed before we got around to naming and tagging them. These aren't marked, since there is no way to really know what was done, if anything. Unless there's a readme or .nfo file inside whatever archive of the game we received specifically mentions some kind of fixing, this tag isn't used.

[bx] = where x is 1-n = bad, best copy available. Simply put, this is all we have to work with until something better comes along. It may also be a case that a game is protected and has not yet been cracked.

[hx] = where x is 1-n = hacked/graffiti "cracked by mr coolDude" type stuff. We wish these didn't exist. Can also include hacks for cheats.

[SW] / [SWR] = Shareware and Shareware, registered respectively. Yes, this collection includes shareware. It wouldn't be complete without it. Any time the author suggests sending a donation, even if there is no nag screen or "unregistered" status on the title screen, it is tagged with [SW]. Games that are dedicated to the public domain as freeware, or commercial, or no mention anywhere of any amount of money or other donations are not tagged at all.

Figuring out the year of release

Being anal about how we want this collection represented, one of the most important items is the year of release for a game. Ideally, we want to use the year presented in the title screen of the game. This is the easiest to verify. If the title screen doesn't show a year, the next best option is to dig through the main executables and see if you can find any year messages embedded in the program. Drag the file into either a dedicated hex editor, or just use notepad. You'll see lots of garbage, but if you do a search for "198" or "199", you very well may find a text string that shows a copyright and year of release. While you're there, look for version information and other publisher data, as that may come in handy too. You might also find year dates for the compiler that was used to create the game. These can be useful to give you a baseline as to when the game was released. (ie, if it says "(c) 1988 Borland International" then there is no way the game could be pre-1988.)


If this method fails, check the file date/time stamp of the files in the game's archive. Often the date/timestamp is wrecked when a game is archived, but sometimes they survive. If the date is somewhere close to the vintage of the game you're checking, odds are pretty good that is the year it was released. Try to find the latest date in the archive, however, be wary of config files or savegames that get stashed inside archives-those can often be very different dates than the game's release. (ie, if you see files dating 1986 and 1987, pick 1987 as the release year. If you also see "hiscore.dat" dated 2004, ignore it, as it obviously was archived considerably later.)


If this method also fails, try google for more information. If you find that most all the abandonware sites that host this particular game all use the same year, the best thing to do is place the game in that year's folder, but name the top-level filename with "(198x)" or "(199x)" just to signify that the year is impossible to determine, but we're using the accepted year of release.


If a game has multiple years listed, such as "1985,86,88" then please use the LAST year listed, as that year should represent the year the game was physically released to the public.


There are some fuzzy rules here as well to be aware of. Use your best judgment for these. If a game was released in, say 1985, for a different platform (maybe an Apple II or something) and then the game was ported over to DOS at a later date, technically, we should be using the year that it was available for the PC. If, however, the game's title screen clearly has a date in it, then, well, we tend to be using the screen date. It's ugly. B1 Nuclear bomber in 1980 is an example of this. Games in which the author has dug out the source code to a game he wrote when he was in highschool and then released 2 decades later onto the internet are also examples of these. These rules may have to be re-visited sometime in the future.

Games that have been re-released, perhaps due to an additional graphics update, or even the "remastered" games (like digger, jumpman, etc) should be put in the year that the re-release was made available to the public. Ie, jumpman was released originally in 1984. It was remastered in 2003. The remastered game should be placed in 2003, but it should also have a (remastered) tag attached to the top level filename to denote the difference.

Tools

A new tool has emerged from this project which can help people with large collections of DOS games sift and sort through them and compare their own files against the ones we have already named.
DOSCenter is a windows based tool that loads in our collection's .dat file and allows you to scan your zip file collection against it, showing a score of how close of a match your particular zip is to one of the known zip files in our torrents. DOSCenter also has a zip merging tool (if you have, apb1thg.zip apb2thg.zip and apg3thg.zip you can merge them into 1 zip) among other renaming and file displaying options. It is still a work in progress, but has certainly show its usefulness early on. (and yes, it was named in honor of ROMCenter because the similar functionality. ROMCenter is not quite built to handle the files that a DOS collection generates, so a new app was written.)

Methodology of collecting/listing

Because of the magnitude of this project, it was easier to break down the task of collecting and verifying the titles by year of release. This gave us a quick sense of accomplishment during the early years, and gave us a decent mode of operation for verifying and naming each title. Here's how the operation works in a nutshell:

1) All titles for a given year are pulled from the master list, and given a dedicated wiki page.

2) A volunteer then take a year's list, locate the games in it, and test each one individually.

3) As titles are tested and top level filenames adjusted to the convention described above, odd/interesting/troublesome titles are discussed in the forum. Titles that have the year of release incorrectly listed in the master list are placed at the bottom of the wiki page for the project secretary to pick up and change the master list-thus pushing a title back into the master list and into another year to be pulled later.

4) When titles are verified to be in the correct year, the [genre] information is added to the title, and that title is considered to be "finished". Finished titles are updated back into the master list by the secretary.

5) Eventually, all the titles in the master list will be pushed into the correct year, until the [genre] data is added to all of them, and the project will then be complete!

6) Titles that cannot be located via torrents, sites or personal collections, get added to the missing list for that particular year, and the total missing list, which has missing titles from all the years of the project. Likewise with games that are non-working, uncracked, etc. Simply list those titles in a different heading in the wiki page and eventually we will get around to finding another source for the game, or fixing it.



What should be included in the each archive?

We're trying to keep the games "pure" whenever possible. In a perfect world, anything that was not released with the game on the original media it was released on (if it was released on media) should be the only thing inside each .zip file. This means that docs, walkthroughs, save games, configuration files, even source code and key codes should be removed. All of these additional files would ideally be released as separate "addon" pack torrents to be released at a later date. All of these additional items would then be named using the same naming convention as the game itself, so that everything relating to a given title could be located at once and kept together. It's not a perfect world yet, so some of these items do get slipped into the mix occasionally. At a minimum, BBS ads, .nfo files from pirate groups and "this file passed through..." types of files have got to go!

We are also trying to keep a balance between being 100% original copies of games, with doc checks and key disk protections intact, and having the game actually be playable without bringing out your 286 to play it on. This project errs on the side of playability over accuracy. Due to the sheer number games, and the almost impossible task of trying to source original, untouched copies of each of them to take perfect copies of, it makes the archive more consistent to have all the games be "unzip and play" vs. "unzip, extract image file, write image file to media, perform installation routine, launch game, locate documentation for check, enter doc check, and play"

That said, we are interested in "pure" copies as they become available. The key to all good complete sets are [!] (known good dump) files, and we will certainly have a share of them in this collection. If an uncracked .exe file is available along with a cracked copy, we rename the uncracked one to .ex_ and include it inside the archive. Games that can be verified against two known, original copies of a game, produced on different computers, and each file matches a CRC, will get renamed to [!] at the end of the filename. Since there is still plenty of archival work to be done on this project, focusing on [!]'s is not a priority at the moment.

Ripped games (games in which audio tracks or video files have been removed to make for a quicker download) are to be REJECTED from this project. If no complete copies of the game are available, please tag the title as [b1] and make a note in the notes page as to why this is the "best copy available". Exceptions to this rule would be games that were re-bundled onto CD-ROM and packaged with demos of other games, and perhaps documentation. Since no part of the game itself is ripped, it is ok to archive up only the game to be released in this archive. (this rule may be revisited as we get into later years)

Final Goals

The ultimate goal for this project is to have a tool such as ROMCenter, CLRMAMEPro or GoodTools which can verify using CRC32's that your DOS collection is in order and properly named.

In order to reach this goal, 4 phases (at least) are in order:

1. Collect the pieces. This is our current phase. Using the master list, assemble all the possible titles we can, and test them.
2. Rename the top level files. Also part of the current phase. Whenever a new title is discovered, attempt to properly name the top level filename to a standard.
3. Once a naming standard has been completed, compare contents of the files inside the archive and map them using checksums. Build a ROMCenter/CLRMAMEPro datafile to verify and assist in content checking.
4. Once multiple sources for the same game can be obtained and contents can be verified, move the top level file names to a [!] (known good dump). Comparing dumps from different sources will help us weed out such things as savegames, config files, and externally added files to an archive.

Beyond that, imagine if you will, a GUI based application (similar to MAME32) which displays a list of titles on the left, sortable by year, publisher, etc, and when you click on a given title, the right side of the screen shows a box cover, screenshot, documentation, etc. 2x click on a title, and DOSBox launches and puts you directly in the game. Since we have standardized naming across our game titles, screenshots and other meta data, it would be a simple task to populate such a front end by merely downloading update packs. Since our titles are also named to match mobygames.com, it would also be easy to tie our GUI directly to the web and utilize the database that is already online. Anyone interested in working on such a front end for us?

We will be going through the files contained in this project multiple times, to help weed out known bad copies of the games, promote other copies to [!]'s and continue the cleanup and renaming process. The work is never done, and we're ok with that. Most of us have been doing this type of work in one way or another for decades anyway.

The Dump

The Dump can be accessed here. The dump is a list of all games that were at first included in our masterlist but, for some reason (web research, programmer contact, game testing), were found not to be true DOS games or not existing games at all. We store these titles in a dump so we can always remember that there is still a possibility that these games exist, and if you have any information on these games, please provide it to us.

Now my Idea to archive and play them with eCC
1. This Collection is ready for DL on UG for the Years 1981-1989.
2. If you aren't allready a member on UG - Hurry Up ;)
3. Let eCC parse the *.zip files the way, that Phoenix has written in this Thread :

Code: Select all

[EXTENSIONS]
zip = 1
[PARSER]
FileParserZip#dosbox = zip
4. now, we all have the same dumps/crc32 of the Games

next question is, how to start the zipped dumps on eCC ?
my idea :
- let us use the "ID-Info-Field" (same field as for the VIC20-script) in eCC to start the *.conf with DOSbox !

example on one Dosgame :
Game is : $100,000 Pyramid, The (1988)(Box Office) [Trivia]
Make a *.conf-file and name it : $100,000 Pyramid, The (1988)(Box Office) [Trivia].conf

(Put all *.conf-files in a seperatly Folder e.g. "Config")

Now, type $100,000 Pyramid, The (1988)(Box Office) [Trivia].conf in the ID-Info-Field.

then we need a script that :
1. extract the zipped-game to the unpack-folder
2. start dosbox with config "XY" (ID-Info-Field)


Hope you understand this idea...i know, my english is terribly :D

Is this possible ???

If yes, we are able to make an Imagepack & ecc-Datfile for our lovely DosGames :swing:
(an the guy's on UG can use eCC for their Project ! )

What do you think ?

btw.
I know, we also need a solution for savegame-files, when using the auto-unpack-folder.....

@ te_lanus :
Can you try, to write this script ? Phoenix is a little busy with his EDC-Projekt (think so) ;)
Attachments
master.zip
The MASTER-LIST :
[compl. Gamelist for this UG-Projekt]
(182.07 KiB) Downloaded 43 times
imagepacks:
Done: Vic20, N64, Philips VG-5000 G7000 G7400, SordM5, Amstrad GX4000, Enterprise 64/128
Progress: Atari 8bit, Dosbox, Exelvision EXL 100
Vicman's eCC-Clips on You Tube

User avatar
Phoenix
ECC Developer 2006-2016
Posts: 9057
Joined: 27 Aug 2006, 01:17
Location: Deventer, The Netherlands
Contact:

Re: New DOSbox idea...

Unread post by Phoenix » 12 Dec 2009, 04:19

Vicman wrote:next question is, how to start the zipped dumps on eCC ?
my idea :
- let us use the "ID-Info-Field" (same field as for the VIC20-script) in eCC to start the *.conf with DOSbox !

example on one Dosgame :
Game is : $100,000 Pyramid, The (1988)(Box Office) [Trivia]
Make a *.conf-file and name it : $100,000 Pyramid, The (1988)(Box Office) [Trivia].conf

(Put all *.conf-files in a seperatly Folder e.g. "Config")

Now, type $100,000 Pyramid, The (1988)(Box Office) [Trivia].conf in the ID-Info-Field.
No No, no need to put this data in the ID-info-field :no: :lmfao: , a simple script will do it, it can look for this configfile in a config folder automaticly, so really no need to copy/paste names here

p.s. i also like the idea, and i have also thought about extending it with something else....more soon!
Sebastiaan Ebeltjes | ECC Developer 2006-2016 | ECC Forum Admin | Phoenix Interactive WebMaster
[- ECC programs -]
eccUpdate, eccScriptSystem, GtkThemeSelect, DatFileUpdater (DFU), ImagePackCenter (IPC), eccDiagnostics, 3dGallery, iccImageInject
eccKameleonCode, eccCreateStartmenuShotcut, eccThirdPartyConfig (TPC), EmuMoviesDownloader (EMD), eccVideoPlayer
MobyGamesImporter (MGI), ECC Amiga GetGemusConfig.

User avatar
Vicman
PLATINUM member
Posts: 1233
Joined: 17 Jun 2007, 16:12
Location: Germany,Bochum

Re: New DOSbox idea...

Unread post by Vicman » 12 Dec 2009, 08:27

Phoenix wrote: i also like the idea, and i have also thought about extending it with something else....more soon!
sounds nice ! waiting for your idea ;)
imagepacks:
Done: Vic20, N64, Philips VG-5000 G7000 G7400, SordM5, Amstrad GX4000, Enterprise 64/128
Progress: Atari 8bit, Dosbox, Exelvision EXL 100
Vicman's eCC-Clips on You Tube

User avatar
Phoenix
ECC Developer 2006-2016
Posts: 9057
Joined: 27 Aug 2006, 01:17
Location: Deventer, The Netherlands
Contact:

Re: New DOSbox idea...

Unread post by Phoenix » 12 Dec 2009, 13:59

p.s. how do we handle DOS games that need to be installed first??, do you install them?, and then ZIP them again?, examples:
Attachments
dos_box_installgame01.png
dos_box_installgame01.png (25.81 KiB) Viewed 1016 times
dos_box_installgame02.png
dos_box_installgame02.png (25.71 KiB) Viewed 1016 times
Sebastiaan Ebeltjes | ECC Developer 2006-2016 | ECC Forum Admin | Phoenix Interactive WebMaster
[- ECC programs -]
eccUpdate, eccScriptSystem, GtkThemeSelect, DatFileUpdater (DFU), ImagePackCenter (IPC), eccDiagnostics, 3dGallery, iccImageInject
eccKameleonCode, eccCreateStartmenuShotcut, eccThirdPartyConfig (TPC), EmuMoviesDownloader (EMD), eccVideoPlayer
MobyGamesImporter (MGI), ECC Amiga GetGemusConfig.

User avatar
Vicman
PLATINUM member
Posts: 1233
Joined: 17 Jun 2007, 16:12
Location: Germany,Bochum

Re: New DOSbox idea...

Unread post by Vicman » 12 Dec 2009, 17:39

Don't know, what the Guy's on UG do.....(if we use their Collection)

I would say, installing & rezipping.
But, do we get then diff. CRC32 ?
imagepacks:
Done: Vic20, N64, Philips VG-5000 G7000 G7400, SordM5, Amstrad GX4000, Enterprise 64/128
Progress: Atari 8bit, Dosbox, Exelvision EXL 100
Vicman's eCC-Clips on You Tube

User avatar
Phoenix
ECC Developer 2006-2016
Posts: 9057
Joined: 27 Aug 2006, 01:17
Location: Deventer, The Netherlands
Contact:

Re: New DOSbox idea...

Unread post by Phoenix » 12 Dec 2009, 18:16

Hi Vicman,

Yes, you get different CRC32, please note that these projects and most dos games are PURE, that means in most cases there is only an installer for the game, so the game has to be installed first before you can play it!, we have to find a solution for this...
Sebastiaan Ebeltjes | ECC Developer 2006-2016 | ECC Forum Admin | Phoenix Interactive WebMaster
[- ECC programs -]
eccUpdate, eccScriptSystem, GtkThemeSelect, DatFileUpdater (DFU), ImagePackCenter (IPC), eccDiagnostics, 3dGallery, iccImageInject
eccKameleonCode, eccCreateStartmenuShotcut, eccThirdPartyConfig (TPC), EmuMoviesDownloader (EMD), eccVideoPlayer
MobyGamesImporter (MGI), ECC Amiga GetGemusConfig.

User avatar
Vicman
PLATINUM member
Posts: 1233
Joined: 17 Jun 2007, 16:12
Location: Germany,Bochum

Re: New DOSbox idea...

Unread post by Vicman » 12 Dec 2009, 20:11

Phoenix wrote: Yes, you get different CRC32.........
Well....this is a (big) disadvantage of CRC32-based Imagepacks :oops:
imagepacks:
Done: Vic20, N64, Philips VG-5000 G7000 G7400, SordM5, Amstrad GX4000, Enterprise 64/128
Progress: Atari 8bit, Dosbox, Exelvision EXL 100
Vicman's eCC-Clips on You Tube

Locked

Who is online

Users browsing this forum: No registered users and 1 guest