ECC ImageInject freezes with larger amounts of roms

Posts that are done/fixed/closed or aged! for this section are placed in here to keep the forum in shape & more viewable!
Tormentor667
Posts: 111
Joined: 09 Jun 2013, 13:00

ECC ImageInject freezes with larger amounts of roms

Unread post by Tormentor667 » 09 Jun 2013, 19:09

Another one... maybe I am just too dumb or a very good bug-spotter :P

When feeding my database with images (via Kameleon and also via EmuMovies) it works fine with smaller amounts of roms (e.g. 32X) but I always run into problems with larger amounts of roms (e.g. GameBoy, NES, SNES, Genesis). It always freezes while searching images, about 50% (more or less, I can't say that for certain). The only thing that I can do then is to CANCEL the current progress, but starting again also starts from zero - so I can't finish it.

I am not sure if one of the roms us buggy (even though I doubt that) or if the crawler has problems with larger amounts because of the cache. The questions now are: Can I scrape only a selection instead of only one or all roms? And Is it possible to stop, save what I have, and then continue with the rest? That way I could at least finish it.

Thanks already for the help.
snap01.png
snap02.png

User avatar
Phoenix
ECC Developer 2006-2016
Posts: 9059
Joined: 27 Aug 2006, 01:17
Location: Deventer, The Netherlands
Contact:

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Phoenix » 10 Jun 2013, 00:46

Hi,

I will test this soon with large quantity's of images, since i am redoing the ICC database atm!
Sebastiaan Ebeltjes | ECC Developer 2006-2016 | ECC Forum Admin | Phoenix Interactive WebMaster
[- ECC programs -]
eccUpdate, eccScriptSystem, GtkThemeSelect, DatFileUpdater (DFU), ImagePackCenter (IPC), eccDiagnostics, 3dGallery, iccImageInject
eccKameleonCode, eccCreateStartmenuShotcut, eccThirdPartyConfig (TPC), EmuMoviesDownloader (EMD), eccVideoPlayer
MobyGamesImporter (MGI), ECC Amiga GetGemusConfig.

Tormentor667
Posts: 111
Joined: 09 Jun 2013, 13:00

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Tormentor667 » 10 Jun 2013, 01:15

Ok then, thanks, looking forward to this.

Tormentor667
Posts: 111
Joined: 09 Jun 2013, 13:00

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Tormentor667 » 10 Jun 2013, 15:48

By the way, the same happens with EmuMovies as source
emumov.jpg

User avatar
Phoenix
ECC Developer 2006-2016
Posts: 9059
Joined: 27 Aug 2006, 01:17
Location: Deventer, The Netherlands
Contact:

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Phoenix » 10 Jun 2013, 16:43

1) So it does import some images...then it freezes?
2) does the freeze thing always happen on the same game/rom (crc32)?

Could be a hard bug to track since there is no error message shown...whats wrong.... :detective:
Sebastiaan Ebeltjes | ECC Developer 2006-2016 | ECC Forum Admin | Phoenix Interactive WebMaster
[- ECC programs -]
eccUpdate, eccScriptSystem, GtkThemeSelect, DatFileUpdater (DFU), ImagePackCenter (IPC), eccDiagnostics, 3dGallery, iccImageInject
eccKameleonCode, eccCreateStartmenuShotcut, eccThirdPartyConfig (TPC), EmuMoviesDownloader (EMD), eccVideoPlayer
MobyGamesImporter (MGI), ECC Amiga GetGemusConfig.

Tormentor667
Posts: 111
Joined: 09 Jun 2013, 13:00

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Tormentor667 » 10 Jun 2013, 21:44

Phoenix wrote:1) So it does import some images...then it freezes?
Exactly
Phoenix wrote:2) does the freeze thing always happen on the same game/rom (crc32)?
It doesn't freeze on a certain rom, it freezes randomly, but always about 50% of the whole roms.

Tormentor667
Posts: 111
Joined: 09 Jun 2013, 13:00

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Tormentor667 » 11 Jun 2013, 19:20

Sorry for double posting but maybe you might be interested in this:

I played around with it further more, and it doesn't seem to have something to do with the 50%, actually it sometimes freezes around 5% or 60%, anything is possible. But - it always freezes I have the feeling. I still wasn't able to complete my rom database and this is kinda sad I have to admit, so a fix would be awesome. I guess it doesn't help that the error seems to occure randomly, but maybe it's something you can work with.

User avatar
Phoenix
ECC Developer 2006-2016
Posts: 9059
Joined: 27 Aug 2006, 01:17
Location: Deventer, The Netherlands
Contact:

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Phoenix » 11 Jun 2013, 22:01

Hmm i cannot recreate the bug, it works fine over here!

Please try again today, the server was doing some backup etc last days, so it was terribly busy....is the problem still there?, if yes could you answer these:

1) Wich OS are you using? Windows XP/Vista/7/8?
2) Are you doing something else while ECC is busy? (like browsing?)
3) Harddisk full maybe? (or C disk?)
Sebastiaan Ebeltjes | ECC Developer 2006-2016 | ECC Forum Admin | Phoenix Interactive WebMaster
[- ECC programs -]
eccUpdate, eccScriptSystem, GtkThemeSelect, DatFileUpdater (DFU), ImagePackCenter (IPC), eccDiagnostics, 3dGallery, iccImageInject
eccKameleonCode, eccCreateStartmenuShotcut, eccThirdPartyConfig (TPC), EmuMoviesDownloader (EMD), eccVideoPlayer
MobyGamesImporter (MGI), ECC Amiga GetGemusConfig.

Tormentor667
Posts: 111
Joined: 09 Jun 2013, 13:00

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Tormentor667 » 11 Jun 2013, 22:14

Tried it again today without any improvements.

1) Windows 7 Professional
2) I shut down any single program to make sure nothing interfers, even the Firewall and Antivirus has been deactivated in the time during the download sequence
3) Any harddisk has at least 5GB left, I doubt that this might be a problem.

What else could I try? Or what programs does ICC need to run correctly? Java maybe? Maybe I need to update something.

One suggestion though: What about the option to mark several roms at once and then download the images via the right-click dialog? This would be a nice workaround for those who can't make the "Get all images" function to work propably. I could mark batches then and finish it up that way.

User avatar
Phoenix
ECC Developer 2006-2016
Posts: 9059
Joined: 27 Aug 2006, 01:17
Location: Deventer, The Netherlands
Contact:

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Phoenix » 11 Jun 2013, 23:26

ICC is Autoit based, just runs the Autoit3.exe with the AU3 script in "ecc-core\tools" :smartass:

Multiple select is and cannot be possible with the PHP-GTK engine :no: , maybe i could create a "add to todo-list" menu option and then it downloads all the images when initiated with ICC, this will not work with EmuMovies (only namebased per platform)

Ps. in the ROM context menu there is also a option to download images form ICC, did you know?, maybe not batch but a few clicks more to download the remaining images for now?

For the problem... i was thinking that it could be some ASCII characters in a filename...but that doens't make sense, because it also hangs with ICC, wich downloads CRC32 based...

I will post a newer autoit and a older autoit version, maybe you can replace the current .exe and see if that changes anything....

Stay tuned!
Sebastiaan Ebeltjes | ECC Developer 2006-2016 | ECC Forum Admin | Phoenix Interactive WebMaster
[- ECC programs -]
eccUpdate, eccScriptSystem, GtkThemeSelect, DatFileUpdater (DFU), ImagePackCenter (IPC), eccDiagnostics, 3dGallery, iccImageInject
eccKameleonCode, eccCreateStartmenuShotcut, eccThirdPartyConfig (TPC), EmuMoviesDownloader (EMD), eccVideoPlayer
MobyGamesImporter (MGI), ECC Amiga GetGemusConfig.

Tormentor667
Posts: 111
Joined: 09 Jun 2013, 13:00

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Tormentor667 » 11 Jun 2013, 23:32

Okay, understood.

I actually new about the context menue to download single entries, but in my case, with several hundreds of roms, this is still a very time-consuming solution. It's better than nothing, but I'd prefer something less click-intensive :) I already fear the pain in my left hand :P The real strange thing is, that it happens randomly... and that makes it very hard to track I guess.

I will replace the files as soon as you post them and see if that helps.

User avatar
Phoenix
ECC Developer 2006-2016
Posts: 9059
Joined: 27 Aug 2006, 01:17
Location: Deventer, The Netherlands
Contact:

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Phoenix » 11 Jun 2013, 23:37

Ok, before you continue, please make a backup of the file: ecc-core\thirdparty\autoit\AutoIt3.exe wich is stable v3.3.8.1

Now, replace AutoIt3.exe with this newer (beta) version:
AutoIt3_v3.3.9.4.zip
(449.73 KiB) Downloaded 20 times
Any improvents?, no?, try this older version:
AutoIt3_v3.3.4.0.zip
(411.37 KiB) Downloaded 17 times
Sebastiaan Ebeltjes | ECC Developer 2006-2016 | ECC Forum Admin | Phoenix Interactive WebMaster
[- ECC programs -]
eccUpdate, eccScriptSystem, GtkThemeSelect, DatFileUpdater (DFU), ImagePackCenter (IPC), eccDiagnostics, 3dGallery, iccImageInject
eccKameleonCode, eccCreateStartmenuShotcut, eccThirdPartyConfig (TPC), EmuMoviesDownloader (EMD), eccVideoPlayer
MobyGamesImporter (MGI), ECC Amiga GetGemusConfig.

Tormentor667
Posts: 111
Joined: 09 Jun 2013, 13:00

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Tormentor667 » 12 Jun 2013, 00:12

Tried both, the new one doesn't improve anything - the bug still happens, the old one doesn't run because of this error:
auto.jpg

User avatar
Phoenix
ECC Developer 2006-2016
Posts: 9059
Joined: 27 Aug 2006, 01:17
Location: Deventer, The Netherlands
Contact:

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Phoenix » 12 Jun 2013, 00:52

if you have some time...You could try to change the code @ line 68:

Code: Select all

$IdtRead = FileOpen($EccIdtFile)
into:

Code: Select all

$IdtRead = FileOpen($EccIdtFile, 0)
Since the older version needs it (it's optional in the newer version)
See here: http://www.autoitscript.com/autoit3/doc ... leOpen.htm
Sebastiaan Ebeltjes | ECC Developer 2006-2016 | ECC Forum Admin | Phoenix Interactive WebMaster
[- ECC programs -]
eccUpdate, eccScriptSystem, GtkThemeSelect, DatFileUpdater (DFU), ImagePackCenter (IPC), eccDiagnostics, 3dGallery, iccImageInject
eccKameleonCode, eccCreateStartmenuShotcut, eccThirdPartyConfig (TPC), EmuMoviesDownloader (EMD), eccVideoPlayer
MobyGamesImporter (MGI), ECC Amiga GetGemusConfig.

Tormentor667
Posts: 111
Joined: 09 Jun 2013, 13:00

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Tormentor667 » 12 Jun 2013, 10:20

I have absolutely no idea how to do that :) Which file do I need to alter? Please consider that I am not a programer.

User avatar
Phoenix
ECC Developer 2006-2016
Posts: 9059
Joined: 27 Aug 2006, 01:17
Location: Deventer, The Netherlands
Contact:

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Phoenix » 13 Jun 2013, 21:15

Tormentor667 wrote:I have absolutely no idea how to do that :) Which file do I need to alter? Please consider that I am not a programer.
The file like stated in your error messagebox :yes_head: "ecc-core\tools\iccImageInject.au3", you can use "ecc-core\thirdparty\notepad++\notepad++.exe" as editor :sorcerer:
Sebastiaan Ebeltjes | ECC Developer 2006-2016 | ECC Forum Admin | Phoenix Interactive WebMaster
[- ECC programs -]
eccUpdate, eccScriptSystem, GtkThemeSelect, DatFileUpdater (DFU), ImagePackCenter (IPC), eccDiagnostics, 3dGallery, iccImageInject
eccKameleonCode, eccCreateStartmenuShotcut, eccThirdPartyConfig (TPC), EmuMoviesDownloader (EMD), eccVideoPlayer
MobyGamesImporter (MGI), ECC Amiga GetGemusConfig.

Tormentor667
Posts: 111
Joined: 09 Jun 2013, 13:00

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Tormentor667 » 14 Jun 2013, 19:32

Tried it but it doesn't improve anything. The problem still persists.

Though, I tried it on a different computer, and there it works fine. It has something to do with my machine obviously, but it's still impossible to track for me. Isn't there something like a LOG file?

User avatar
Phoenix
ECC Developer 2006-2016
Posts: 9059
Joined: 27 Aug 2006, 01:17
Location: Deventer, The Netherlands
Contact:

Re: ECC ImageInject freezes with larger amounts of roms

Unread post by Phoenix » 17 Nov 2013, 23:48

I've looked again...i cannot trace the bug, so this must be a (OS) system thingy, since it also works on your other computers...

Closing...
Sebastiaan Ebeltjes | ECC Developer 2006-2016 | ECC Forum Admin | Phoenix Interactive WebMaster
[- ECC programs -]
eccUpdate, eccScriptSystem, GtkThemeSelect, DatFileUpdater (DFU), ImagePackCenter (IPC), eccDiagnostics, 3dGallery, iccImageInject
eccKameleonCode, eccCreateStartmenuShotcut, eccThirdPartyConfig (TPC), EmuMoviesDownloader (EMD), eccVideoPlayer
MobyGamesImporter (MGI), ECC Amiga GetGemusConfig.

Locked

Who is online

Users browsing this forum: No registered users and 1 guest