Question about helping

A place to talk about whatever you want.
Post Reply
emclinux
Posts: 3
Joined: Sun Oct 07, 2018 1:38 am

Question about helping

Post by emclinux » Sun Oct 07, 2018 3:00 am

I had some questions about helping this site out. I play when I can retro games (2600, nes, gen, etc) and it's not too much trouble to run a script that takes a screenshot every 5 minutes while I play. If I do this how many screenshots do you want? Best 2-3? I doubt you would want 12 from an hour of playing Driver 2 or 1942.

Is there any bonus to helping the DB like more requests per month? I like to tinker with my system about as much if not more than I play so there have been a few times I have had to start from scratch and would not like to do the screenscraper run again and with this site it would take me close to a year to get all my scrapes done (1k per month * 11k roms) . I know I know backups work wonders just sometimes I try something and don't want to stop to backup and next thing you know...

Is there an easy way (or less than easy way with curl or something) to get a list of games that need screenshots filtered by system. Say I feel like playing some n64. Is there a way to get a list of titles for the n64 that need some screenshots?

User avatar
Zer0xFF
Posts: 240
Joined: Fri Apr 20, 2018 9:18 am

Re: Question about helping

Post by Zer0xFF » Mon Oct 08, 2018 9:17 am

Hi @emclinux,

Thanks for wanting to contribute to the site.
If I do this how many screenshots do you want? Best 2-3?
we currently have a maximum of images per category, aka max of 5 screenshot, and 5 fanart etc etc, beside that its up to you if you want to upload 1 or 5.
Is there any bonus to helping the DB like more requests per month
I'm afraid at the moment we have no such incentive.
with that said, contributors are very important to the site, as you know its purely crowd sourced, while this might not be evident, as of the new site release, we've actually been keeping a record of every changes a user makes, and we plan with that eventually having a contribution/revision list.

I've also noted that you dont have access to the API, so i presume you're using some else scrapper? if not, and you're able to write your own scrapper, I'd recommend cloning the entire Database, then using your own server for your scrapping needs, that way, we'd have a reduced overall server load, while you'd have unlimited api access.
I have had to start from scratch
this is something we'd like people to try and avoid, if you ever used the old site/api, you would have realized how unreliable the service was (if you didnt, you probably never had to experience, at times a page or an api request taking up to 5 minutes to go through), that was because of indiscriminate scraping.
Is there an easy way (or less than easy way with curl or something) to get a list of games that need screenshots filtered by system.
right now, there isn't an easy way, the closest we have is this, however i see the value and convenience of having something like this, so i will look at adding a platform filter to that page

Edit: just quickly prototyping https://thegamesdb.net/missing.php?type ... tform_id=3, i will in time add an option to make this easier to use, but this gives you a quick idea
Regards
Zer0xFF

emclinux
Posts: 3
Joined: Sun Oct 07, 2018 1:38 am

Re: Question about helping

Post by emclinux » Wed Oct 10, 2018 2:16 am

That will work perfectly. Here is what I have sorted out in case anyone comes across this thread later:
Here are the platform ids:
1 PC
2 Nintendo Gamecube
3 Nintendo 64
4 Nintendo Game Boy
5 Nintendo Game Boy Advance
6 Super Nintendo (SNES)
7 Nintendo Entertainment System (NES)
8 Nintendo DS
9 Nintendo Wii
10 Sony Playstation
11 Sony Playstation 2
12 Sony Playstation 3
13 Sony Playstation Portable
14 Microsoft Xbox
15 Microsoft Xbox 360
16 Sega Dreamcast
17 Sega Saturn
18 Sega Genesis
19 NO SYSTEM
20 Sega Game Gear
21 Sega CD
22 Atari 2600
23 Arcade
24 Neo Geo
25 3DO
26 Atari 5200
27 Atari 7800
28 Atari Jaguar
29 Atari Jaguar CD
30 Atari XE
31 Colecovision
32 Intellivision
33 Sega 32X
34 TurboGrafx 16
35 Sega Master System
36 Sega Mega Drive
37 Mac OS
38 Nintendo Wii U
39 Sony Playstation Vita
40 Commodore 64
41 Nintendo Game Boy Color

There is no system 19 as far as I can tell. I then go to https://thegamesdb.net/missing.php?type ... id={NUMBER ABOVE} (For snes is would be https://thegamesdb.net/missing.php?type ... tform_id=6) and find a game I want to play that has missing screenshots. On my retropi/recalbox type system, I install raspi2png ( https://github.com/AndrewFromMelbourne/raspi2png ) and also install ImageMagick so that we can remove the black outline from the image.

Code: Select all

sudo apt install imagemagick
Next I create a script called:

Code: Select all

/usr/local/bin/getscreenshot
and in that I put the following:

Code: Select all

#!/bin/bash
GAMENAME=$1
for i in 1 2 3;
do
  raspi2png -p $GAMENAME-$i.png -d 120;
  convert -trim $GAMENAME-$i.png $GAMENAME-$i.png
done
I then run the above script like so when I start playing the game:

Code: Select all

getscreenshot NAME_OF_GAME
For example if I am playing Link to the Past I run

Code: Select all

getscreenshot link_to_the_past
This will take a screenshot every 2 minutes named link_to_the_past-2.png link_to_the_past-2.png link_to_the_past-3.png. You can then sftp them off your raspberry pi and upload them here. If you want to change how often your screenshots are captured change the -d number (in seconds between screenshots) but I probably would not go lower than 2 minutes (120 seconds) so that the screenshots have some diversity.
Last edited by emclinux on Wed Oct 10, 2018 3:26 pm, edited 1 time in total.

emclinux
Posts: 3
Joined: Sun Oct 07, 2018 1:38 am

Re: Question about helping

Post by emclinux » Wed Oct 10, 2018 2:20 am

As for the scrapper thing I use skyscraper. How would I mirror the DB? Do you have any instructions on this? I have web servers that have I can use or local systems that I can use for this and I can just modify the Skyscrapper code to point to my stuff. Don't really have the time to write my own scrapper but I can modify existing code to suit my needs (yay FOSS)

User avatar
Zer0xFF
Posts: 240
Joined: Fri Apr 20, 2018 9:18 am

Re: Question about helping

Post by Zer0xFF » Wed Oct 10, 2018 9:07 am

I don't know if any scrapper has that ability, as this is something that would usually be done server sided(most people dont post their server sided code).
but effectively, (you'd need to request access to API) you'd use the API and request all games within a console, loop through the pages, and save the results,
then have your scrapper use that data to populate the emu or whatever you're using.
Regards
Zer0xFF

Post Reply