Having Trouble Understanding the Update Process

Need help with your application? Ask here.
Post Reply
balderdashian
Posts: 8
Joined: Tue Aug 07, 2018 5:49 am

Having Trouble Understanding the Update Process

Post by balderdashian »

I would like to download the whole database using my private key doing Games/ByPlatformID, as has been suggested here. My dev-use-only scraper tool will save these out as json files, one per platform.

How would I then go about doing daily or weekly updates while using as few requests as possible? Edits and new entries, both games and platforms?


balderdashian
Posts: 8
Joined: Tue Aug 07, 2018 5:49 am

Re: Having Trouble Understanding the Update Process

Post by balderdashian »

Yes, of course I saw that but many things are still unclear.

What is a last_edit_id?

If the last_edit_id is based on time, what is the time query?

Does this include new additions or only edits? What about removals?

What about platform edits, additions and removals?

What about edits, additions, and removals for developers, genres, and publishers?

User avatar
Zer0xFF
Posts: 330
Joined: Fri Apr 20, 2018 9:18 am

Re: Having Trouble Understanding the Update Process

Post by Zer0xFF »

that API endpoint returns "edits", so the last_edit_id is the last update you've received for that api... (not edits can still represent addition/removal of a game)
the time param is obsolete since that can return all the updates that happened since let say 5 days ago when you made the last update, but using last_edit_id should be more accurate.
What about edits, additions, and removals for developers, genres, and publishers?
these can be deduced from game edits,example if you encounter developer id in a game that doesnt match any in you cache, then you can poll for the new info, but no way to find if one was removed or edited as of now.
for now you could expire your cache every month or so, as those are rather rare.
Regards
Zer0xFF

balderdashian
Posts: 8
Joined: Tue Aug 07, 2018 5:49 am

Re: Having Trouble Understanding the Update Process

Post by balderdashian »

Thanks for the clarification. That clears up almost everything. The only thing I'm still unclear on is the last_edit_id itself.

If I understand correctly, after my first update, I use the result's last_edit_id for the second update, etc. So the second update and on, that makes sense. But how do I do the first update? Let's say I download a cache of all games today looping through pages of Games/ByPlatformID. Then a week from now I want to run my first update. What do I put for last_edit_id?

User avatar
Zer0xFF
Posts: 330
Joined: Fri Apr 20, 2018 9:18 am

Re: Having Trouble Understanding the Update Process

Post by Zer0xFF »

you should probably call update api endpoint with time param right after you finish complete clone, with time in minutes to the time you started the cloning process (time=60 if the cloning took an hour), in doing so you will make sure you've the latest database, and also get an id to use for the next update request
Regards
Zer0xFF

balderdashian
Posts: 8
Joined: Tue Aug 07, 2018 5:49 am

Re: Having Trouble Understanding the Update Process

Post by balderdashian »

Ok thanks that makes sense.

balderdashian
Posts: 8
Joined: Tue Aug 07, 2018 5:49 am

Re: Having Trouble Understanding the Update Process

Post by balderdashian »

What do I put in for last_edit_id in my first update (which it says is required)? I tried this:

https://api.thegamesdb.net/Games/Update ... &time=3000

It seems to have ignored the time parameter and started at the beginning.

EDIT:
Nevermind it seems to work fine if I don't put in a last_edit_id parameter (even though it says it's required).

Post Reply