Most effecient query to mirror DB, and then update?

Need help with your application? Ask here.
Post Reply
gamesibeat
Posts: 12
Joined: Sun Jul 22, 2018 7:50 pm

Most effecient query to mirror DB, and then update?

Post by gamesibeat » Sun Nov 04, 2018 10:01 pm

Hello Everyone,

I am looking for an efficient way to simply mirror the database and then do small updates once a week. I am unsure how to do it with the current setup. Anyone have a good solution they are willing to share?

Thanks,
gamesibeat

gamesibeat
Posts: 12
Joined: Sun Jul 22, 2018 7:50 pm

Re: Most effecient query to mirror DB, and then update?

Post by gamesibeat » Sun Nov 04, 2018 10:08 pm

Actually looking at the DB it appears that the games still have the same ID's from before. So I no longer need to mirror the database. I just need an effective way to add the recently added games past a certain ID.

User avatar
Zer0xFF
Posts: 238
Joined: Fri Apr 20, 2018 9:18 am

Re: Most effecient query to mirror DB, and then update?

Post by Zer0xFF » Mon Nov 05, 2018 12:35 am

Hi there,
while the IDs haven't changed, the data structure has changed in such a way that it would warrant a full remirror.

should you choose to do so, you should query for platforms, then games by platform and loop through all platforms and all pages.
then after you're done, you can query all images based on ID you've collected.
please check and double check your code and query, as your private key won't be reset.

With that all said, if you're ok with a hybrid database as you slow populate your DB with new entries look at the update api endpoint and use last_key_id=0, which should give you the entire DB history since the new site went up


Regards
Zer0xFF
Regards
Zer0xFF

gamesibeat
Posts: 12
Joined: Sun Jul 22, 2018 7:50 pm

Re: Most effecient query to mirror DB, and then update?

Post by gamesibeat » Mon Nov 05, 2018 1:02 am

Okay, I can test it out and then practice with a .json file. For the /Games/ByGameID what is the maximum game id's you can query?

User avatar
Zer0xFF
Posts: 238
Joined: Fri Apr 20, 2018 9:18 am

Re: Most effecient query to mirror DB, and then update?

Post by Zer0xFF » Mon Nov 05, 2018 10:23 am

gamesibeat wrote:
Mon Nov 05, 2018 1:02 am
Okay, I can test it out and then practice with a .json file. For the /Games/ByGameID what is the maximum game id's you can query?
I'd recommend using `/Games/GamesByPlatformID` instead, and iterate through pages.

but to answer your question, as much as URL query limit would allow (google suggest 512 characters, so roughly 100 ids), however, each page would only return 20 results per page.
Regards
Zer0xFF

gforceca
Posts: 4
Joined: Tue Jan 15, 2019 4:42 pm

Re: Most effecient query to mirror DB, and then update?

Post by gforceca » Fri Feb 01, 2019 3:10 pm

That's what I did and couldn't mirror all games. Ended up with close to 30k games, wich is logic because of the 1500 queries limit * 20 results.

Leo_Pride
Posts: 309
Joined: Mon Apr 23, 2018 2:10 am

Re: Most effecient query to mirror DB, and then update?

Post by Leo_Pride » Sun Feb 03, 2019 7:24 am

This is where you're supposed to use your private key.
The public key is for testing and end-user usage, and has a significantly lower limit.
We're curious to see how new projects use TGDB API.
If you have a new public project, please provide a link to it so we can highlight cool new applications! 8-)

Post Reply