My thoughts about the API, 2 days in.

Need help with your application? Ask here.
Post Reply
tanderson
Posts: 13
Joined: Wed Jul 10, 2019 2:43 pm

My thoughts about the API, 2 days in.

Post by tanderson » Sat Jul 13, 2019 3:44 am

I like this API better then the last, the move to JSON is a good change.

This API doesn't appear to be versioned... IE there is no version number in the URL. It looks like the plan is to replace this API with the next new API version. This will break all existing applications. This will scare developers away, its best to leave at least a few prior API versions running for compatibility then age them out slowly. I have rewriten my code for thegamesdb api 3 times now when the API has evoloved, lucky for me I have written a wrapper so i just need to edit the wrapper but still... painful.

I am able to get every platforms info along with all of the platform images in only 14 API calls. That being said I had trouble figuring out the URL for playform icons. The other APIs for images contain a list of BaseURLs however the platform list API does not. I was able to find the url for the one size by looking directly at the main site and looking at the folder it was using but i feel there should be more sizes... maybe there are.

When using /Games/ByGameID i can only include boxart and platform, but platform doesn't include as much information as calling the platform API call does. Ideally i would also like to be able to include genres, developers, and publishers so i dont have to call the API again to get that information.

originally i though 6000 requests would not be enough for a month but it is more then enough to scrape the entire DB if you do it efficiently using the API calls. Then cache that data on your own server for future calls.

The update API is the one i had the most trouble with.
  • Games can have update API but platforms/Genres/developers, etc cant? What if a new platform is added, where are the updates for that? I think the update api would be better off as a generic API with another variable that indicates if its a publisher, genre, playform, game, etc.
  • When items are in the list its not clear if they are being added, removed or edited. I think another field in the API for the type of update would be ideal
  • The best way i can find to update is to get the update list, create a list of GameIDs, deduplicate the list then pull all of those entries completely, then delete my cached data and resync it.
  • the Update API needs the time in seconds but the time/date format for the rest of the API's is YYYY-MM-DD, i found this inconstancy a little odd.
But so far so good!

User avatar
Zer0xFF
Posts: 253
Joined: Fri Apr 20, 2018 9:18 am

Re: My thoughts about the API, 2 days in.

Post by Zer0xFF » Sat Jul 13, 2019 11:06 am

tanderson wrote:
Sat Jul 13, 2019 3:44 am
This API doesn't appear to be versioned... IE there is no version number in the URL.
no, but it will be.
this thought dawned upon me shortly after release, by then, moving it to let say `/v1/` would have caused an API (url) break, i redirect MIGHT work, but most who use the api dont factor that into their code, and frankly its not worth the hassle so it remained. so that will have to remain as base url, until and if its ever removed.

we've already revised a call https://api.thegamesdb.net/v1.1/Games/B ... e=platform since `include`->`platform`->`data` (data tag was skipped in the v1 version) https://api.thegamesdb.net/Games/ByGame ... e=platform. so the versioning issue is mostly cosmetic really.
tanderson wrote:
Sat Jul 13, 2019 3:44 am
That being said I had trouble figuring out the URL for playform icons.
this API needs some work, there are 2 different icon at 2 different location, that doesnt really fit into image cdn directory path pattern we currently use and also different image set associated with it.
tanderson wrote:
Sat Jul 13, 2019 3:44 am
When using /Games/ByGameID i can only include boxart and platform, but platform doesn't include as much information as calling the platform API call does. Ideally i would also like to be able to include genres, developers, and publishers so i dont have to call the API again to get that information.
you can include all of those, ofc you'd only get IDs associated with them, and all of that is by design, and you should have noticed all those API endpoints (genres, developers, and publishers) aren't paginated, but would actually return all the data associated with them, that was done so you can cache the results.
tanderson wrote:
Sat Jul 13, 2019 3:44 am
Games can have update API but platforms/Genres/developers, etc cant? What if a new platform is added, where are the updates for that? I think the update api would be better off as a generic API with another variable that indicates if its a publisher, genre, playform, game, etc.
while somewhat of a problem, these updates can be implied. aka if you see a genre, publisher, dev, platform that is not in your database, then you can imply an update, at which you can request a complete list, or just check for that missing id. this also can't be addressed without a complete revision of update API.
tanderson wrote:
Sat Jul 13, 2019 3:44 am
When items are in the list its not clear if they are being added, removed or edited. I think another field in the API for the type of update would be ideal
if a game was added, the 1st instance of that ID would always contain the tag `[NEW]`, the same goes for deleted `[REMOVED]`, an edit is then implied for everything else, also if you think about it, the minute you see `[NEW]` you can create an entry for that game and have it blank, then treat all subsequent calls as edits, which will edit and populate that entry, and if there wasn't a `[NEW]` tage, that logic behind your update would work exactly the same.
tanderson wrote:
Sat Jul 13, 2019 3:44 am
The best way i can find to update is to get the update list, create a list of GameIDs, deduplicate the list then pull all of those entries completely, then delete my cached data and resync it.
not very efficient and will just eat away at your limit, you should stick to just the update api.
tanderson wrote:
Sat Jul 13, 2019 3:44 am
the Update API needs the time in seconds but the time/date format for the rest of the API's is YYYY-MM-DD, i found this inconstancy a little odd.
as i mentioned in my other posts, you should avoid using time, that remained mainly as legacy, and would likely to be removed in an api revision.

To summaries, We are aware of most of the limitations and short comings you mentioned, however, I work on this project in my free time, and unfortunately time isn't free and it will take a while before I can addressing those issues.
Regards
Zer0xFF

tanderson
Posts: 13
Joined: Wed Jul 10, 2019 2:43 pm

Re: My thoughts about the API, 2 days in.

Post by tanderson » Sat Jul 13, 2019 4:13 pm

thanks for the reply.

the API is great and your definitely headed in the right direction! I have worked on projects like this before and feedback from people that were new to the documentation/service always helped

I will rewrite my update module to use the NEW and REMOVED tag, I had not noticed that logic before but now that you describe it to me it makes sense.

Thanks for spending your free time on this project!

User avatar
Zer0xFF
Posts: 253
Joined: Fri Apr 20, 2018 9:18 am

Re: My thoughts about the API, 2 days in.

Post by Zer0xFF » Mon Jul 15, 2019 11:42 am

feedback is always welcome and thanks for taking the time to post them.
Regards
Zer0xFF

Post Reply