Remaining monthly allowance

Need help with your application? Ask here.
Snogs
Posts: 12
Joined: Fri Jun 29, 2018 9:06 am

Remaining monthly allowance

Post by Snogs » Wed Jul 04, 2018 2:54 pm

So, it appears that the API has a monthly allowance for queries set to 1000...

"remaining_monthly_allowance":981

Isn't that a bit low? Depending on the application, that can be consumed in less than 1 day.

Just to get the genre list and platform list is 2 queries. Granted that doesn't need to be called every time they are needed, but it's still 2 queries deducted every time they are called.

User avatar
Zer0xFF
Posts: 234
Joined: Fri Apr 20, 2018 9:18 am

Re: Remaining monthly allowance

Post by Zer0xFF » Thu Jul 05, 2018 12:45 am

We're asking dev to be considerate of their usage, the previous api was massively abused that neither the main site nor the api was in a usable state due to very occasional timeouts and incredibly long response times. the limit will be reviewed again in the future based on the usage and performance on the new api.
Regards
Zer0xFF

Snogs
Posts: 12
Joined: Fri Jun 29, 2018 9:06 am

Re: Remaining monthly allowance

Post by Snogs » Thu Jul 05, 2018 1:54 pm

Zer0xFF wrote:
Thu Jul 05, 2018 12:45 am
We're asking dev to be considerate of their usage, the previous api was massively abused that neither the main site nor the api was in a usable state due to very occasional timeouts and incredibly long response times. the limit will be reviewed again in the future based on the usage and performance on the new api.
I'm all for rate limiting. Usually rate limits are per second, not per month.

With a limit of 1000 calls per month, you're limiting people to a little over 16 calls per day (30 days, 2 queries per game, 1 for main info, 1 for screenshots/fanart).

IMHO that is extremely low.

Leo_Pride
Posts: 269
Joined: Mon Apr 23, 2018 2:10 am

Re: Remaining monthly allowance

Post by Leo_Pride » Thu Jul 05, 2018 9:24 pm

160 calls per day by that logic, each request can process 20 calls.
We're curious to see how new projects use TGDB API.
If you have a new public project, please provide a link to it so we can highlight cool new applications! 8-)

Snogs
Posts: 12
Joined: Fri Jun 29, 2018 9:06 am

Re: Remaining monthly allowance

Post by Snogs » Thu Jul 05, 2018 9:42 pm

Leo_Pride wrote:
Thu Jul 05, 2018 9:24 pm
160 calls per day by that logic, each request can process 20 calls.
Unfortunately my application only processes one request at a time. It's not something where multiple games can be retrieved at once.

It's based on the user inputing the TGDB link, it extracts the game ID from the link and processes it for other uses.

Leo_Pride
Posts: 269
Joined: Mon Apr 23, 2018 2:10 am

Re: Remaining monthly allowance

Post by Leo_Pride » Thu Jul 05, 2018 9:47 pm

If you're not using arrays because of the app structure, that's something we can consider for the per-key limits. Right now we're mostly shaking out the bugs and globally curtailing excessive use, but in your unique case, they do sound a bit low.
We're curious to see how new projects use TGDB API.
If you have a new public project, please provide a link to it so we can highlight cool new applications! 8-)

Snogs
Posts: 12
Joined: Fri Jun 29, 2018 9:06 am

Re: Remaining monthly allowance

Post by Snogs » Thu Jul 05, 2018 9:59 pm

Unfortunately I couldn't use the other type of key.

The application gets distributed to other web sites, and I couldn't include my key if it were the other type of key.

The application builds discussion threads about individual games in forums.

I honestly don't see a site going over the limit, but the possibility exists. There are some active sites out there.

Leo_Pride
Posts: 269
Joined: Mon Apr 23, 2018 2:10 am

Re: Remaining monthly allowance

Post by Leo_Pride » Thu Jul 05, 2018 10:11 pm

Ideally speaking, the app should draw from its own mirror, with API calls limited to syncing actually new data. That way, single-use "full syncs" could be used to bring the apps current, then API calls could keep them current, without forcing end-users to sync the entire site on every new installation. That's not even counting the scrapers that just parsed the entire site, resulting in massive performance hits.

The other benefit to having a dedicated mirror is that at no time is the app key out of your control, the app simply syncs to your mirror.
We're curious to see how new projects use TGDB API.
If you have a new public project, please provide a link to it so we can highlight cool new applications! 8-)

Snogs
Posts: 12
Joined: Fri Jun 29, 2018 9:06 am

Re: Remaining monthly allowance

Post by Snogs » Thu Jul 05, 2018 10:31 pm

Leo_Pride wrote:
Thu Jul 05, 2018 10:11 pm
Ideally speaking, the app should draw from its own mirror, with API calls limited to syncing actually new data. That way, single-use "full syncs" could be used to bring the apps current, then API calls could keep them current, without forcing end-users to sync the entire site on every new installation. That's not even counting the scrapers that just parsed the entire site, resulting in massive performance hits.

The other benefit to having a dedicated mirror is that at no time is the app key out of your control, the app simply syncs to your mirror.
That would just make the limits worse if I were to hold the info for hundreds of sites. The individual sites grab the data once from here and don't call the site again unless there's info missing, then a cron task checks for missing info once per day using the updates call.

Example done with old API...
https://yaketys.com/threads/diablo-iii- ... ouls.7755/

Leo_Pride
Posts: 269
Joined: Mon Apr 23, 2018 2:10 am

Re: Remaining monthly allowance

Post by Leo_Pride » Thu Jul 05, 2018 10:44 pm

That's how top-down distribution works.
You mirror us, we limit you, your users mirror you, you limit your users.
You don't let your users hammer the site because you can't or won't maintain a proper mirror.

This whole idea that a centralised database is a permanent thing with infinite resources is fundamentally flawed, and the time where we let app design issues override the needs of well-behaved clients is over.

Remove the plank from your own eye before complaining about the speck in ours.
We're curious to see how new projects use TGDB API.
If you have a new public project, please provide a link to it so we can highlight cool new applications! 8-)

Post Reply