apikey stopped working

Need help with your application? Ask here.
muldjord
Posts: 22
Joined: Sun Apr 22, 2018 2:32 pm

apikey stopped working

Post by muldjord »

I was implementeing the new API in Skyscraper yesterday using the API key you've provided a while back. Now today, it suddenly gives me "This route requires and API key and no API key was provided." even though I have my key in the call:

https://api.thegamesdb.net/Games/ByGameID?id=88&apikey=[my key]&fields=game_title,players,release_date,developer,publisher,genres,overview,rating,platform

I can't find anything about my key being deprecated anywhere. What's going on?

Leo_Pride
Posts: 630
Joined: Mon Apr 23, 2018 2:10 am

Re: apikey stopped working

Post by Leo_Pride »

Pinged Zero for an official response.
Unofficially, I know some debugging has been ongoing, but how (or even if) this is related is not yet clear.
We're curious to see how new projects use TGDB API.
If you have a new public project, please provide a link to it so we can highlight cool new applications! 8-)

User avatar
Zer0xFF
Posts: 330
Joined: Fri Apr 20, 2018 9:18 am

Re: apikey stopped working

Post by Zer0xFF »

Hi muldjord,

This key you where provided was a temporary beta one, please read the announcement for the new key.

Regards
Zer0xFF
Regards
Zer0xFF

muldjord
Posts: 22
Joined: Sun Apr 22, 2018 2:32 pm

Re: apikey stopped working

Post by muldjord »

I did, and I now use the public key, but it only has a 1000 request limit which seems extremely low to me for a monthly limit. 3000 with the beta key already seemed low as it was. To clarify, I KNOW this is on a IP-basis, but 1000 for a month is still almost unusable. Do I have the option to raise this with a Skyscraper specific key soon? Or do you then need me to mirror the site as you've proposed (which I can't).

Seems to me there's no middle ground here: Am I being unreasonable? What is going on with this? I am not asking to raise the limit to 50000 requests per month. But something like 10000 per-IP per month. I don't know anything about your server load, obviously, so that is of course your call. The next step from 1000 right now, as I understand it, is for me to run an entire server mirror myself. Which is just a huge time and probably money investment and I would bet for this option to not be used by anyone unless that is their direct interest. I am just a scraper developer, not a server administrator.

As I've stated repeatedly, I have no issue with limits, but 1000 a month? I guess that answers my question regarding the stance towards automated scrapers. I would not want my users to scrape thousands of roms a month. But 1000 is just... Almost unusable.

EDIT: Yes, I am getting a bit aggravated about this. I'm trying to figure out what it is you're trying to do, playing catch-up with the whole situation trying to implement this API. I'm spending loads of time on this, and I'm doing this having no idea if it will at all be usable in the end. Stuff breakds, the old site is down. Your information is spread out in several different forum posts instead of being collected in the API documentation. It's not optimal, is what I'm trying to say. And I keep asking myself why I even bother.

Leo_Pride
Posts: 630
Joined: Mon Apr 23, 2018 2:10 am

Re: apikey stopped working

Post by Leo_Pride »

You're not limited to 1000 entries per month, each call can pull 20 results.
That's 20,000 basic results, or 10,000 results with full images, per IP per month.
We're curious to see how new projects use TGDB API.
If you have a new public project, please provide a link to it so we can highlight cool new applications! 8-)

muldjord
Posts: 22
Joined: Sun Apr 22, 2018 2:32 pm

Re: apikey stopped working

Post by muldjord »

Yes, I am. My app is designed to search single games, it's not designed to pull in 20 games at a time. 1 request to search for the filename, and 1 to pull in the data for the actual game. I'm even trying to guess the artwork url's right now, in order to save 1 request per game. Otherwise I'd use 3 per game.

Leo_Pride
Posts: 630
Joined: Mon Apr 23, 2018 2:10 am

Re: apikey stopped working

Post by Leo_Pride »

So what you're telling us is you're not interested in users queuing their downloads like everyone else, or that you're not able to do so because... reasons?

Every unique IP associated with your key has a monthly limit of 1000 calls; if you're farming those through your app to mask IPs yet somehow a mirror is too costly, then no - that usage is not welcome.

As it stands, it sounds like you're the only one experiencing this issue with your app, so maybe you should be asking your users if they're having trouble, rather than personally hammering our servers to try to get a response?
We're curious to see how new projects use TGDB API.
If you have a new public project, please provide a link to it so we can highlight cool new applications! 8-)

muldjord
Posts: 22
Joined: Sun Apr 22, 2018 2:32 pm

Re: apikey stopped working

Post by muldjord »

"Interested?" "Like everyone else?" I am telling you I am not able to do so because of how Skyscraper is designed. I am not changing my core design because of one single scraping source. I support several sources and by the nature of scraping roms for EmulationStation and Attractmode, I take one file from my pool, search for it at the sources to get the best result, then fetch the information from that result. In order to make use of 20 calls per request I'd have to queue this up somehow, which can't be done, as I don't have the id's of the games until I have searched for them. Which is a request in itself.

Farming? What a hugely negative word to put in my mouth. I do NOT appreciate that. I am simply trying to convey to you what issues your new api is giving automated scrapers. I never asked for unlimited access. I never asked for much in the first place. If you don't want to provide this service, fine. It's your server, as I keep saying. Then just tell me if automated scrapers aren't welcome and I can stop spending so much time and aggravation on this.

muldjord
Posts: 22
Joined: Sun Apr 22, 2018 2:32 pm

Re: apikey stopped working

Post by muldjord »

I've decided to stop supporting your site. It will be a relief to put this behind me. I do wish you guys luck with this project, as I know how much work you've put into it. I hope for your sake that the "mirror is everything" approach works out. It is not an option for me, and with the limits it is unfeasible to use for anything more than a few game scrapings. I understand the problem with server load, I understand the necessity for solutions to that problem. So we can agree to disagree on that approach. I will remove the 'thegamesdb' support in the next Skyscraper release. Thank you for your time.

Leo_Pride
Posts: 630
Joined: Mon Apr 23, 2018 2:10 am

Re: apikey stopped working

Post by Leo_Pride »

Every response you've made has been veiled threats and needlessly antagonistic, when ZerO explained it neatly to you.

The key is limited by IP.
You, in testing, will not ever be able to scrape thousands of entries repeatedly, other than described previously - by queuing downloads or running a mirror.
It will not happen.
You've given no indication that your individual members need or want that kind of ability, only that you are unable to make use of your tool in that manner.

If your members are EACH scraping thousands of entries per month, something is very seriously wrong, and either you need to find out why they're doing this, or implement some sort of limit.
People thinking it's okay for a single IP to pull 50,000 entries per day EVERY DAY is a big reason why this limit was introduced in the first place, and as it sits now, anything that needs that kind of overhead should either be mirroring or using proper calls, which - despite your claims to the contrary - limit the entry responses to 20,000 basic entries, 10,000 with full images.

The "problem" you're having with the service is wholly manufactured on your end, and if you're going to use that to divide your users from our content, you're not welcome here.
We're curious to see how new projects use TGDB API.
If you have a new public project, please provide a link to it so we can highlight cool new applications! 8-)

Post Reply