Someone's deleted over 1000 games recently

  • They seem to mostly be limited and collector’s editions, but as far as I know, those are fully supported. What’s going on? These items will inevitably be recreated by someone

  • moderators

    That was me, and official word from smidley is that those aren’t supposed to be included as they’re functionally identical to other titles (apparently also extending to unlicensed combo paks of licensed games).

  • That makes it really difficult to use the data for tracking purposes. If there were controls to prevent people from recreating the data and ways to create edition info, regioning info, alternative covers, etc, I’d be all on board because it could be reliably modeled.

    But as it stands, it’s just a long battle. You’ll see there’s already collector’s editions being re-added.

  • Unlicensed combo packs I agree shouldnt be in there.

  • I just don’t think it was wise to make a purge of that magnitude without having anything in place to address the reasons those entries were being added in the first place. All of that information, (what comes with those, whether it be physical items, additional dlc content, etc…) is now irrevocably lost. As is accessing it in a programmatic fashion.

  • moderators

    Oh, I’m well aware they’re being re-added, which is why I agreed to track Recent Additions in the first place - I knew there’d be some tough enforcement decisions to minimize the impact of duplicate database entries. That said, I did provide “ALSO AVAILABLE” suffixes for my earlier moderation attempts such as Call of Duty: Black Ops III, but I believe that this is something any user can implement, and making any one user personally responsible for merging the contents of over 1000 entries isn’t sustainable.

    At this point, I’m really just waiting on direct access to the www directory, so I can at least try to fix the majority of pain points. Expanding “Other Versions” with an “Alternate Editions” field might be a nice addition.

  • I was wondering if it would be better if we wrote game insertion rules in the database (maybe we should put this on the wiki or any other place of easy access) . In this way, I believe we can prevent these unnecessary re-insertions in the database.

    Maybe we could do the same with their names like this naming convention example on Convention

    What do you think?

  • My point was that merging them is not even viable given that there are no ways to mark the data in the first place. You can’t select a check box and say “has collectors edition”, “available in japan and here is its cover”, “this overview material only applies to game of the year addition”.

    That means at best merging would just create a larger overview entry, but no analysis such as “this is the release date of the greatest hits” or anything programmatic whatsoever. Merging at this point would just mean lost data, from tons of community driven manwork.

    And that’s what’s happened. Tons of valid info put in by volunteers that may never return is lost forever now, with nothing in place to replace it.

    If the data was managed in such a way that this information could be input on single records, then the community would gladly re-fill it in. Instead, a large portion of the community will now feel scorned and feel it is probably pointless to enter in data.

    And apps like my own (gameye) now have to make some serious considerations about how I’m going to have to handle the data. I will no longer be able to rely on thegamesd as being authoritative and will have to augment the data myself. It’s gonna blow.

  • @filipe_figueredo it would be a huge improvement if the duplicate checker amd search functions checked the alternate titles. And also if the broken search functions (if its all numbers or has punctuation, search fails) were fixed.

    Those two issues are why most duplicates are created in the first place (ie, fzero vs f zero vs f-zero)

  • moderators

    I’m not a huge fan of sitting around with my thumb up my arse when I could and should be implementing fixes to ease the blow of these removals, and maybe I could’ve held back some, but my directive is to not allow collector’s editions as their own entries. Having fourteen international versions of Pokémon mixed in with various hacks - none of which are presently allowed - wasn’t really helping.

    That said, I’m not totally against having some sort of hack/mod section(s), either.

    If it helps any, I’ve been working on international functionality for the Big Three (US, Japan, Eurozone) as a start for the past month. I need to get back on that, but having access to the www directory would improve my confidence that I’ll be able to rollout the updates.

  • That’s great that you’re working on region support, and personally I didn’t like seeing 15 pokemons so I am in complete agreeance there. But I do feel that without COMPLETELY rewriting the current model, that there will be no way to support multiple SKUs. As mentioned before, there is a big difference between original release, GOTY edition, collectors edition, limited edition, etc… They can have different physical content, different levels, different release dates, different overviews, etc…

    A simple, “what regions does this support” and “is it a hack” does not address the editions issue. Even the regioning is severely complex, and if not modeled right will just produce headaches. Like NES pal-a vs pal-b and GB having completely different games coming out in AUS vs EUR vs GER, different release dates for different regions, etc…

    Heck, I’m not even sure that adding a spot for multiple SKU (i.e., collector’s edition vs regular vs limited etc…) within the same game ID makes sense. Then anyone trying to cross reference additional information such as barcodes, prices, etc… has a much worse problem to model because you can’t simply cross-reference via ID. It will be cross-reference ID + SKU number etc…

    That’s why the only that’s working is multiple gameIDs for separate SKUs. Then those SKUs should have region flags, etc…

    As it stands, from what I’m understanding, it will be a severely long time before tgdb can appropriately handle this information again.

    And I agree with “not sitting around with my thumb up my arse” but it seems like a lot of wasted effort when simple fixes like fixing search is a much bigger deal. Or even removing the 1000 hard limit on updates info. That’s just one line of code for christ sake! Heck, if the db was updated to handle the information well, the community would flag and delete these doubles FOR you. It just seems like wasted and unnecessary effort, AND with a negative cost to users.

  • Or fixing the thumbnail bug (thumbnail is never updated after it is generated once. So when someone fixes a cover, they have to download the large image and thumbize it themselves). Another darn near one-liner. The tickets for these have been up for years, along with solutions, pull requests, etc…

  • Locking down platforms (or at least putting them behind mod approval for changes/additions/updates) is another big one. Like the NES listing has been complete for years, but gets violated everyday with home-brew. Locking it would (or mod approval-ing it) would literally end that.

  • I’m sorry if I’m going on and on this is just very frustrating and I had to go into emergency recovery mode to back cycle all the lost data (which is also no longer updatable) or risk having a gimped, barcode less, SKU less and non-price matched generic listing for every game.

    I’m having to hold a meeting this weekend with some friends to see what my options are to move to tgdb being a source and no longer being an authority. i.e., I’m about to have to recreate the data layer entirely and move to where my dataset is the authority. The paradigm today is for developers to be able to source data somewhere else so they can focus on the product, but now I’m being forced to consider becoming the data source.

  • I understand the issue, however, as a collector with 1900 ish different games in my collection, I think this is ridiculous. Collector’s and Limited Edition titles are valued differently without doubt thus justifying the existence of a separate entry. As stated above mass deleting the content without some backup plan in place is basically a smack in the face to everyone who has taken their time to contribute to the database. I personally have entered a large amount of variants as I have them in my collection. This just killed the reliability and usefulness of this database IMO at least.

  • moderators

    I don’t disagree, and I’ve been asking questions of the administration along the way to try to provide a solution for what’s an ongoing problem.

    My personal idea is that - at least initially - the Big Three could be represented by region pages (example: Castlevania X could do something like 1553?region=usa/jpn/eur (with options for additional ROW regions), which would pull Dracula X / Akumajou Dracula XX / Vampire’s Kiss as appropriate). Implementing that and fixing aliases would help a lot with preventing duplication.

    Additionally, I’m seeing Alternate Editions as sort of additional content for an overarching title entry, hence why I’d like them to appear under Other Versions. Given the proper implementation, at least theoretically, infinitely many editions could be added, each with their own Keyword (suggesting but not requiring defaults of GOTY, Collector, Limited) and Bonus Content fields to cut down on needless redundancy. In other words, if Castlevania X had Collector’s Editions, something like 1553?edition=GOTY/Collector/Limited could be used to pull additional details as a functionally unique SKU.

    Beyond that, again, having my hands tied is my biggest problem, as technically a few fixes (including the cover art issue) are already greenlit. As for the 1000 limit, I can honestly say this is the first I’m hearing of this, but I’m interested in knowing more.


    Note i thought this was an absolutely critical bug so i was talking fire and brimstone lol. It’s also got an arbitrary 30 day limit

  • I dont disagree that you could add additional skus to an overarching title but since there currently isnt, in the meantime there is a large gap of info and it cant be recovered. Everyone could’ve pitched in on a merge once that was available.

    But if youre going to go that approach, youll probably also want every other piece of metadata except pub/dev to be able to be filled out independently for those skus (release date, overview, etc). Which basically at that point pushes down all that metadata under a new layer anyway. Which means to get at any piece of info youll still need two keys, the gameid and the sku id. Which just begs the question if since only 1.5 percent or so of the games are affected this way, why not just have seperate ids altogether?

  • moderators

    That’s just it, technically they’re not unique titles.

    That said, I’m considering a variety of ways to handle this, and it may just be a presentation issue that ultimately dovetails into unique Game IDs presented differently between the site and API (there’s no reason I can think of why additional Game IDs couldn’t be used in lieu of keywords, but having them top-level on the site is really neither sustainable nor elegant). Unfortunately, for that to happen, the Editions would ultimately have to be re-entered anyways. I could’ve left the Band-Aid on, but that only hides a deeper wound that needs a more concentrated effort to heal. Perhaps I could move with less haste after a revised site is implemented, though; I just hate wasting time doing effectively nothing of consequence.

  • I agree, there needs to be a way to keep track of collection editions. There should exists reasons for multiple listings of the same title, alternate copies and etc. I use gameye to track my collection and notetate them on the notes portion of the listing. I don’t want to scroll through duplicate listings either but deleting them without installing a specific method to catalog them isint the answer in my opinion. They are being added because people are looking to track them. I don’t own many collectors edition so it doesn’t impact me that much. just feel like I needed to voice my opinion on the matter, as I did loose a title that I listed.

    If the main games page had a way to choose alternate cover variations and pieces I could see that as a good step in the right direction.

  • moderators

    Yeah, I’m scaling back enforcement on Editions until I can implement a better solution; hacks and direct-to-ROM will still be nuked on sight. Not having an alternative, I tried to get my point across about Editions being largely meaningless clutter, but perhaps that deserves further consideration.

Log in to reply