FM Database Update Project

In anticipation for FM26 I'm working on collecting data to add a large amount of clubs and teams to the game that I don't think will be in the base database for the main game.

At the moment I am focusing on adding college teams in the USA but I am open to suggestions of leagues or teams to add that are not in the vanilla game.

The US teams have proven to be relatively easy to add because of the amount of data available online.

The German lower leagues are also quite well documented online so I am planning on collecting data for the Regionalliga and below once I've finished updating the US.

Northern Cyprus and Greenland also have leagues that I will add due to neither being in the game.

Please leave suggestions below and I'll be able to work on collecting the data for the leagues or clubs you suggest.

In addition to this I'll be updating a large amount of the cities + regions of the game.

This will be my main project for the FM26 life cycle and I'll be updating my progress here.

I've discovered a new method to extract data from the FM editor so it will make updates much easier.

Hey can you give more information on how you're planning to go ahead with this?

It would probably be alot easier if multiple people are collaborating on it.

As you probably already know I already have extensive tools for doing this sort of thing in bulk e.g. generating every player from Greenland or Northern Cyprus based on data from TransferMarkt, SofaScore, FotMob etc

First I am going to make the teams themselves and after that I will generate the players if there is sufficient data available.

I've been collecting data for teams with python scripts then add that to a larger database which l'll eventually add in bulk to the editor and assign to their correct league.

I'm trying to make sure as much of it is accurate as possible.

Team data needed is:

• Name 
• Short name
• Abbreviation
• City
• Primary Colour
• Secondary Colour
• Year Founded
• Division
• Stadium 
• Stadium Coordinates
• Stadium Capacity 
• Head Coach
• Logo png
• Etc.

If you are able to share any advice for adding in bulk to the editor I'd appreciate it.

Players are relatively simple to add by name but what is time consuming is adding their detail which took me quite a while to achieve with my San Diego update so I will try to see if there are methods of making this quicker.

What do you mean by “their detail” i'm not sure which attributes you're referring to?

I'm not sure what advise there would be to add, it's just a case of writing the XML, of course there is a huge amount of work mapping all the property ID's.

This site has all of that, but unfortunately it's all in PHP scripts which are very tightly coupled to the site's CMS, so it's not really shareable the way a small Python package would be.

But I would very much suggest allowing collaboration, with forms that anyone can submit to, rather than a forum thread where you would still have to do a huge amount of manual work to do.

I want to set something that up myself, obviously we already have 20,000 new players added, but new clubs, stadiums and competitions are a little more tricky.

Either way we should definitely try and collaborate on this.

Are you also interested in women's football? I believe there will be a huge amount of work to do to add all the women's leagues as well

Details would be ethnicity, hair colour, skin tone, 2nd nationality, playing attributes, position, club history, etc.

Position would just be parsing from the string available on the scraped websites right?

Club history is simple so long as you match up the ID's i've already updated hundreds of thousands here: https://sortitoutsi.net/content/74126/2526-player-history-update-for-fm24 

2nd nationality should also not be a problem?

Ethnicity, hair colour and skin tone, yeah I have no idea how to go about those, I haven't found any reliable sources.

Could you share more about the problems you're having with these details? Is it just matching the ID's?

I didn't realise you could add player history with scripts 

I added them all by hand when I added players haha

Which is why I am dreading adding players again right away.

I've downloaded the details for 6000 NCAA D1 players already and was intending to add that by the FM Editor Generator once all the clubs are made.

On the college roster pages they have all the players details and even season stats but not their ages, only their college year.

Some of them have a transfermarkt page so I was trying to cross reference their name to transfermarkt to find that but my script didn't work how I wanted.

For now I'm just getting as much teams sorted as I can.

For college soccer I've got data for 1800 clubs, 23,000 high schools which I can add with the details I've got and assign them to their correct conference.

I also have their logos saved by name so when they are imported into the game I can assign their UID. 

I'm not the most tech savvy with coding so I'm not sure I'm using the best methods.

You have to be very careful with this, if you delete a player, the change the order in which they're loaded or combine with other files then the ID's will all change.

I would not suggest adding anything by hand other than the Advanced Rules 😂

Feel free to upload an example, it would help alot with understanding what you're trying to do.

that's true.

I'm not assigning any graphics yet for this reason.

At the moment I'm just collecting data for clubs. So the only examples I'd have to show would be a datasheet. 

I'll begin adding the teams once I've got a good amount of data for the teams.

My first batch of teams to add will be in the USA after which I'll be able to share and you can get an idea of what I'm doing.

Something that's quite time consuming by hand is adding history for club, league and players so if you could provide methods of adding those it would make a lot of difference.

That's exactly what I was suggesting, for you to share one of the data sheets, not the final editor file.

I assume you are aware you can use File > Export to export XML. You can just add one history entry and the see then structure of the XML and recreate those XML files for each row from your datasheet.

I'm not sure what other “methods” I can provide. Like I said I have scripts to do it in bulk but they're tightly coupled to the sites CMS. All I could do is create a form for you to upload a CSV (or JSON or anything structured) that then creates the FMF.

Happy to answer any questions you have, but it's hard to offer advice with the vague information you've given so far I don't really understand what the blockers are.

Methods of adding history in bulk.

Still working on this project.

I've collected the data to add all the NCAA and college teams then was able to import that into the FM editor.

At the moment I am working on a few kits for the NCAA D1.

I have a method to scrape the club roster pages so I can get the latest squads and staff.

So I have actually been working on this project still and it has become bigger than I expected.

I have been recently working on the city data in the existing database after extracting it all.

At the moment I'm creating regions for every nation then aligning all the cities to their corresponding regions and fixing any coordinates that are completely wrong in the editor (there's a lot).  also adding a lot of cities to the editor but I'm not sure if this will make the game slower so I can make a version without and without new cities.

New weathers, languages, stadiums, etc all added to match with newly added objects.

USA has so many clubs that don't exist in the editor who all all compete in different disconnected leagues so that was a bit of a headache to create all the teams but managed to get a lot of those.

New clubs will be made for

USA (Lower Leagues, Colleges, High Schools)

Greenland

Northern Cyprus

Falkland Islands

Guernsey

Isle of Man

Making clubs in languages I don't speak could be a bit challenging so if people want to help me collect the data to add to the database it would help.

Adding lower league English teams and German teams would probably be my next step but also finding accurate data could be tough.

I realised that to add history for clubs, competitions, players you need their long id which is easy to get once you have the conversion number so I should be able to add a lot histories.