Scripting Runtime Information

[WildW]

As many of us are now aware, the scripting runtime used in Civilization VII is based on JavaScript. (More specifically, Google's V8)

There appears to be at least two V8 isolates (execution environments), Tuner and App UI. Though, I suspect there may be one or two more for gameplay scripts and map scripts. When scripts are loaded, they are loaded into their own V8 context (execution scope) within their respective isolate, keeping their global namespaces separate from another. There may also be a mod action that allows you to load multiple scripts into a single context, thus sharing the same global namespace.

Civilization VII uses the Coherent UI library, you can find its website here: https://coherent-labs.com/Documentation/cpp/index.html. Coherent also has a remote debugger which you can attach to Civilization VII and details on how to use it are in the Coherent documentation.

The ways I know of doing communication between scripts so far is by doing one of the following:

• Commands and operations event drivers e.g. Game.PlayerOperations.sendRequest(GameContext.localPlayerID, PlayerOperationTypes.EXECUTE_SCRIPT, args)
• Object properties. e.g. Game.setProperty("some_property", someValue) and Game.getProperty("some_property")
• Coherent UI event driver e.g. engine.trigger("some-event", param1, param2, ..., paramN)
• Persistent data e.g. engine.createJSModel('g_SomeGlobal', someValue) (accessed directly via global variables in other contexts of the same isolate)

No unsafe access through something like ExposedMembers as far as I can tell.

Looking through the files, you have probably also noticed some TypeScript files, typically these aren't run directly, but rather are compiled into JavaScript ahead of time, which appears to be how they are handled here, too. The content of these files should never actually be read by the game, and only seem to be included in the game's install for modders to reference and maybe down the line, use for type information in their own mods using TypeScript; when replacing base-game scripts, you do not need to replace any TypeScript code, only JavaScript.

The purpose of TypeScript is to help developers spot bugs and write code more confidently using duck-typed annotations that integrate with code editors (through Intellisense, linting, etc). A promising implication of its integration into their scripting codebase is that native functions exposed to the scripting environment, such as Units.get or Game.setProperty, are likely documented in a definitions file that we haven't been given access to yet, rather than requiring us to document them manually as we did in the past. I hope we will receive access to these definitions with the release of the modding SDK.

Until the modding SDK comes out, I would suggest that you do not write mods in TypeScript, it may be tempting but it will be a lot more trouble than it's worth, given that we don't yet have the type information for any new native functions.

Attached is a script that you can run to dump up to more or less an entire JavaScript context, and the output I got from running it through Firetuner in the Tuner and App UI contexts. You can use this information to help you write mods in JavaScript, until we have a better solution for documentation. UPDATE: an improved version of the script is now attached as well, but I haven't got the time yet to run it on the game, be aware, it may take a while if called on global this.

To get information on an arbitrary object, you can invoke the dump function with reference to the object and call JSON.stringify on the result. e.g. JSON.stringify(dump(Players.get(GameContext.localPlayerID)), refReplacer()) to dump information on the Player class. If you are in a UI context you can also use the UI.setClipboardText function with the dumped output as argument to copy it directly to your clipboard. Otherwise just minify the code and run it through firetuner to have the return get outputted in the console window, or separate the response into several 100 character segments and console.log it.

UPDATE: Attached are lists of the many internal events which can be accessed in scripts. Reporting events are not yet well understood, but the other kinds of events can be subscribed to with engine.on. e.g. engine.on("NaturalWonderRevealed", onNaturalWonderRevealed).

I will update this post with new information as it becomes available and discovered.

 

[Gedemon]

great work, thanks !

 

[WildW]

Update 1

• Added lists of the internal events you can subscribe to and invoke
• Updated scrapeGlobalThis.js to be a lot more thorough

 

[Craimasjien]

@WildW thanks for your amazing contribution, I've added your findings to a personal reference so I can easily find this in the source for my mod. Hopefully with your permission I pushed it to my repository as well.

 

[adamk33n3r]

How do you run this scraper? Tried different methods of calling `stringifyAllJSON` but getting different errors, looking like `this` was undefined.

Will this let me see all functions we have access to? Like `GameplayMap.getResourceType`? etc? Looking at your `App UI.json` the `GameplayMap` dump was empty, but perhaps the v2 would work I just don't know how to run it correctly.

 

[EmperorFool]

I'm coming at this as a TypeScript/JavaScript developer having read maybe 100 lines of the UI JS code looking for how to solve the permanent red religion icon or why the chain icon sometimes disappears from conquered settlements after too many hours playing, so I'm not a Civ7 modder yet.

• JavaScript is the code Google's V8 engine executes. As JS is an interpreted language, the engine reads the code, parses it, and builds the executable bytecode that gets run post-load.
• TypeScript is JavaScript with type declarations and annotations to assist development and promote type safety--think "Don't give me a city name when I ask for the science produced in it." This provides autocomplete for function parameters and return values, which helps when exploring the API and determining the available methods.
• The TypeScript is transpiled to JavaScript by Firaxis in their build process, essentially removing the type declarations that the V8 engine ignores. This is why you should extend the JavaScxipt files. The TypeScript files are included so modders can know what's what. Ideally, we'd be given the script to build the JS from the (modified) TS, but that would make monkey-patching extremely difficult.
• Monkey-patching is when you replace one or more class functions (technically object prototype properties) with your own, patching or extending the original class/object/monkey. This is done to the final JS code, and doing so before transpilation would be challenging to make work with multiple mods.
• JSON is a data format that is typically used to serialize JavaScript data. It will not work with functions or classes. "Stringifying" is the process of turning an object into JSON data, so an object with foo and bar properties becomes {"foo":"hello","bar":"world"}. This is used to store game data to a save file or send over the wire to be turned back into JS objects on the other end. Think of it as CSV (comma-separated values) that can be directly interpreted as a JS value once parsed.

I hope these high-level definitions help. I've already seen some very detailed guides on monkey-patching (look for code referencing a prototype), so you're all in good hands.

 

[MantisMaestro]

As for adamk33n3r above, I must be missing something obvious, but I can't get the script to run for the life of me. If you have time a few steps to reproduce your output would be much appreciated

 

[MantisMaestro]

nvm, found the discussion on the modding discord, for others refrence: https://discord.com/channels/1328178941592080425/1332182099993821304/1341212411763494982a very simple way to use it is to use a JS minifier to get a one-line version of the whole file, paste it into firetuner's console, run it, and then access the defined functions on each subsequent use of the console

a very simple way to use it is to use a JS minifier to get a one-line version of the whole file, paste it into firetuner's console, run it, and then access the defined functions on each subsequent use of the console
then you can type JSON.stringify(dump(Cities.get(someCityId)), refReplacer()) into firetuner and it will output the dump of the city object
copying the output is inconsistent though, so select it, then spam CTRL+C and right click a bunch of times until you get it, then you can paste it into your code editor and use an auto formatter to make it look pretty
You can't console.log the value because it truncates messages, though there might be another logging function that doesn't do that

 

[WildW]

How do you run this scraper? Tried different methods of calling `stringifyAllJSON` but getting different errors, looking like `this` was undefined.

Will this let me see all functions we have access to? Like `GameplayMap.getResourceType`? etc? Looking at your `App UI.json` the `GameplayMap` dump was empty, but perhaps the v2 would work I just don't know how to run it correctly.

I'm planning on doing something perhaps with a firetuner panel to make use of the script more straightforwards, but you can reference the message above for now.

Attached is the dump of GameplayMap in App UI using v2.

 

[leonardify]

I'm planning on doing something perhaps with a firetuner panel to make use of the script more straightforwards, but you can reference the message above for now.

Attached is the dump of GameplayMap in App UI using v2.

Awesome; how did you export the json from the game?
Right now I've been able to dump some basic info in the game console, but the output is truncated

PS: I'm trying to connect to the remote debbuger, but I didn't find a way. In-game console shows the address to be localhost:9444, which is the same used by Coherent UI even for Cities Skylines 2, so seems to be the correct one. Anyway I can't find a way to connect through Chrome Inspector

 

[WildW]

Awesome; how did you export the json from the game?
Right now I've been able to dump some basic info in the game console, but the output is truncated

PS: I'm trying to connect to the remote debbuger, but I didn't find a way. In-game console shows the address to be localhost:9444, which is the same used by Coherent UI even for Cities Skylines 2, so seems to be the correct one. Anyway I can't find a way to connect through Chrome Inspector

Using civ 6's Firetuner, the evaluation of any expression you write into the REPL is outputted into the tuner console. I do this rather than using console.log. However, copying values from the console window in firetuner is a bit inconsistent, so you may have to CTRL+C and right click your selection a bunch of times (I'm not sure why this is the case).

And in regards to the Coherent Debugger, I haven't tried using it myself yet, but make sure you have it enabled in AppOptions.txt. You can find it in your local Civilization VII appdata. (C:\Users\<USER>\AppData\Local\Firaxis Games\Sid Meier's Civilization VII)

 

[leonardify]

Got it, need to fiddle more with Firetuner, thanks. About the debugger yes, I set `UIDebugger 1` in the AppOptions but unfortunately it doesn't seem enough.
I tried enabling a bunch of other options but nothing worked.

I think it could be related to some Command Line arguments maybe, but it's possible it's just disabled at the moment

 

[WildW]

Got it, need to fiddle more with Firetuner, thanks. About the debugger yes, I set `UIDebugger 1` in the AppOptions but unfortunately it doesn't seem enough.
I tried enabling a bunch of other options but nothing worked.

I think it could be related to some Command Line arguments maybe, but it's possible it's just disabled at the moment

I doubt this is the issue you're experiencing but some people didn't realize they had to remove the semicolon at the start of the line to uncomment it when people were just figuring out how to enable generating the debug databases within AppOptions.txt, so I figure I should mention that.

 

[leonardify]

Sure it's an important thing to note, hope it helps someone reading the topic; unfortunately it is correctly uncommented right now.
I'll try with -UIDebugger 1 from Steam command line arguments option, I'll post here my findings

 

[IanMoon33]

Game.PlayerOperations.sendRequest(GameContext.localPlayerID, PlayerOperationTypes.EXECUTE_SCRIPT, args) Where does this snippet come from? I understand the game uses sendRequest (and the precheck canStart()) requests to talk to the game engine and modify the various databases. I can't seem to find this function ever referenced in the scripts. The closest I've come is UNITOPERATION_EXECUTE_SCRIPT, which is a unitoperation type. I would like to be able to call a script on an event from a js file and edit a database value but I don't know how to structure my args object. I could change it to use Game.UnitOperations.sendRequest(myUnit.ID, "UNITOPERATION_EXECUTE_SCRIPT", *How do I setup args? What are the object properties?!*) args = {}; args.????? = ????? (How to call an xml database modifier, eg. MyMod.xml?)

 

[LilBudyWizer]

Chrispresso's Debug Console is a mod that gives you access to all this. It has autocomplete so you can explore and experiment.

Update: Not quite exactly, or even close to exactly. Apparently you can't JSON.stringify a function. This does a more extensive context while the debug console either picked which contexts or which objects within contexts to, presumably, what interested the developer. Between the two this seems to best give the values to use in function calls while the debug console gives the functions to use.

 

[leonardify]

I was having some issues outputting all the data from `stringifyAllJSON` in FireTuner;
I've found in the end that using UI.setClipboardText(stringifyAllJSON()) allows me to copy the output in clipboard (warning: it's 5mb!)

 

[KayleeR]

I was having some issues outputting all the data from `stringifyAllJSON` in FireTuner;
I've found in the end that using UI.setClipboardText(stringifyAllJSON()) allows me to copy the output in clipboard (warning: it's 5mb!)

Well that's a lot easier than what I was doing and splitting the string into 100 character segments to output across multiple log calls then piecing back together.

 

[LilBudyWizer]

It sucks being a newbie. I was like stringifyAllJSON? Where did that come from then realized was from the scripts at the start of the thread. Duh.