Assembling the team

[Caro-Kann]

Hey, this is Lucien from Discord and Github. For now I'll post under this old account I found, but I might switch to a new one that has my actual name once that gets approved. Just to be more consistent.

Anyway, I'll try to make a post tomorrow about what I've been doing in the QueryCiv3 section of the repo. My plan is that, now that biq data is fully populated, I'll work on doing the same for Sav data. I have a feeling that Sav files are going to be a lot trickier to parse, but I'll see what progress there is to be made.

 

[Puppeteer]

I have a feeling that Sav files are going to be a lot trickier to parse, but I'll see what progress there is to be made.

They are, but a lot of it is figured out. I think my most complete decoding is in c3sat/luaciv3 in Go, although there may be some translation needed as at different iterations of decoding I'm used the offset from the start of the 4-char section name, and at other times the end of it.

The tech list and cities were difficult to get decoded (and both rely on the particular BIC in use which is why I ensure we have a reference to the BIC when decoding the SAV), but they are successfully parseable with that code. I don't recall exactly how much info I'm getting out of CITY, but just figuring out where each city begins and end was a pretty big feat. I think LEAD has some good info, too. I definitely recall being able to decode gold, figure out war status, and providing an non-spoiler trade availability list in Lua. (Spoiling info is of course even easier.)

Actually I seem to be parsing UNITs, too, but not yet health and experience level. That may have been what I was working on most last.

The luaciv3 code is different of course because it's in Go, but it's also loading values into a Lua environment directly where the QueryCiv3 code is styled as convenience getters for the raw SAV data.

 

[WildWeazel]

Welcome Lucien!

That reminds me, someone had suggested we ask ainwood about CivAssist II code but I don't think he's around anymore. The tool itself could be helpful too, in deriving the logic.

 

[Puppeteer]

I'm looking at my code trying to remember the tricks to CITY. I don't seem to have commented it as well as I had hoped. There are a couple of loops that needed some figuring out. One was a loop for the number of citizens. The other was a loop the number of buildings that are defined in the BIC, with no other local indicator of how many elements there are in the loop. I'm looking at the code and definitely not seeing what I just said. I have a strong memory of a loop based on the BIC count rather than a local value, but I'm just not seeing it in the code right now.

That may be the BINF section which I always assumed meant "bin food" but seems to be "building information" with culture, built date, and built-by info for every possible building.

Another quirk/feature of the Go/Lua code that won't directly translate to QueryCiv3 is that in luaciv3 I was looping and reading and just pushing offset ahead as I went. Of course I was also learning how to parse the section as I coded, so maybe after knowing how to parse through all the city bits it will be easier to calculate the offset in a getter.

 

[Quintillus]

CITY is one of the relatively few areas of the SAV where my Java code is fairly complete and relatively easy to read; it is of course also one of the more complex areas. BINF is indeed building information.

Check your PMs for some additional documentation of the SAV format that I just sent out. It's the most complete SAV documentation I am aware of, and is from another private forum at CFC (which, come to notice it, our forum still describes itself as a private forum even though it isn't anymore). It also includes the vBulletin CFC Christmas theme, RIP.

 

[Caro-Kann]

CITY is one of the relatively few areas of the SAV where my Java code is fairly complete and relatively easy to read; it is of course also one of the more complex areas. BINF is indeed building information.

Check your PMs for some additional documentation of the SAV format that I just sent out.

I'm looking through the mht files now, and yep, they look to be pretty comprehensive. Between those and your Java files I should be able to put the Sav structs together in the same way as Biq. Great resources. There'll be some gaps, but it looks like cities, units, terrain, and terrain modifiers are all there, which are the most important things and the ones that are wanted for the Babylon iteration.

 

[Lanzelot]

I don't recall installing it, but it's there. That must be the magic to make the Godot SDK references work.

I also have that plugin installed (that was the reason I upgraded to VS 2019 in the first place: the plugin doesn't support 2017...)
But perhaps my problem is, that the VS projects reference Godot 3.3, while I downloaded and installed 3.4? I couldn't find 3.3 (probably well hidden in their archives, as they want users to always use the latest and greatest...) I'll experiment with that. Ah, no, never mind, I found it! The target framework was set to 4.7.2 in C7.csproj. After I changed that to netstandard 2.0 (like the rest of the projects), it now finds the Godot DLLs ok, and compiles! (And runs...!)

Should I check in my changes to the project files, so that other people on Windows can also benefit from a fully functional VS project?
You will probably need to lend me a hand for that so I don't mess up the proper review process etc.
Also is there a way to diff the files I modified, before checking them in? In perforce I usually do a "right-click --> diff against head revision" for all files in my current change list, to verify that I'm not checking in something stupid or incomplete... Would be nice if there is a Git command for that as well. (So far I didn't get to installing a Git UI, only have the bare-knuckle CLI tools.)

The Godot SDK exposes to the C# environment the C++ objects in Godot, so whenever we're using a Godot namespace object it's *probably* just a bridge to the C++ code which may have some edge case implications we bump into but for the most part should be able to ignore. (e.g. We can't necessarily expect dropping all references to a Godot object to free its memory, but in most cases that probably doesn't matter.) So on export, Godot compiles the C# code into Mono dlls, then bundles the project folder and drops it alongside a precompiled redistributable Godot binary.

I'm not sure exactly how that C# to Godot bridge works or if that's even the right way to talk about it, but maybe that's why we cant seem to build directly from VSCode or VS.

I think, here I can contribute a bit knowhow. We often have to call C/C++ libraries from C# projects and it's actually quite easy (much easier than for example JNI (Java Native Interface), which we also have to use a lot). Microsoft provides two technologies for that, "P/Invoke" and "C++ CLI Interop". I'm pretty sure Godot simply uses this for providing a C# wrapper around their C++ native binaries. Probably they use P/Invoke as that is easier to use when it has to work on Windows, Linux and Mac at the same time.

Anyway, as we know now, that was not the reason why VS could not compile the project...

Godot can be configure to open VSCode (and I think Visual Studio) ...

Yep, I did this and it works fine. Language Server looks like it is meant for remote debugging (running Godot on one host and attaching your Visual Studio Debugger from another). Haven't figured out, yet, how it works, though... (It could be that remote debugging isn't supported by the free Visual Studio Community edition that I installed. Could also be that I just didn't find the right button, as I haven't used that feature for a couple of years...)

 

[Lanzelot]

OK, Visual Studio had a Git plugin, and that had a decent diff tool, so I went ahead and made a local Commit.
However, when I tried to push, login worked ok (somehow had to enter user & pwd twice?!), but then it said "Failed to push to remote repository. See the Output window for more details."

But the output window is empty?!

Ok, when I try from the CLI, I get this:

remote: Support for password authentication was removed on August 13, 2021. Please use a personal access token instead.
remote: Please see https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/ for more information.
fatal: Authentication failed for 'https://github.com/C7-Game/Prototype.git/'

I read the "for more information", but it's not yet completely clear, what I need to do. Access token, to-factor authentication, etc. Getting too late now, will continue tomorrow...

 

[Quintillus]

Ah yes, the SSH requirement to push code. They removed the username/password support since a lot of people use insecure, shared passwords, but SSH requires a lot more knowledge to set up. Just searching for a good guide right now, one of the top results was a developer with 12 years of experience who argued it was the toughest part of setting up source control for a project; I'd definitely put it near the top.

I still forget the details between when I have to set up new SSH authentication fairly often, and I haven't done it since last winter, so I forget the details again. But you can go to https://github.com/settings/keys and it has a guide for how to generate the keys. I don't think I used their procedure though, I am pretty sure I used Pagaent and Putty instead. But if their procedure works for you, it's probably simpler to follow it.

For what it's worth, I still get an error when pushing code on my laptop (using Pagaent and SSH), but it still works. Haven't been able to figure out why despite searching for an answer. It works without an error on my desktop with seemingly the same configuration and the same key.

(If I wanted to make the maximum impact I could for developers, making more user-friendly tooling might be a good candidate. Although, more user friendly tooling is not necessarily what is adopted)

 

[Puppeteer]

But perhaps my problem is, that the VS projects reference Godot 3.3, while I downloaded and installed 3.4?

Godot 3.4 is the right version. I also noticed the SDK attribute still says 3.3 for Godot 3.4. It works.

Also is there a way to diff the files I modified, before checking them in? In perforce I usually do a "right-click --> diff against head revision" for all files in my current change list, to verify that I'm not checking in something stupid or incomplete...

From the command line you would git add your changed files (individually, with wildcard, or all changed files not omitted by .gitignore with git add -A) before committing with git commit -m "commit message" or git commit which then opens an editor to add the message.

At any point you can get a status of what's staged and what's changed since the last commit with git status .

You can see differences in the current working directory tree with git diff , staged changes with git diff --staged (this is what you'll probably be most interested in), and you can also compare *any* two commits by specifying commit IDs, tags, or branch names which is handy.

I would think VS has this ability built-in. VSCode has a source control sidebar for diffs, staged files (with diffs), the commit message and commit button, and also a pull/push sync button.

So your workflow will be: edit files, add/stage, commit, push. (If you weren't in your own branch or repo then pull before pushing to be sure you've merged the latest updates.) You may want to check diffs at any point in there.

 

[WildWeazel]

staged changes with git diff --staged (this is what you'll probably be most interested in), and

I didn't know this one! That's handy

 

[Lanzelot]

Ok, getting an access token (PAT) took only a few clicks. But then trying to figure out what to do with this token, took hours, none of the instructions I found seems to work, not even the stuff on Stackoverflow, which is usually quite good. (https://stackoverflow.com/questions/18935539/authenticate-with-github-using-a-token )
Either it would not log me on, or it said the repository cannot be found...

Finally I found a way, where (I think) I am logged on successfully and hit upon the right repository name, but now I'm getting

$ git push --set-upstream origin Development
remote: Permission to C7-Game/Prototype.git denied to Ulrich-Schmidt.
fatal: unable to access 'https://github.com/C7-Game/Prototype/': The requested URL returned error: 403

Does someone still have to give me write permission?!

 

[WildWeazel]

Yes, you have to be approved to push to the repo. I'm afk but @Puppeteer or @Quintillus can add you to our organization. What's your GitHub username?

 

[Lanzelot]

Ulrich-Schmidt

 

[Puppeteer]

I either can't or don't know how to add anyone to the project, but I can add write permission to the repo, so I just now did that for @Lanzelot /Ulrich-Schmidt as I had previously done for Flintlock/maxpetul .

And come to think of it, we may want to keep a map of GitHub usernames vs other usernames, be they CFC and/or Discord. It already took me a minute to remember who maxpetul was when looking at the permissions list, and I added him myself. I'm not sure where that belongs, but it should be documented somewhere.

 

[Quintillus]

I have added Lanzelot to the C7 organization. For future reference, that can be done from this page: https://github.com/orgs/C7-Game/people

 

[Lanzelot]

Thanks, looks better now. But I'm still struggling...

First I got

hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.

so I did a git pull. It pulled in some more changes, and I tried to push again. This time I got

hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Integrate the remote changes (e.g.
hint: 'git pull ...') before pushing again.

As I had just done a pull, I studied the git docs for half an hour until I believed I understood something and that it said I had to do a "git pull --rebase". So I tried that, got some message that I did not really understand (and of which I wasn't sure, whether it was a success or an error message...), anyway I tried to push again, but I'm still getting the message that my branch is behind.

So far I have spent more time with git than with writing code. I think I better take some time off this weekend, play some chess, whatever, because I have a hunch that if I continue with this for a little longer, I'm going to do harm to my poor innocent monitor or go insane and jump out of the window...

 

[Quintillus]

Sadly that's a great example of how Git is unfriendly to newcomers. git pull was probably the right thing to do after the first message, and that message is normal if you are working on the same branch as someone else, and they did work in the interim. But why you got a similar message the second time is the surprising part.

Two questions to try to figure out what state things are in.

1. If you run git branch, which branch has an asterisk next to it? My best guess is it will be *Development, but it might be a branch you've created (although in that case the messages you mentioned above would be less expected).
2. If you run git log, which are the first few commits?

For the second question, in theory the most recent one that you didn't write will be Merge pull request #55 from C7-Game/BarbariansOnTheProwl from QuintillusCFC (me), if you are on *Development, but it could be an older one if you are on a different branch.

Hopefully the answer to one of these questions is illuminating. Git problems can be tough to diagnose without seeing the behavior in real time/being able to run queries on it.

Definitely don't continue moving in the direction of insanity and jump out a window.

 

[Lanzelot]

What puzzles me even more is, that no one has touched the two files that I modified (C7.sln and C7.csproj). So why can't git just let me submit them?!?

 

[Lanzelot]

_______________________________________________________________
uschmidt@old-firehand MINGW64 /d/Sprachen/C7/Prototype (Development)
$ git branch
* Development

uschmidt@old-firehand MINGW64 /d/Sprachen/C7/Prototype (Development)
$ git log
commit 8eaf3863ad506fa8a1f7ae834a4212a694412c47 (HEAD -> Development)
Author: Ulrich-Schmidt <u.schmidt@sap.com>
Date: Fri Dec 17 00:18:07 2021 +0100

Update project to compile completely from Visual Studio

commit 3182552facef7f86d2e64e6597020fa41df2b519
Merge: e9d7f19 3931799
Author: Quintillus <5287977+QuintillusCFC@users.noreply.github.com>
Date: Tue Dec 14 10:01:11 2021 -0500

Merge pull request #53 from C7-Game/ZoomWithZKey

Allow setting the zoom to 100% or 50% with the Z key.

commit e9d7f19d81db52dcfe060d21c96767c43d1e29e0
Merge: e8e2514 9637e36
Author: Quintillus <5287977+QuintillusCFC@users.noreply.github.com>
Date: Sun Dec 12 15:45:49 2021 -0500

Merge pull request #52 from C7-Game/PrototypeCityProduction

Add some basic city production and growth
_____________________________________________________________

So looks like I'm still missing #54 and #55?