Thoughts on use git for Civ5 DLL mod sharring

Impaler[WrG]

Civ4:Col UI programmer
Joined
Dec 5, 2005
Messages
1,750
Location
Vallejo, California
I've been thinking about ways that those of us who will be doing DLL modding can collaborate better. I'd brought up the idea of reviving the old CCCP (Civilization Core Community Project) earlier but it was pointed out that people found it hard to continue that and other DLL projects. Even those most of us make our code public the best tool available to us before was SVN and SourceForge but its still quite a pain to make and create such projects and pulling in code you want from another mod is a whole bunch of Winmerging. I've also been thinking that the old CCCP concept was too centralized, it's not practical to have that many people working on one code base. I think their may be a better way.

It seems that the popular unix source control program git has finally got a windows GUI front end now. It's called msysgit and while its still in beta looks like it could do the job. If we put a copy of the raw SDK along with any patches/expansions as the 'root' and we all then fork off it we can have many little branches with fixes and improvements. People can then cherry pick and pull in these branches to their own DLL's and best of all we can update our DLL's when new official patches/versions are available by pulling in from the trunk. And this can all happen without any central clearing house or bottle neck.

Now I don't have much experience with Git so this is all somewhat speculative as to the benefits but from what I've read of git the kind of uncoordinated community development is exactly what its geared towards. If anyone more familiar with git and how it really works in practice could put in their two cents? Would it be worth the time to learn and use? Are their any major hurdles I'm not seeing?
 
This is one of those ideas that sounds good, but ultimately is full of fail. Such a project is guaranteed to produce an unstable and unreliable dll with alot of features, but will never be able to get through a full game. See WoC as an example.

I only trust a few modder's work, and for good reason. And even the best of the best still release buggy code from time to time. Without reliable people willing to take responsibility for unavoidable errors that occur in all coding projects, nothing is ever going to get fixed. And that will be the end fate of this idea if implemented.
 
The goal isn't to produce 'a' DLL at all, like I said a single centralized DLL isn't viable, it's not even really a project either, more of a community wide standard for hosting open source mods. This is completely the opposite approach to having a central project, everyone can make their own branches and choose to pull-in what ever code they wish into it, so for example you could pull code in only from the people you trust to produce good code. I'd imagine a collection of DLL lines would evolve for specific goals such as better AI or Combat rules, re-creating some Civ4 features etc etc.

P.S. Looks like their is a TortoiseGit client as well, I've always like the Tortoise clients windows explorer integration and ease of use
 
I do like this idea. Being able to pick and choose which branches I want to make my own trunk would be handy, rather than having to rely on a single DLL to rule them all.

But phungus does make a point about inherited bugs. I believe his comment in part relates to me ;) but as we all know, un-paid modders do disappear into real life and just leave there code. I had to do it with my BtS work, and there are many others.

But overall I like the idea. Of course, there would have to be a dedicated person to administer it and make sure that inserted code isn't malicious or just plain crap. I don't know where you'd find that person though.
 
I like this idea. As Dale mention we would definitely need someone manage the overall project.

Another thing which I think is important is the ability for the creator of a branch to limit those who have editing rights along a branch. Other branches can of course merge any branch into their own branch, but we could has chaos if anyone can go in and edit any branch they wish. Does anyone know if Git allows control of who has editing rights to branches?
 
mathewv:

Best thing would be to have one branch per mod. Once locked it can't be changed by anyone, but anyone can take that branch to create their own base to work from.
 
Actually I think a branch for each 'atomic' piece of a mod would be best. So for example Dale your Civ4 combat mod has a number of parts that could stand alone, such as the Air bombing mission and the stack-attack system. If you had a branch for each standalone feature ware it's developed to completion you could then pull them all into a compilation branch for the DLL you would actually upload as a finished mod. This would allow other people to cherry pick the component they want by pulling the desired branches into their own branch rather then taking or leaving the whole thing.

Also it dose appear that github can allow multiple account/people to work in one branch, you are not forced to create multiple branches for each person though this seems to be a popular way to use git.
 
Impaler[WrG];9072740 said:
I've been thinking about ways that those of us who will be doing DLL modding can collaborate better. I'd brought up the idea of reviving the old CCCP (Civilization Core Community Project) earlier but it was pointed out that people found it hard to continue that and other DLL projects. Even those most of us make our code public the best tool available to us before was SVN and SourceForge but its still quite a pain to make and create such projects and pulling in code you want from another mod is a whole bunch of Winmerging. I've also been thinking that the old CCCP concept was too centralized, it's not practical to have that many people working on one code base. I think their may be a better way.

It seems that the popular unix source control program git has finally got a windows GUI front end now. It's called msysgit and while its still in beta looks like it could do the job. If we put a copy of the raw SDK along with any patches/expansions as the 'root' and we all then fork off it we can have many little branches with fixes and improvements. People can then cherry pick and pull in these branches to their own DLL's and best of all we can update our DLL's when new official patches/versions are available by pulling in from the trunk. And this can all happen without any central clearing house or bottle neck.

Now I don't have much experience with Git so this is all somewhat speculative as to the benefits but from what I've read of git the kind of uncoordinated community development is exactly what its geared towards. If anyone more familiar with git and how it really works in practice could put in their two cents? Would it be worth the time to learn and use? Are their any major hurdles I'm not seeing?

Doomed to failure. Before you bash me, here is why:

1.) Too Large of a Scope. (See WoC)
2.) You have no paid dev's, pure volunteer labor. With only about 30 people on CFC that regularly do C++ modding of any depth, you lack the manpower.
3.) Lack of interest. Crappy and buggy scripted python mods are much more popular than most SDK mods, just because it's faster to ship out python than clean working C++ code. With Lua, this will be even more the case.
4.) Need, or lack of. In Civ4, most SDK mods worked together to some degree. BBAI is in basically all SDK mods, and so are a lot of other features.
5.) Negative feedback, from the start. If this is the public reaction now, before anything has been even decided, it can only get worse from here.

I hate to sound like that Bible preacher on the side of the street shouting about the Armageddon, but I will. This project will fail, mark my words. And worse, like the WoC project, it will waste thousands of hours from talented modders for code that will never see the light of day.
 
Doomed to failure. Before you bash me, here is why:

1.) Too Large of a Scope. (See WoC)
2.) You have no paid dev's, pure volunteer labor. With only about 30 people on CFC that regularly do C++ modding of any depth, you lack the manpower.

Both these points seem to imply that this is a 'project' with specific goals, that's not the case, in fact it's your critique of flaws in CCCP I'm trying to address here. Git is a system designed for massively distributed development with little or no central control, individual branch authors (which can be anyone github allows unrestricted branching) can create what ever scope they wish for their branches. I mentioned CCCP more as a foil and to avoid exactly this confusion but I apparently not been clear enough, this is just a proposed community wide standard for source code hosting.

3.) Lack of interest. Crappy and buggy scripted python mods are much more popular than most SDK mods, just because it's faster to ship out python than clean working C++ code. With Lua, this will be even more the case.

From what I can tell Civ4 SDK modders grew in number over the years and the SDK mods got more elaborate. I think the SDK modding group will transition easily into Civ5 SDK modding ware as Lua modding will have to start from scratch. A lot depends on how powerful the Lua API is, though it will never be as powerful as the full SDK.

4.) Need, or lack of. In Civ4, most SDK mods worked together to some degree. BBAI is in basically all SDK mods, and so are a lot of other features.

Not sure what you mean here, SDK mods can indeed usually be combined without a problem but merging and diffing code between different SVN repositories is a pain, if you want someones mod they need to send you a diff or you need to check out their whole project and make it yourself. Git looks to be an easier solution to the code sharing hurdle. Also the ease of creating branches in git would encourage many small branches of limited scope and complexity.

5.) Negative feedback, from the start. If this is the public reaction now, before anything has been even decided, it can only get worse from here.

Actually the feedback seems mixed at this point rather then completely negative, though admittedly the negative responses have been very negative. I have to say I think this is due more to poor explanation on my part and resulting misinterpretation. So far no one seems to have anything bad to say about git or github itself, I've love to get some feedback from people who use it.
 
Best thing would be to have one branch per mod. Once locked it can't be changed by anyone, but anyone can take that branch to create their own base to work from.

I fail to see how this is different from the current SVN setup most large mods have. If it's just a pool to draw from, its seems like you'd get the best and the worst of everything. And if there is one thing I've learned from Civ4 modding, it's that most inherited code is buggy. It doesn't matter who wrote it (even myself). It's going to have bugs. With Git, wouldn't we be increasing the 'infected' pool, so to speak? Of course, this is a rather lame argument, I admit; it's like arguing against airplanes because they spread disease. :lol:

I just don't see the advantages over the current system. I never found merging a big pain. (Except Python. I hate python). How would this be different?
 
I'd actually be more enthused if they:
A> Fixed the utterly broken serialization/deserialization system (you cannot add, or remove fields without completely breaking save game compatibility!)

B> Enforced class boundaries strongly,

C> Allowed for the dynamic loading of a class from a 3rd party DLL. (via one of many methods -- a simple naming scheme, dynamic loader, and an entry point that returns a pointer to an instance of the class with defined constructors would do it).

That would allow you to drop in a replacement CvUnit (or CvCity, etc) into an otherwise vanilla DLL.

You could even go further than this, and allow for modular method replacement, but C++ doesn't make that as easy.
 
C> Allowed for the dynamic loading of a class from a 3rd party DLL. (via one of many methods -- a simple naming scheme, dynamic loader, and an entry point that returns a pointer to an instance of the class with defined constructors would do it).

That would allow you to drop in a replacement CvUnit (or CvCity, etc) into an otherwise vanilla DLL.
That would be very useful, yes.
 
Git is much better than svn for branch management. I daresay that branch management is what svn does worst in fact.
I think the proposal has a lot of merit, but it requires the community to actually use the git trunk. The best thing to do would probably be to actually ask those who own the trunk to provide us access to it, or a copy of it. Namely Firaxis. After all, they want to provide the best modding tools to everyone, including source code, why shouldn't they also provide a git repository, or a svn one, or whatever they like, and maintain it? It would make it easy for everyone to branch from that and to actually keep up to date when they release patches and expansion packs.
If Firaxis is going to provide in-game access to uploaded mods, they could also provide access to branches too?
 
It would only require one person to upload the SDK and any patches too it and that's quite simple to do, it would be nice if someone did that for us but it's hardly a make or break. I'd volunteer to upload the code if no one else wants to do it.
 
C> Allowed for the dynamic loading of a class from a 3rd party DLL. (via one of many methods -- a simple naming scheme, dynamic loader, and an entry point that returns a pointer to an instance of the class with defined constructors would do it).

That would allow you to drop in a replacement CvUnit (or CvCity, etc) into an otherwise vanilla DLL.

You could even go further than this, and allow for modular method replacement, but C++ doesn't make that as easy.

DLL Hell, anyone?:rolleyes: No thank you.

Can someone please explain (I do really want to know) why git is better than what we have now? The tools for git are all in beta for windows, while SVN is well established. I don't need nor trust automated merging, so that isn't a feature. Easier forking sounds nice, but the current system of just checking out someone elses source code is not very hard. I'm not a huge fan of SVN (I still have issues with it to this day), it shouldn't be hard to convert me, if someone can try. ;)
 
DLL Hell, anyone?:rolleyes: No thank you.

I'm inclined to agree with you on that, theirs way too much interaction between the core objects like unit/city/plot for simply substitution to be viable.


Can someone please explain (I do really want to know) why git is better than what we have now? The tools for git are all in beta for windows, while SVN is well established. I don't need nor trust automated merging, so that isn't a feature. Easier forking sounds nice, but the current system of just checking out someone elses source code is not very hard. I'm not a huge fan of SVN (I still have issues with it to this day), it shouldn't be hard to convert me, if someone can try. ;)

The tortisegit tool should be entirely stable by the time Civ5 is in stores, considering the high activity level on the project, the level of completion it's already at and the considerable time window I don't anticipate any trouble their. I'm strongly attached to tortise style explorer integrated tools so for me this was a must have to even consider git.

I'll try to get myself more familiar with git so I can convert you, I'm sure their is some 'killer app' in git that will convince you. Perhaps if you elaborated on what you don't like about SVN that would help. Also what other VCS have you used besides SVN? I still wish we could hear from some git users (so long as they don't try to tell us how Linus is Gawd)
 
Impaler[WrG];9085372 said:
The tortisegit tool should be entirely stable by the time Civ5 is in stores, considering the high activity level on the project, the level of completion it's already at and the considerable time window I don't anticipate any trouble their. I'm strongly attached to tortise style explorer integrated tools so for me this was a must have to even consider git.

Ditto. Reverting to the command line would be a deal breaker for me. Nothing says easy to use like having to use the command line in Windows... :rolleyes:
Impaler[WrG];9085372 said:
I'll try to get myself more familiar with git so I can convert you, I'm sure their is some 'killer app' in git that will convince you. Perhaps if you elaborated on what you don't like about SVN that would help. Also what other VCS have you used besides SVN? I still wish we could hear from some git users (so long as they don't try to tell us how Linus is Gawd)

Subversion is the only source control I've used, ever. As for what I don't like about SVN... well, it's slower than I like, but I suppose that applies to all apps, everywhere. I seem to have to use the cleanup command more often than I should, likely because SVN & Win7 x64's new filesystem are partially incompatible... I guess I don't have any real major complaints; isn't it your job to tell me what I should like and dislike? :mischief:
 
Top Bottom