JCivED - a toolbox for Civilization

[rhaul]

Yes, I also wanted it for my mod of the civilization of Spain.

 

[rahorvat]

Hi there,

It's one the difficult challenge I have been working on, but sound code is really not easy to reverse engineer, especially for old games like this.

I remember finding some interesting data in the additional General MIDI patch released for CIV (GSOUND.CVL), but since I am not very familiar with MIDI, there is still work before being able to export/import as MIDI files.

So, sadly I advise you to not expect this feature very soon.

You can't !

The sounds are hard coded into sound drivers. The only possibility is a new .cvl driver which accepts wav files, or modifying the existing one. But I don't know if this is worth the effort, as I hope that in a few month's the OpenCiv1 will be ready for a release.

I was looking at the RSOUND.CVL code (MT-32 version), but it seems that after the code analysis the memory block (which is a mix of ASCII and binary) starting with string '[MPS Civilization]' seems like a memory dump from Roland MT-32 internal memory. I hope that someone did a good MT-32 ROM analysis so we will be able to decode the MIDI stream from there. There is also AdLib/Sound Blaster version in ASOUND.CVL which is software based wav player, but the sound quality is poor.

 

[rahorvat]

DOS games will often interface with sound cards directly rather than playing back files, which makes it really tough to change things. Your best bet is to hope @rahorvat 's decompilation reaches a point where enhancements are possible (or play the windows port, you can switch out low quality wav files to your heart's content )

It's hot a hope anymore, I'm progressing nicely. The things work, and I think that the game is truly playable in this state but without music for now as my priority is game first and sound second.

Of course, I envisioned the multiple language feature later.
About custom Music /Sound effects we can discuss also later.

Cheers

 

[darkpanda]

You can't !

The sounds are hard coded into sound drivers. The only possibility is a new .cvl driver which accepts wav files, or modifying the existing one. But I don't know if this is worth the effort, as I hope that in a few month's the OpenCiv1 will be ready for a release.

I was looking at the RSOUND.CVL code (MT-32 version), but it seems that after the code analysis the memory block (which is a mix of ASCII and binary) starting with string '[MPS Civilization]' seems like a memory dump from Roland MT-32 internal memory. I hope that someone did a good MT-32 ROM analysis so we will be able to decode the MIDI stream from there. There is also AdLib/Sound Blaster version in ASOUND.CVL which is software based wav player, but the sound quality is poor.

Sometime ago I found a dump of Roland MT-32 based MIDI files, although I cannot find them anymore...

However I did contact the person who made them, and he explained that he plugged an MT-32 emulator into a DOSBox version of CIV - or more precisely the "CIV jukebox", that I never had my hands on, but apparently can play any CIV song individually.

That way he simply captured MIDI events sent out through the MT-32 emulator and generated MIDI files this way.

I still have that person contact and e-mails, but cannot find the MIDI files at the moment...

 

[rahorvat]

Seomtimes ago I found a dump of Roland MT-32 based MIDI files, although I cannot find them anymore...

However I did contact the person who made them, and he explained that he plugged an MT-32 emulator into a DOSBox version of CIV - or more precisely the "CIV jukebox", that I never had in my hands, but apprently can play any CIV song individually.

That's great!

That way he simply caputred MIDI events sent out throughte MT-32 emulator and generated MIDI files this way.

Well, as I understood, MT-32 not only plays MIDI files, but stores synthesized sound models. So you could, for example load new piano or guitar synthesized sound and then play song with that sound. I think that's also a problem, as MIDI generally stores only name of instrument and notes.

Don't know for sure, as it's not my expertise.

Maybe Blake00 knows more details as he owns actual MT-32?

But then we are still stuck on two choices: Make a custom cvl sound driver, or modify an existing one, possibly with wav input for simplicity?

 

[darkpanda]

That's great!

I found it again, and the actual source where I found it: http://www.mirsoft.info/gmb/music_info.php?id_ele=MTQzMzQ=

Well, as I understood, MT-32 not only plays MIDI files, but stores synthesized sound models. So you could, for example load new piano or guitar synthesized sound and then play song with that sound. I think that's also a problem, as MIDI generally stores only name of instrument and notes.

Yes MT-32 was a synth board created by an actual top brand in synthesizers (Roland), however they used their very own flavour of MIDI which does not follow the "General MIDI" standard that came around several years later.
MIDI is first and foremost a music industry standard, and as such only includes information such as notes, durations, velocity (volume ?) and instruments' names, letting the actual sound devices (usually a synthesizer) generate the sound.
In that way, it is much simple to modify than actual sound samples (such as WAVs).
Windows includes a default Gsynth Midi Synthesizer, but there are tools that can new MIDI synthesizers to Windows to get a better sound (see VirtualMIDISynth), some with loadable alternate soundbanks - including MT-32-like soundbank

Don't know for sure, as it's not my expertise.

Maybe Blake00 knows more details as he owns actual MT-32?

But then we are still stuck on two choices: Make a custom cvl sound driver, or modify an existing one, possibly with wav input for simplicity?

I can't even imagine the effort it'd take to program WAV support into a 16-bit CVL supported by CIV.EXE !!!

However you got me into diving into the General MIDI lead again

 

[Lord_Hill]

The traditional way of getting MT32 playback in software is including MUNT as a dependency. This is what dosbox does I believe.

GitHub - munt/munt: A multi-platform software synthesiser emulating pre-GM MIDI devices such as the Roland MT-32, CM-32L, CM-64 and LAPC-I. In no way endorsed by or affiliated with Roland Corp.

A multi-platform software synthesiser emulating pre-GM MIDI devices such as the Roland MT-32, CM-32L, CM-64 and LAPC-I. In no way endorsed by or affiliated with Roland Corp. - GitHub - munt/munt: A...

github.com

We're going round in circles, Blake also mentioned that site with the MT32 rips! And I linked to DarkPanda's old post about Civ Jukebox.

Another example that might be useful for inspiration, the Rigel Engine project to rewrite Duke Nukem II for modern systems uses dosbox's adlib emulation for original classic sound, plus a separate system to mod in modern wav based sound. Something similar might work here.
https://github.com/lethal-guitar/RigelEngine/wiki/Architecture-overview

I'm going to try and get hold of a copy of Civ Jukebox, I'm curious as to how it works.

 

[Blake00]

Yeah I was gonna say you could have saved yourself some time Darkpanda as I literally posted that link in this thread yesterday only a handful of posts before yours lol!

And nope, I definitely don't have a Roland MT-32 rahorvat sadly! From the sound of things you guys know way more than I do about how all that works too.

Great to hear things are getting that far with OpenCiv1 now!

 

[Lord_Hill]

I used wayback machine to reach the microprose ftp site (circa 1998!), here it is. It even lets you choose what sound driver you want to use.

 

[darkpanda]

Yeah I was gonna say you could have saved yourself some time Darkpanda as I literally posted that link in this thread yesterday only a handful of posts before yours lol!

And nope, I definitely don't have a Roland MT-32 rahorvat sadly! From the sound of things you guys know way more than I do about how all that works too.

Great to hear things are getting that far with OpenCiv1 now!

So sorry, my forum reading is very erratic
I must have thought you were using WAVs of MT32 playback inside WinCiv, and didn't care to explore further...

 

[darkpanda]

I used wayback machine to reach the microprose ftp site (circa 1998!), here it is. It even lets you choose what sound driver you want to use.

Excellent ! I'm on my phone but will check it out as soon as I am back on PC

 

[rahorvat]

I can't even imagine the effort it'd take to program WAV support into a 16-bit CVL supported by CIV.EXE !!!

However you got me into diving into the General MIDI lead again

Well, the MUNE in DOSBox will not play regular MIDI so that appears as a dead end. I tried to play those midi files but they are in MT-32 format it seems.

The 8-bit stereo would be fine in at least 32000, 44100 would be the best. DOSBox seems to support 44100, 16bit stereo, so that seems like it should work?

It wouldn't be so bad, there is a lot of Sound Blaster doc around to do the job.
Well, I could do it as I'm not a stranger to assembly language. There is also a ton of details about a sound driver in OpenCiv1 code.

But again, I'm not sure if the effort is worth the result when I expect to have OpenCiv1 release in a few months.

P.S. I forgot about the DOS memory constrictions, probably it wouldn't work as the samples would be too big to fit in 640K!

 

[Blake00]

I must have thought you were using WAVs of MT32 playback inside WinCiv

I am lol, I guess you didn't read my next post after that too where I said I wanted the playable midi format files for other things.. sigh..

 

[darkpanda]

I am lol, I guess you didn't read my next post after that too where I said I wanted the playable midi format files for other things.. sigh..

... looking for a neurologist right now

 

[darkpanda]

nah sadly for me it's not about listening as I have the roland mt-32 recordings for that.. I wanna add the midis to my Civ1 mod for Civ2 and also use/edit them for some other things too.

Just to mention that I have posted 18 MIDIs extracted from the Civ Jukebox using General MIDI driver with DOSBox capture, in @Lord_Hill Jukebox-dedicated thread: https://forums.civfanatics.com/threads/the-civilization-jukebox.684866/

Most of them need a re-cut because CIV and the Jukebox both seem to loop endlessly when playing them.

 

[Blake00]

ahh brilliant! Thanks heaps!

 

[darkpanda]

Messing around with CivNet sav VS CivWin sav, there doesn't seem to be a huge amount between them.

As per this post, CivWin SAV has an initial byte followed by FE FF as some kind of signature.



CivNet has a 10 byte header which reads in ASCII as 'CIVNET!' followed by three zeros, then the initial byte then a different signature, F5 FF.

Curiously, I was able to 'convert' a CivNet save by removing the 10 byte header, opening it as a CivWin save (which worked somehow) then resaving it in JCivEd as a CivWin save. I tried this with a second save and JCivEd locked up, so presumably it's a case of very close but not quite the same! I did not attempt to actually play with this save, so there may well be other problems. But it's a start.

Strangely enough it doesn't *crash* JCivEd (which means somehow JCivEd can partially parse the CivNet SAV data), but rather gets it stuck in an endless loop... At first sight something to do with looping references in "unit stacks" IDs... Investigating.

 

[Lord_Hill]

I was hoping it was close enough that a hex editing newbie like me could make sense of it, but it quickly became apparent that it was quite different. There are a number of extra preferences and features in civnet that presumably all need a few bytes in the save file, so I shouldn't be surprised!

 

[Lord_Hill]

Hoping someone can point me in the right direction with PIC editing. I'm just trying a simple edit to change the settler unit, and it doesn't seem to matter how I save the gif image (I've even done it using Paint Shop Pro 7 so I'm 99% sure it isn't mangling the palette when I save), I always get a conversion to PIC with broken colours. Then I thought maybe I'm supposed to load the correct SP257 palette over the top, but however I do this, the colours stay broken. What am I missing here?

EDIT - of course I figure it out as soon as I post. I needed to change the palette import to 'substitute palette colours', I was doing it with 'cast'.

 

[darkpanda]

Glad you could figure it out PIC editing hasn't much changed for years, so it's still pretty raw and obtuse, sorry for that !