View Full Version : CVS issues


12monkeys
May 12, 2006, 07:32 AM
Ok, I started this thread here to have a single thread for CVS issues. I would like to begin, that the CVS at Source Forge is down since 3 days. This is not only a problem for our projects its a problem for amyn projects at SF. Somewhere else, I told how frustrating it is that we don't get any infroamtion from Source Forge. Well, they did hear me. I just received an email from SF about the problem :

-----------------------------------------------------------
Greetings,

You are receiving this mail because you are a project admin for a SourceForge.net-hosted project. One of our primary services, CVS, suffered a series of interrelated, critical hardware failures in recent weeks. We understand how frustrating this CVS outage must be to you and your users; however, our top priority remains preservation of the integrity of your data.

The series of CVS hardware failures prompted us to expedite the deployment of planed improvements to our CVS infrastructure, drawing upon much of the knowledge that we gained from our Subversion deployment. Our improved CVS service architecture, which we plan to deploy tomorrow afternoon (2006-05-12), will offer greater performance and stability and will eliminate several single points of failure.

The Site Status page (https://www.sf.net/docs/A04) will be updated as soon as the new infrastructure is rolled out. In the interim, please read the important information provided below to learn about how these changes will affect your project.


Summary of changes, effective 2006-05-12:

1. Hostname for CVS service

Old: cvs.sourceforge.net

New: PROJECT_UNIX_NAME.cvs.sourceforge.net

This change will require new working copies to be checked out of all repositories (so control files in the working copy will point to the right place). We will be updating the instructions we supply, but instructions that your team has written within documentation, etc. will need to be updated.

cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/gaim co gaim

would be changed to

cvs -d:pserver:anonymous@gaim.cvs.sourceforge.net:/cvsroot/gaim co gaim



2. ViewCVS

We are moving from ViewCVS to its successor, ViewVC. ViewVC is currently in use for our Subversion service.



3. Sync delay

Old: CVS pserver, tarballs and ViewCVS provided against a separate server which is a minimum of three hours behind developer CVS.

New: ViewVC will be provided against developer CVS (it will be current). CVS pserver will be provided against a secondary server (not developer server) with a maximum expected delay of two hours.

Follow-up work is planned (this infrastructure takes us 80% of the way) to essentially eliminate the sync delay.



4. Read-only rsync service

As a new service offering, we are now providing read-only rsync access against developer CVS. This allows projects to efficiently make on-demand backups of their entire CVS repository.

All projects should be making regular backups of their CVS repository contents using this service.



5. Nightly tarball service

Nightly tarball service is being dropped in lieu of read-only rsync service. Projects which currently depend on nightly tarballs for repository backups will need to begin using rsync to make a backup copy of their repository contents.

We see this as a major functional improvement. For a number of reasons, tarballs have fallen out of sync with the data in the repository at times in the past few years. Tarballs required a substantial amount of additional disk, and I/O to generate. The move to read-only rsync allows backups to be produced on-demand, with an update frequency chosen by the project.



6. Points of failure

In the past, developer CVS service for all projects was provided from a single host. CVS pserver service was provided from individual backend heads based on a split of the data.

Under our new design, developer CVS and most of our CVS-related services are provided from one of ten CVS hosts (count subject to increase with growth). Each host is independent, and makes a backup copy of the repository data of another host (which is used to provide the pserver
CVS service).

Failure of a single host will impact only the availability of data on that host. Since the data is split among a larger number of hosts, the size of data impacted by an individual host outage is substantially smaller, and the time required for us to restore service will be substantially shorter.

This rapid architecture change has been made possible specifically using the research we performed for our recent launch of Subversion service. We've applied our best practices, produced a substantial amount of internal documentation, and kept an eye toward maintainability. This effort has allowed us to deploy this new architecture quickly once hardware was received, and will permit us to quickly scale this service horizontally as growth and demand requires.



Many other minor improvements have also been made to improve the service offering and make it less trouble-prone. The most important of which are
listed above. For a full description of the new service offering, and for information on how to use the services described above, please refer to the site documentation for the CVS service after the service has been launched: https://www.sf.net/docs/E04 (https://email.t-online.de/index.php?ctl=dereferer&to=aHR0cHM6Ly93d3cuc2YubmV0L2RvY3MvRTA0)


Thank you,

The SourceForge.net Team
-----------------------------------------------------------

Impaler[WrG]
May 12, 2006, 09:41 PM
Well I am glad to hear that things are on the mend, though it seems our timing was terrible (starting to use SF just before it crashed, burned and began an upgrade cycle aka second round of crashing and burning). If these changes turn out to be for the good in the long term then thats great and I would like too see use use the SF CVS in the future. But we must be prepared (even more so now then ever) to do without it for the first version of the CCP. All haste should be taken getting that Spoiled Fruit Server up (hint hint). If absolutly nessary we will have to just designate some unlucky person to mail our source code to and have said person merge it all themselves.

12monkeys
May 13, 2006, 04:50 PM
OK, CVS is up again. You have to check out anything again by using the following root

cvs -d:pserver:<yourname>@civ4ccp.cvs.sourceforge.net:/cvsroot/civ4ccp

Also the access by web breowser is possible now (just viewing).

BUT :
all those old mess is there again. We have to clean up again. I suggest the following :
SimCutie did the only changes ight now. He DLs all his stuff. I will clean up the CVS again tomorrow night. Then SimCute can UL his changes again.

Pleae comment.

The Great Apple
May 13, 2006, 06:04 PM
Excellent. I can view it on my web browser from here, and the directory structure dows look a bit crazy.

I'll send out that PM I was going to send out when we had the CVS back up tomorrow.

Impaler[WrG]
May 13, 2006, 08:50 PM
I'm glad to see its finaly working :goodjob:

SimCutie could you skip all of that #ifdef stuff this time, your earlier code (which I see now through the web view) is full of these and their realy not not nessary or desirable as was discussed earlier. Also your tags need more descriptions, a minimum of some kind of mod identification is required even if its just "Initial Commit" or something.

SimCutie
May 14, 2006, 06:02 AM
']I'm glad to see its finaly working :goodjob:

SimCutie could you skip all of that #ifdef stuff this time, your earlier code (which I see now through the web view) is full of these and their realy not not nessary or desirable as was discussed earlier. Also your tags need more descriptions, a minimum of some kind of mod identification is required even if its just "Initial Commit" or something.
OK I will try to avoid #ifdef as when it looks too complicated. I will use comment out old code instead and insert version number in new code. I did not used version numver bacause I don't had such numberig, in parctice.
But differnciating commit number is meaningless because the number is quckely get out of sync by frequent commit and the version number in CVS is not so meaningful per se..
I will use release numbdering/identification instead.

12monkeys
May 15, 2006, 02:57 PM
Daily news from CVS :

I tried to clean it up and had only half success. I Can cleanup the directory contents, but I was not able to remove empty directories. After trying a lot and reading even more I found out that it is not possible to remove a directory with CVS. It can only be removed at your local harddisk but not from the repository.

That means, we allways will see all those directories at the CVS repository when using the browser view. Because really I don't like that I did update our support request on SF and asked them to reset the repository as soon as possible.

So, be warned ! It could happen each day, that the repo is reset. As soon as this happens, I will upload the files again. After discussing it with SimCutie in the chat from saturday night (or sunday morning for him), I will do this as follows:

Directory structure :

Root:
civ4ccp_v161

Subdirectories :
Assets
CvGameCoreDLL

In the subdir Assets, the Python and XML subdirectories from standard Assets folder from version 1.61 will be uploaded including their subdirs. They will be uploaded with version number 1.1.1.1 and last modification date as file date. After this, we will delete them again, so that they became the version 1.1.1.2. As soon as we upload a file we modied, it will get the version 1.1.1.3. The advantage is, that we can now compare the version 1.1.1.1 with 1.1.1.3 and CVS will disaply us the changed lines.
If we need other subdirectories from Assets in the future we can add them at any time.

In the subdir CvGameCoreDLL we will upload the SDK files from the SDK directory CvGameCoreDLL_v161\CvGameCoreDLL. There will be no subdirectories. The files will aslo be uploaded with version 1.1.1.1 and file modification date as file data.

12m

SimCutie
May 15, 2006, 08:30 PM
Assets in Assets_v161 folder? Rather confusing..
Old Top dir name looks good ( civ4ccp_161 ).
I would like to suggest one of following structures.
I prefer former because of shallow tree depth.


Assets_v161 -+---- Python
|
+---- XML
|
+--CvGameCoreDLL -- *.cpp
|
+--- CvGameCore.dll (single binary file)

civ4ccp_v161 -+-- Assets ----+-- Python
| |
| +-- XML
| |
| +-- CvGameCore.dll (single binary file)
|
+---CvGameCoreDLL -- *.cpp

Impaler[WrG]
May 15, 2006, 10:35 PM
I like the second structure best

civ4ccp_v161 -+-- Assets ----+-- Python
| |
| +-- XML
| |
| +-- CvGameCore.dll (single binary file)
|
+---CvGameCoreDLL -- *.cpp

you can grab the top level folder of SF and just drag-n-drop the Assets subfolder into a mod to play it without renaming.

SimCutie
May 15, 2006, 11:46 PM
2nd structure is also looks good to me.
It has same structure with game installation folder.

@12m:
If CVS management gives you too much burden or headache,
I am willing to take the responsibility. Think about it..

12monkeys
May 16, 2006, 01:41 AM
Assets in Assets_v161 folder? Rather confusing..
Old Top dir name looks good ( civ4ccp_161 ).


Ooops! Sorry, my fault. Of course if should be civ4ccp_v161. I corrected my post.

12monkeys
May 16, 2006, 01:50 AM
2nd structure is also looks good to me.
It has same structure with game installation folder.

@12m:
If CVS management gives you too much burden or headache,
I am willing to take the responsibility. Think about it..

I agree with second structure as well.

@SimCutie : that'll be fine. Its not the headache, its the time I have to spend with it. CVS became a very time consuming activitiy the last days/weeks. Although I'm sure it'll stop as soon as its running fine, I would very much appreciate if you take reponsibility for CVS.
Also I have to travel for buisness the next 2 days and will be offline anyway. Therefore I think its best for the project if you take it over. You seem to have some experiences with CVS anyway, which will be bonus as well.
If you need some information or help, just contact me.

SimCutie
May 16, 2006, 04:45 PM
I agree with second structure as well.

@SimCutie : that'll be fine. Its not the headache, its the time I have to spend with it. CVS became a very time consuming activitiy the last days/weeks. Although I'm sure it'll stop as soon as its running fine, I would very much appreciate if you take reponsibility for CVS.
Also I have to travel for buisness the next 2 days and will be offline anyway. Therefore I think its best for the project if you take it over. You seem to have some experiences with CVS anyway, which will be bonus as well.
If you need some information or help, just contact me.

OK. Just PM / E-mail me if you have some CVS maintenamce work to do.
I will do the work for you. Is there any work to to on CVS while you are on business travel?
I will check mail/PM more frequently while you are away from home.

Opinion on top level directry:
How about to drop "_V161" version numbe surffix and keep next expansion/patch version of SDK in same place?

When new expansion/patch and new version of SDK come out we just update major number
( ex) 1.xx.xx.xx -> 2.xx.xx.xx ) and keep the new source in same place... with tag like "Firaxis_patch_188", "Warlord_209"
Then we can compare / view difference of the old 1.61 (Firaxis) version source and new patch /expansion version by Firaxis in same place with CVS web.

.

Impaler[WrG]
May 16, 2006, 07:42 PM
I vote for keeping the Suffix and creating a new modual when the game is patched next by Firaxis. That might occur before the expantion or with it, if its the later they will also release a new SDK download which we will need to integrate into the project. In any event we will need to destinquish between them and keep things seperate.

Do we have an ETA on getting all the garbage that in their now removed so we can finaly commit changes and get this first release moving?

SimCutie
May 16, 2006, 09:49 PM
']I vote for keeping the Suffix and creating a new modual when the game is patched next by Firaxis. That might occur before the expantion or with it, if its the later they will also release a new SDK download which we will need to integrate into the project. In any event we will need to destinquish between them and keep things seperate.

Do we have an ETA on getting all the garbage that in their now removed so we can finaly commit changes and get this first release moving?
That can be handled with CVS branch. we keep always most up-to-date version ( by new Firaxis SDK release or patch ) as MAIN branch. And continue other old version as non-main branch.
One branch per new SDK release.
And we can easily continue maintain old version while starting new job on new version of SDK. "Keep things separate" LOGICALLY is what the CVS are designed for. Do you know what the "CVS" stands for? it is "CONCURRENT Versions System".
We can easily check out and "current" version of old branchs (1.61). or any branch other than MAIN.


Keeping things separete PHYSICALLY will reduce most benefits of using CVS anyway. Proper use of CVS Tag, branch, release will eliminate any need of keeping them physically separate.
Even larger projects with multiple team or project can work with single repository of CVS. why we should have separate repository for project of this magnitude?
What is the exact advantage of keep separate repository for each release? I can hardly figure out.
I strongly recommand you all to carefully consider why we use CVS, anyway. instead of multiple ZIP/tar files.

Impaler[WrG]
May 16, 2006, 10:22 PM
I dont know much about Branching under CVS, I just know Commiting and Checking out which is what I want to do. If your going to be able to handle these issues then by all means create such things. All I want is to be able to get in their and make changes without a big hastle and to be able to checkout and run the changes easily. Having Assests folder with XML Python and Binary in it should do that. I will leave it in your capable hands and just bug you if their is a problem. SimCutie are you now going to be cleaning up the CVS and making it ready for Commits, this is most urgent if we are to make release date even remotly on time.

12monkeys
May 18, 2006, 04:33 PM
I just got an email from sourceforge support, that they now start to work on our request. Its an automatic generated email, but maybe we do have a clean CVS repository within the next hours.

12m

Impaler[WrG]
May 20, 2006, 12:14 PM
I see that our felow moders over at the Civ4AC project have their's up and running, they are using Subversion rather then CVS (what adsactly is the difference?) perhaps we should request some help from them.

SimCutie
May 20, 2006, 12:34 PM
Why does it take so long time?
A week has passed without any progress in CVS problem
Resetting whole CVS directory won't take more then a minutes for SF staff.
How about we start our project in other name in sourceforge?
I think that it would be faster than waiting CVS problems fixed.

SpoiledFruits's backup server does not seems to be working, too.
I can not connecte to ftp server and CVS server of SpoiledFruits.
( I was able to connect to it yesterday with FTP but not CVS)
I guess that we can not use it as production use for the time being,
before it get stable..


.

The Great Apple
May 20, 2006, 01:04 PM
Why does it take so long time? Resetting whole CVS directory won't take more then a minutes.
How about we start our project in other name in sourceforge?
I think that it would be faster than waiting CVS problems fixed.
I heard they had a 2 week wait at the moment for new projects...

SpoiledFruits's backup server does not seems to be working, too.
I can not connecte to ftp server and CVS server of SpoiledFruits.
I guess that we can not use it as production use for the time being,
before it get stable..
Me neither.

Can you give us a vague guess at a maximum and minimum time that it'll take before it's working properly SpoiledFruit?

If all else fails we could always post our code up on the forum, combine and compile, and then stick it all on a download server.

12monkeys
May 24, 2006, 08:01 AM
Good news! The CVS at SF has just been reset.

SimCutie, your last mail to them may had lead to a reaction.

12m

Impaler[WrG]
May 24, 2006, 10:49 AM
Yep Its all cleaned out, better late then never.

Now I sugjest we be very carfull this time around, SimCutie can set up a nice file structure and we might also consider using a version of Spoiled Fruits individual folders for uploading of incomplete Code, I realy like this idea because it will improve our peer review ability tremendously.

I recomend each user folder have a structure that parralels the main trunk, a directory that servers as a ready to play Mod "NAME_MOD" containing binary and any other assets nessary to activate the mods effects. This alows us to keep a copy of each other changes under our Mod folders and fire them up on a moments notice. Another seperate directory containing your hopefully ready to compile code which again can be checked out to help each other resolve compile problems.

I hope we can get this set up later today/night as I am still unable to get into SpoiledFruits server (despite using FileZilla) and I realy need a place to get my code up for peer-review and intigration. Now having said that I remind everyone to hold your horses and not try to set it up themsevles, let Sim our designated CVS master set up the directory structure so we dont end up with another mess like lasttime.

SpoiledFruit
May 24, 2006, 06:24 PM
Sorry but its not you. The power went out here and I had to reset all my servers. It should be up now but with a different IP. The IP is (Removed)

Impaler[WrG]
May 25, 2006, 10:46 PM
I see Sim has uploaded 1.61 blank to the CVS, I could try making a commit to it but I think its a nice idea to use the individual folders as well. Perhaps a system with individual folders and testing/unstable stuff at SpoilFruits server an our "should be working properly" stuff on the CVS. Lets discuss. In any event I will be Pasting on the Forum tonight.

SimCutie
May 25, 2006, 11:49 PM
I will post steps for doing experimental version branching tonight.
Individual developer will have his/her own branch for development in CVS.
Each branch will be merged periodically on agreed date on Integration_test branch,.
On final release, all branch vesion will be meged and tested,
Then, it will be finally meged as new version into main (HEAD) branch for release.
After release, all developer will have experimental branch again from the released version.
This process will be repeated for each release.

SimCutie
May 26, 2006, 09:24 AM
Step for expermental branch and integration.

Now all original Firaxis version sources are uploaded in CVS. as 1.1.1.1. But please don't check-in your version into CVS HEAD(=main) branch, yet.

I have create a special branch named "dev".
First, all individual develoeper should create individual developer's branch from the "dev" branch. You should create individual developer branch from this "dev" tagged branch. Use yor login name as new branch name. I will use "yourname" as example. First letter of name should be alphabet.

I put few other tag for easier refernce of the files. Currently, we have three existing tag other than default "HEAD" tag.
"Firaxis_161"/"Base_161" is original virgin version with SDK and all file "Assets" files.
"Start" is tag is put on the version with all "Assets" files removed.

Then check out the "yourname". When you check out, you shoud specify the tag "yourname" as barch tag to check out. If you don't not specify the tag, main branch will be checkout which is not what we want. If checked out correctly, You will have a civ4ccp directory. In WinCVS or in Tortoise CVS, you can see that each file and directories are all marked as having "yourname" tag.

Next, you start to edit the file, and when all editing on the sources are finished, commit the files to "yourname" branch. Then it will have version number 1.1.1.2 and have "yourname" tag (sticky). If You finished all you experiment in 1.1.1.8 version, now it is time to start merge. You can start merge anytime when you feel it time to show your result to other developer. You are recommended to do merge periodically to show your current work-in-progress.
Specify source branch as "yourname" and destination branch as "dev" and start merge.
If there is source line modified by two developer (conflict), You have to manually edit and resolve the conflict. If you finished merging process then "dev" branch will have version number 1.2. It will be tested. We repeat such process few times until merging all develper's branchs are merged into single "dev" branch.

Now "dev" branch has good copy of fully tested source to be released in public.
Then I will merge the finished "dev" branch into main "HEAD" branch. I will put tag "release" on the finished version in HEAD branch. Then we announce the release to public and all other Civ player/moder will start to download the "release" version.

This is full cycle of a relase version. After the public release, another cycle will begain.
Again, each developer will create individual "yourname" from main "release" or "dev" version, and start develpe next version based on the previous release version. This process will be repeated for each new release.
If Firaxis announce new release of the SDK, then the version also merged into the "vendor" branchand labeled as "Firaxis_XXX" for our reference.

If you have question on this matter, don't hesitate to ask question.

The Great Apple
Jun 02, 2006, 04:06 PM
Sorry but its not you. The power went out here and I had to reset all my servers. It should be up now but with a different IP. The IP is <IP Removed>You can probably remove this IP now, else you'll forget it's here...

SpoiledFruit
Jun 02, 2006, 06:35 PM
Thanks for reminding me. IP removed.

SimCutie
Jun 03, 2006, 08:41 AM
Source version commited by Impaler 2days ago is moved to "dev" branch.
To checkout the version, specify "dev" as version tag to check out.
And please do futher work on the "dev" branch.
If it is finished it will be merged into HEAD branch.
Please, don't commit modifiied source in MAIN branch before it is ready to release in public.