1. We have added the ability to collapse/expand forum categories and widgets on forum home.
    Dismiss Notice
  2. Photobucket has changed its policy concerning hotlinking images and now requires an account with a $399.00 annual fee to allow hotlink. More information is available at: this link.
    Dismiss Notice
  3. All Civ avatars are brought back and available for selection in the Avatar Gallery! There are 945 avatars total.
    Dismiss Notice
  4. To make the site more secure, we have installed SSL certificates and enabled HTTPS for both the main site and forums.
    Dismiss Notice
  5. Civ6 is released! Order now! (Amazon US | Amazon UK | Amazon CA | Amazon DE | Amazon FR)
    Dismiss Notice
  6. Dismiss Notice
  7. Forum account upgrades are available for ad-free browsing.
    Dismiss Notice

CVS issues

Discussion in 'Civ4 - Community Enchancement Project' started by 12monkeys, May 12, 2006.

  1. 12monkeys

    12monkeys Chieftain

    Joined:
    Nov 24, 2003
    Messages:
    440
    Location:
    Germany, Europe
    Ok, I started this thread here to have a single thread for CVS issues. I would like to begin, that the CVS at Source Forge is down since 3 days. This is not only a problem for our projects its a problem for amyn projects at SF. Somewhere else, I told how frustrating it is that we don't get any infroamtion from Source Forge. Well, they did hear me. I just received an email from SF about the problem :

    -----------------------------------------------------------
    Greetings,

    You are receiving this mail because you are a project admin for a SourceForge.net-hosted project. One of our primary services, CVS, suffered a series of interrelated, critical hardware failures in recent weeks. We understand how frustrating this CVS outage must be to you and your users; however, our top priority remains preservation of the integrity of your data.

    The series of CVS hardware failures prompted us to expedite the deployment of planed improvements to our CVS infrastructure, drawing upon much of the knowledge that we gained from our Subversion deployment. Our improved CVS service architecture, which we plan to deploy tomorrow afternoon (2006-05-12), will offer greater performance and stability and will eliminate several single points of failure.

    The Site Status page (https://www.sf.net/docs/A04) will be updated as soon as the new infrastructure is rolled out. In the interim, please read the important information provided below to learn about how these changes will affect your project.


    Summary of changes, effective 2006-05-12:

    1. Hostname for CVS service

    Old: cvs.sourceforge.net

    New: PROJECT_UNIX_NAME.cvs.sourceforge.net

    This change will require new working copies to be checked out of all repositories (so control files in the working copy will point to the right place). We will be updating the instructions we supply, but instructions that your team has written within documentation, etc. will need to be updated.

    cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/gaim co gaim

    would be changed to

    cvs -d:pserver:anonymous@gaim.cvs.sourceforge.net:/cvsroot/gaim co gaim



    2. ViewCVS

    We are moving from ViewCVS to its successor, ViewVC. ViewVC is currently in use for our Subversion service.



    3. Sync delay

    Old: CVS pserver, tarballs and ViewCVS provided against a separate server which is a minimum of three hours behind developer CVS.

    New: ViewVC will be provided against developer CVS (it will be current). CVS pserver will be provided against a secondary server (not developer server) with a maximum expected delay of two hours.

    Follow-up work is planned (this infrastructure takes us 80% of the way) to essentially eliminate the sync delay.



    4. Read-only rsync service

    As a new service offering, we are now providing read-only rsync access against developer CVS. This allows projects to efficiently make on-demand backups of their entire CVS repository.

    All projects should be making regular backups of their CVS repository contents using this service.



    5. Nightly tarball service

    Nightly tarball service is being dropped in lieu of read-only rsync service. Projects which currently depend on nightly tarballs for repository backups will need to begin using rsync to make a backup copy of their repository contents.

    We see this as a major functional improvement. For a number of reasons, tarballs have fallen out of sync with the data in the repository at times in the past few years. Tarballs required a substantial amount of additional disk, and I/O to generate. The move to read-only rsync allows backups to be produced on-demand, with an update frequency chosen by the project.



    6. Points of failure

    In the past, developer CVS service for all projects was provided from a single host. CVS pserver service was provided from individual backend heads based on a split of the data.

    Under our new design, developer CVS and most of our CVS-related services are provided from one of ten CVS hosts (count subject to increase with growth). Each host is independent, and makes a backup copy of the repository data of another host (which is used to provide the pserver
    CVS service).

    Failure of a single host will impact only the availability of data on that host. Since the data is split among a larger number of hosts, the size of data impacted by an individual host outage is substantially smaller, and the time required for us to restore service will be substantially shorter.

    This rapid architecture change has been made possible specifically using the research we performed for our recent launch of Subversion service. We've applied our best practices, produced a substantial amount of internal documentation, and kept an eye toward maintainability. This effort has allowed us to deploy this new architecture quickly once hardware was received, and will permit us to quickly scale this service horizontally as growth and demand requires.



    Many other minor improvements have also been made to improve the service offering and make it less trouble-prone. The most important of which are
    listed above. For a full description of the new service offering, and for information on how to use the services described above, please refer to the site documentation for the CVS service after the service has been launched: https://www.sf.net/docs/E04


    Thank you,

    The SourceForge.net Team
    -----------------------------------------------------------
     
  2. Impaler[WrG]

    Impaler[WrG] Civ4:Col UI programmer

    Joined:
    Dec 5, 2005
    Messages:
    1,750
    Location:
    Vallejo, California
    Well I am glad to hear that things are on the mend, though it seems our timing was terrible (starting to use SF just before it crashed, burned and began an upgrade cycle aka second round of crashing and burning). If these changes turn out to be for the good in the long term then thats great and I would like too see use use the SF CVS in the future. But we must be prepared (even more so now then ever) to do without it for the first version of the CCP. All haste should be taken getting that Spoiled Fruit Server up (hint hint). If absolutly nessary we will have to just designate some unlucky person to mail our source code to and have said person merge it all themselves.
     
  3. 12monkeys

    12monkeys Chieftain

    Joined:
    Nov 24, 2003
    Messages:
    440
    Location:
    Germany, Europe
    OK, CVS is up again. You have to check out anything again by using the following root

    cvs -d:pserver:<yourname>@civ4ccp.cvs.sourceforge.net:/cvsroot/civ4ccp

    Also the access by web breowser is possible now (just viewing).

    BUT :
    all those old mess is there again. We have to clean up again. I suggest the following :
    SimCutie did the only changes ight now. He DLs all his stuff. I will clean up the CVS again tomorrow night. Then SimCute can UL his changes again.

    Pleae comment.
     
  4. The Great Apple

    The Great Apple Big Cheese

    Joined:
    Mar 24, 2002
    Messages:
    3,361
    Location:
    Oxford, England
    Excellent. I can view it on my web browser from here, and the directory structure dows look a bit crazy.

    I'll send out that PM I was going to send out when we had the CVS back up tomorrow.
     
  5. Impaler[WrG]

    Impaler[WrG] Civ4:Col UI programmer

    Joined:
    Dec 5, 2005
    Messages:
    1,750
    Location:
    Vallejo, California
    I'm glad to see its finaly working :goodjob:

    SimCutie could you skip all of that #ifdef stuff this time, your earlier code (which I see now through the web view) is full of these and their realy not not nessary or desirable as was discussed earlier. Also your tags need more descriptions, a minimum of some kind of mod identification is required even if its just "Initial Commit" or something.
     
  6. SimCutie

    SimCutie Chieftain

    Joined:
    Dec 10, 2005
    Messages:
    197
    OK I will try to avoid #ifdef as when it looks too complicated. I will use comment out old code instead and insert version number in new code. I did not used version numver bacause I don't had such numberig, in parctice.
    But differnciating commit number is meaningless because the number is quckely get out of sync by frequent commit and the version number in CVS is not so meaningful per se..
    I will use release numbdering/identification instead.
     
  7. 12monkeys

    12monkeys Chieftain

    Joined:
    Nov 24, 2003
    Messages:
    440
    Location:
    Germany, Europe
    Daily news from CVS :

    I tried to clean it up and had only half success. I Can cleanup the directory contents, but I was not able to remove empty directories. After trying a lot and reading even more I found out that it is not possible to remove a directory with CVS. It can only be removed at your local harddisk but not from the repository.

    That means, we allways will see all those directories at the CVS repository when using the browser view. Because really I don't like that I did update our support request on SF and asked them to reset the repository as soon as possible.

    So, be warned ! It could happen each day, that the repo is reset. As soon as this happens, I will upload the files again. After discussing it with SimCutie in the chat from saturday night (or sunday morning for him), I will do this as follows:

    Directory structure :

    Root:
    civ4ccp_v161

    Subdirectories :
    Assets
    CvGameCoreDLL

    In the subdir Assets, the Python and XML subdirectories from standard Assets folder from version 1.61 will be uploaded including their subdirs. They will be uploaded with version number 1.1.1.1 and last modification date as file date. After this, we will delete them again, so that they became the version 1.1.1.2. As soon as we upload a file we modied, it will get the version 1.1.1.3. The advantage is, that we can now compare the version 1.1.1.1 with 1.1.1.3 and CVS will disaply us the changed lines.
    If we need other subdirectories from Assets in the future we can add them at any time.

    In the subdir CvGameCoreDLL we will upload the SDK files from the SDK directory CvGameCoreDLL_v161\CvGameCoreDLL. There will be no subdirectories. The files will aslo be uploaded with version 1.1.1.1 and file modification date as file data.

    12m
     
  8. SimCutie

    SimCutie Chieftain

    Joined:
    Dec 10, 2005
    Messages:
    197
    Assets in Assets_v161 folder? Rather confusing..
    Old Top dir name looks good ( civ4ccp_161 ).
    I would like to suggest one of following structures.
    I prefer former because of shallow tree depth.

    Code:
    Assets_v161 -+---- Python
                 |
                 +---- XML
                 |           
                 +--CvGameCoreDLL  -- *.cpp
                 |
                 +--- CvGameCore.dll (single binary file)
    
    civ4ccp_v161 -+-- Assets ----+-- Python
                  |              |
                  |              +-- XML
                  |              |
                  |              +--  CvGameCore.dll (single binary file)
                  |
                  +---CvGameCoreDLL  -- *.cpp
    
     
  9. Impaler[WrG]

    Impaler[WrG] Civ4:Col UI programmer

    Joined:
    Dec 5, 2005
    Messages:
    1,750
    Location:
    Vallejo, California
    I like the second structure best

    civ4ccp_v161 -+-- Assets ----+-- Python
    | |
    | +-- XML
    | |
    | +-- CvGameCore.dll (single binary file)
    |
    +---CvGameCoreDLL -- *.cpp

    you can grab the top level folder of SF and just drag-n-drop the Assets subfolder into a mod to play it without renaming.
     
  10. SimCutie

    SimCutie Chieftain

    Joined:
    Dec 10, 2005
    Messages:
    197
    2nd structure is also looks good to me.
    It has same structure with game installation folder.

    @12m:
    If CVS management gives you too much burden or headache,
    I am willing to take the responsibility. Think about it..
     
  11. 12monkeys

    12monkeys Chieftain

    Joined:
    Nov 24, 2003
    Messages:
    440
    Location:
    Germany, Europe
    Ooops! Sorry, my fault. Of course if should be civ4ccp_v161. I corrected my post.
     
  12. 12monkeys

    12monkeys Chieftain

    Joined:
    Nov 24, 2003
    Messages:
    440
    Location:
    Germany, Europe
    I agree with second structure as well.

    @SimCutie : that'll be fine. Its not the headache, its the time I have to spend with it. CVS became a very time consuming activitiy the last days/weeks. Although I'm sure it'll stop as soon as its running fine, I would very much appreciate if you take reponsibility for CVS.
    Also I have to travel for buisness the next 2 days and will be offline anyway. Therefore I think its best for the project if you take it over. You seem to have some experiences with CVS anyway, which will be bonus as well.
    If you need some information or help, just contact me.
     
  13. SimCutie

    SimCutie Chieftain

    Joined:
    Dec 10, 2005
    Messages:
    197
    OK. Just PM / E-mail me if you have some CVS maintenamce work to do.
    I will do the work for you. Is there any work to to on CVS while you are on business travel?
    I will check mail/PM more frequently while you are away from home.

    Opinion on top level directry:
    How about to drop "_V161" version numbe surffix and keep next expansion/patch version of SDK in same place?

    When new expansion/patch and new version of SDK come out we just update major number
    ( ex) 1.xx.xx.xx -> 2.xx.xx.xx ) and keep the new source in same place... with tag like "Firaxis_patch_188", "Warlord_209"
    Then we can compare / view difference of the old 1.61 (Firaxis) version source and new patch /expansion version by Firaxis in same place with CVS web.

    .
     
  14. Impaler[WrG]

    Impaler[WrG] Civ4:Col UI programmer

    Joined:
    Dec 5, 2005
    Messages:
    1,750
    Location:
    Vallejo, California
    I vote for keeping the Suffix and creating a new modual when the game is patched next by Firaxis. That might occur before the expantion or with it, if its the later they will also release a new SDK download which we will need to integrate into the project. In any event we will need to destinquish between them and keep things seperate.

    Do we have an ETA on getting all the garbage that in their now removed so we can finaly commit changes and get this first release moving?
     
  15. SimCutie

    SimCutie Chieftain

    Joined:
    Dec 10, 2005
    Messages:
    197
    That can be handled with CVS branch. we keep always most up-to-date version ( by new Firaxis SDK release or patch ) as MAIN branch. And continue other old version as non-main branch.
    One branch per new SDK release.
    And we can easily continue maintain old version while starting new job on new version of SDK. "Keep things separate" LOGICALLY is what the CVS are designed for. Do you know what the "CVS" stands for? it is "CONCURRENT Versions System".
    We can easily check out and "current" version of old branchs (1.61). or any branch other than MAIN.


    Keeping things separete PHYSICALLY will reduce most benefits of using CVS anyway. Proper use of CVS Tag, branch, release will eliminate any need of keeping them physically separate.
    Even larger projects with multiple team or project can work with single repository of CVS. why we should have separate repository for project of this magnitude?
    What is the exact advantage of keep separate repository for each release? I can hardly figure out.
    I strongly recommand you all to carefully consider why we use CVS, anyway. instead of multiple ZIP/tar files.
     
  16. Impaler[WrG]

    Impaler[WrG] Civ4:Col UI programmer

    Joined:
    Dec 5, 2005
    Messages:
    1,750
    Location:
    Vallejo, California
    I dont know much about Branching under CVS, I just know Commiting and Checking out which is what I want to do. If your going to be able to handle these issues then by all means create such things. All I want is to be able to get in their and make changes without a big hastle and to be able to checkout and run the changes easily. Having Assests folder with XML Python and Binary in it should do that. I will leave it in your capable hands and just bug you if their is a problem. SimCutie are you now going to be cleaning up the CVS and making it ready for Commits, this is most urgent if we are to make release date even remotly on time.
     
  17. 12monkeys

    12monkeys Chieftain

    Joined:
    Nov 24, 2003
    Messages:
    440
    Location:
    Germany, Europe
    I just got an email from sourceforge support, that they now start to work on our request. Its an automatic generated email, but maybe we do have a clean CVS repository within the next hours.

    12m
     
  18. Impaler[WrG]

    Impaler[WrG] Civ4:Col UI programmer

    Joined:
    Dec 5, 2005
    Messages:
    1,750
    Location:
    Vallejo, California
    I see that our felow moders over at the Civ4AC project have their's up and running, they are using Subversion rather then CVS (what adsactly is the difference?) perhaps we should request some help from them.
     
  19. SimCutie

    SimCutie Chieftain

    Joined:
    Dec 10, 2005
    Messages:
    197
    Why does it take so long time?
    A week has passed without any progress in CVS problem
    Resetting whole CVS directory won't take more then a minutes for SF staff.
    How about we start our project in other name in sourceforge?
    I think that it would be faster than waiting CVS problems fixed.

    SpoiledFruits's backup server does not seems to be working, too.
    I can not connecte to ftp server and CVS server of SpoiledFruits.
    ( I was able to connect to it yesterday with FTP but not CVS)
    I guess that we can not use it as production use for the time being,
    before it get stable..


    .
     
  20. The Great Apple

    The Great Apple Big Cheese

    Joined:
    Mar 24, 2002
    Messages:
    3,361
    Location:
    Oxford, England
    I heard they had a 2 week wait at the moment for new projects...
    Me neither.

    Can you give us a vague guess at a maximum and minimum time that it'll take before it's working properly SpoiledFruit?

    If all else fails we could always post our code up on the forum, combine and compile, and then stick it all on a download server.
     

Share This Page