Multi-Developer-Setup with SVN-VC and remote test-servers for each developer. Best practices?

H

4

5

I would like to have some input on how a professional development setup with the following requirements might look like.

several PHP-developers (say PHP)
each developer belongs to one group
each group has one team-leader who delegates tasks
each developer works on one Windows 7 machine
and developes either with NetBeans or Eclipse
each developer 'owns' one virtual test-server where he can run the code
the VCS in use is SVN
there is a staging server where the product is ultimately tested before it gets released/deployed

I gave some specific technology to not be too abstract and b/c I also would be interested in concrete suggestions for plug-ins etc.

There are several questions coming to my mind in that setup.

1) So every developer will work on personal branch.

2) This branch is checked out in a working copy.

Now ... this working copy is edited locally on the PC with the dev's IDE and executed/tested on the server.

What would be in that case the best/usual way to do that? I mean - how do you get your edited code on the server without causing too much overhead?

Would the dev have the code on his local disk at all? Or would it be better to have the IDE write on the remote virtual server through a tunnel or via a specific protocol?

3) Every day a dev will commit his work into his personal branch which resides in a central repository.

Is there a best practice on where the repository is supposed to be located? A seperate server?

4) Then after a dev finished his task either s/he or the team-leader merges the new code into the respective main-branch or trunk.

The most confusing part is about what I wrote between 2) and 3). Because so far I only worked with a local server. For example a VM with a server running a code which is located in a shared folder so I will be able to edit it directly. I'm not sure how to bridge the gap efficiently when the server is now actually remote. Efficiently means not having to upload manually via FTP for example.

Also external sources or book recommendations are welcome.

edit

My question/s is/are aiming at a quasi-standard / best-practice. I think this is pretty much a standard development scenario so there must be a 'usual' solution.

edit 2

Okay ... so lets try with a picture: the setup

V is the virtual test-server for one or more developers D. C and C' are the two code-versions. They should be kept as identical as possible.

Two solutions come to my mind:

1 : Edit C, then upload it to C', then execute C', then commit C.

2 : No C existant. Just C' which is edited through some tunnel technology and executed and committed.

My guts tell me both solutions are semi-optimal. So what would be "professional" / most efficient / fastest / most convenient / most friction-less / least error-prone / best practice / industry standard?

Any questions?

Hwahwan answered 8/4, 2011 at 13:47 Comment(5)

Does an SVN access exist on the Server where to put the result onto? Means can the server checkout from SVN? – Manipur 8/4, 2011 at 15:4

Well, I think there should be no restrictions. If you got the credentials you could connect to the SVN-server/-repo from the dev's personal test-server as well as from his/her machine. I am not sure what would make more sense. – Hwahwan 8/4, 2011 at 15:8

I would suggest that on the test-server a branch will be put online via svn checkout/svn update to make a more Q&A test. – Manipur 8/4, 2011 at 15:11

you mean checking out a working copy onto the dev's test server? how would you edit the working copy with your IDE? – Hwahwan 8/4, 2011 at 15:15

The developers don't develop locally and than checkin and the real test will be done on the test-server? – Manipur 8/4, 2011 at 17:15

P

1

I'm sure everyone has different ways of doing things but here are my thoughts.

"Best Practice" is probably "Continous Integration" ie each developer doesn't have their own branch but checks in to a common development branch. This forces them to handle conflicts and coordinate with each other early and often to avoid the lead developer from managing a huge train wreck merge later down the road. Take a look at cruisecontrol if you really want to go that route.

The best way is if they have a local apache web server and full php stack. You can use the Zend_Server community edition to get up and running on windows fast. Most standard php code will run just fine on both Windows and Linux, but if you are doing lots of file manipulation or cron job or cli stuff, or need memecache, etc you'll run into incompatabilities. If thats the case and the Linux only stuff is going to bite you use VMWARE or VirtualBox to run local linux instances and install the IDE inside those and just make sure they have goobs of RAM to deal with it.

Each developer needs to run a syncronize inside of Eclipse, basically an svn update, deal with any conflicts with the other developers right then and there, do local testing and commit their changes.

I setup a post_commit hook on the svn server that calls and /autobuild.php on my web server. autobuild.php runs svn update and gets the latest code changes as well as does any chown or chmod file permissions stuff and resets any server specific config files config.php. Its a little tricky to get it setup so that the apache user can run svn update, but once you do your beta/testing server always has the latest committed code. CruseControl, and several others can also help you do this sort of thing and add unit testing, etc

Now your Lead Developer still has a job to do merging the Development Branch into the Production One, testing on the dev server, and reviewing the commits of the others and deciding how and when to push out a release, but your not putting the burden on him of resolving every conflict and merging every change.

Your developers are not ftping files or ssh remoting into servers, they just work locally in their IDE and interact with each other through svn (and email, phone, chat, etc) updating to get the new code and commiting as they finish things.

I don't see any good coming out of having a seperate branch for each developer using SVN. Merging those branches might work in Git but with SVN your lead developer will be hating life very quickly with that type of setup.

Pycno answered 13/4, 2011 at 21:43 Comment(1)

I think you should reconsider creating all those standalone servers with different configurations, it could be a real pitfall. Instead, manage the server(s) remote and hook on the filesystem/network share only (where the developer can check out code). – Creditor 29/12, 2012 at 18:58

M

3

Maybe its not of great help but GIT sounds like a perfect fit to your problems, i recommend to take a look to the GIT features. And if you got time check Linus Torvalds him self talking ablout GIT. http://www.youtube.com/watch?v=4XpnKHJAok8

Mindful answered 8/4, 2011 at 14:25 Comment(8)

Git is indeed more effective when dealing with several branches, merging back to trunk with svn can be problematic – Zelazny 8/4, 2011 at 14:32

Git is no option here as development is essentially centralized. And Git wouldn't solve the central problem I stated in the OP. – Hwahwan 8/4, 2011 at 14:38

git/dvcs doesn't prevent centralized development? It only opens up alternative workflows, it doesn't prevent ones available via svn. – Carbo 11/4, 2011 at 2:4

I know, but Git is not what I need. – Hwahwan 11/4, 2011 at 8:59

@Raffael1984 I think maybe you do not fully understand what GIT is capable of doing – Lampyrid 11/4, 2011 at 19:27

Git can do everything SVN can do and much more ... nonetheless, this is of no importance regarding my question. – Hwahwan 11/4, 2011 at 21:10

Look Git is nice, but it's not the cure for the soggy doughnut. Git can result in many problems if you're not careful. Developers can go off on their own code lines and not merge changes back until right before deployments. Patches can be moved from location to location, and it gets very difficult to know what code line has what patch, or if a patch gets applied more than once. For a centralized development environment (most private companies), it can become a nightmare of merge hell. As a CM, I have found doing more babysitting that actual work at sites that use Git. – Acarid 15/4, 2011 at 18:57

@Raffael1984: From what you have described it is very clear that you need a DVCS. What you are trying to achieve using SVN in such an elaborate manner can be achieved easily if you can use any DVCS (Git, Mercurial, Bazaar, etc). Have you considered any DVCS (not just git) and chose SVN or you have no other option but to use SVN? – Frost 15/4, 2011 at 18:58

B

2

The standard procedure as you describe is more or less the same. I also you this approach for my team. It can also be called staged application development.

Here is how I am doing it, I use a remote SVN host (ex: assembla.com, unfuddle.com) to store all my codes. My team members store the information there on these remote svn servers. You can also buy an VPS and setup SVN there and user the same approach.

Best practices is to test locally and commit and commit as many times as you can but every commit must solve a problem or include a significant segment that adds any new feature.

Once the commit is done by everyone the lead developer then can login to the staging server via SSH using tools like PuTTY. First time the lead developer has to checkout the code into the folder where the codes are to be located. During this phase file conflict may arise if multiple developers edits same segment of a file. The lead developer should then resolve the code first and then proceed with the checkout. Once checked out, there onwards the lead developer will only need to do a svn update on the staging server to make the code up to date.

Basic idea is to get the code working on local setup then commit and update the staging for testing the application on a simulated scenario and then commit it to the live site.

There are a lot of if's and but's here which will need me to write a chapter on :) but in short this is the zest.

Tools (you can use to work under this setup): - Tortoise SVN Manager - PuTTy - NetBeans

hope it helps :)

Battologize answered 12/4, 2011 at 4:23 Comment(6)

This blog-post by me elaborates how such a setup can be made - bit.ly/cEnfgx – Battologize 12/4, 2011 at 4:27

Thanks for your input (+1). Though it doesn't touch the vital part of my question. When I understand you correctly then your devs commit to SVN after new functionality has been implemented. Before they can do so they have to run the code and on your blog you write about a local server instance. The core issue of my question is this ... how do you work when you have no local server but exclusivly a remote one. And how do you technologically bridge the network-gap between your IDE (to write) and the server (to execute). – Hwahwan 12/4, 2011 at 8:11

Hi, The solution to your scenario is to use tortoise SVN in your local PC/Laptop and host the code on SVN. The developer will code and commit to SVN and you can then have multiple branches of the same code in your remote server. The idea is to emulate a 2 stage development scenario on your remote server. Example: dev.yourdomain.com points to var/yourdomain.com/branches/dev/ and www.yourdomain.com points to var/yourdomain.com/trunks the concept is to initially test the code on development server then after everything is tested update the trunk folder with the latest code to make it live. – Battologize 13/4, 2011 at 7:30

You will gain better understanding if you study more on Subversion management. The code issue network-gap between your IDE and server is bridged with the commits to the SVN server that stores every update, which you can apply to server whenever you like. – Battologize 13/4, 2011 at 7:34

I am very well aware of this option. There is one aspect about it that I don't like. In this scenario, to test you have to commit. But committing should be reserved for added functionality and not be misused for testing. And TortoiseSVN certainly is no serious option for software-development purposes anyway as switching from IDE to TortoiseSVN is too much overhead. – Hwahwan 13/4, 2011 at 12:49

Hi Raffael1984, I think the overhead issue with IDE and TortoiseSVN is resolved by David W's comments. Also you may look into continuous integration as well introduced here by Brian :) – Battologize 21/4, 2011 at 5:17

A

2

I don't like working with personal branches. I worked with ClearCase for almost 15 years and even though ClearCase probably handles personal branching better than most, it was still a big pain. Even worse, personal branches encourages people to not commit their work until the last minute -- usually a day or two before a major release.

For that reason, and to force developers to stay on track with each other, I highly recommend everyone working together on a single branch (or on the trunk) as much as possible. I keep telling developers to take small bites when they make changes.

What you sound like you need is a way to automate the deployment. That is, I make changes on my local machine, and with a single command, I make sure that the server has a duplicate copy of the code. You also want the deployment to be efficient. If you change a single 2 kilobyte file of a 2 gigabyte, 10,000 file deployment, you only want to copy over that one file, not 10,000 gigabytes. For that, I would recommend you write a deployment script in Ant.

Your developers can modify files, then deploy those files via an Ant script. The developers don't have to remember what files they had updated because Ant will automatically handle that. In fact, Ant can even modify files to make sure they contain the right environment information as they get copied over. And, of course, Ant can rearrange the files if the setup on the server is different from the setup in the source repository. And both Netbeans and Eclipse can execute Ant scripts right in the IDE.

So:

Have your developers modify code on their local machine.
Run an Ant script to make sure the server and the local machine are in sync.
Test on the server.
Then, check in their changes once they're happy with the results on the server.

Someone mentioned a Continuous Build System like Jenkins. That actually would be a good idea anyway even though it doesn't solve this particular issue. Jenkins could have its own server and database. Then when you commit your code, Jenkins would update the server and run automated tests. Jenkins can then create a report. It all gets displayed on Jenkin's webpage. Plus, you can archive your deployments on Jenkins, so if you tell someone to test "Build #20", they can simply pull it off of Jenkins where its easy to find.

Acarid answered 15/4, 2011 at 18:49 Comment(2)

"And both Netbeans and Eclipse can execute Ant scripts right in the IDE." ... interesting fact! (+1) Thanks! – Hwahwan 17/4, 2011 at 15:55

Totally agree on 'no personal branches'. Branches should be functional/for functionality, adding features, fixing bugs. After completing they'll be merged into mainstream. If later somethings broken you don't want to get back to "Jason's-branch", you want to get back to the branche that the bugging features was changed. – Creditor 29/12, 2012 at 19:5