Version Control Systems

The Blackfin open source project keeps its main project in Subversion (SVN). Some older projects may use CVS instead.

Subversion is touted as a replacement for CVS.

Improvements include:

• Better file movement handling
• Better branch and merge features
• More suited to work groups
• Atomic commits
• True version History

An online book is available http://svnbook.red-bean.com.

If you are behind a firewall, and can not access the subversion port, you can replace all the below instructions with a different machine name/port. To test if this will be in the situation you are in - this is a direct connection to the internet (no firewall):

$ telnet sources.blackfin.uclinux.org 3690
Trying 204.50.165.247...
Connected to sources.blackfin.uclinux.org.
Escape character is '^]'.
( success ( 1 2 ( ANONYMOUS ) ( edit-pipeline svndiff1 absent-entries ) ) ) Connection closed by foreign host.

This is the same with a firewall:

$ telnet sources.blackfin.uclinux.org 3690
Trying 204.50.165.247...

By replacing the sources.blackfin.uclinux.org with firewall-sources.blackfin.uclinux.org:80 the following command should work:

$ svn checkout svn://firewall-sources.blackfin.uclinux.org:80/svn/u-boot/trunk u-boot

And here is telnet with the proper firewall address (behind the firewall):

$ telnet firewall-sources.blackfin.uclinux.org 80
Trying 204.50.165.246...
Connected to firewall-sources.blackfin.uclinux.org.
Escape character is '^]'.
( success ( 1 2 ( ANONYMOUS ) ( edit-pipeline svndiff1 absent-entries ) ) ) Connection closed by foreign host.

Some firewalls do content checking rather than simple port filtering. If that's the case, then attempts to do svn over port 80 will probably fail. The firewall may let through encrypted traffic on port 443 though.

$ telnet firewall-sources.blackfin.uclinux.org 443
Trying 204.50.165.246...
Connected to firewall-sources.blackfin.uclinux.org.
Escape character is '^]'.
SSH-2.0-OpenSSH_3.9p1

If you see the SSH version string (exact version information may differ), then you can probably access via the anonymous user (password is anonymous) and SSH. You will need to create an entry in your ~/.ssh/config file so it knows to go to port 443.

$ svn checkout svn+ssh://anonymous@firewall-sources.blackfin.uclinux.org/svn/u-boot/trunk u-boot

Example ~/.ssh/config entry:

Host firewall-sources.blackfin.uclinux.org
    Port 443

If all of these methods fail, then sorry, there isn't anything we can do to help you. Complain to your IT department.

The source code for subversion itself is found at its homepage http://subversion.tigris.org.

Most popular distributions already have packages available for you.

The sub directory will be called “u-boot”:

$ svn checkout svn://firewall-sources.blackfin.uclinux.org:80/svn/u-boot/trunk u-boot

or

$ svn checkout svn://sources.blackfin.uclinux.org/svn/u-boot/trunk u-boot

• find out what version you have checked out:

  $ svn info

• find out what version upstream is at:

  $ svn info svn://firewall-sources.blackfin.uclinux.org:80/svn/linux-kernel/branches/2008R1

• look at the files that were updated upstream:

  $ svn log -v svn://firewall-sources.blackfin.uclinux.org:80/svn/linux-kernel/branches/2008R1 | less

• bring your working copy up-to-date with the repository

  $ svn update

• bring your working copy up-to-date with a specific revision in the repository (this can upgrade or downgrade, depending on the revision you give):

  $ svn update -r 5316

• check the status of your local copy:

  $ svn st
        'A' Added
        'C' Conflicted
        'D' Deleted
        'I' Ignored
        'M' Modified
        'R' Replaced
        '?' item is not under version control
        '!' item is missing (removed by non-svn command) or incomplete

• see what has changed in your local version, and why:

  $ svn log -v | less

• to get unified diff output of your local changes:

  $ svn diff 

• commit a new version into the repository:

  $ svn commit

Say you want to checkout a repository from a fast local mirror and then use that checkout to commit to the original repository (so you don't need to check things out twice from different servers). SVN provides a method for switching the repository your local checkout works against.

$ svn co svn://some.fast.mirror/project/trunk project
$ cd project
$ svn switch --relocate svn://some.fast.mirror/project/trunk svn://some.slow.server.to.commit.to/project/trunk

Once you've made your checkins, you can of course switch back to the fast mirror by doing the inverse of the above switch command.

While CVS requires you to use magic commands to create tags and branches, SVN has actual directories in the top level.

So to create a branch in svn, you would:

$ svn cp trunk branch/branch_R07R1
$ svn commit -m 'add new 2007R1 branch' branch/branch_R07R1

And to tag a release:

$ svn cp branch/branch_R07R1 tags/R07R1-RC1
$ svn commit -m 'tag new R07R1-RC1 release' tags/R07R1-RC1

Since the svn connection from blackfin.uclinux.org can be slow at times, the best way of speeding this connection up, is to use a local mirror. To set up a local mirror, you can use rsync. This will provide you a local copy of the entire repository, which you can access much faster than blackfin.uclinux.org. This is the recommended way of doing things, as it puts less stress on the server, and will decrease the overall bandwidth requirements.

To set up a local repository, do the following:

1. Check to ensure you can talk to the rsync server on blackfin.uclinux.org

   $ rsync blackfin.uclinux.org::
   toolchain
   jtag
   ltp
   multimedia
   u-boot
   uclinux-dist
   linux-kernel
   xmame
   dokuwiki
   

   If port 873 is blocked on your firewall, there is no way to connect to the rsync server.

2. Do an initial rsync the projects you are going to create local repositories of -- remember that you only need to mirror the projects you actually care about. The first time that this is done, it can take a over two hours per repository.

   /usr/bin/rsync -azv --delete --force blackfin.uclinux.org::u-boot /svnroot/u-boot
   /usr/bin/rsync -azv --delete --force blackfin.uclinux.org::toolchain /svnroot/toolchain
   /usr/bin/rsync -azv --delete --force blackfin.uclinux.org::jtag /svnroot/jtag
   /usr/bin/rsync -azv --delete --force blackfin.uclinux.org::ltp /svnroot/ltp
   /usr/bin/rsync -azv --delete --force blackfin.uclinux.org::multimedia /svnroot/multimedia
   /usr/bin/rsync -azv --delete --force blackfin.uclinux.org::u-boot /svnroot/u-boot
   /usr/bin/rsync -azv --delete --force blackfin.uclinux.org::uclinux-dist /svnroot/uclinux-dist
   /usr/bin/rsync -azv --delete --force blackfin.uclinux.org::linux-kernel /svnroot/linux-kernel
   /usr/bin/rsync -azv --delete --force blackfin.uclinux.org::xmame /svnroot/xmame
   /usr/bin/rsync -azv --delete --force blackfin.uclinux.org::dokuwiki /svnroot/dokuwiki
   

3. after this is complete, you can check out code on the local machine with

   svn -q checkout file:///svnroot/toolchain/trunk toolchain
   

   Doing this will take less than 3 minutes, 38 seconds (on my machine, which uses RAID0), but it is much faster than downloading over the network.

1. If you want to set svn up to give people local access to the repository via a svn: protocol you must set up svn as a deamon to listen on your network.

   svnserve -d -r /svnroot/ -R
   

   This causes svnserve to run in daemon (-d) mode (in the background) this will accept TCP/IP connections on the svn port (3690, by default - since 3690 is not a secure port, it does not need to be started as root) with the virtual root (-r /svnroot/) for repositories served by svnserve to be set to /svnroot (pathnames in URLs provided by the client will be interpreted relative to this root, and will not be allowed to escape this root), and to be read-only (-R) forcing all write operations to be forbidden (afterall it is a mirror).

2. This allows checkouts to be done with

   svn -q checkout svn://192.168.0.4/toolchain/trunk toolchain
   

   Doing this will take less than 3 minutes, 25 seconds, showing there is no appreciable difference between network, and local file access. If it is possible to check out a local mirror, please do so.

1. To keep rsync up to date, you can run this script.
   Because rysnc can sometimes take over 60 minutes to complete, and we don't want to be running more than one rsync at a time, this small shell script will make sure that this is the only rsync connecting to a site with “blackfin” in the name (to make sure that the network does not become congested) before starting the network transfer. It times out in 59 minutes, as we will be starting it every 60 minutes with a cron job. Make a new file /usr/local/bin/blackfin_rsync

   #!/bin/sh
   # This needs 1 argument
   if [ $# -ne 1 ]; then
       echo 1>&2 Usage: $0 repository_name
       exit
   fi
   
   # Set initial variables to zero
   I=0
   
   # This checks to see if rsync is running.
   # $RUN will be set to 0 if it is not running
   #
   RUN=`/bin/ps -C rsync | grep -v PID | wc -l`
   while [ "$RUN" -gt 0 ]
   do
       # This will put all the rsync process ID into a tmp file
       # This assumes that the first parameter from ps is the PID
       #
       /bin/ps -C rsync | grep -v PID | awk '{print $1}' > /tmp/tmp_file
   
       # This will read the file into the pid[] array
       typeset -i cnt=0
       while read line
       do
           pid[$cnt]=$line
           ((cnt = cnt + 1 ))
       done < /tmp/tmp_file
   
       typeset -i temp=0
   
       # This checks to see if the rsync is running is going to a site
       # with 'blackfin' in the address. If so, wait. Otherwise, skip it.
       while [ "$temp" -ne "$cnt" ]
       do
           RUN=`ps ${pid[$temp]} | grep -v PID | grep blackfin | wc -l`
           if [ "$RUN" -gt 0 ] ; then
              break
           fi
           ((temp = temp + 1))
       done
   
           # Wait 1 minute
           sleep 60
           I=$[I+1]
           if [ "$I" -gt 59 ] ; then
                   # If waiting 59 minutes stop
                   rm /tmp/tmp_file
                   exit
           fi
   done
   
   rsync -az --delete --force blackfin.uclinux.org::$1 /svnroot/$1
   # rsync options are:
   # -a        archive mode, equivalent to -rlptgoD
   # -r        recurse into directories
   # -l        copy symlinks as symlinks
   # -p        preserve permissions
   # -t        preserve times
   # -g        preserve group
   # -o        preserve owner
   # -D        preserve devices
   # -z        compress file data
   # --delete  delete files that don't exist on sender
   # --force   force deletion of dirs even if not empty
   

   Ensure that this file has execute permissions with chmod +x /usr/local/bin/blackfin_rsync

2. Add to the crontab, with crontab -e. The following lines should be in your crontab file:

   #min  hour  day  month  dayofweek   command
   #0-59 0-23  1-32 1-12   0-7
   0        *     *    *     *         /usr/local/bin/blackfin_rsync jtag
   10      *     *    *     *          /usr/local/bin/blackfin_rsync jtagtools
   20      *     *    *     *          /usr/local/bin/blackfin_rsync uboot533
   30      *     *    *     *          /usr/local/bin/blackfin_rsync uclinux533
   45      *     *    *     *          /usr/local/bin/blackfin_rsync gcc3
   

   To reduce load on the server - use different times than this. If everyone uses the same times, it will have periods of intense activity (and be slow for everyone), and then periods where it is sitting idle.

3. Now your localhost mirror should be set up, so you can replace blackfin.uclinux.org with localhost. Not only can this be used for a local mirror, but you now can tell other people on the same subnet as you, that they can use your server to pull from. This will increase their bandwidth, and reduce the overall load on the main server.
4. You should use this script to checkout from your local lan:

   #!/bin/sh
   
   #Home
   CVSSERVER=192.168.0.4
   
   CHECKOUT=~/blackfin/trunk
   mkdir -p $CHECKOUT
   
   DIST=uClinux-dist
   KERNEL=$DIST/linux-2.6.x
   UBOOT=u-boot
   TOOLCHAIN=toolchain
   
   cd $CHECKOUT
   
   rm -rf $CHECKOUT/$DIST $CHECKOUT/$UBOOT  $CHECKOUT/$TOOLCHAIN
   
   svn checkout --ignore-externals svn://$CVSSERVER/uclinux-dist/trunk $CHECKOUT/$DIST
   svn checkout  svn://$CVSSERVER/linux-kernel/trunk $CHECKOUT/$KERNEL
   svn checkout  svn://$CVSSERVER/toolchain/trunk $CHECKOUT/$TOOLCHAIN
   svn checkout  svn://$CVSSERVER/u-boot/trunk $CHECKOUT/$UBOOT
   
   $CHECKOUT/$TOOLCHAIN/buildscript/BuildToolChain -s $CHECKOUT/$TOOLCHAIN -k $CHECKOUT/$KERNEL -u $CHECKOUT/$UBOOT/u-boot-1.1.6 -o $CHECKOUT/gcc4 -b $CHECKOUT/gcc-build -c 4.1
   

   The --ignore-externals flag will ensure that the kernel repository at http://blackfin.uclinux.org will not used, but the local version will be checked out, and put in the proper place.

git is a fast distributed scm that is used in many open source projects but most notably, the Linux kernel. Unlike SVN/CVS, it has no centralized repository.

All of the blackfin projects have begun mirroring the SVN repository in git at https://blackfin.uclinux.org/git/.

Here's a quick reference for common commands and their counterparts in CVS, SVN, and git. Most commands are (purposefully) pretty much the same, but the little details are what nag you.

Other than per-project repositories, people with accounts may create as many per-user git or svn repos as they like. However, the interface is command line only (no website interface to this).

By default, the repo is created with permissions such that only you may access it (read or write). If you want, you can change the permissions to let any one read it (but you still are the only one who can write it).

The interface is simple (to use the svn version, simply change git-publish to svn-publish):

$ ssh $USER@sources.blackfin.uclinux.org git-publish -h
Usage: git-publish <-c|--create> <repo> <description>
Usage: git-publish <-p|--make-public> <repo>
Usage: git-publish <-P|--make-private> <repo>
Usage: git-publish <-S|--make-semi-public> <repo>
Usage: git-publish <-d|--set-desc> <repo> <description>
Usage: git-publish <-r|--remove) <repo>

Public:  world readable (git/rsync) and via http viewgit
Semipub: world readable (git/rsync) but not via http viewgit
Private: readable only by you

So to create a new git repo and then check it out, you do:

$ ssh $USER@sources.blackfin.uclinux.org git-publish -c foo
$ git clone ssh://$USER@sources.blackfin.uclinux.org/git/$USER/foo.git

Or for svn:

$ ssh $USER@sources.blackfin.uclinux.org svn-publish -c foo
$ svn co svn+ssh://$USER@sources.blackfin.uclinux.org/svn/users/$USER/foo

The git interface has a gitweb interface, but there is nothing for svn.