What is a Delta Build?

A Delta Build consists only of the changes (delta) that have taken place after a certain version of the version control (e.g. version of the last release).

What is a Delta Build used for?

A Delta Build allows you to only deploy those assemblies and files that have actually been changed since the last release. This procedure is also referred to as Delta Deployment.

Who needs it?

In large systems it can be necessary to only deploy deltas (e.g. minimal down times) or it can be required to identify what assemblies and files of the system actually have changed since the last released version.

Configuration management tools like CA Harvest support this feature.

What is the issue?

Out of the box Visual Studio Team System and Team Build does not support Delta Builds. This blog post presents one solution to facilitate Delta Builds using standard Team Build extension capabilities.

What is the solution?

The solution is shown in the graph bellow. First the GET operation gets the source code version of the last release version into the team build workspace. The sources are compiled and the build result is stored into a safe location for later reference.

After the 'clean' BUILD has finished an additional GET operation is executed. This time the GET is executed as an incremental operation - only source files that have changed since the currently in the workspace existing file version are replaced by the get. After the workspace got updated a additional BUILD is executed. Since the build results (and temp files) of the first BUILD run do still exist within the workspace, this BUILD is executed in an incremental build fashion. Only those projects that are 'out of date' are getting compiled.

Finally we have the results of the first (clean build) and second (incremental build). All we need to do now is a comparison of assembly versions. All assemblies that are of later date in the second build result than in the first build result are part of our Delta Build result.

How is it done?

All extensions to the default build process are done within an additional targets file (DeltaBuild.targets). 

TFSBuild.proj

In order to get executed the targets file needs to be imported into the TFSBuild.proj file. This allows to identify all changes and additions to the original team build template very easily an allows to export the functionality to other team builds efficiently.

 

 

 

 

DeltaBuild.targets

Inside the DeltaBuild.targets file the GetVersion property is overwritten (line 6) to the changeset ID of the last released software version. The changeset ID given here is the version your delta is going to be build on.

Also the AfterCompileSolution Target is overwritten (line 19). This target gets executed after the compile of each solution defined inside the TFSBuild.proj file. Whenever this happens four additional Targets will be executed (lines 12-15). [This code and text were edited due to feedback from Sayed Ibrahim Hashimi in his blog post. Tanks a lot Sayed!]

 

 

TFSBuild.proj

In order to build our solutions twice within one team build (and one configuration) a trick needs to be used. The solutions have to be duplicated within the SolutionToBuild item group (lines 135-166 TFSBuild.proj). Since VSTS 2008 it is possible to provide custom properties to each solution execution. To facilitate Delta Builds a BuildRun property is used to identify if the first (CleanBuild) and second (IncrementalBuild) build execution. Don't forget to insert a ";" in front of your property, otherwise it will not be available (bug).

 

 

DeltaBuild.targets

The SafeClenBuildResult target is only executed if the property BuildRun is set to CleanBuild. When executed an item group containing the current build results is created (line 24). Then all files of the item group (CleanBuildResult) are copied to a folder named after the build configuration and what build result it contains (line 29).

 

 

 

 

The GetLatestVersion target is also only executed if the property BuildRun is set to CleanBuild. Inside this target the MSBuild task is used to execute the GET operation needed to update the team build's workspace with the latest source version. The MSBuild tasks allows to define properties to the executed "target". It is not necessary to set the GetVersion property to "T" (= latest version) since this is the default value, but it is important to set the ForceGet property to False, since otherwise a full get is executed and a calculation of the delta between both builds will not be possible.

 

The CollectIncrementalBuildResult is only executed if the BuildPun property is set to IncrementalBuild (line 55). The only thing done inside this target is the creation of a item group containing the build results of the incremental build run (line 59).

 

 

The CreateDeltaBuildResult target is only executed if the BuildRun property is set to IncrementaltBuild (line 68). This target uses the so called transforms functionality of MSBuild allowing to compare the 'age' of input files (Inputs, line 69) and output files (Outputs, line 70). As "Inputs" the IncrementalBuildResult" is used, created beforehand inside the CollectIncrementalBuildResult target. As "Outputs" the transforms syntax requires to state the location of the files to compare dates against. In this case the clean build (first build executed) is given as reference. The transforms mechanism no reduces the item group to only those members that are older in the "Inputs" location than in the "Outputs" location. The build log file shows this behavior in line 1950.

 

Inside the target all files remaining in the item group after the transformation took place are copied into the DeltaBuildResult folder. The following screen shot shows the folders after the team build has finished.

The target file can be downloaded from here.

If you have any questions let me know or send me email at thomas <dot> janssen <at> seria-mummert.de

Stay tuned and enjoy your delta builds!

Tom

.

This week I held a talk at CMConf in Munich. The talk was about how to migrate from Visual Source Safe to Team Foundation Server Version Control. You can download the PPTs here or keep on reading.

History Migration

Thinking about migration the first question is: "What gets actually migrated during the process?". Well let's have a look.

Migrated information:	Not migrated information:
Add File or Folder	Sharing
Delete File or Folder	Branching
Undelete File or Folder	Pinning
Rename File or Folder
Edit File
Modify Label
Move Folder

Further readings: Visual SourceSafe Converter Limitations

Lets spend some words about the not migrated VSS history information:

Sharing

Sharing is not supported by TFS version control. If you might ask why, the answer is that sharing is considered to bring more confusion and faults into projects than it brings advantages. In TFS VC we use a certain branching and merging method to allow a comfortable way of sharing sources. I will wirte about this in the future.

Branching

Branching has always been very difficult for VSS users. VSS did not create a branch/merge relationship between the source and target branch the way TFS VC does it today. Therefore every merge operation was a baseless merge, as we call it in TFS VC. This meant the least possible support for your merge operations in the past.

Pinning

Pinning is another concept that is not supported by TFS Version Control. In TFS VC you can use several workspaces (local working folders) that can contain dedicated versions of your projects sources. To archive this you can use a number of options within the "Get Specific Version" operation. The other option like in sharing is to use a branching and merging method to orchestrate your projects sources to a certain combination of versions.

Migration Process

In our migration projects, as a best practice, we follow a six step process:

STEP 1: Planning

Before going to action we conduct a assessment of what actually has to be migrated. The questions of interest at this point are:

• What size and quality has the VSS DB?
• Are there any know issues or inconsistencies?
• Is there any dead code?
• How often do user's access the VSS DB?
• What date shall the migration take place?

There are also three decisions to be made during this phase:

• Migration or integration of the VSS DB into TFS
• What code sections are relevant for migration
• What part of the history is worth the migration effort

Before you go to action you should always ask yourself: "Do I really need to migrate this code?" There are some facts that might influence your answer to this question: You can use VSS as a version control within TFS. If you only need to access your VSS sources every once in a while it might be good enough to switch over to the VSS version control plug-in within Visual Studio.

Make sure not to waste your time migrating sources that are not needed any more for further development purposes and think about the length of history that is relevant for you to migrate.

STEP 2: Prerequisites

For a migration you need the following software:

• Visual Source Safe 2005
• VSSConverter.exe (latest version!)
• List of VSTS User
• Team Explorer
• SQL Server (Express) (if you don't execute the migration on your TFS Data tier machine)

If you are able to it's recommended to execute the migration on the TFS server machine. This allows you to use SQL server as well as the good performance of your team foundation server. In most cases this will not be possible nor allowed.

STEP 3: Preparation

Prepare your VSS DB for migration. Make sure to

• Get access to VSS DB admin  (user) and password
• Get access to TFS Team Project admin (user) and password
• Check-in all checked out files into VSS
• COPY the VSS DB twice - for BACKUP and MIGRATION

Further reading: Walkthrough: Preparing to Migrate from Visual SourceSafe to Team Foundation

STEP 4: Analyze VSS DB

If possible, execute the VSS analyze.exe on your TFS server. If you only whish to migrate only a certain history segment of your VSS DB use the archive feature to archive the section of history you are interested in (The migration of partial VSS histories has caused trouble in the past and might not be possible in every case.)

Execute the analyze.exe with the following parameters:

analyze.exe –fp –v4

• fp repair and remove invalidly named physical files
• V Highest verbosity of output

Try to solve any errors and warnings that get shown in the analyze.exe output!

Backup your VSS DB one more time after you have executed the the VSS analyze.exe.

Further reading: ANALYZE Utility, How to: Find and Repair Data Corruption

STEP 5: Migration

The migration itself takes place in two steps: analyze and migrate.

Analyze

First you have to create a config file that tells the VSSConverter.exe what sources to analyze for migration. The project source "$/" means that you want to analyze/migrate the complete VSS DB you specified.

During the analysis of the VSS DB the VSSConverter detects what users used the DB in the past. As a result you receive tow files

• UserMapping.xml
• VSSMigrationReport.xml

UserMapping.xml allows you to specify how VSS DB users are mapped to TFS users. The VSSMigrationReport.xml tells you what warnings and errors the VSSConverter has detected.

Try to solve all errors and warnings! One thing that often helped in the past was to identify folders for migration separately.

Fill in TFS user names into the UserMapping.xml file. Make sure to give also the domain of TFS users.

Migrate

Add the migration target to the config file and then it is time the call VSSConverter for the migration:

VSSConverter migrate <config file name>

The execution of the migration will take 5 - 10 times longer than the analysis did! Put this into consideration for your planning.

Further reading: Walkthrough: Migrating from Visual SourceSafe to Team Foundation

 

STEP 6: Quality Assurance

After the migration has finished make sure to verify the quality of the migration result. If any warnings or errors where given during the migration try to resolve them and repeat the migration one more time.

If you have any questions please let me know and drop me a line at

thomas <dot> janssen <at> steria-mummert.de

Good luck and stay tuned!

Tom

 

Great news! SPALM out SharePoint Application Lifecycle Management Solution is life on CodePlex! You can download a free version of SPALM that allows you to use a limited set of its functionality within your projects. If you you like to get a full version of SPALM or want to see a life demo of SPALM drop me a line.

Here is the link to the download.

Stay tuned.

Tom

Great news! VSTS Development Edition and Database Edition are joining together to become one VSTS edition. If you are either Development or Database Edition MSDN subscriber you have access to both edition beginning tomorrow, Oct. 1st.

So Oct. 1st also seems to be the release date of Database Edition 2008 since it was only available as CTP so far.

Got Questions? - Ask Christian - our DB Edition expert!

If you have any questions regarding VSTS Database (or Development) Edition don't hesitate to write my college Christian at Christian <dot> meusel <at> steria-mummert.de - he is our DB Edition expert.

Read here about the original announcement.

Official MSFT announcement:

Better Together – Visual Studio Team System Development Edition and Database Edition
In recognition of the increased need to integrate more of the lifecycle members together, we will provide a unified Development and Database product in Visual Studio Team System 2010. Beginning October 1, 2008 Development Edition and Database Edition MSDN subscribers will have access to both products.

Stay tuned.

Tom

Torsten held a session about our latest improvements on the SharePoint Application Lifecycle Management (SAPLM). What the participants thought about his talk you can read here. Rainer Ganser a SharePoint specialist himself writes about his impressions on SPALM (in German).

Torsten will publish his PPTs soon for download, he promised. Meanwhile you can try to download the ppts form this site.

Stay tuned.

Tom

I'm already on the way back to office from Mainz. We had a great time on the first ALM day at this year's BASTA! conference. Thanks to all participants!

I'm already working on the next upcoming event - CMConf in October. So you will read my resume within the next days.

Here are the PPTs of my talk together with Neno: Software Entwicklung im Griff: Application Lifecycle Management (ALM)

Stay tuned.

Tom

Exciting news for next year's TeamConf just arrived! Ken Schwaber - one of the inventors (fathers) of SCURM is coming to Munich for a key note at TeamConf 2009. This is going to be a big thing - you defiantly should not miss it. Just recently I have watched some of his talks over the Internet and he really is a great speaker that knows how to transport a message to his audience.

If you like some links to find out more use the one's in the official MSFT announcement:

"Die TeamConf 2009 findet vom 5. Mai bis 7. Mai 2009 in München zum zweiten Mal statt. Ziel der Konferenz ist es, den Teilnehmern durch Anwenderberichte und Fachvorträgen eine breite Kommunikationsplattform rund um das Thema Microsoft Visual Studio Team System in der Praxis zu bieten.

Bereits jetzt hat  Ken Schwaber, der „Erfinder“ von SCRUM und Mitbegründer der Agile Alliance, seine Teilnahme als Keynote Speaker zugesagt."

See you at TeamConf 2009!

Stay tuned.

I hear users every once in a while complain that "Get Latest" on Team Foundation Version Control doesn't really do it's proper job. Instead of writing about causes that can lead to this assumption I would like to link to Jason Barile's Blog. He explains how it can happen that you don't end up with the latest version of a file in your workspace when you actually expect to do so.

If you want to read more about it have a look here:

• MSDN forums:  Get Latest Version not bring all the files to local computer. bug?
• MSDN Documentation:  How To:  Get the Source for Your Team Project

If this explanations won't cover your case of experiencing a false behavior when executing a "Get Latest" operation please let me know!

Stay tuned.

Tom

Finally it arrived - Service Pack 1 for Visual Studio 2008 and Team Foundation Server 2008 is available for download here.

If you want to know what you actually get with Service Pack 1 have a look at Willi's blog posts - he did an excellent job in presenting the new features.

• Send selection in Mail
• Office Ribbon
• Source Explorer Enhancements

Stay tuned.

Tom

If you use Team Foundation Server 2008 as you probably know - you also have to use the 2008 Build Agents. What you might not know - since it is not activated by default - is the fact that 2008's  Team Builds can be run in parallel processes on your build agent. Therefore the build (compile) time can be reduced by a significant figure.

How do I activate parallel builds on my 2008 Build Agent?

What you need to do is configure your build agent's "Visual Studio Team Foundation Build" (display name in process explorer) service's coninguration file to use more than one process to execute your team build jobs. You should find the config file on your build machine in a location similar to this:

<Drive:>\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies\tfsbuildservice.exe.config

Within the tfsbuildservice.exe.config file you have to look for the the "MaxProcesses" key and increase the value to the number that fits for your build agent - e.g. number of cores available.

After you save you changes to the tfsbuildservice.exe.config file you need to restart the "Visual Studio Team Foundation Build" on your build machine using the windows services explorer.

After the restart the service runs with your new configuration setting.

What has actually changed now?

After your change in the tfsbuildservice.exe.config team build will build different ConfigurationToBuild's of you team build in parallel. This should not cause any problems four your existing team builds since different configurations are treated independently from each other during the build process.

If you don't want your team build to build ConfigurationToBuild in parallel you can always add a BuildConfigurationsInParallel property to your team build script preventing team build to do so.

Not only ConfigurationToBuild will be build in parallel - also your SolutionsToBuild will be build in parallel.

This can cause errors when executing team builds that have dependencies between different solutions. Therefore you might have to insert a BuildSolutionsInParallel property to you team build preventing team build from building solutions in parallel.

So what has left?

Even if you don't build different configurations or you are not able to build your solutions in parallel due to inter solution dependencies team build will still speed up compilation of you solutions. The projects of your solutions are build in parallel if dependencies allow it - this way you will still have a gain in build speed and decrease the time it takes for your team build to finish.

I hope this description is clear enough for you to "Speed up your Team Builds". If you have any questions drop me a line please.

Stay tuned.

Tom

This week - finally - the agenda for the Application Lifecycle Management Day on Monday, September 22nd in Mainz, Germany went online! The ALM Day is one of BASTA!'s special days this year. Here the most important information about it.

Here the agenda:

Here the links to Torsten's (still not online :-0) and mine sessions together with Neno Loje.

In our first Session Neno and I will talk about ALM in general. In the second session we will give an example on how ALM can be realized using VSTS as your ALM platform of choice.

Torsten will present the latest developments within Steria-Mummert's SharePoint Application Lifecycle Management (SPALM). Torsten and colleges have put a lot of effort into SPALM and used it already very successful in several customer projects. Read Torsten's blog here.

If you want to increase your productivity or quality, reduce costs or time-to-market within your company's software development department(s) join us on BASTA!'s ALM Day. This day will give you a perfect overview about what is possible today with ALM.

Hope to see you on Monday, Sept. 22nd in Mainz.

Stay tuned.

Tom

I was just searching for team build property reference on MSDN - I knew I had seen this once. Actually true, but not on MSDN - it was Aaron Hallberg's great blog post with all the information you need in place.

Here is the link.

Stay tuned.

Tom

During my time as a configuration and build manager (I stress out the "manager" ;-) I found this to be a very useful practice. Separation of concerns means that you separate you team build scripts into different files and keep together those parts of the script that belong together.

Let's start with the tfsbuild.proj file - within this file I keep all properties that "configure" my team build. If there is another solution to build, to add unit testing, switch on auto deployment (I come to this in a later post) etc. I do it within this file.

Overrides.targets file - within this file I put all custom override targets top down in the order of execution. This file allows me to control the timely behavior of the team build. I don't actually override the targets in this file, I only insert my custom targets to the "DependsOn='...' attribute. Whenever I have to change the order or add another target it is very easy to do it. This way you don't have to wonder if a target is overridden on any later location of the team build.

Pojects.targets file - within this file I keep all project related targets (custom targets). Here you find the actual targets you execute during the team build. The nice thing is that I bring a bunch of targets with me to every team build I have to create and don't have to change anything within this file. Properties - that are used to "configure" the build and the order of the targets are all set outside this file.

This way I am able to keep a strict order within the team builds and know every time where to do necessary changes without looking for hours and wondering why my changes are not working - even after I checked them in into version control. This really works out good if you have to take care of 10+ team builds.

Btw. you connect the different team build files by using the <import> task.

What do you think? Drop me a line if you have an opinion on this.

Stay tuned.

Tom

Ed Glas has posted twice this week about great resources for you if you want to dive into web, load or unit testing.

• His first post lists links to a number of MSDN Videos you also find here within MSDN.
• His second post updates his earlier posted link list - a collection of vital information for doing testing with VSTS.

If you are looking for more videos also check my earlier blog post.

Enjoy the movies - stay tuned.

Tom

On Monday I received notice that I have the honor to hold two sessions at this years CMConf conference, taking place from October 14th - 16th in Munich. CMConf is held for the first time this year and addresses topics as configuration and change management, process integration, product line strategies, KPIs and  optimizations.

Talk one will be about do's and don'ts in source code migration when migrating from Visual Source Safe to Team Foundation Server Version Control. Talk two will be about ways to realize a automated staging process based on a version control system as TFS Version Control.

My company will also have a booth during the conference days (15. and 16.).

I would love to see you at CMConf 2008 and have a chat with you.

Stay tuned.

Tom

My college Torsten Mandelkow will also have a session at this years BASTA!2008 conference in Mainz. He will talk about our solution for developing SharePoint Platforms under use of Application Lifecycle Management. SPALM is therefore an example of how ALM can be customized to certain and individual requirements.

Dreamteam SharePoint und Team System: MOSS-Entwicklung mit einer Software Factory

SharePoint 2007 bietet unzählige Möglichkeiten der Anpassung und Erweiterung: .NET Code, ASP.NET-Controls, WebParts, Site templates, Content Types usw. Torsten Mandelkow zeigt, wie Steria Mummert Consulting die SharePoint-Entwicklung weitestgehend automatisiert und welche Rolle dabei Application Lifecycle Management mit Visual Studio Team System und eine SharePoint Software Factory spielen.

You can learn more about SPALM here.

Torsten's session will be held on Monday September 22nd within the ALM Day at BASTA!2008 in Mainz.

I will post the agenda of the ALM Day as soon as it is released.

Stay tuned.

Tom

Last night Neno Loje and I finally completed the work on the abstracts for our common sessions about ALM. In total we will have three sessions, one about ALM in general and two sessions about ALM in day to day practice.

I am really excited to be on stage together with Neno for three times during BASTA!2008's ALM Day on Sep 22nd in Mainz, Germany.

As soon as the agenda is online I promise to post the link. Here the abstracts for our sessions:

Softwareentwicklung im Griff: Application Lifecycle Management (ALM)

Wer Softwareentwicklung professionell betreibt ist auf einen reibungsfreien Softwareentwicklungszyklus (= "Application Lifecycle") angewiesen. Einfacher gesagt als getan. Lernen Sie den ALM kennen und erfahren Sie welche Herausforderungen und Möglichkeiten im Projektalltag bestehen - von der ersten Minute bis zum erfolgreichen Release. Erfahren Sie, wie man Projekte erfolgreich steuert, kontinuierliche Wertschöpfung und Qualitätsstandards sicherstellt, Ordnung und Übersicht in die Sourcen und die verschiedenen Versionen bringt. Profitieren Sie von in der Praxis erprobten Best Practices und den Erfahrungen aus einer Vielzahl an Projekteinsätzen.

ALM mit Visual Studio Team System realisieren – Teil 1 & Teil 2

Den ganzen Lebenszyklus "verwalten". Ohne geeignete Werkzeuge, die Sie dabei unterstützen kaum vorstellbar. Tauchen Sie ein in die Welt des Application Lifecycle Managements und erleben sie live mit wie ALM durch den Einsatz von Visual Studio Team System (VSTS) in der Praxis realisiert wird. Sehen Sie, welche Unterstützung Ihnen VSTS als ALM Plattform im Projekt in den einzelnen Phasen eines Projektes bietet. Holen Sie sich Anregungen und Best Practices für Ihre eigenen Projekte und profitieren Sie von der Erfahrung aus Vielzähligen Projekteinsätzen.

I hope to see you at BASTA!2008 and the "Application Life Cycle Management Day", September 22nd in Mainz.

Stay tuned.

Tom

If you are looking for some guidance on how to use VSTS to increase and secure the quality of your web projects here is a good hint for you. On the ASP.net web site you find up to 30 short clips between 6 to 15 minutes long. Being shorter than regular webcasts they are more focused on certain questions such as "How Do I: Tune Web Application Performance with Profiling?".

Have a look at the site here.

Enjoy the movies - stay tuned.

Tom

If you want to learn more about VSTS and how it helps you, your team and company to accomplish day to day tasks you can find a list of available VSTS white papers here.

If you read about application lifecycle management and it's benefits in these white papers and you wonder how mature your team is about ALM you can accomplish a ALM self assessment on the German ALM MSDN web site. If you answer all questions of the ALM self assessment you might even win a day of free ALM consulting. In case you do win we might even get to know each other in REAL life, just request me as your favorite "Team System Quality Board" member.

Have fun answering the questions - stay tuned.

Tom

Either if you are interested in Visual Studio Team System and Team Foundation Server or you want to develop customizations or extensions to VSTS and TFS in an isolated environment you should know about the pre installed and configured VSTS Virtual PC's. All you need to do is download the VSTS VPC files and run the included exe file. Total size for the download is about 5 GB.

WARNING: Do not keep data inside the VPC image. This is a full featured Team Foundation Server with a Visual Studio Team Suite pre installed - but is timebomed. Therefore it will not be able to access the VSTS VPC after 12/31/2008! All data kept inside the VPC will not be accessible after new years eve.

You can download a VSTS 2005 and VSTS 2008 VPC. If you want to try the features of the next VSTS Version called Rosario you can also download a Rosario VPC from here.

Have fun - stay tuned.

Tom