With a number of SharePoint 2010 projects running in production and a few projects that are currently in their second stage, I thought it was about time to write something about how we organized these projects. In this post I will describe how we applied DTAP in our projects. It took us a year and half and a few revisions to get where we are now. In this post I will describe which environments we have and how we use them. And here is the disclaimer: it is a setup that works pretty good in our organization. It might work in your organization, but it also can be complete overkill. It depends. Most important is to think about this before you build your street. And in this post I hope to give you some insights in our situation that might help you thinking about yours.
Thinking about how many environments and how to use them is important, because it is good for everyone to have a clear understanding of the purpose of each environment. Also make sure everybody understands the processes around each environment. Who’s doing deployments, who to call when it is down, where to find the latest build, things like that.
We started working with SharePoint 2010, doing an internal proof of concept project around knowledge management. We had good discussions on how to organize things in Visual Studio, how to deploy our software and how to manage versions. A year and a half later things have grown a bit and we needed to re-think our strategies to make the growth possible.
I start off with giving you a schematic overview of our street. A few things to note:
- Besides intranet projects we also started to run internet facing sites. We have a separate SharePoint production farm for those websites. And this also led to 2 acceptance farms, 2 test farms etc
- Multi server farms are showed as 3 servers. This is not the real topology of these farms. The real topology is out of scope for the purpose of this post. Just the fact that a farm is built using multiple servers is important.
- Every server is virtualized.
In the following paragraphs I will talk about each environment, starting with the Development and moving forward to Production. For every environment I will discuss the following subjects:
- Usage – The purpose of the environment. Who is using it, and what for.
- Topology– Single server or multi server.
- Installation and management– Who is installing the servers and who is responsible for maintaining them. How do we install SharePoint and tooling.
- Deployment – How are projects deployed to the environment.
- Topology– All development machines are standalone farms. Our development machines are not dedicated for one of the farms. To be flexible when people change teams, new projects start and other projects end, we want to be able to switch our development servers from intra- to internet and vice versa.
- Installation and management– Initial Installation of the OS and SharePoint is done by our IT-Pro team. We have a set of PowerShell scripts that install SharePoint and prerequisites and all tooling that we need. Day to day management is done by the developers themselves. They are responsible for installing service packs etc. They also have a script to change their dev server from an intranet to an internet development server. Main difference are the service applications that are used and the service accounts, that are dedicated for the type of farm. Initially developers get a fully functional machine, with the SharePoint farm up and running. Creating web applications, site collections, etc. is the first thing they do in their projects.
- Deployment – Software and SharePoint configuration we build in our projects are deployed to the local farm through Visual Studio deployment and through our Project Installer. This is a set of PowerShell scripts that installs WSP’s, configures SharePoint and adds content. It probably is the subject of a future post.
For the developers there is 1 simple ps1 file to kick off. This cleans up the machine and re-installs everything that is created by the whole team. This way front-end developers who don’t know anything about SharePoint are also able to do and test their work in a SharePoint environment.
In projects that are in their second stage, we still use this procedure to install the project to dev machines. Developers are also responsible to build the upgrade path to their latest version, but this is not tested in the development farm. In the setup we currently have it is not very easy to go back to a specific, to test the upgrade path over and over again.
Test – Development (T-D)
- Topology– All T-D machines are standalone farms. They are the same machines as development machines. This way we are flexible when setting up new teams. T-D machines can become D machines and vice versa. After a project team stops, D and T-D machines are recycled for new projects, either in the intranet or the internet farm.
- Installation and management– Same story as for development machines.
- Deployment – Our Project Installer is used to install software to SharePoint and configure the environment for testers to be used. Generally one of the developers in the team has the task to install a new version to this environment. Most teams do this every day. The developer gets the latest code from Team Foundation Server, builds and packages everything and runs a re-install on his own machine. This is to validate everything that is checked in the previous day is installable. If this leads to a working site, he/she runs the same re-install (1 big PowerShell script) on the T-D server. If something is wrong, the team has time to fix it and testers still have yesterday’s build to continue testing. This way testers always have a working test environment, with just a short break when the installer runs. We started off by always installing the daily build, but this turned out not to be a good idea. Having 3 testers asking every 10 minutes when they can start testing is not very good for your blood pressure .
Build – Development (B-D)
- Topology– Both B-D farms are multi-server, with a single web frontend.
- Installation and management– Same story as for development machines. Our development department is also responsible for managing this environment, just like the development servers. The challenge here is that multiple project teams are dependent on this farm. They need to work together to ensure test environments are available when needed and coordinate fixing of their daily builds.
- Deployment – Our Project Installer is used to install software to SharePoint and configure the environment. This is done by kicking off the PowerShell script for the project installer. This is done using the Windows Task scheduler on the application server, after the TFS teambuilds for all projects are finished. We are currently using TFS2010 with the MSBuild based teambuild. We haven’t yet upgraded the teambuild processes to the new workflow based teambuild. One day we will move to the approach as described by Mike Morton and Chris O’Brien.
- Topology– Both T farms are multi-server.
- Installation and management– SharePoint and everything it needs is installed to the servers by our scripts. Of course without the development tooling. Developers do not have access to this server. They just have read permissions to the diagnostic logging folder. Everything else is done by admins.
- Deployment – Our Project Installer is used to install software to SharePoint and configure the environment. This is done by an IT Pro. Installation is done based on release notes. When projects deliver for deployment to T, they also deliver this document. It describes all required parameters, prerequisites, how to run the Project Installer and, if needed, manual configuration steps. Projects themselves decide their deployment schedule to the test environment. Some do it every (scrum) sprint, some do it weekly and some do it after a number of sprints. It is the scrum master’s task to ensure a time slot and admin are available to do the installation.
We have a special mode for our Project Installer; it does not just do every action, but asks whether or not to run the action. For admins this makes it easy to see what happens and find errors easier. Installations in T are done with the help of one of the developers.
- Topology– Both A farms are multi-server.
- Installation and management– SharePoint is installed to the servers by the samen PowerShell scripts, but with different parameter sets.
- Deployment – A deployments are done by admins, based on the release notes.
|Usage – Not a lot to tell here. It’s what it’s all about. We do all of the above to get everything we create to these farms.|
- Topology– Both production farms are multi-server.
- Installation and management– SharePoint is installed to the servers by the same PowerShell scripts, but with different parameter sets. The big advantage of these PowerShell scripts, is that it is way easier to ensure all environments in our streets have the same setup. So if we have a multi tenant farm, all our development machines are also running the same multi tenant setup. And if we need Performance Point, the development machines also have the PPS service application running. From a quality perspective this is pretty important.
- Deployment – Production deployments are done by admins, based on the release notes. By the time we get here, the deployments have been tested a number of times, so they should run without problems. Our goal is to automate as much as possible by using and continuously improving our Project Installer. This way there is very little manual configuration left by the time we get to the P-installation. This greatly improves the quality of our installations.