This is 5 part blog series, on how to “Cost Optimization for On-demand Private Build Server for SharePoint”. Although I specifically write this for SharePoint’s build server, however the concept and ideas is also applicable for anyone looking for optimizing their cost running private CI/CD (Continuous Integration/Continuous Delivery) in Visual Studio Online. The series in these articles are:

1. Part 1, Introduction to Visual Studio Online Build and Release Agent
2. Part 2, Cost Analysis and Optimizing Cost
3. Part 3, Step-by-Step Creation Private Build Server for SharePoint
4. Part 4, Azure Automation Service
5. Part 5, Integrates Azure Automation Service in Build Pipeline

Introduction to Visual Studio Build and Release Agent

Many of you might already know about Team Foundation Server (TFS) the on-premise source code management, work item management, reporting, and developer’s collaboration tools. Since the booming of cloud services, Microsoft also bring TFS to online – known as Visual Studio Team Services (VSTS). The VSTS is a collection of hosted DevOps services for application developers, which is managed by Microsoft. Using VSTS provides additional benefits compare to TFS, because of some reasons:

1. You consume it in per-user-license model
2. You do not need to configure any “server side” feature or setup virtual machines
3. You do not need to worry about feature update, patches, etc.
4. You will have new feature update, much earlier than TFS since Microsoft adopt cloud first concept. Cloud first, means the new feature/update will arrive in cloud first before it will be shipped to on-premise version.

If you haven’t subscribed for VSTS, please heads on www.visualstudio.com and subscribe for it. And the good news is, its free for small team up to 5 users, and its free for MSDN subscriber. I know many of use, loves free goodies – so if you are running a small team just give a shot. There are many features that we can explore in VSTS, but let’s focus on Build and Release Agent. Together with the free subscription, you will receive up to 240 minutes hosted agent and maximum 30 minutes job duration for free, per month. This calculation will reset on the monthly billing cycle. You will also receive 1 free private pipelines. All of these information is available when you access Settings -> Build and Release -> Resources limits .

Build and Release Agent

Build and Release are tow of the DevOps services in VSTS and TFS that help to manage continuous integration (CI) and continuous delivery (CD) of applications. CI is automatic merging and testing of the code, it helps to catch bug early in the development cycle, which makes them less expensive to fix. CD is automatic process where the code is built, tested and deployed to one of more environment. Automatic delivery drives quality since this process can be executed multiple times in different environment with no or less human intervention – hence it consistence process across different environment. To build the code or deploy the application, you will need at least one agent. There are two types of agents in VSTS:

1. Hosted agents Hosted agents are provided and managed by Microsoft in the cloud. It is subscription based and provided as part of VSTS subscriptions add-ons. First 240 minutes every month is free.
2. Private agents Private agents are your own server that run build/release agent and connect to your VSTS tenant. Since you prepare and managed the private build server, then you have more control to install dependent software needed. For example, you might need proprietary software to build your application – then you will need to create private build server.
   

One of the reason to create private build server is because you need to consume more than 240 free build/release minutes per months and your build job is more than 30 minutes. If build minutes is the main reason, then you actually have an alternative by buying more build minutes from Visual Studio Marketplace (additional hosted pipelines, here: https://go.microsoft.com/fwlink/?LinkId=832651 ). So, how does the cost comparison between adding private build server vs subscribing build minutes in marketplace? For the comparison I will assume that we will run the 1 private build server in Azure VM vs 1 additional hosted pipelines.

Cost running 1 private build server in Azure VM

There are many VM configuration options in Azure VM, for this purpose we are looking for the cheapest configuration and yet still operational. I will use Windows OS VM since I will install SharePoint 2016 bit in the server, with Pay As You Go price. I choose Pay As You Go price comparison, so it will be same as VSTS hosted pipelines monthly subscription. B1S VM ( USD 12.41 / month) Initially I tried B1S configuration (1 CPU, 1GB Memory, 2GB temp storage) – this configuration is too slow for the build server. I have spent hours of waiting the internal operation of the server before I can install or open a console to run the script. Even after I have successfully configured the VM, it takes very long time for the service to appear in VSTS build agent readiness status. B1MS VM ( USD 23.36 / month) I thought it was memory issue, and I thought I need more memory for build server. Then I choose B1MS with slightly more memory then B1S. B1MS has 1 CPU, 2GB Memory, 4GB temp storage in its configuration. However, this configuration is also too slow for the build server. I don’t feel any performance different while configuring and connecting this configuration to VSTS. A1 v2 VM ( USD 39.42 / month) If it was not memory issue, I thought using faster disk will be better. Then I choose A1 v2 that has faster disk with comparable configuration with B1S VM. The A1 v2 VM has 1 CPU, 2 GB Memory, 10GB temp storage and faster disk. The pricing guidelines said, that it is suitable for build servers, code repositories, low-traffice websites and web application. Although this VM is better than B1S, however it is still too slow for the build server. B2S VM ( USD 47.45 / month) My next experiment is B2S VM, which double resource configuration from B1MS counterpart. This VM has 2 CPU, 4GB Memory and 8GB temp storage. This time, I can feel that overall installation experience was sufficient – it is functional and I don’t need to spend my time waiting for the Operating System to work in the background. Effectively, I can install all software which are needed for my build server. I believe that the CPU does matter, the build server will require at least 2 core CPU. I check pricing guidelines for other VM’s and B2S VM is the most cost effective and operational.

Cost of 1 additional hosted pipelines

Pricing for hosted pipelines is available here : https://marketplace.visualstudio.com/items?itemName=ms.build-release-hosted-pipelines#pricing . It costs USD 40/month per hosted pipelines.

Comparison

Private Build in Azure

Hosted Pipelines

Pros

You can install third party component for build

Provided and managed by Microsoft

Cons

Independent hosting and managing server

Can not install third party component for build

Configuration

2 Core CPU, 4GB RAM, 8GB Temp storage

Unknown

Price / month

USD 47.45 / month

USD 40.00 / month