AWS Experience: How to Reduce Costs and Increase the Effectiveness

AWS Experience
Table of contents

In our company work the developers that do their best to develop qualitative software. We use the test-driven development approach and write the tests before the actual code to make it as reliable and effective as possible.

Additionally, we use automated builds for running the tests on every project when making a branch on Gitlab.

However, with the growth of our company, the infrastructure couldn’t handle the number our developers uploaded into DVCS. We came up with several solutions for this challenge.

Thus, we tried to run our tests faster than it was on the personal stations of our developers. Read this article to find out how we implemented this solution.

We also tried to resolve an issue with our infrastructure with an affordable price for its support.

Before moving all the infrastructure to Autoscale runners, we used self-hosting hardware located in our office. Among the expenditures were power costs. One machine cost was about 2-3$ per month. They functioned around-the-clock even if they were not loaded.

Moreover, because of loading, we couldn’t run the necessary number of builds concurrently. As a result, the average amount of build’s timeout could take up to 10 minutes. All the builds were terribly slow and unstable. All of these significantly slowed down the development process.


Our DevOps department reached a solution to start using Autoscale EC2 instances. This provided us with a possibility of using the necessary “power” as well as keeping the builds on in the needed moments.

So we got rid of our self-hosted solutions and start using AWS EC2. This was the beginning of our new infrastructure optimization.

We had 4 machines that were used as Gitlab Runners. The cost of every machine was around $500. All other vacant machines were given to our new developers so that we could save much money on buying the new ones.

We used the following runner’s settings.

Based on the settings, we could concurrently run 20 builds on 10 machines with 1 GB memory on each one.

We used packer for creating our personal AMI . Here’s how a config looks like.

This config allows creating AMI that is suitable for our work. There can be found the settings for ElasticSearch, SWAP as well as our keys for connecting with GitLab.

The Result

Using this configuration we significantly increased the effectiveness of our runners. Moreover, we solved an issue with builds lagging.

Thanks to Knapsack, all the builds have been parallelized to the great amount of machines.

Using Knapsack and fail-fast strategies, we were able to reduce the average amount of build on 25-30%. And now let the numbers talk for themselves.

The total infrastructure’s cost: $23

In contrast:

4 machines:

Monthly ~$12-14 + time spent waiting for the builds

1-time payment: $2000 cost of unused machines

For 1 Year: $2156

Autoscale Runners:

Monthly: ~$23 with no time for waiting

For 1 year: $276

As you can see, we successfully reduced both the company’s expenditures and time for the code delivery.

Share on facebook
Share on twitter
Share on linkedin

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

Share This Post

Share on facebook
Share on twitter
Share on linkedin
Share on whatsapp
Share on email
Share on vk

Complete step-by-step guide

How to find the best app development company for your App?

Freelancer vs Agency. Pros & Cons
How & Where to look for developers
9 signs of a good and bad developer
11 questions to ask any developer

Looking for a solution to your problem?

Please fill out this quick form and we'll be in touch with lightning speed

More To Explore

Save the link to read the article later.

Send material to:

Share on facebook
Share on twitter
Share on linkedin
Share on whatsapp
Share on email
Share on vk
Share on telegram
Share on skype