Methodology

The word review dates back to the 15th century. It’s a noun denoting a formal inspection of military or naval forces.

Term “review” originates from obsolete French reveue, from revoir – ‘to see again‘.

In modern times review is “a formal assessment of something with the intention of instituting change if necessary”.

https://en.oxforddictionaries.com/definition/review

VPN service reviews come from all directions, but nothing makes us happier than finding a leakproof provider who has made an effort for its customers and of course the hidden gems – services that are truly undervalued.

It all starts with a list of companies that have a ready-to-use product and could be considered as trustworthy. Trust is the core of all VPNs.

1. Digging Into Company Profile

We search their court case history, logging, location, whether they have separate companies for payment processing, who are the owners, reputation, and more. We sometimes even use lawyers to take a look at their documents.

Company profiles usually give us an overview of the companies’ business practices. Privacy policy and terms of service are the first to look into.

HotSpot Shield VPN logging practices

2. Customer Support

We ask them a wide range of questions to see how they interact with customers. There have been cases when a provider had a live chat feature, but the operators never reacted to the initiated conversation.

Sometimes it takes a lot of back-and-forths.

Some companies have flawless support. The agents are personal and interested in making the customer feel at ease. Yet, some have enormously strict rules about responses, which makes us feel like we’re talking to robots.

Fun fact: Out of 91 contacted VPN services Cloak VPN took the longest time to respond to our e-mail – 11,841 minutes.
(Nonetheless, there are companies which never respond.)

3. We Buy Their Service

We buy their service for the purpose of testing.

4. Setup Files are Virus-Tested With VirusTotal

VPNinja Virustotal.com test

5. The Pandora’s Box is Opened

You’ll never know what’s coming..

Our laptop desktop.

Setting up the software is part of the user experience and it’s important to take note of the time spent on configuring the client. Not all VPNs have a standalone program. If the configuration files have to be integrated with VPN implementing software like OpenVPN, we note it as a drawback.

Usability comes next. We will tune every knob and checkbox and see which of them gives errors.

6. Speed Tests

ShadeYouVPN. Maybe a bit too low.

We do one test without VPN and right after that try 3 countries from their server list. The countries include USA, UK, and China, though we have had to use Japanese servers due to lack of available servers.

This gives an overview of speeds on 3 continents.

However, note that the speed varies by ISP and location. The speed tests are mainly done to get an overall comparison of every VPN.

7. DNS leak Tests.

Leak tests are done once we’re connected to the provider’s service (random server). We use 5 main web tools to detect DNS leaks.

8. Netflix and Torrenting Availability

We test at least 3 servers, if downloading a torrent from a random server fails the first time, we will do 2 more tests with other servers.

Torrenting for testing purpose

9. Partner in China Tests the Great Firewall of China

Tests are conducted with China Unicom Shandong (CHINA UNICOM china169 Backbone) ISP.

10. Usability Tests

If the provider offers service on iOS devices, we also do a simple usability test to see how it feels for the customer.

Example: PIA lets you choose between light and dark mode on your iOS device.