Yahoo
Advertisement
Advertisement
Advertisement
PC Mag

How We Test VPNs

Justyn Newman
VPN conceptual art
VPN conceptual art - Credit: René Ramos

Too many security products trade on fear, uncertainty, and doubt among customers and the media. At the same time, recommending a flawed product risks putting people's privacy and even their safety in danger. This is especially true for virtual private networks, or VPNs. When we test VPNs , we consider their performance and available features to write factual and useful reviews for our readers.

This is harder than it sounds. If we relied entirely upon objective measurements, it would be trivial for a vendor to inflate stats like server count or the number of simultaneous connections. If we relied only on subjective observations, we'd miss the features that make it unique. Combining the two—objective measurement and subjective analysis—is messier but leads to better and more comprehensive results.

A Note About Ethics

In an era of phony reviews and mounting concern over pay-for-play content, it is important for readers to understand how PCMag earns money and how all of our reviews are written. At the top of every review on PCMag, VPN or otherwise, is the following statement:

Advertisement
Advertisement

PCMag editors select and review products independently. If you buy through affiliate links, we may earn commissions, which help support our testing.

In practice, this means that PCMag may earn a commission from either the company whose product is being reviewed or another entity. It's a common practice among review sites. All handling of affiliate commissions is entirely separate from our editorial process and managed by a completely separate staff. By design, reviewers do not have any knowledge of the specific ways in which a particular review is monetized. Nor do reviewers or editors receive a cut of that monetization. Reviewers, both full-time and freelance, are paid for their work and do not earn commissions or bonuses for the reviews they produce.

Similarly, we at PCMag must be as transparent as possible about our relationships with vendors. Our parent company, Ziff Davis, is nearly a century old and has expanded to encompass many ventures beyond publishing, and has acquired several software companies, including some VPNs. We always disclose whether a product is owned by our parent company whenever it is mentioned. Moreover, these companies do not receive any special treatment or access to our work at PCMag. We extend only one courtesy to Ziff Davis companies: we inform them when a review will be published. Nothing else.

Ziff Davis also owns one of our speed-testing partners, Ookla . Best known for its Speedtest.net bandwidth benchmarking tool, we've been granted access to the backend of Ookla's network to ensure consistent and reliable tests each time. This removes any variables that could affect running the bandwidth test in a browser or other app, including unseen page elements that might impact download speeds. Each time we use the tool to test a VPN, the relationship between Ziff Davis and Ookla is disclosed in the copy.

Importantly, companies—even those with affiliate relationships with PCMag or owned by our parent company—do not dictate the outcome of reviews. This is assured by both PCMag's code of ethics and our reviewers' collective bargaining agreement, which is a legally binding document. The members of our editorial staff value their reputations and would not stake them on corporate nepotism.

Advertisement
Advertisement

Why are we focusing so much on our editorial ethics? First, we're proud of it, and it bears repeating. Second, it is especially relevant in the discussion of VPNs. Security software attracts readers who are concerned with fairness and transparency. The VPN market is awash with suspicion. Some of it is paranoia; pay-for-play "review" sites have created some, and some—purportedly—are stoked by VPN companies. Rest assured, we hear it when readers worry about the ethics of VPN reporting, and it's important to us, too.

Our Testing Criteria

As PCMag's primary VPN reviewer, when I evaluate the privacy-protecting abilities of a VPN, we look at:

  • The privacy policy with a focus on readability and uncovering what information is logged

  • The technology it uses (connection protocols)

  • The servers it makes available (RAM-only, number of server locations, physical vs. virtual servers, etc.)

  • The presence of privacy-enhancing tools (primarily multi-hop connections and additional obfuscation tools)

  • The measures a company takes to ensure that the VPN itself does not become a threat to user privacy

This last point can include everything from third-party audits to adopting no-logging policies to using an anonymizing login system.

Streaming is a large concern for some readers, but that’s not my main focus. A VPN’s ability to access a specific regional catalog can change daily due to server blocklist and new VPN-blocking measures from streaming services. I test each VPN against five regional Netflix catalogs (US, UK, Australia, Japan, and Canada) and compile the results into comprehensive instructions for readers interested in using a VPN for streaming. However, a VPN’s success or failure in accessing a given catalog doesn’t weigh heavily in my scoring process.

Pricing and Plans

My main consideration when evaluating a VPN's pricing is the monthly cost, excluding discounts or deals. Deals change all the time, and long-term discounts come with some caveats that I don’t think are worth the initial price savings. My goal is to report that base-level price and evaluate the VPN’s features with that monthly cost in mind. I also pay close attention to price transparency—if you get a promotional rate as a new user, then the full renewal cost should be clearly stated before the payment page.

Advertisement
Advertisement

Below are all the prices of the VPNs I have tested—monthly, yearly, and beyond—recorded at the time of review:

Another reason I report the price of a VPN's monthly plan is to encourage readers to start with a free trial or a short-term subscription. Too often, readers reach out to say they spent $60 or more on a year-long VPN subscription only to discover it doesn't work for them. It's far better to try out a service for a month or two and decide later to pay for a long-term, discounted plan when you're certain you want to keep it.

Additional Features

With more VPN services popping up, companies have begun adding more and varied features to their offerings to stand out. In my reviews, I report as many features as possible, but focus on those that I believe most protect user privacy and reflect the service's value. The number of devices the VPN service allows you to connect simultaneously, for example, is a concrete measure of value and a point we always mention in our VPN reviews.

Each VPN review also notes the most significant add-ons available from a VPN service. These add-ons usually include static IP addresses, additional simultaneous connections, and so on. I generally do not test these add-ons and instead focus on the core product being sold to consumers.

Advertisement
Advertisement

Beyond standard add-ons, many of the top VPNs now position themselves as all-in-one security suites rather than just a VPN service. In these instances, I collaborate with other members of PCMag's security team to deliver a combined review of the services being offered.

Server Numbers and Distribution

A key differentiator among VPNs is server distribution. If a VPN company offers no servers where you are or where you wish to spoof your location, it won't be very useful. I tend to refer to server locations and their geographic spread as "geographic diversity." In general, I prefer services with servers in many different parts of the world. It's particularly important for frequent travelers and users abroad, since a VPN server closer to your location will likely yield a faster, more reliable connection. For users in the US, more server locations mean more opportunities to spoof your location.

I do not test every single connection to ensure it is functional. This is one place where I must assume companies are telling the truth about their products, since services can have tens of thousands of servers. However, I investigate whether a server is unavailable or not routed to the correct location.

Most VPN companies offer servers in Asia (sometimes excluding China, as explained below), Australia, Canada, the US, and Western Europe. Better services include servers in Africa, Eastern Europe, the Middle East, South America, and Southeast Asia.

Advertisement
Advertisement

Each review includes the current number of servers the VPN company provides. With more servers available, the VPN company can automatically assign fewer people to each or give those connecting manually the ability to spread themselves out more evenly across the network. Ultimately, this means a bigger slice of the bandwidth pie for anyone connecting during peak usage hours.

This figure, however, is just part of the story. Most VPN companies spin up new servers as needed to meet demand, so the exact number of servers can change frequently. It also doesn't make sense for a small company with only a few thousand subscribers to have as many servers available as a company with a million subscribers. A company might also seek to inflate its server count by using numerous virtual servers, which I explain in greater detail below. I try to balance these considerations in reviews since server networks between large and small providers often aren’t directly comparable.

When talking with VPN companies, I ask about the number of virtual locations and virtual servers. Virtual servers are software-defined, meaning a single hardware machine can host many virtual servers. Virtual locations are servers configured to appear somewhere other than where they are physically located.

Neither is inherently bad. Virtual servers allow VPN companies to quickly respond to user demand and keep their networks ticking over nicely. Virtual locations can also expand a company's reach and sometimes provide VPN protection in regions where it's unsafe to host servers physically. I look for transparency. It is important that virtual locations are clearly marked and that the company has standards and practices regarding the security of its server infrastructure.

VPNs and Censorship

VPNs are especially useful for people living or working in countries whose governments have chosen to restrict information and punish dissent. Recommending a VPN that doesn’t work as advertised in such a country could lead to dire consequences for the user. Given those stakes and the fact that I cannot physically test from these locations, I believe it would be unethical to choose a service that would be "best" for circumventing censorship.

Advertisement
Advertisement

I do appreciate and elevate companies that contribute to a free and open society by giving no-cost access to users. My reviews note whether or not a VPN company offers servers in countries with particularly restrictive internet policies. Just bear in mind that I cannot fully test the efficacy of these servers.

When I write stories about a VPN that would be "best" for a region with repressive policies, I base it on the availability of local servers, the region, and any documentation provided by the VPN company. In these stories, I try to make it clear that readers seeking to circumvent censorship do so at their own risk, and my evaluations cannot guarantee safety.

User Experience

Even the best, most secure VPN needs to be easy to use. Nobody wants to deal with a frustrating, poorly thought-out user interface with messy menus or an unappealing design. A well-designed security product that average consumers can actually use is better than a perfect security tool that is only accessible via the command line.

I go through the setup process for each VPN on various platforms to see how easy it is for a general user to install. I also poke around the different menus and see if advanced features are clearly explained. It's important that you, as a reader, have a sense of what using a given product will be like from reading my reviews.

Advertisement
Advertisement

Sometimes, an excellent user experience makes a mediocre product better. Conversely, a poor user experience undercuts the value of an otherwise stellar product. In general, I place great emphasis on a product being easy to use and accessible to users with all levels of experience. At the same time, I cannot deny the importance of technical excellence, especially when it's combined with value.

VPN Protocols

There are several ways to create a VPN connection, but not all are equal. I prefer to see open-source, audited protocols like WireGuard . It also has a reputation among security professionals for providing better speeds and more reliable connections. OpenVPN is another good choice, as it uses newer, more secure technology than older protocols such as SSTP, PPTP, IKEv2, and IPSec.

Some services have also deployed their own VPN protocols alongside a selection of the classics. I’ve spoken directly with representatives of these companies about their protocols, and I've learned that they're built on established, validated tools.

While most proprietary protocols are based on existing technologies, I recommend proceeding with caution. Any number of changes could result in compromised data due to an unforeseen vulnerability. I prefer to see VPNs that use proven, open-source protocols. While yet unproven in real-world applications, I also pay close attention to proactive services that have deployed post-quantum encryption. I cover what post-quantum technology is and its importance in this overview.

VPN Speed Testing

I test VPNs using an Intel NUC 12 Enthusiast Kit ('Serpent Canyon') benchmark PC located in Portland, Oregon, on a CenturyLink 1Gbps simultaneous up/down connection.

Advertisement
Advertisement

Many readers are concerned about a VPN's impact on internet speeds . That's understandable since most VPNs increase latency and slow your internet connection. This happens because a VPN takes your internet traffic and runs it through extra steps to encrypt it.

While I evaluate each VPN’s speed and performance, I do not consider it a core criterion, since performance scores can vary widely from user to user. Base internet speeds and your distance from the closest server play major roles in the performance you experience when connected to a server. Due to this variance, I only draw attention to performance when a service scores exceptionally well or performs below the norm.

To assess a VPN’s speeds, I run the Ookla SpeedTest tool 10 times with the VPN active, then 10 times with the VPN inactive. I then take the median of each set of results and compare them to find the percentage change. The Ookla test returns latency, upload, and download speeds, so I use those metrics as well.

The chart you see below appears in all VPN reviews and updates automatically with the latest results:

Advertisement
Advertisement

(Note: Ookla is owned by Ziff Davis, PCMag.com's parent company. For more, see the ethics policy in our Editorial Mission Statement .)

Despite gathering numerous test results, it remains a single data point and is insufficient to definitively judge a service's overall network performance. Because of this limitation, I present speed test results not as the final word in a VPN's performance but as a snapshot. It is meant to say that on this given day and time, this VPN performed this way.

Trust and Privacy

A VPN company has the same level of insight into your online activities as your ISP does when you connect to its network. Because of that level of access, it's important that you trust the VPN company you sign up with and that you are comfortable with the potential pitfalls of using a VPN.

I read each service's privacy policy. In particular, I look for what information a company gathers about its customers and their behavior, how the company protects user information, and how the company responds to requests for information from governments and law enforcement. I also directly ask VPN companies to explain their policies and disclose the legal jurisdiction under which they operate, how they make money, and the name of any parent company.

It is, of course, entirely possible for a VPN company to lie to the public in their privacy policy and lie to me during interviews. The goal of these questions is to put them on record in case contrary information surfaces in the future.

I judge VPN companies by the steps they take to protect their customers. All companies will have to respond to legal requests for information somehow. The best will present nothing, or very little, because they have no user-identifiable information. I prefer that companies be transparent about their efforts to protect customers, and also their interactions with law enforcement. Companies should issue transparency reports that record requests for information and the company's responses.

While I note a company's location and factor in local surveillance laws, I do not judge a company solely on its country of origin. Rumor and fear-mongering are not unheard of in the security industry, nor is using baseless fears over race, class, and other factors. For example, China and Russia have been accused of numerous cyberattacks against the US and are known for fostering oppressive environments domestically. Because of this, some consumers refuse to use security products from these countries, believing they are inherently compromised.

I do not penalize a product solely for its country of origin. I present the information and provide context while encouraging readers to make their own choices. The only time I discuss the country of origin in detail is to present local laws on data retention. For example, Panama is famed for its hands-off approach to the prosecution of many different industries, which is part of the reason you'll see some big companies call it home.

Meanwhile, most Western nations are part of the Five Eyes intelligence-sharing network, which allows them to trade classified intelligence for investigative purposes. If your VPN keeps logs and you used its network for illegal activities in, say, the UK, but later moved to the US, both governments could openly share intelligence on your activities without much bureaucratic friction over your rights or citizenship.

No company is immune to attacks or data breaches . Instead of holding companies to an impossible standard, I examine how a company responded to a breach and what the service did to protect its users. I must also give companies space to change. If a shifty product is reborn and bad past practices abandoned, I view that change favorably, if skeptically.

My testing process assumes that the VPN companies being reviewed are good actors that are operating in good faith. In addition to my own research, I rely heavily on the work of security researchers who have unmasked some of the worst behavior among VPN providers, and on the robust security community that is quick to point out the flaws in any product.

VPNs Beyond Windows

Windows remains the dominant desktop operating system, so the bulk of my VPN testing and analysis is performed on a Windows machine. Speed testing is carried out entirely on my Windows 11 NUC. Still, it's critical that a VPN perform well across all platforms.

I also install native VPN apps on Android , iOS , and macOS devices . My main consideration is for consistency in design and features. I then assess platform-specific features and ensure the VPN works as expected on each supported operating system.

Each platform has its own design language, and VPN apps should speak it fluently. On mobile devices, you interact via a touch screen rather than a keyboard and mouse/trackpad. iOS and Android look as different as Windows and macOS, and VPN apps should blend in with their surroundings. A VPN app should be visually and functionally similar across all platforms but tailored to each platform—whether it's for Android or iPhone. Ease of use is a major criterion for an excellent VPN, especially for mobile VPN apps.

The Evolution of VPN Testing

Our aim is to create informative, meaningful reviews that are based on objective, reproducible testing. We strive to make each review understandable without advanced security or privacy knowledge, so all our readers can get digestible advice. As such, our reviews take a general-audience approach that may not include every merit or demerit of a service, in favor of lucidity.

We regularly evolve and refine our testing procedures. This approach can and will change as developments in the VPN space and the broader security landscape unfold. Perhaps a new technology will completely upend what makes a VPN worthy. Whatever the case, the VPN reviews you read here will always be as accurate and useful as we can make them.

Advertisement
Advertisement
Mobilize your Website
View Site in Mobile | Classic
Share by: