If you’re watching the awards being doled out to AT&T, T-Mobile and Verizon for their 5G networks and your head is spinning, don’t worry. You’re not crazy for wondering what’s going on, or even why it’s still going on.
In the space of a week, each operator was awarded the crown for “fastest” 5G network. Ookla got the ball rolling last week by naming AT&T for its 4G and 5G speeds. OpenSignal named T-Mobile as the fastest on Monday, the same day RootMetrics announced that Verizon remains top dog for its network prowess.
Of course, each network analysis firm uses their own methodology, which explains some of the discrepancy. PCMag provides an explanation here. Yet questions still linger as to how the results can differ so vastly.
Lynnette Luna, principal analyst at GlobalData Technology, doesn’t put a lot of stock into these reports when it comes to ranking each operator. “They make for excellent marketing though,” she told Fierce. AT&T can point to Ookla, Verizon can point to RootMetrics and T-Mobile can point to OpenSignal. “It seems to have always been this way. If every company tested exactly the same way, I think it would be more meaningful.”
Verizon: It’s simple
To be sure, the reports serve as marketing for the operators and are popular fodder for press releases. Indeed, during its fourth-quarter earnings report on Tuesday, Verizon highlighted RootMetrics results as well as yet another J.D. Power award for network quality. In RootMetrics' tests, Verizon registered a median download speed across the U.S. of 40 Mbps.
Verizon CTO Kyle Malady sought to “demystify” the confusing array of tests during Verizon’s Up To Minute company meeting after fourth-quarter earnings were announced. “It’s actually very, very simple,” Malady said. “What we want to do is we want to drive test our networks so we can understand where the problems are” and engineer the network to make it better for customers.
RELATED: Editor’s Corner: Is Verizon losing its network advantage?
He referred to RootMetrics and J.D. Power as the “premiere” tests. While the J.D. Power survey measures customers’ perceptions, RootMetrics uses drive tests to compare devices from the various carriers in different parts of the United States. It tests networks side-by-side, at the same time and in the same locations.
It’s an “apples-to-apples comparison,” Malady said, adding that it’s a scientific method that was invented decades ago. In fact, years ago, Verizon started sending its engineers out to drive test the network to figure out where improvements were needed.
He explained that over the past several years, a new set of tests have come up based on crowd-sourced data by the likes of OpenSignal, Ookla and many others. He described those as interesting but “not a scientific method that we’re used to where you can compare apples to apples,” and more like comparing oranges to apples. “That’s why we put more credence in the RootMetrics and the J.D. Power awards,” he said. “While we think crowd source is interesting and there is useful sources of data that we use,” it’s not how Verizon chooses to refine its network. “We’re really happy to win again,” with RootMetrics and J.D.
(If Verizon won OpenSignal and Ookla tests, it probably would be happy about that, too, but that’s not what he said.)
For its part, Ookla says it has a team of data scientists dedicated to ensuring its claims about networks are accurate, representative and fair. Opensignal says it’s all about combining real-world measurements with scientific analysis to provide independent insights on mobile connectivity globally.
RELATED: 5G download speeds in U.S. at meager 47-58 Mbps range, per Opensignal
Analysts point out that RootMetrics’ data includes rural areas, which is where Verizon historically holds an advantage. It also takes longer to collect the drive test data and it could be perceived as out of date by the time it’s reported. That’s part of what T-Mobile President of Technology Neville Ray complained about back in 2016.
AT&T gets some love
Given the back and forth between Verizon and T-Mobile that has dominated headlines in recent months, what’s the deal with AT&T? It hasn’t received a lot of attention lately for its network prowess, but it was the fastest mobile operator during the fourth quarter of 2020, according to the Ookla Speedtest, scoring 50.27. T-Mobile was second and Verizon was fourth. (Ookla will report results for T-Mobile and Sprint separately until their networks are fully integrated; Sprint came in third.)
AT&T has been able to deploy a significant amount of disparate spectrum bands across multiple low and mid-frequency bands, which might be a contributing factor to its overall improvement in download speeds over the past few quarters, according to Ookla’s in-house mobile network expert Milan Milanovic. It’s also worth noting that Ookla tracks FirstNet separate from AT&T, so that’s not what’s boosting AT&T’s results.
RELATED: AT&T scores fastest 5G network, Verizon’s availability drags down speed
In addition, the proliferation of flagship devices with modern chipsets capable of advanced LTE features and carrier aggregation of four or more frequency bands (in some markets) allowed AT&T to elevate its median download speeds, according to Milanovic. In parallel, AT&T has rolled out a non-standalone low-band 5G service on top of its existing mid-band LTE anchor bands.
Movers & shakers
Some of the discrepancy between the various reports could be chalked up to how fast the U.S. operators are moving. A lot has happened in the past three to six months. Verizon launched Nationwide 5G in October using dynamic spectrum sharing (DSS) and lower band spectrum, and that’s bringing down its speeds in some areas. T-Mobile was busy rolling out 5G using 2.5 GHz from the Sprint acquisition, and that’s boosting performances.
RootMetrics and others have seen, across all carriers, that 5G in the low bands is about on par with 4G in terms of throughput speeds and in some cases, it’s not as fast as LTE, noted Doug King, director of Customer Success at RootMetrics. “You don’t get the real impact of 5G and the blazing fast throughput speeds until you’re on the high frequencies,” he said. “It’s kind of early on with 5G.” Verizon boasts the “world’s fastest 5G” in its Ultra Wideband markets where it uses millimeter wave spectrum, but that’s extremely limited in coverage/availability.
T-Mobile will further benefit from the 5G deployment of the 2.5 GHz spectrum, and that will be nationwide by the end of 2021. The industry expects to see great things once that’s more fully deployed, and so does RootMetrics, King said. “We haven’t seen really that significant use of it yet or coverage of that in the tests that we’ve done,” he said. “We do expect there to be continued improvements there.”
Interestingly, in T-Mobile’s home state of Washington, Verizon was named the Rootscore winner, with T-Mobile in third place. Specifically for the Seattle market, Verizon also came out on top, with T-Mobile in second place.
Not mentioned in the spate of recent reports is Tutela, which publishes an annual report in September and monthly snapshots along the way. Verizon won the most recent title for excellence in consistent network quality based on Tutela’s results. Based on last month’s data, AT&T was in first place in the category of download throughput. Tutela, which uses network performance testing software that runs in the background on phones, also measures latency, jitter and other things.
There’s a lot of discussion around which methodology is the best and fairest, acknowledged Chris Mills, head of industry analysis at Tutela. But the methodology differences only go so far.
“Ultimately, if you’ve got lots of competition between the operators and very close network performance, you get the kind of results that we see now, where some operators are winning some reports but then there’s reports where they’re not sweeping every market or every metric,” he said.