Subscribe to Newsletter
Tell a Friend
Print this Page
11/21/2006Novarum Compares Networks, Cuts Through Confusion
Strategic-consulting firm Novarum has launched a survey of municipal wireless networks in North America, focusing on Wi-Fi, WiMAX and 3G cellular data. Novarum’s research aim is to “cut through the confusion of this overlapping range of broadband, IP-based, licensed and unlicensed wireless data technologies,” says Phil Belanger, a wireless networking veteran who is cofounder of Novarum with Ken Biba. Belanger spoke with The W2i Report about the company’s work in the California communities of Anaheim, Galt, Palo Alto, Santa Clara, Sunnyvale and Cupertino.
How do you describe your research program?
Novarum is doing an ongoing wireless-broadband study based on the premise that these networks will be changing over time, and comparing the performance of citywide Wi-Fi networks with 3G and 2.5G cellular services. Eventually we will add WiMAX. What’s available today for people to use as wireless broadband is going to change over time. More municipal Wi-Fi clouds will be put in, but more people will use them.
Our perspective is from the user point of view. Are these networks really broadband? What’s the experience like? Which service is better — free Wi-Fi or cellular data service? We can also make some inferences about how these networks are designed. In addition to our quarterly review and city network analysis, we’ll also offer a service level agreement benchmarking service. Cities say: “We’ve done an RFP, now how do we measure and ensure the level of service”? So we offer a custom version of the testing and analysis to validate the performance and compare it to other networks we’ve tested.
Do you have a starting premise?
Our starting premise revealed a funny bias that we (and many other people) have: The cellular data services are ubiquitous. To do an interesting comparison, we will take a city with a metro Wi-Fi network and test all the wireless broadband networks from the same locations within the advertised coverage area of the Wi-Fi network. We assumed that cellular was everywhere, but what we’re finding is that those cellular data services aren’t really ubiquitous. Combined 3G and 2G cellular data service was available from about 80% of the locations we tested in the third quarter. High performance 3G service was only available in 35% of the locations we tested. For example, in Galt, California there is no 3G service at all, so the cellular services did not perform well there. Silicon Valley has good 3G service form one of the carriers but not all. In Anaheim, the cellular was mixed — some 3G and some 2G. This may be fine for cities if the low data rates are OK for them. EDGE networks — 2G and 2.5G — have pretty high availability, and if it’s a text-based application to a smart phone, that’s fine.
But many cities obviously want more than that.
If they’re thinking about streaming low-quality video from intersections, for example, that would be a problem for sure, and they would be better serviced by a municipal Wi-Fi infrastructure. Of course, municipal Wi-Fi has service-availability issues as well. Only one of the networks we tested — St. Cloud — had 100% service availability in the coverage area. For the cities we tested in Q3, the Metro Wi-Fi networks had an average of 60% service availability.
How do you define service availability?
Service availability means you can find the network, log in, and do some useful work. One of the funny things about Wi-Fi is that you can see it over a big area, but whether you can do something useful on the network is another story. So far, the biggest challenge for the Metro Wi-Fi networks we tested has been service availability.
Comparing these networks, do you see a rationale emerging, one way or the other, for single versus multiple networks?
Certainly multiple networks are going to exist. There’s no technical problem with that if they’re operating at different bands. Why you would use one or the other comes down to commercial reasons. This industry has gone through the first phase of municipal Wi-Fi — the early-adopter phase has ended — and we’ve validated the technology. But what we’re finding is a variation in performance. Each network feels different, and that’s driven by the business models and the economics of how they’re building the network out.
For example, MetroFi’s network in Santa Clara has only 50% availability in its coverage area. We talked to them about this, and their expectation is about 50%. In other words, this meets their business objectives — a free, ad-based service with a premium offering as an option. The network meets their economic goals. It meets their infrastructure investment, and it’s good enough to keep the thing rolling. But when we go test it, it feels like a thin network to us. In that area, the best wireless service was cellular. Cingular was the best performer in Santa Clara.  When you arrive in a community, what’s your methodology on the ground?
We have a two-pass approach to testing in a city. First we drive throughout the entire area we will analyze. We’re continuously testing, taking radio measurements of the infrastructure, and discovering what the real coverage area is. We’re also learning about the Wi-Fi temperature — how many APs are there. We’re driving along and continuously recording what we’re hearing, and taking lots of low-level measurements — signal strength, RSSI, GPS coordinates, etc. Simply reporting on these low-level technical measurements is not enough, and we think that kind of thing only contributes to the confusion in the market. So we also test from the user perspective.
Second, we pick 20 locations in the area to go and do performance testing. At each of these 20 locations we use IxChariot to measure download throughput and upload throughput for all of the available wireless services. We run the exact same script on all of the services so we can make apples to apples comparisons. We also measure roundtrip delay. Some of these metro Wi-Fi networks want to be used for voice. Understanding their delay profile helps us assess the ability of the networks to support voice, and we know that one of the challenges for wireless mesh is delivering low delay. In general, the delay for the Wi-Fi networks was less than half the delay for cellular networks, but there is a lot of variance in delay from one Wi-Fi network to the next.
We’re not doing a mobile throughput test at 50 mph, but we are measuring nomadic and portable performance. That is, if I create an account and log in and go somewhere else, do I still need to log in?
Have you tested any WiMAX-type networks yet?
Not yet. Our testing of Wi-Fi and cellular has already shown that there’s some overlap there, so what’s the role of WiMAX? Where does WiMAX fit? Improved cellular technologies are rolling out all the time. Will WiMAX be higher performance than 3G? Will it be more available than metro Wi-Fi after it gets deployed? We are anxious to test WiMAX-based wireless broadband services and report on how they work from a user perspective.
One of the things that’s happening is that we’re expanding the type of client device we’re using as we go and test more cities. Initially, the key focus was on PC laptops. Now we are also testing smart phones and higher power Wi-Fi clients that simulate CPE devices used for residential broadband. If you’re going to do residential broadband over a Wi-Fi cloud, then you need a CPE device to help get the Wi-Fi signal through the walls and back out to the infrastructure.
What’s the takeaway for cities from all this?
We don’t know how they’re going to interpret the data. We seek to cut through some of the hype. We are an independent voice describing the real performance and utility of these networks. We are not promoting a particular technology or vendor. If we have an agenda, it’s to help make the expectations of cities and end users more reasonable. We hope cities will make informed choices about investing in these networks.
back
Related Items:
• Molfetta, Italy
• WiMAX Spotlight Shifts to India
• Funding Rural Broadband Wireless: Alvarion Achieves "Buy American" Status for Its Mobile WiMAX Solutions
• Glenn Strachan, Project Director, Macedonia Connects
Comments
No records were found.
Post new comment:Only registered users can add comments.
Please Log-in
|
