Here is a Wireless Link Test from a few months ago. Much earlier software versions would give around 220Mbps. This was a live/test network set on a tower on my family land (you can see the live data flowing in the graph). It was just my home, my parents home, and another family members home. This was our first 3000AP we acquired for testing before we mounted any for customer use. Actually... it is still like this We got busy mounting 3000's on other towers and installing customers and never moved this AP. We will be mounting it at the top of the tower in the next few weeks and migrating other customers to it.
Here is a real world graph from a sector in town. This was taken about 2 months ago. At this time there was only 4 customers on this AP. One customer could MU-MIMO with the other three. The other three could only MU-MIMO with him. Two customers are on 40Mbps packages, one on 25Mbp packages, and the one who can MU-MIMO with everyone else was set at 100Mbps. The guy with a 100Mbps package owns a PC repair store on Main Street. We told him we would leave him at 100Mbps until sector started to fill and then we would back him down to 40Mbps (or if he continued to need 100Mbps we could set something up for him since this tower has fiber). This was in exchange for him advertising us to his customers.
He takes some of his work home with him. In the graph below you can see he starts a massive download that last for around 6.5 minutes and traffic shoots to between 130-140Mbps during this time.
Here is a screen shot of his SM during this download. I almost forgot to get a capture of it, but you can see he is reaching 100Mbps consistently during his download, all while not affecting the others traffic.
Awesome details CWB. I especially like your second image, and it shows something interesting and important. That is that among the keys for when the AP will MU-MIMO and when it won't... one of those factors is ''if it needs to''. So, if it can handily deliver the data using SU-MIMO, then it won't necessarily bother MU-MIMOing. That can make it difficult to test artificially, and make it difficult to test on low-CPE sectors - even though in the real world, the heavier a sector is loaded, and the more bandwidth demands on that sector, the more likely MU-MIMO will be.
So, I particularly like your example, where your sector is going along delivering 30-40 Mbit to clients, and when another client needs 100 mbit... the AP decides to kick into MU-MIMO mode and essentially uses 'free' bandwidth to deliver this - and then goes back to SU-MIMO mode when the bandwidth peak is over. Very cool, very useful, and very spectrum efficient.
Awesome details CWB. I especially like your second image, and it shows something interesting and important. That is that among the keys for when the AP will MU-MIMO and when it won't... one of those factors is ''if it needs to''. So, if it can handily deliver the data using SU-MIMO, then it won't necessarily bother MU-MIMOing. That can make it difficult to test artificially, and make it difficult to test on low-CPE sectors - even though in the real world, the heavier a sector is loaded, and the more bandwidth demands on that sector, the more likely MU-MIMO will be.
So, I particularly like your example, where your sector is going along delivering 30-40 Mbit to clients, and when another client needs 100 mbit... the AP decides to kick into MU-MIMO mode and essentially uses 'free' bandwidth to deliver this - and then goes back to SU-MIMO mode when the bandwidth peak is over. Very cool, very useful, and very spectrum efficient.
This is exactly why I used this particular example.
Here's another partially real-world, partially synthetic result for everyone to consider. 3k AP with 19 subscribers - 13 Force 300 25 and 6 Force 200. I used the mikrotik bandwidth test from my head end to 4 of the subscribers with mikrotiks on their side of things to generate additional traffic. One of these subscribers was un-throttled and the rest others are set for 12 or 18Mbps service
Subscribers are between 3 and 8 miles out on this sector, and the furthest few out are fairly poor links. I suspect if I could have tested to more grouped STAs simultaneously, the MU numbers would have been even higher. Either way, I consider 120+Mbps of real traffic in such a situation pretty impressive on a noisy 20Mhz channel.
I had a moment to play with this again today on the same sector with a legacy force 200 client doing about 25x10Mbps at the same time. I un-capped the 4 same test subscribers as the prior image and let them fight over the airtime.
I observed peaks of ~194Mbps during this brief test. I'm not sure if the extreme saw-tooth pattern was caused by the 100Mbps ethernet ports on the mikrotiks causing a buffer issue, or if it was something else. Also, these 4 force 300 cpe are all grouped but probably not very evenly over the sector. In fact, 3 of the 4 are probably on the "left" side of the sector pattern.
I observed peaks of ~194Mbps during this brief test.
Very nice results Jacob, thanks for sharing that. :) Just to clarify, that's 194Mbps only testing download, correct? Aggregate DL + UL speeds would likely have been over 200Mbps of aggregate data in a noisy 20 Mhz channel then?
You are correct @ninedd the 194Mbps was only download traffic. Aggregate was over 200. On a noisy 20Mhz channel, with a few poor legacy clients at long distance.
Thanks for the help, I am from Brazil I have a wisp provider NETSIM TELECOM and I am in a dilemma to deliver speeds of 30 to 100 mega today most of the guys ran to fiber and I in some places will not have leaving I will need a solution and my doubt if the scrap account epmp3000 I am following the posts to see if it really is a solution to my problems, sorry the english google translation :) but you helped a lot @jacob
Here is a MU-MIMO test we did on one of our 3K AP’s with the 4x4 sector. 465 Mbit aggregate Throughput in a 40 Mhz channel
So, what I did was that I shut off QOS (so that I could push more than 60 Mbit to these client, which was the plan most of them choose). For MU-MIMO to really kick in, the AP needs to figure it can’t really do what is asked of it in SU mode, so it’s important to stress it a bit in order for it to choose to use MU mode.
Anyway, these two clients agree to help test. We did downlink & uplink tests, and we saw 390 Mbit download, and 75 Mbit upload, for 465 Mbit throughput in 40Mhz with MU-MIMO.
Ive never seen more than about 220Mbps to an client running 40Mhz. Ive got clean spectrum where my gear is the mumimo throughput is always no more than about 15mbps. Was this test done using the link test tool or was it really traffic passing to the customer’s LAN? The goal of my network was to deliver 300mbps to customers, which is why I chose cambium. I suffered through 24 months of terrible firmware to achieve this goal and I feel like it may be about to pay off, which is exciting. It’s possible my APs just haven’t been pushed hard enough, but i’d expect to see mumimo cranking like this when using linktest …
Cheers
Forgot to ask - whats your latency like? I can’t get any better than 22ms with epmp3k - f300-25. The 2.5ms tdd option went way and never came back.
Update: I hadn’t tested mumimo on new fw - just got 330 aggregate download. Not bad. Latency still terrible though, yours?
Thanks for the info. This is wonderful feedback. What distance were the clients that were used for testing? What site/sites were used for the test? What was the load on the AP at the time of the test? Is the AP using GPS timing? What is the DL/UL (75/25 etc) link ratio on the AP? These answers will help us understand more about the results and what to expect on our testing. Any other information you may find pertinent will also be helpful.
Hi. Yes, these are good clients. Both of the ones I coordinated to test with were both DS9 modulation clients, so this wouldn’t’ necessarily be the case with all clients in all situations for sure. If you have crappier connections, you’ll have crappier results for sure. My point is however – I think MU-MIMO is a game changer, and I think it under appreciated.
But in my experience, MU-MIMO gives more and more and more, the more you throw at it. For me, it’s a lot like the Multi-Core CPU and GPU’s that computers have now. The manufacturers learned that it is basically impossible to just keep increasing the CPU’s clock speed past a certain point, so they instead went to dual-core and quad-core and oct-core CPU’s… and now there are Intel i9’s with what… 12 cores and 24 threads?
And likewise with todays 12 core CPU’s, if you run a single-thread benchmark, these 12 core CPU’s don’t seem to fare any better than an i9 from 5 years ago. But, throw a BUNCH of real-world stuff at them all at once, and they just keep screaming along.
So to me - MU-MIMO is a LOT like that. And to me, it’s a game changer and it’s the way of the future. With the 8x8 ePMP4000 in the pipeline too, this will get even better and better going forward.
I’m travelling, so I’ll see about capturing a SA later. Yes, this is a relatively rural community and we’re trying really hard to get everyone’s signals as good as we can. But there are ~65 homes there, and everyone has a router, etc, etc, etc – so there is still noise, and we still do need to be careful about that noise, and need to pick channels carefully, and we need to make sure our signals and SNR are good for sure.
A mini update to this thread just because - For me, there have been a number of really frustrating issues with the 3k line (that didn’t exist with 1k/2k), and I’ve on many occasions been tempted to rip and replace my entire 3k install base of around 300 radios due to those frustrations. I have actually lost a few customers due to these issues. I’ve never lost customers due to equipment issues that couldn’t be solved in the past.
What keeps me from forklifting this gear for something else though, is that the 3k gear continues to impress me when things are good. I regularly see 3k mu-mimo sectors with 30+ subscribers delivering 100Mbps+ to the group of subs, even though they are 8 miles from the AP and in noisy conditions on a 20Mhz channel.
Example - This AP has 40 subs connected and I get no complaints, except when it had a tendency to shed all subscribers before the most recent few 4.6.1 RC firmwares. It isn’t even breaking a sweat at these throughput levels. 20Mhz and subs out 8+ miles. A mix of N and AC subscribers also:
You might see similar performance in a vacuum with UI LTU gear, but not anywhere around other wireless gear, and you certainly wont see similar performance from any other vendor’s AC gear in this sort of environment.
There are so many frustrating things - but I agree with you… the performance with MU-MIMO is undeniably remarkable. The 3000’s MU-MIMO talks to 2 SM’s at the same instant, and it’s like free bandwidth, or free spectrum. And for us, the ability to upgrade to the ePMP 4000 at some point, and to get backwards/forwards compatibility, and to get 8x8 MU-MIMO - is the killer feature for Cambium. Plus, the more you load it up, the more MU-MIMO performance it gives back.
My point is however – I think MU-MIMO is a game changer, and I think it under appreciated.
