Force 200 with management and customer vlan. Private IP address to customer.
Customer is using a Ubuntu server as router and server. Customer hosts its own voip and coin mining operation. Was formerly on a 3Mbps DSL connection that degraded over time to 1Mbps or less. Changed to a Tmobile cell modem. Server and network worked fine on both. Recently changed to our service for higher throughput.
Network consists of one NIC installed in server for WAN and one NIC installed in server for LAN, which feeds a unifi switch.
Here is where things get weird. These test are made directly on the server, but is constant across the network. Customer can pull up speedtest.net in Firefox, run a speed test and gets his full package (25/3). Customer can not pull up almost any other web page. Voip service does not work. After a few minutes, full internet activity and voip is restored for 15-30 seconds, then it all dies again for quite a while (except speedtest.net). Windows pc's work as expected when connected directly to Force 200.
We do not know enough about Ubuntu to trouble shoot this issue. The customers son built this server and network. He is in Iowa (we are in NC) and was hard to reach while we were at customer location. Son later tried to log into the server using Team Viewer, but my understanding is it was unsuccessful (duh).
Before I go doing a ton of research on the web, has anyone seen this before, or have any ideas on what it could be? MTU, DNS resolution in server, etc....??
We went this afternoon and installed a cnPilot R200 that is connected to cnMaestro so we could track connectivity and run tests. Unit has been up for over 8 hours now and we have no problems running ping or traceroutes from the router out onto the net. Tablets and PC's that were connected to the R200 had no issues while we were there as well.
While there owner said now when internet connectivity dies when connected to the Ubuntu server, it all dies. No more speedtest.net working.
From what we have seen tonight, it is definitely with Ubuntu. All other devices connected to ethernet cable of Force 200 keep internet connectivity.
We are still open to any ideas anyone may have. If we can not figure this out, we are going to discuss with the owner using a Mikrotik for routing and let the server be that...just a server. Hopefully the Ubuntu server will play well with the Mikrotik.
What hardware are they running Ubuntu on? Could it be that the NIC's in the hardware are not well supported by Ubuntu and are having intermittent issues? We typically only recommend and use high end Intel based NIC's for any Linux VM's.
This is a very reasonable deduction. When install guys went to this location, I asked them to test before an actual install since customer was using hardware and software we had never tested our network on before. They went up about 40 ft in the bucket and connected to Ubuntu server with about 100' of ethernet cable. They -said- all went well, web pages, phones, etc. They only tested for a few minutes then went ahead with permanent install.
Permanent install is 50 feet in a tree and end of customer building. They are running close to 300' of ethernet cable. There is also a surge suppressor (1Gbps) at bottom of tree and one at entrance to building. Good quality ethernet cable (Vertical Cable; shielded w/double jaket) was used.I did notice while I was there testing later that one of the laptop PC's only connected at 100Mb (has 1G port) while the others connected at 1G. Maybe the NIC card in the server is not sensitive enough and is having issues. I will program a Mikrotik as a switch and take it out shortly. Connect Force 200 and server to the "switch" so the server will see the "switch" as its ethernet source. Hopefully this will work.
Well...that was not the problem. With the Tik acting as a switch, we stil had the same problem. I ended up connecting the R200 that was in router mode directly to the Unifi switch. ALL his conventional equipment worked fine, 4 Microsoft PC's and anything connected to his several Ubiquiti access points (tablets, cell phones, printers, etc). He called his son about our findings and it did not go well.
Looks like we will be using a Tik for routing and he will be using a cloud server for his phones and files...or however he wishes to do things.
Still not sure what the issue is, but as I told him, my job is to bring in reliable internet. We have done that.
Any of the Mikrotik interfaces involved show FCS or CRC errors? All of them linked up at the correct speeds? I guess this could also be some sort of MTU issue, although I'm not an expert on tracking down those types of issues.
No ethernet errors with TIk or F200 at all and ALL other devices on network work perfectly when we bypass Ubuntu DHCP server, and use a R200 as the router (connect it to Force 200 and then to Unifi swith, with Ubuntu server completely out of the picture).
I am leaning toward a MTU issue as well, but his son became defensive when that was mentioned stating server worked fine with earthstink and Tmobile. Our network is set for MTU of 1528. When his daugher pulled up MTU for Ubuntu server in a terminal window, it came back as 1500. They were careful for me to not see too much. They did mention something about an issue with loopback 127.0.0.0. They may have not known what they were talking about...who knows.
I am not a network guy. I can set up small networks with vlans, etc..but nothing on a large scale involving OSPF, MPLS, BGP etc. We use a network firm a few states away for our core routers. They were all at Wispapalooza and would not answers calls or text. Kind of upset me with the money we have spent with them and a few big projects we have in the works with those guys. We ran into this same $h!t when getting started. Knew what we wanted and only got lip service from all the recommended guys on the Tik site. Told us how awesome they were...over several phone calls... but could never give us a clear SOW (Statement of Work) for what we would get for x amount of dollars. So...if anyone knows any reliable network engineers, please PM me.
Stepping off the soap box....Using Firefox on the Ubuntu server, I can reach the GUI of the AP it is connected to, tower router, and switch with no problems. It just will not pull internet. This is why I told him we have done our part. Brought you internet that works. He said he understood and hopefully he and his son could work the issue out on their end.