Just a heads up for anyone that runs into this.
We just finished off upgrading our entire network to 4.3.2. Had a bunch of XM's that didn't come back. After more digging into the AP we found the Guard Interval setting under "Radio". This was set to short. It must be set to Long for any XM to connect.
After changing this the rest of the XM's connected. So check that out if you're having issues.