Having problems connecting to your friends on Xbox Live for gaming or even just chatting? Here is a tip that could help you determine the source of the problem, and then hopefully fix it.
Go to the System blade, select Network Settings and then Test Xbox Live Connection. You’ll be able to see what sort of NAT your system is behind, and if it’s set to Moderate or Strict then you will only be able to connect to certain other users.
To fix this, set your NAT to Open by opening the following ports on your router: 88 (UDP), 3074 (UDP), and 3074 (TCP).
Source: Xbox.com Forum