One thing that C# could do and Visual Basic could not was development for Windows Phone 7. I will not get too much into whether this is even an issue, but I will notice that it is no longer true.
Visual Basic developers can now download Microsoft Visual Basic CTP for Windows® Phone Developer Tools.
CTP stands for Community Technical Preview and it is closest to something in alpha stage. It will not be usable for production environment and it will take a while for final version to come. However, it is a big step forward.