No matter how good your programming is, there will be bugs. Once exception is thrown (hopefully your have centralized exception handling) you need to do something with it. Here is what I found that works best for me.
1. Trace it. In cases where other steps fail, this will at least give you basic information although you will need to use some tool like DebugView in order to see it. Think of it like fail-safe mechanism.
2. Write report to file. I just create file named ErrorReport.something.txt in temporary folder. Here I collect all information that is available to me. Time, message, stack trace, whatever you can think of. This same report will be used in next step so it is not waste of time.
3. Show message to user. Don't go into details, just be nice and apologetic.
4. Send collected information. Be sure that user knows what you will do. They will not take kindly if their firewall detects data going out and they haven't authorized it. If sending is successful, you may delete report file from step 2.
5. Exit application. Be sure to use exit code different than zero for this. Console guys can get dangerous if you don't.