Therefore I started used Au Group Electronics connector soldered to simple cable that plugged into PICkit. By using pogo pins I could put six through-hole pads on borad and have connectivity. Works nicely and it is as cheap as it goes.
That did worked but connector was weak spot. No matter how you hold it, there was need to re-solder broken wire every few weeks. Not too annoying, I guess...
Well, I found myself a new personal favorite.
Mill-max has a nice pogo connector "clearly" named 829-22-006-20-001101. Out of box it comes as 90° PCB-mountable connector but, with the help of pliers, I straightened those pins and they fit perfectly into PICkit 3 (they actually hold better than straight pins). Now I can press onto my board's pads with whole PICkit and that feels much better.
There is only small issue of fitting into snug spaces. With cable you can get into any space, with this "connector" things work better if connector is at board's edge. But it is small price to pay.
P.P.S. Yes, connector works with both PICkit 2 and PICkit 3.