If you want the most stable, longest supported version use 12.04. It is a LTS (Long Term Support). I prefer to never do an inplace upgrade with any OS including Linux. Too many variables that can go wrong. I think it best to do a clean install when upgrading OSs. Dual booting is not as difficult as some want to make it sound.
And I see my directions (I think you used mine) to install to the external and place GRUB in MBR of external have indeed worked just fine and simply as they are made to do. The foul up came when you did an in place upgrade. In software sources you can disable the notification to inform you when a distro upgrade is available.
Now if you have multiple internal hard disks consider this. You can do as you did with the external installation. Make the disk that does not have Windows on it first in hard disk boot order in BIOS. Install Ubuntu to that disk. You will have the same setup as what you have now except that you don't have to play with the boot order to boot either Linux or Windows. And the beauty of this is your windows disk's MBR is untouched and your linux disk's MBR has GRUB. Win-Win for the faint of heart.
That external you had would be able to boot from any BIOS that is capable of booting from USB. I think you have it down now.