Start here, http://browse.deviantart.com/?.....fice+icons
Download (preferably) in .ico, right click the existing icon and 'Change'.

@FMZ: That is because the Microsoft Office shortcuts are specical Windows Installer shortcuts and some options like Change Icon are disabled for them. You will have to create shortcuts to the exes in C:\Program Files\Microsoft Office\OfficeXX to get shortcuts with a Change Icons button.
Note: This would only affect the shortcut. When the program is open, the taskbar button and title bar will still have the normal icon.

Cool, Txs JD. Is there a way to change the actual icon in the start menu, task bar button and title bar?

I'm not sure of a way to change all of them.