I know what you are thinking. Can't I just make a shortcut? Of course you can. But using this tweak you can recreate the native Internet Explorer icon the way it used to be in prior versions of Windows.
The native icon gives you two benefits: First, you can right-click and get directly to the Internet Options panel, and secondly it doesn't have the shortcut icon arrow.
Update: This doesn't work in Windows 7. See this Microsoft KB article for answers, but the short version is that you'll need to just create a regular shortcut instead.
Manual Registry Tweak
Open up regedit.exe through the start menu search or run box, and then browse down to the following key:
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\
HideDesktopIcons\NewStartPanel]
Find the key named "{871C5380-42A0-1069-A2EA-08002B30309D}" on the right-hand side, and then set the value to 0. If the key does not exist, then create a new 32-bit DWORD value with the same name. To remove the icon, you could set the value to 1 instead.
Note that if you are using the classic start menu, you should create this value under the ClassicStartMenu key instead of the NewStartPanel key.
Download Registry Hack
Even easier, you can just use one of the registry files I've prepared. Just unzip and double-click to enter into the registry.