ScottW said "That's probably what you saw -- multiple recycle folders due to different OS handling."
Windows 9X and Windows NT/2000/XP using the FAT file system (which external USB drives use by default) create a folder named "RECYCLED".
Windows NT/2000/XP using the NTFS file system create a folder named "RECYCLER".
The reason for this is explained at http://blogs.msdn.com/oldnewth.....20225.aspx
Windows Vista/7 using the NTFS file system create a folder name "$RECYCLE.BIN".
I can't remember where but I know I read somewhere that the reason for the new name was to enable new functions like Recycle Bins for Network Folders.
ScottW said "I don't know why it would be upset about this. I would think that the deleted folder would just be re-created the next time a recycle bin deletion was needed."
I don't have any idea either. This is how the system would normally handle it if I recall right (I've messed with the Recycle Bin briefly before).
The only thing I can think of it that the Recycle Bin had system files in it when the topic op deleted it that the computer relied on.
Which of course is very unusual as the system will never put system files in the Recycle Bin itself. If it happened, then the problem was most likely caused by third-party software.
Looking at the line in the original post "now I get a rundll32.exe error", I know see the problem.
I think what happened is that a third-party program installed into a directory named "RECYCLE" or similar on the hard drive.
It uses rundll32.exe to function (A common host program for small things like shell extensions).
The "rundll32.exe" is from Windows attempting to load this program/shell extension.
Here is what you need to do to solve this issue:
Download Sysinternals AutoRuns and check for entries pointing to rundll32.exe and uncheck ones pointing to the folder you deleted (D:\recycle\something.dll or similar).
Should that not help then you will need to use ShellExView to disable third-party shell extensions one by one until you find the program. I can post more on this should you need to use it.