HDD - Love those Seagates, even a late model. Quiet, reliable, and a 5-year warranty -- you can't go wrong.
Mobo - I'm no expert on AMD chipsets, but that 690G is certainly loaded with features. With built-in graphics and HD audio, you pretty much have a whole new system here. One thing that jumped out at me: only 4 USB ports ready, with 3 headers on the board. I would need more than 4 to get by, so I hope you have places to connect the other (up to) 6 ports.
Proc - Again, no AMD expert, but this one seems a little behind the times. 90 mm fab and 2x1MB L2? Maybe that's how AMD keeps the price down. If so, that's fine.
RAM - with that mobo, it's DDR2 800 max. From what I have seen, DDR2 DIMMs are pretty cheap these days. Pick a trusted brand and the best timings you can get within your budget. You could even drop back to DDR2 667 with tight timings if it saves money. 4 GB sounds good, especially if you plan to use that integrated 690G graphics. The integrated graphics will take it's RAM from main memory -- up to 1 GB!
Since you will be running Ubuntu and not Windows, these parts should be plenty. Here's something I never thought about: can you run 64-bit Ubuntu to use all 4 GB of memory? If you use only 32-bit Ubuntu, what's the max memory available to the OS?