First you understand that torrents are not hosted in any servers. its shared(seeded) by other peoples, so you can only download at that speed of the SEEDING man's upload speed.
Speeds go down gradually over a period of time. And it is again at peak speed if I restart utorrent. So I think this is not because somebody else is using my internet.
Also, if I download from a direct HTTP download, speeds remain consistent (450 KNPS-600 KBPS).
Cheers
Nitin