Yeah id go with Tipsy ( now that i am awake again !) Ping the router from your computer when the connection goes down. If the ping works it's between the router and the ISP .
If you can ping from your router ( you can on some domestic one's ) to your ISP's IP number , then it could be a router problem or some sort !
Just in case , to ping go to Start> Run , and type "cmd" ( less the quote's!) in command .exe type "ping x.x.x.x"
Where x.x.x.x = your router's IP address ( usually 192.168.0.1 or 192.168.1.1 , check your manual or google for the make model ) .
If the connection is good you will get a reply , if not it will time out or get a "destination host unreachable" message.
If the ping works log on to your router ( again check the manual for details) and check if there is a utility to ping available , if so ping your IP address given to you by the ISP ( you should find this number on the router in the main config page) .
Also check the connection statistics of the router during the loss of connnection .