Really stop Arduino is possible, it's called sleep mode, but the UNO is designed so badly with respect to power consumption that you may as well keep it running.
To have it done something at a specific time you could connect a real time clock (RTC) breakout board (SparkFun has a couple). Use a 1 second timer interrupt to read the clock and compare it with (a table of) times where you want things done. A table could hold the time, an output (e.g. for a relay) and the new status (on or off). If you have just a couple of those you can store them in the RTC's battery-backed up RAM.
edit
You can use your PC's time, all the hardware you need is the USB cable. On the PC you run a program that sends the current time to the Arduino. (There are Processing examples of how to communicate with the Arduino on the Net.)
Note that without an RTC your PC will have to be on all the time and connected to the Arduino. You could use the PC just to sync the Arduino's clock and keep time yourself, but the clock of the Arduino is not very accurate.
Getting the time from the Internet is also possible, but then you'll need a Wifi or Ethernet shield to make the connection.