I recently picked up a voice recorder that has a USB port. Its an Olympus, and I got it through Amazon for $43 IIRC.
Be sure to read the reviews on any you select. One device that had a USB port and plugged directly into the PC, it got bad reviews because the weight of the device caused the USB to disconnect.
You want one with fairly large sized recording capability, voice activated recording, and USB port.