I asked a lawyer about this a while back.
In public (places where you have no expectation of privacy, like anywhere outside your home) you may record anything you want... including other people talking.
In private places (including inside someones house or on the phone) at least one party must know that they are being recorded. So if you are talking to someone on the phone and recording it, your knowing that it is being recorded is sufficient.
Basically I believe that you just can't secretly tap into peoples houses and phones. If you have a recorder in your pocket and the police show up and dont know about it thats their problem.