From what you're describing, it sounds as though it is one of two things. First and most likely, you have something blocking the microphone for the car so they can't hear it. The microphone is located in the overhead console near the map lights and sunglass holder. Between the two are a series of small ridges and in those you should be able to see holes on the driver's side that lead to the microphone. If you have anything blocking those holes, whomever you are talking to will not be able to hear you.
The second possibility is that your phone may not be fully compatible with the bluetooth system on your vehicle. If you go to
www.toyota.com/letstalk it will let you walk through the type of phone and carrier you have along with the model of vehicle and sound-system you have to see if it is fully compatible. You may need to re-connect the bluetooth to ensure it's a good connection if this is the case.
Hope this helps.
Ken