Yeah sounds like a shifter cable retainer came loose (U clip) or shifter rubber end link tore,ripped, deteriorated.
If the U clip is missing you can use any clip from any Toyota with a manual trans.
These work well if you find your bushing is torn up.
http://www.twosrus.com/catalog/produ...products_id=78