You want the max possible signal voltage swing at the ptx/resistor junction as the encoder wheen spins, but it will never be zero nor will it be Vcc. If you're getting Vcc, check your connectiobs & ptx polarity. Then adjust your resistors. For a reflective sensor, you need lots of LED current, so a low LED resistor, and a high ptx resistor. 1-4 volts is pretty good signal swing, and the PicAxe's schmidt triggers do a good job of filtering: http://www.youtube.com/watch?v=PfMnl...3&feature=plcp
But as far as why your PicAxe isn't reading your 5V input signal, must be a bad electrical connection.
Thanks for your help. I arrived at the resistor values through experimentation to get the biggest voltage swing. I have 330 ohms on the LED resistor and 10K Ohms on the ptx resistor. I followed your suggestion to check the polarity of the ptx. I thought it was right but I swapped it anyway. Now I am really baffled! I get the same result with the ptx either way round! With an opaque blocker between the led and ptx I read close to zero volts on the output side. When I remove the blocker, the voltage jumps to near +5 V. I repeat, I get this result no matter which way the ptx is inserted. How do I determine which side is which? I assumed the flat (short lead) was to go to the 0V side of the circuit.