There seems to be three options for powering a PICAXE - running from the PV cell, running from the rechargeable battery or running from the LED. All of which have their pro's and con's and possible limitations; PV only works when light is present, battery may be too low a voltage and PICAXE current may extend charging time, LED may be pulsed rather than steady output.
There also seems to be three types of solar lights - Original with lots of transistors, IC with light sensor, IC which uses PV as light sensor - plus two and one battery types. The ones I picked up in my local PoundLand store have a very tiny rechargeable battery, about 1cm in length, but I've no idea of the rating.
Given they are often so cheap they are ideal candidates for 'try it and see' experimenting.