Handheld GPS units are useful navigational tools as long as you know how to use it and understand its limitations.
What the GPS unit does is it gives you your position using satellites, most GPS units these days also have mapping on them so as well as giving you the grid reference which you could plot on another map they actually show your position on a map on the screen. All GPS units come with mapping, mostly it's fairly basic, it's a mistake people often make when they know their GPS has a map and they think that'll do, then they discover that it just does not show enough detail for serious navigation.
Standalone GPS come with road maps and again these are no use in the hills and assume it'll get you where you want to be the other factor with these of course is their battery life, batteries can run out especially in cold weather, so you need to carry spare batteries.
I would suggest always having two methods of navigation so that if one fails you've still got the other and I think an ideal combination is a GPS and a traditional map and compass, but you need to know how to use both of them properly.