The handbrake shoes will take a few dergrees of drum rotaion to fully energize. There are three separate adjustments required. Few people adjust them correctly. Never adjust the cable before first adjusting the shoe to drum clearance. What you describe is fairly normal, shouldn't be anywhere near as much as an army Landrover though as the transmission mounted handbrake includes diff backlash as well.