If you dissect striated muscle out of most-any organism, the actual contractile apparatus works over a wide range of temperatures. So that's at the single-muscle-fiber scale. The muscle itself continues to work at all (thawed) temperatures below body temperature -- the problem comes with its regulation.
The shivering response -- a centrally controlled involuntary contractile pattern -- overwhelms voluntary muscle control. So the mechanism behind loss of muscle coordination in hypothermia is the same as the shivering mechanism. When the core temperature drops, the midbrain starts over-riding voluntary control of the muscles. When the core temperature drops far enough; around 32C, the shivering often slows down or stops. Voluntary movement becomes compromised probably because the brain simply isn't working; neuron firing rates are so slow that sensation, processing, and motor responses are all critically impaired.
The feeling of numbness does not actually directly accompany a loss of muscle contractility. You can walk pretty much indefinitely on frozen feet if you can keep your balance (and you keep your core temperature up.) Lots of people survive severe frostbite of their feet (their feet do not often survive, however.) The reason why it seems your hands don't work when they get cold is that you can't feel what you're doing (note; your hands can be much colder than your core body temperature.) But the muscles themselves work right up until they freeze solid.
UPDATE:
Here's a paper that directly addresses the scenario posed by OP -- the decrease in grip strength with temperature. Figure 1 of that paper illustrates their experimental setup; they measure the contractile strength of the index finger while manipulating the temperature of the rest of the hand. They show that contractile function is impaired with temperature and look at temperatures as low as 12C.
They measure as much as a 50% impairment on twitch tension upon cooling to 12C. It's interesting that they review results suggesting that some of this effect is intrinsic to the muscle fiber (not neurological), showing that I should refine what is meant by "continuing to work" in my opening paragraph. (I meant having an ability to generate contractile force when equilibrated in solution containing sufficient ATP and Ca$^{2+}$, not the ability to contract optimally.) For fun, I linearly extrapolated the final arm of Figure 5 and found that the 'voluntary tension' approached 25% at 5C. This suggests that total failure of the voluntary contraction happens somewhere below the freezing point of water (muscle would freeze at a temperature lower than 0C because of colligative effects.)