etymology - Where did the term 'moral fiber' come from?

Fiber:

a. Something that provides substance or texture.

b. Essential character: "stirred the deeper fibers of my nature" (Oscar Wilde).

c. Basic strength or toughness; fortitude: lacking in moral fiber.

Source: American Her. Dic.

It's use in a figurative sense has been common for a while. I think it refers to the threads that compose a texture, which in this case is likened to the morality of the character.