lord of the rings - Why didn't Gollum kill Frodo and Sam?

To begin with, Gollum does not know Frodo and Sam are trying to destroy the Ring, only that they want to enter Mordor. In fact he reacts very angrily when he learns they want to destroy his Precious. In the film, Frodo tells him just outside Shelob's lair; in the books, Gollum doesn't work it out until he attacks them on the slopes of Mount Doom itself:

"Wicked masster!" it hissed. "Wicked masster cheats us; cheats
Smeagol, gollum. He musstn't go that way. He musstn't hurt Precious.
Give it to Smeagol, yess, give it to us! Give it to uss!" [Return of
the King, Book 6, Chapter 3, "Mount Doom"]

To answer your main question, Frodo and Sam do not trust Gollum. In the books they take it in turns to sleep, with one remaining awake to watch:

"Look here!" Sam whispered to Frodo, not too softly; he did not really
care whether Gollum heard him or not. "We've got to get some sleep;
but not both together with that hungry villain nigh, promise or no
promise." [The Two Towers, Book 4, Chapter 2, "The Passage of the Marshes"]

If Gollum attacked the hobbit on guard, he would wake the other; and Gollum is not strong enough to defeat both the hobbits at once, especially since they are armed and he is not. This is hinted at in the films as well, along with the fact that Frodo hardly sleeps at all.

Finally, it is suggested that possession of the Ring gives Frodo a power to command Smeagol/Gollum:

For a moment it appeared to Sam that his master had grown and Gollum
had shrunk; a tall stern shadow, a mighty lord who hid his brightness
in grey cloud, and at his feet a little whining dog. Yet the two were
in some way akin and not alien; they could reach one another's minds.
[The Two Towers, Book 4, Chapter 1, "The Taming of Smeagol"]

As Frodo says in the film, "The Ring is treacherous. It will hold you to your word." Of course, Gollum himself is treacherous and will take any opportunity to seize the Ring for himself, but he still finds it difficult to defy Frodo.