game of thrones - Why didn't Jon take Ghost with him?

Shevliaskovic's answer is correct, based on the tv-show.

As an aside: I know that you have asked the question in regards to the tv-show; but I will offer some further insight from the books.

[A Storm of Swords and A Feast for Crows spoilers]

Jon Snow is sent by Ser Alliser Thorne and Janos Slynt to "treaty" with Mance. This is after a prolonged tortuous stay in a cold room.

They see Jon as a threat to their leadership as well as a thorn [pun intended] in their sides; so they send him in hopes that Jon will be able to get deep into their lairs and slay Mance or die trying in a suicide mission.
Note that it was Mance who requested someone to treaty with, as he had an important message ('let us in, we don't want to kill you, we just want to hide behind your wall'); but Slynt and Thorne take the opportunity and use it to their advantage.

In the books, Jon does take Longclaw (his sword) in order to do this.

Jon does not rejoin with Ghost until later (after Stannis' arrival and Mance's capture), hence why they are not together at this scence!

So, essentially, in the books, Jon was not with Ghost, therefore in the tv-show, they must have come up with an idea to keep that part of the story consistent [IMO].