lord of the rings - How well do Aragorn and Saruman know each other?

In the chapter Flotsam and Jetsam in the The Two Towers, Aragorn has this to say of Saruman:

‘Once he was as great as his fame made him. His knowledge was deep, his thought was subtle, and his hands marvellously skilled; and he had a power over the minds of others. The wise he could persuade, and the smaller folk he could daunt. That power he certainly still keeps. There are not many in Middle-earth that I should say were safe, if they were left alone to talk with him, even now when he has suffered a defeat. Gandalf, Elrond, and Galadriel, perhaps, now that his wickedness has been laid bare, but very few others.’

This indicates that Aragorn was fully aware of the abilities and nature of Saruman. Was this information second hand via Gandalf/Elrond or is there in any work indications that they had crossed paths previously?

We know Aragorn has never been to Isenguard:

I have never been in Isengard, but I have journeyed in this land, and I know well the empty countries that lie between Rohan and the Shire.

But that doesn't exclude the possibility that they met elsewhere.

How well do Aragorn and Saruman know each other personally, had they ever met or had dealings prior to the attack on Rohan?