ag.algebraic geometry - Relation between l-adic and l'-adic geometric monodromy

Suppose $X$ is a smooth family of algebraic varieties over the base $B:=mathbb{P}^1backslashlbrace0,1,inftyrbrace$ over $overline{mathbb{Q}}$; then we can form the relative $l$-adic cohomology on the base, which will be (having fixed some cohomological degree of interest) an $l$-adic sheaf; we can think of it as a $mathbb{Q}_l$ vector space $V_l$ equipped with a $pi_1(B)$ action. (Algebraic $pi_1$, of course.)

We can do the same with $l'$-adic cohomology for some other prime $l'$, getting a vector space $V_{l'}$ over $mathbb{Q}_{l'}$ equipped with a $pi_1(B)$ action.

As far as I can see, there ought to be a relatively tight connection between these two groups. For instance, it seems morally like we ought to be able to choose pro-generators $alpha,beta$ for $pi_1(B)$, and find some matrices $M_alpha,M_beta$ over $overline{mathbb{Q}}$, such that, w.r.t suitable bases of $V_l,V_{l'}$, $alpha$ acts via $M_alpha$ on both $V_l$ and $V_{l'}$ (thinking of $M_alpha$ as determining an $l$- and $l'$- adic matrix in turn) and similarly for $M_beta$. Can this indeed be done? If not, can you do something close? If it can, what's a good reference for a precisely stated theorem---rather than just intuition---for these things?