Is there a way to define Hecke operators "inherently" as certain endomorphisms of the Jacobian?

Philip remarks that he wants to define the Hecke operator as an endomorphism of the Jacobian of "any Riemann surface $X$ such that the endomorphism ring is defined over $mathbf{Q}$", and at the same time he wants the Hecke operator to reduce to ${rm Frob}_p + p{rm Frob}_p^{-1}$. I think that for a generic Riemann surface $X$, the endomorphism ring is $mathbf{Z}$. However, ${rm Frob}_p + {rm Ver}_p$ is very unlikely to be an integer, so for a general $X$ it is highly unlikely that there exists an element of ${rm End}(X)$ that lifts ${rm Frob}_p + {rm Ver}_p$. (I'm using ${rm Ver}_p$ to denote the dual of Frobenius.)

Here is another closely related point. When $A$ is an abelian variety with good reduction at a prime $p$, there is a natural map ${rm End}(A) to {rm End}(A_{{mathbf F}_p})$. (See my remark here). I think this map is injective (consider the induced map on Tate modules at some good prime $ell$). Thus you could define the Hecke operator $T_p$ to be the unique (if it exists!) lift of ${rm Frob}_p + {rm Ver}_p$. That's intrinsic and makes no reference to any moduli space.