RemoteTap uses its own communication protocol, however the packets itself are not encrypted (besides the login password). In cases where additional security is required, e.g. for connections over the Internet or mobile data network, RemoteTap can be run over a VPN.
On the iPhone, use the build in VPN client located at Settings -> General -> Network -> VPN. The iPhone supports the L2TP, PPTP, and IPSec protocols.
On the Mac, Mac OS X 10.5 and 10.4 actually ship with VPN server software, although there is no GUI to configure it. You can either do this manually
if you are a Unix geek, or use a VPN GUI like iVPN
to do the work for you.
After establishing a VPN tunnel between the iPhone and your Mac, use the Macs local IP address (not the public one of the dsl router) as the IP/DNS address in RemoteTap.