iftop accepts pcap-filter formatted filters on the commandline with the -f flag. Below is a table of some of the filers you might want to use with iftop:

dst host host src host host
dst net net src net net
dst port port src port port
dst portrange start-end src portrange start-end
gateway gateway
ip proto protocol

For example, to view only traffic going from your local machine to www.itkylin.com over eth0, you could run:

iftop -i eth0 -f “dst host www.itkylin.com

Or to see only ssh traffic over wlan0:

iftop -i wlan0 -f “dst port 22″