From how I've heard the word used, the difference between pan- and polysexual is that pan is open to dating folks of all sexes and genders, while poly is open to many, definitely more than two, but not all.
E.g., if you're not open to, say, dating transmen (for whatever reasons), but open to...