One of your first stops should be Rye Heritage Centre. Here, you'll be able to find your bearings and get lots of useful information before you start exploring.
To explore this tourist information center, head to Rye. You'll find plenty of other worthwhile attractions in the area, such as Camber Sands, Rye Harbour Nature Reserve, and Winchelsea Beach.