If you'd like to inject a little more culture into your trip, put Vincent House Museum on your itinerary. See if there are any guided tours or just look around in your own time.
Located in Edgartown, this museum is well worth including on your itinerary. But you'll find tons more to see and do in the area. Inkwell Beach, Oak Bluffs Town Beach, and Martha's Vineyard Museum aren't too far from Vincent House Museum.