See if it's possible to attend a worship service at Temple Gemiluth Chassed. You'll learn about some of the traditions and practices of the Jewish faith.
You're spoiled for sightseeing choice in Port Gibson. Temple Gemiluth Chassed is a good place to kick off here, but there are numerous other interesting sights to check out. Port Gibson Methodist Church, Port Gibson Baptist Church, and A.K. Shaifer House are close to the Jewish house of worship and will top off your day nicely.