Buy tickets and join the fans at a game at Shielfield Park. If there's nothing happening while you're in the area, you might like to check if the stadium offers a guided tour of the grounds.
You'll be spoiled for sightseeing choice when in Berwick-upon-Tweed. Shielfield Park is an ideal place to start here, but there are many other fine sights to discover. Old Bridge, Chain Bridge Honey Farm, and Haggerston Castle are not far from the sporting facility and and will cap off your day nicely.