You needn't be of strong faith to appreciate the peaceful atmosphere of Kaunas Cathedral. Check when the next service starts or when it's open for general viewing.
Old Town is where you'll find Kaunas Cathedral. Admission is free, which means you'll have more cash in your pocket to explore! Other worthwhile attractions near the principal church include Kaunas Castle, Ninth Fort and Vytautas the Great War Museum.