Kapp attended Iona Preparatory School in New Rochelle, New York where he participated in soccer and football. Kapp earned All-City and All-State honors during his Junior and Senior years at Iona Prep. He also participated in the 2011 High School All-American Game and was chosen to train with the USA U-17 men's national team. At Iona, Kapp also played as the placekicker for the football team. Kapp also developed as a player by playing for FC Westchester Academy. Kapp attended Boston College where he Redshirt (college sports) his freshman year before playing for 3 more years. For his final season in college, Kapp transferred to Creighton University.
Kapp was selected by Atlanta United in the 4th round of the 2017 MLS SuperDraft with the 68th pick. Atlanta United chose not to sign Kapp, however, so he remained as a free agent until he was signed by Minnesota United on August 8, 2017.
