In 1950–51, James Rainwater and Aage Bohr had developed Models of the atomic nucleus which began to take into account the behaviour of the individual nucleons. These Models, which moved beyond the simpler liquid drop treatment of the nucleus as having effectively no internal structure, were the first Models which could explain a number of nuclear properties, including the non-spherical distribution of charge in certain nuclei. Mottelson worked with Aage Bohr to compare the theoretical Models with experimental data. In three papers which were published in 1952–53, Bohr and Mottelson demonstrated close agreement between theory and experiment, for Example showing that the Energy levels of certain nuclei could be described by a rotation spectrum. This work stimulated new theoretical and experimental studies.
He moved to Institute for Theoretical Physics (later the Niels Bohr Institute) in Copenhagen on the Sheldon Traveling Fellowship from Harvard, and remained in Denmark. In 1953 he was appointed staff member in CERN's Theoretical Study Group, which was based in Copenhagen, a position he held until he became professor at the newly formed Nordic Institute for Theoretical Physics (Nordita) in 1957. He was a visiting professor at the University of California, Berkeley in Spring 1959. In 1971 he became a naturalized Danish citizen.
Rainwater, Bohr and Mottelson were jointly awarded the 1975 Nobel Prize in Physics "for the discovery of the connection between collective motion and particle motion in atomic nuclei and the development of the theory of the structure of the atomic nucleus based on this connection".