The idea of holding a Pan American Games was first raised at the 1932 Summer Olympics in Los Angeles, where Latin American representatives of the International Olympic Committee (IOC) suggested that a competition among all the countries in the Americas should be created.

The first event called the Pan American Games took place in Dallas in 1937 as part of the Greater Texas & Pan-American Exposition, but it attracted so little attention it has never counted in the records of the competition.

At the first Pan American Sports Congress, held in Buenos Aires in 1940, the participants decided that the first games should be held in Buenos Aires in 1942.

The plans had to be postponed because of World War II. A second Pan American Sports Congress held in London during the 1948 Summer Olympics reconfirmed Buenos Aires as the choice of host city for the inaugural games, which were held in 1951.

The games began on February 25 of that year, and offered 18 sports. Countries that were part of the Commonwealth of Nations such as Canada did not compete at the first Pan American Games.The second games were held in Mexico City, Mexico. Competitions started on March 12 and included 2,583 athletes from 22 countries, competing in 17 sports. The Pan American Games have been held subsequently every four years.