In the early 1900s, there was growing apprehension in the U.S. towards immigration, including immigration from Japan. While the settlers' children were United States citizens, Sakai and his fellow immigrants could not become naturalized citizens until the 1950s.

By the Second World War, few of the Japanese settlers remained. In 1942, not long after the Pearl Harbor attack when anti-Japanese sentiments were at a peak, the Federal government confiscated their land -over 6,000 acres - to create an Army Air Corp training base, ending the Yamato colony.

Situated on the site today is Boca Raton's airport and Florida Atlantic University. A former Yamato Colony settler, George Morikami, farmed in Delray Beach until the 1970s. He eventually donated his land, which is today preserved as the Morikami Museum and Gardens.