The games' organizers say Muslim prayer rooms will be available at the athletes' village but not at stadiums or other competition venues.
The mobile mosques are expected to help address the concerns of Muslim visitors, who must pray 5 times a day, facing Mecca.
A group of firms working on the project unveiled a model on Monday in the city of Toyota, Aichi Prefecture, NHK reported.
The group renovated a truck by expanding the container on both sides.
The air-conditioned prayer room can fit 50 worshippers. Developers also plan to install a compass that points to Mecca.
The truck costs about 100-million yen, or about 900,000 US dollars, but the group is aiming to lower the price.
Project chief Yasuharu Inoue says he hopes to provide a place where both athletes and spectators can pray.
A car maker in Yanai in the western prefecture of Yamaguchi has also developed a similar vehicle.
It sought the advice of foreign Muslim students at Yamaguchi University who tested the truck.
The company's president Tetsuro Akimoto says he hopes the vehicle will be stationed at Olympic and Paralympic venues as a show of "omotenashi," a Japanese word for hospitality.