As early as 1826, Jamae Mosque already existed, a testimony to the large number of Tamil Muslims in Chinatown. The present brick and plaster building was completed between 1830 and 1835. It is not a coincidence that the street on which it is located is named Mosque Street and this mosque is always referred to as “Periya Palli” meaning “Big Mosque” 

Jamae Mosque was built by Chulia Muslim merchants from India's Coromandal Coast, under the leadership of Anser Sahib. Thus, it is sometimes referred to as 'Masjid Chulia'. A number of these Chulia Indians (Indians from the Chola Kingdom of South India), mostly traders and moneychangers, settled in Kling Street, which was eventually named Chulia Street.

Perhaps this explains why the mosque is also known as ' Kling Mosque'. “Kling” is the nickname for Indians from the Kalinga Kingdom in India. They came and set up the three Islamic heritage buildings in the heart of Chinatown within a short period: Jamae Chulia Mosque was the first. The other two are Al-Abrar Mosque and Nagore Durga that are located at Telok Ayer Street.

On 21 November 1881, the Governor of the Straits Settlement granted the mosque a 999-year land lease. In 1894, it came under the management of the same panel of court appointed trustees that managed all other mosques in Singapore.

Jamae Mosque became a waqf mosque in 1917.  Waqf means putting aside the original property and donating its benefits for the sake of Allaah. What is meant by the original property is something from which benefit may be derived whilst its essence remains, such as houses, shops, gardens, etc. What is meant by benefits is beneficial produce that comes from the original property, such as crops, rents, provision of shelter, etc.  In 1968, it was put under the charge of MUIS (Majlis Ugama Islam Singapura), the Islamic Religious Council of Singapore.

The mosque’s architectural style is eclectic. While the entrance gate is distinctively South Indian, the two prayer halls are Neo-Classical style typical of George Drumgoole Coleman. This unique appearance has made the mosque a well-photographed landmark, seen in postcards from the 19th century to the present day. It was gazetted as a national monument on 29 November 1974, and is believed to be the oldest of four other gazetted mosques in Singapore. Happily, what stands today is the original 1830s mosque.