The flooding has also destroyed or damaged at least 200 homes in Jos city, the Red Cross said on Monday.
"We have recovered the bodies of 35 people that drowned in the overnight flooding," said Manasie Phampe, head of the Red Cross in Plateau state, where Jos is the capital.
"About 200 homes have either been submerged or destroyed."
'Searching for bodies'
The confirmed victims so far included a 90-year-old woman and a three-month-old baby, Phampe said, adding that the toll could rise.
"Rainwater and water from the Lamingo dam which overflowed swept across several neighbourhoods in the city," he explained.
"We are still searching for more bodies as many people have been declared missing."
The downpour in Jos began at roughly 9:00pm local time (20:00 GMT) on Sunday when many of the city's residents were at home for the night.
Al Jazeera's correspondent Mohammed Adow, reporting from Jos, said "the waters came gushing as most people were asleep in three of the districts".
The area coordinator for the National Emergency Agency (NEMA), Alhassan Danjuma Aliyu, called the flood "devastating" and said police and other rescue bodies were searching the affected areas in hopes of finding survivors.
NEMA was also trying to quickly bring in added relief materials to care for those who lost their homes, Aliyu said.
"We hope to get these materials to them before the end of today," he said.
"I have lost seven of my children in this disaster," said Alhaji Abdulhamid Useini, who described heavy rain pounding his neighbourhood for nearly three hours, which also swept away some of his livestock.
Much of the country has been affected by heavy seasonal rainfall, including Jos and the economic capital Lagos in the south, where flooding has caused several road closures and other damage.
The rainy season typically runs from March to September.
Last week, at least three people were killed by flood waters about 150km north of Lagos in Ibadan, an area where 102 people died following torrential rains last year.
At least 20 people died from flooding in Lagos last year, while 24 were killed after rains inundated a neighbourhood in Nigeria's largest northern city of Kano