The incident was the second of its kind in recent weeks in Aden, Yemen's second city, where the presence of jihadist groups is growing as the embattled government struggles to spread control.
The gunmen, some of whom were masked, forced the students out of the faculties of administrative sciences, law, and engineering, before locking down the gates, students said.
"They dragged us out of the exam halls," said one of the students. "They detained two students who were filming the incident."
Students said the gunmen shouted: "No mixing. We have warned you before," and added that the authorities did not intervene to stop the extremists.
It was not immediately clear what group the gunmen belonged to but witnesses and local residents said they were loyalists of Ayman Askar, a local militia leader known for his links to both Al-Qaeda and the Islamic State group.
Askar's militia is influential in Aden's district of Madinat Asha'ab where the faculties are located, according to residents.
Last month, radical gunmen also entered the faculty of administrative sciences in Aden and closed it down after threatening to use force against students if they did not observe segregation of the sexes.
Al-Qaeda in the Arabian Peninsula (AQAP), long active in Yemen, and the newly emerged IS appear to be vying for influence in the main port city.
Fighting between Iran-allied rebels and Saudi-backed loyalist forces has plunged impoverished Yemen into chaos, which jihadist groups have exploited to make sweeping gains, particularly in southern regions.
The extremists are occupying government buildings and are frequently seen patrolling several districts of Aden.
They have carried out several deadly attacks, assassinated government officials, and intimidated civilians.
Yemen's capital Sanaa has been under the control of Shiite Huthi rebels and their allies since September last year.