It was not immediately clear how long they had been held, but the army routinely detains civilians who have been living in areas that had been ruled by the insurgents on suspicion that they too might be linked to militant activities. (Fontaine is pictured above with displaced girls in Maiduguri, many of them are in school for the first time)
However, rights groups say there is no proper legal process for such civilians, including the children, since they do not get formally charged and some end up in so-called rehabilitation centres, which the groups say are like prisons. The United Nations says children should not be detained.
"We fear that there are still kids who are being at least temporarily detained because they are being released from Boko Haram areas by the army but then kept for a while," Fontaine told reporters by telephone.
He gave no details of the ages of the children or how long they had been at the barracks, but after President Muhammadu Buhari came to power in May 2015, security forces began an offensive -- backed by neighbouring countries -- to retake territory from Boko Haram, meaning at least some of the children could have been held for a year or more.
Nigerian army officials say they need to question civilians to establish whether they have any ties with the militant group, which has been trying for seven years to set up an Islamic state.
Fontaine also said the conflict, which has killed thousands and displaced more than two million, had separated around 20,000 children from their parents, of which 5,000 had since been reunited with their families.
"Once we get children out, there is a major issue of stigmatisation in the communities," Fontaine said. "There is a sense that children who have been associated with Boko Haram for a while could be, and in some cases we have some evidence, are rejected by the community and people around them."
This was also a problem for the girls recently freed from Chibok, he added. Nigeria this month negotiated, with the help of Switzerland, the release of 21 of over 200 school girls kidnapped by Boko Haram in April 2014.