The first mention of the Mali Kalnik fortress can be found in 1334. At the end of the 14th century, the owner of Mali Kalnik was King Sigismund of Luxembourg. There is information that later Mali Kalnik was owned by Beatrice Frankopan after the death of her husband Ivaniš Korvin, the son of King Matija Korvin.
The legend of Barbara of Celje a.k.a. the "Black Queen", the second wife of King Sigismund of Luxembourg, is connected with Mali Kalnik. As the king often fought throughout Europe in his desire to obtain the crown of the Holy Roman Empire, Barbara waited alone for him, often staying in Veliki and Mali Kalnik. She found solace with her lover, the commander of the forts of Veliki and Mali Kalnik, Sigismund Neuhäusel. Upon finding out of her adultery, her husband punished her by imprisoning her in Melnik Castle in the Czech Republic, where she died. Legend has it that she was an evil, greedy and cruel woman dressed in black. The Mali Kalnik fortress is named "Pusta Barbara" after her.
After the Second World War, the castle housed an elementary school until 1964, and today it is in the hands of a private owner.