This can be explained by it originally being used by critics of certain racial policies, in particular in British or South African context. These critics (like almost everyone at this time) usually did not doubt that different races exist. The term itself implicitly acknowledges the existence of "race", by instead focusing on "racial".
This became problematic for critics, when the race concept itself increasingly started to be attacked. Such critics therefore preferred the newer term racism. For the same reason, supporters of the race concept adopted the term racialism. However, many have switched to the term "race realism", which has advantages such as not being confused due to word similarity with the term racism.