Best fit malfunctioning for certain objects

Sometimes framing an object (Best Fit) shoots me into cosmos, or frames the object from a crazy distance. What might be the cause of that?
The pivot of the object is always at its centre
From my observations, the issue happens mostly on copies and instances of an object, or objects that has been imported to scene.