Whisk together the sweetened condensed milk and cream. Set aside.
Arrange 10 ladyfingers in a 7×11 baking dish; layer about half of the mango slices on the ladyfingers. Pour half of the milk mixture over the mangoes and ladyfingers. Repeat layering. Sprinkle ladyfinger crumbles over top. Cover dish and freeze at least 4 hours.