This article is an attempt to address on a theoretical level an antinomy in postcolonial approaches to the question of temporal difference. Current scholarship tends both to denounce the way in which the others of the Western self are placed notionally in another time than the West and not only analytically affirm but indeed valorize multiple temporalities. I elaborate on the two problematic temporal frameworks—linear developmentalism and cultural relativism—that belong to a colonial legacy and generate the antinomy in question, and then proceed to discuss possible alternatives provided by a Koselleck-inspired approach to historical time as inherently plural. I thereby make two central claims: (1) postcolonial conceptions of multiple temporalities typically, if tacitly, associate time with culture, and hence risk reproducing the aporias of cultural relativism; (2) postcolonial metahistorical critique is commonly premised on a simplified and even monolithic understanding of Western modernity as an ideology of “linear progress.” Ultimately, I suggest that the solution lies in radicalizing, not discarding, the notion of multiple temporalities. Drawing on the Brazilian classic Os sertões as my key example, I also maintain that literary writing exhibits a unique “heterochronic” (in analogy with “heteroglossic”) potential, enabling a more refined understanding of temporal difference.