The negative fitness consequences of close inbreeding are widely recognized, but predicting the long-term effects of inbreeding and genetic drift due to limited population size is not straightforward. As the frequency and homozygosity of recessive deleterious alleles increase, selection can remove (purge) them from a population, reducing the genetic load. At the same time, small population size relaxes selection against mildly harmful mutations, which may lead to accumulation of genetic load. The efficiency of purging and the accumulation of mutations both depend on the rate of inbreeding (i.e., population size) and on the nature of mutations. We studied how increasing levels of inbreeding affect offspring production and extinction in experimental Drosophila littoralis populations replicated in two sizes, N = 10 and N = 40. Offspring production and extinction were measured over 25 generations concurrently with a large control population. In the N = 10 populations, offspring production decreased strongly at low levels of inbreeding, then recovered only to show a consistent subsequent decline, suggesting early expression and purging of recessive highly deleterious alleles and subsequent accumulation of mildly harmful mutations. In the N = 40 populations, offspring production declined only after inbreeding reached higher levels, suggesting that inbreeding and genetic drift pose a smaller threat to population fitness when inbreeding is slow. Our results suggest that highly deleterious alleles can be purged in small populations already at low levels of inbreeding, but that purging does not protect the small populations from eventual genetic deterioration and extinction.