These include the distinction between a single admixture event and multiple Neanderthal contributions to both populations, as well as the hypothesis that reduced Neanderthal ancestry in modern Europeans resulted from subsequent admixture with a spectral population that lacked a component of Neanderthal ancestry (the "dilution" hypothesis). To summarize the asymmetric pattern of Neanderthal allele frequencies, the common frequency spectrum of European and East Asian Neanderthal fragments was compiled and compared with both analytical theory and data simulated in various admixture models. Using maximum likelihood and machine learning, we found that a simple single admixture model did not fit the empirical data, instead favoring a model of multiple episodes of gene flow to both European and East Asian populations. These findings point to a longer, more complex interaction between humans and Neanderthals than previously thought.