When host signals used as cues by exploiters coevolve with exploiter recognition systems, our studies show that evolutionary changes may be continuous and cyclic. Selection changes back and forth between specialization and generalization in the exploiters, and weak and strong mimicry in the hosts, where non-defended hosts use the host investing in defence as a model. Thus, host signals and exploiter responses are engaged in a red-queen mimicry process that is ultimately cyclic rather then directional. In one phase, evolving signals of exploitable hosts mimic those of hosts less suitable for exploitation (i.e. the model). Signals in the model hosts also evolve through selection to escape the mimic and its exploiters. Response saturation constraints in the model hosts lead to the mimic hosts finally perfecting its mimicry, after which specialization in the exploiter guild is lost. This loss of exploiter specialization provides an opportunity for the model hosts to escape their mimics. Therefore, this cycle then repeats.

We suggest that a species can readily evolve sympatrically when disruptive selection for specialization on hosts is the first step. In a sexual reproduction setting, partial reproductive isolation may first evolve by mate choice being confined to individuals on the same host. Secondly, this disruptive selection will favour assortative mate choice on genotype, thereby leading to increased reproductive isolation.

The existence of sympatric speciation is a contentious issue because both empirical support is scarce and the underlying theoretical mechanisms are not as fully understood as we might like (e.g. Futuyma & Mayer 1980; Orr & Smith 1998; Rundle & Nosil 2005), although new evidence supports its occurrence (Barluenga et al. 2006). An obstacle for sympatric speciation is the exchange of alleles between lineages and the homogenizing effect of recombination in sexual reproduction (Felsenstein 1981; Rice & Salt 1988). The basic requirements for sympatric speciation are disruptive selection for evolutionary divergence, correlated through assortative mating with reproductive isolation (Felsenstein 1981; Rundle & Nosil 2005). Orr & Smith (1998) make the distinction between extrinsic and intrinsic barriers to gene flow. Extrinsic factors are physical barriers in the environment that prevent encounters between individuals. Intrinsic factors are genetic traits that increase pre- or post-zygotic reproductive isolation. They define sympatric speciation as ‘the evolution of intrinsic barriers to gene flow in the absence of extrinsic barriers’.

Host races have been defined as populations of a species that are partly reproductively isolated from one another as a direct consequence of adaptation to different hosts (Abrahamson et al. 2001). Host races in phytophagous insects are believed to be precursors to full species, an idea that goes back to Walsh (1864). The speciation process through host races is one of the most likely candidate examples for sympatric speciation. Specialization on hosts is a prerequisite for host races to be reproductively isolated, and if the isolation evolves as a correlated character to specialization, it may lead to sympatric speciation (Rice & Salt 1990).

Among insects, diet specialists are more common than generalists (Jermy 1984; Jaenike 1990; Futuyma 1991). It is still largely unknown what selection pressures lie behind the specialization, and it has been proposed as a major enigma in the evolution of insects (Futuyma 1991). Dietary reasons for specialization have been rejected because many laboratory studies show that larvae feed and grow equally well on plants other than those chosen by the female to oviposit on (Dethier 1947; Ballabeni & Rahier 2000). Other theoretical explanations for host specificity in insects include avoidance of inter-specific competition or predation, reduction of parasitism and increased probability of mate finding, but the empirical support is often circumstantial at the best (Futuyma & Moreno 1988). Recently, host specificity in insects has been suggested to be the result of limitations in brain function, more specifically in the recognition systems that processes information (here host signals) for effective recognition of suitable host plants (Holmgren & Getz 2000; Bernays 2001).

Here, we review some of our work on host-plant selection in insects using artificial neural networks as models for the plant recognition mechanism in insects (Holmgren & Getz 2000; Norrström et al. 2006). We present some new insights from the synthesis of our results and discuss some detailed mechanisms that may be involved in sympatric speciation (Dieckmann & Doebeli 1999). This includes specialization and disruptive selection as a result of evolution on recognition mechanisms, as well as coevolution of the exploiter species and their hosts. Although the model is inspired by insect–plant systems, it may be regarded as an example of a more general exploiter–victim system in which the evolution of the exploiters' recognition systems is critically influenced by the ability to assess resource quality of hosts/victims. As a consequence, exploiter recognition coevolves with defensive (physiological or morphological) adaptations of victims and sometimes leads to sympatric speciation. Our aim is to present a framework for exploring the importance of specialization for speciation, with a view to stimulating further theoretical and empirical work.

A niche-breadth model of exploiter evolution under neural constraints needs to be sufficiently detailed with regard to signals produced by victims and the ability of the exploiters to perceive and respond to these signals to adequately address the questions at hand. The plant victims, for example, produce signals of varying complexity dependent on a few key chemical compounds that occur in plant-specific ratios, but are collectively known as ‘green odour’ (Visser 1986). To keep things simple, we let the plant signals in our model be represented by two odourants, the minimum needed for odour quality to depend on component ratios (Getz & Chapman 1987). Insects, as exploiters for example, perceive these plant signals and compute an output signal coding for a behavioural action—in our models, we take this action to be laying versus not laying eggs on a potential host plant.

For simplicity, we modelled the perceptual system as a perceptron (Haykin 1994), rather than a dynamic neural network (Getz & Lutz 1999), which captures the perceptual constraint feature seen in insect exploiters when selecting among victims with different phenotypes (cf. Getz & Smith 1990; Getz & Akers 1997). The perceptron is a three-layered feed-forward network with an ability to differentiate and categorize input signals, once the perceptron has an appropriate set of weighting values for passing on information from one layer of nodes to the next. The output layer has only one node which state corresponds to an on–off or yes–no response. In our simulations, these weightings are made to evolve intergenerationally through both mutations and the fitness of the response in terms of host-plant (i.e. victim) selection, where fitness is measured as the expected number of eggs that will successfully mature into new adults. In short, we have a mutation–selection algorithm on the synaptic weights of replicated perceptrons (see electronic supplementary material for more details).

We investigated the effects of point mutation probabilities and the perturbation effect of a point mutation. In an initial study, we assumed that the exploiters were represented by a unique perceptron reproduced as haploid clones (Holmgren & Getz 2000); that is, from one generation to the next depending on the fitness of the represented individual, zero to two new perceptrons were created and then mutated with predetermined probabilities and size of weighting perturbations. We noted that clonal reproduction excludes genetic exchange between exploiter lineages, which are thus reproductively isolated. In a follow-on study, described below, we developed a diploid genetic structure coding for the synaptic weights (N. Norrström et al. 2006, unpublished data). Our perceptrons had two inputs implying that victim signals were points in a two-dimensional odour space (figure 1).

Figure 1 A cartoon of the perceptual component of the model. Each plant type produces its own unique signature that stimulates the olfactory receptor cells located on the antenna of individual insects. The response of these cells is processed in the antennal lobes (more ...)

How well the recognition system of an exploiter is performing, in terms of identifying suitable victims, is ultimately determined by the number of eggs in each generation maturing into new adults. We constructed exploiter-fitness functions using an insect herbivore as our leitmotif (see electronic supplementary material for fitness function of exploiters). The fitness of each exploiter is a function of the response to the input signals from its victims. One may think of an insect herbivore as having the option of choosing among a number of different types of plants, where each type produces a characteristic odour. In the model, each insect samples the odours of all the plants in the environment. The response or preference of insect g, g=1,…,G, for the plant type h, h=1,…,H, is identified with the output y_g,h of perceptron g to input signal h. One approach to constructing a fitness function is to assume that the decision to lay a clutch of eggs on a host plant depends on the strength of this response relative to the other plant types. This may seem to be a reasonable approach at first. However, if the absolute signal strength is not factored in, genetic drift decreases the insects' sensitivity to the plant odours because there is no selection on sensitivity (confirmed in unpublished simulations). Thus, we found it biologically reasonable and necessary to add a dependency on the signal strength to the relative response to each plant odour type. Biological interpretations for this dependency on the absolute value of the signal include: (i) signals need to exceed a threshold level to cause spiking neurons to fire and (ii) it is also reasonable to assume that there is selection for an increased, rather than a decreased sensitivity to signals that are critical to the insects' fitness. Thus, we used the expression

(3.1)

If response function (equation (3.1)) is at all realistic, it has some significant consequences for insect diet breadth. The function selects for an all-or-none response of insects to their available hosts. For example, in an environment of two plants, an insect with the intermediate response [y_g,1=0.5, y_g,2=0.5] will lay [e_g,1=0.25, e_g,2=0.25], i.e. in total only half of its egg complement. In contrast, an insect with a maximum response to one plant and a zero response to the other [y_g,1=1, y_g,2=0] will lay [e_g,1=1, e_g,2=0], i.e. its total egg complement on plant one. Thus, intermediate responses have the logical consequence that insects do not lay their full egg complement, and are as a strategy less fit than all-or-none response strategies.

Now consider an environment of host plants where the value of each plant as a resource for the insects is intermediate and varies between plants. In other words, the plants are neither perfect hosts nor completely noxious, but provide some intermediate resource in terms of the number of eggs that can successfully hatch and produce viable offspring. Alternatively, the quality of offspring—e.g. the size or fecundity of mature offspring—may vary with the plant quality. As our simulations illustrate below, there is no way for a single insect generalist, with a maximum response to each plant type, to optimally exploit an environment composed of plants that vary in their relative fitness values to insects (see §4a). This conclusion rests on the assumption that these generalists will distribute their eggs evenly among available plants. Intra-phenotypic competition on the plant of the lowest resource value will limit population growth, thereby leaving the more valuable plants underutilized. Under these circumstances, selection favours guilds of insects with relative numbers selected to match the values and optimally exploit the plant resources in the environment.

In all its simplicity, the hypothesis that insects behave in accordance with both their absolute and relative responses to the plants of different types may explain the observation that many insects are more restrictive in their diet than needs be from a nutritional point of view (e.g. Wiklund 1975; Ballabeni & Rahier 2000). As discussed below, our work suggests that the herbivorous insects exploiting a particular ecosystem have evolved into a guild where the different ecological niches arising from host plant variation are occupied by a number of more or less specialized species.

In order to study the evolutionary process of specialization versus generalization, a range of resources must be included in the model. In an initial study of an exploiter–victim system, set in an ecological background determined by four fixed (i.e. non-evolving) victim types (Holmgren & Getz 2000), we focused on the evolution of niche breadth. In particular, we investigated the evolutionary process in several different ecological backgrounds from both a victim-signalling and victim-resource value point of view. Some of these environments represented a more difficult victim-discrimination challenge than others. The victims reproduced clonally and an insect–plant leitmotif was used to discuss and interpret the results.

Natural selection will favour exploiters occupying empty niches—i.e. the number of individuals produced by these exploiters will increase until the ecological niches are all fully occupied. In analogy to the ideal free distribution (Fretwell & Lucas 1970; Holmgren 1995), we expect the number of exploiters to match the resources. In several different environments with four non-evolving plant types, we showed that insect phenotypes evolved to equilibrium levels (numbers) that matched the plants' resource values (Holmgren & Getz 2000). For example, in the case of the resource values of the four plants being {40, 10, 40, 10}, insect phenotypes evolved to match so that the numbers produced by each plant type was also {40, 10, 40, 10} (figure 2).

Figure 2 The values of the population indices of the phenotypes (as labelled on the graph) in the population are plotted for one of the simulations of the population evolving in the environment four plant types of the values as resources: 40, 10, 40 and 10. The (more ...)

Recalling that the insect phenotypes evolve an all-or-none response (or close to it), we can conveniently represent each insect phenotype by an array of preference digits, one for each plant. For example, a phenotype denoted {1010} will lay eggs on the plant types one and three, but reject the plant types two and four. Simulations were initiated with naive perceptrons, in which the synaptic weights were randomly set to small values. The simulations were then run for 100000 generations (for more details see Holmgren & Getz 2000). In the period of 1000–3000 generations (figure 2), the first niche had 40 insects, 30 of phenotype {1000} and 10 of {1110}, the second niche had 10 {1110} phenotypes, the third niche 40 comprising 10 {1110} phenotypes, 20 {0010} phenotypes and 10 {0011} phenotypes, and the last niche had 10 {0011} phenotypes. Simple arithmetic indicates that the phenotypes of this guild are numerically matching the value of the plant resources.

Interestingly, the above guild begins an evolutionary transformation after 3000 generations so that at 4000 generations, the matching of plant phenotypes is accomplished by 40 {1111} generalist phenotypes in concert with the two specialists phenotypes, {1000} and {0010} of 30 individuals each (i.e. occupying niches 1 and 3, respectively). For reasons discussed below, this latter guild is more stable than the one above that arose first. Note that the resource matching is a predicted equilibrium of any exploiter–resource system unless the spectrum of exploiter types (i.e. guild) gets trapped in a non-optimal solution, from which there is no evolutionary escape if mutational perturbations are absent (as in simulated annealing optimization processes: Haykin 1994).

The geological record indicates that sudden turnovers of whole guilds of species seem to occur (Gould 2002). Some rapid turnovers of extinct guilds may be the result of catastrophes, such as meteorite impacts on Earth (Alvarez et al. 1980). Species compositions in terrestrial and aquatic systems are also known to exhibit rapid turnovers when alien species are introduced (Crooks 2002). In addition to catastrophes and major perturbations, guilds of specialists and generalists, unable to match their resources, are vulnerable to invasions of new species forming new guilds. In figure 2, the guild prevailing from generation 1000 to 3000 is matching its plant environment less robustly or resiliently (Amemiya et al. 2005) than the succeeding guild (after 4000 generations), because the former guild is more sensitive to mutations than the latter and more affected by inter-phenotypic competition. In reality, environments may change catastrophically or gradually, and a guild of species whose interactions are characterized by inter-specific competition may be unable to track those changes. As a consequence, a rapid turnover of species will follow, thereby establishing a new guild. Species in guilds have frequencies that are mutually dependent owing to resource matching. As such, they are resistant to invasion of phenotypes that temporarily disrupts this matching. The more mutations and new exploiter phenotypes required to obtain a new matching, the greater the resilience of the existing guild to persist.

Appearance of a new species may not be sufficient to overthrow an existing guild. Sometimes two or more are required to invade in concert. When an invasion has started, existing phenotypes quickly lose fitness as their interdependence with other phenotypes weakens. Because a simultaneous increase in fitness of new phenotypes and fitness loss in old ones is required, turnover rates of guilds when they occur are relatively fast. This is not a group-selection argument; selection still acts at the individual level although the fitness of individuals is dependent on the composition and frequency of the different species in the guild of competitors.

In reality, deviations from the ideal free distribution can be due to mutations on other phenotypes or changes in resource abundances. Guilds composed of specialists are less affected by inter-phenotypic competition and can more readily track a changing environment. In addition, specialists have a more simple discrimination task than generalists avoiding low quality resources. As such, the perceptual networks associated with simpler tasks are potentially less likely to be hampered by harmful mutations. An exploiter phenotype that utilizes two types of plants with sufficiently similar chemical signatures to be able to lump the two plant types into one perceptual category has no more complicated task than a specialist exploiter that needs to identify a single plant type. The most extreme case of this would be the generalist that treats all the existing plants as one category. For this reason we should expect guilds to be made up by specialists on single plant species (monophages), intermediate specialists utilizing a few similar host plants (oligophages) and indiscriminant generalists (heterophages).

In summary, our simulations suggest that disruptive selection for specialization in a heterogeneous resource environment could arise owing to the following two mechanisms. First, selection for sensitivity to signals produces individuals that have an all-or-none response to the different host phenotypes so that only guilds of insects that specialize to some degree will be able to fill up available resource niches. Second, selection favours resource-matching guilds of exploiter phenotypes that perform relatively simple host-choice perceptual tasks.

In a follow-on study, we allowed the victims to evolve in terms of the signals used by the exploiters to detect these victims (Norrström et al. 2006). As in the previous study, each exploiter was identified with a three-layer perceptron with weights subject to mutations. Again, with focus on specialization and disruptive selection, reproduction was assumed to be clonal for simplicity and to eliminate gene flows between genetic lineages. Specialization and disruptive selection among sexuals is within the scope of future work. Also, victims were still represented by a point in a two-dimensional signal (odour) space. Initially, three groups of victims were introduced, differing in their relative palatability: high, intermediate and low. The fitness of each victim depended on the number of victims within the same group and on the attack rate represented by a weighted sum of all exploiters—the weighting being determined by the exploiter response functions to that particular victim.

Figure 3 Victim-cue (e.g. host plants) phenotypes and exploiter-response phenotypes (e.g. phytophagous insects) are plotted above the two-dimensional signal space. The two bottom axes represent signal strengths of cue phenotype. The surface shows the average response (more ...)

The models described above are based on clonal reproduction in the exploiter population. No attention was paid to the homogenizing effect of genetic recombination among lineages produced by sexual reproduction and recombination (Rice 1984). In a recent study, we extended our model by adding sexual reproduction and diploid genetic coding of the synaptic weightings in our perceptron representations of individual exploiters. We allowed for the equivalent of genetic crossover to occur during meiosis by rearranging genes for perceptron weights among chromosomal-like structures (see electronic supplementary material for more details). A first comparison of simulation output from our original clonal/haploid system described above and our sexual/diploid systems reveal that evolution of specialists takes place under more narrow conditions when reproduction is sexual compared with asexual. In the case of sexual reproduction, the solutions are sensitive to the magnitude of the mutation parameters used in the simulations. If mutational perturbations (range) are too small, evolution may get stuck in local minima and not move to an ideal free distribution (matching the resource; columns 5–8, table 1). However, if the mutation probability is sufficiently high (and no crossover occurs) it counter-balances the low mutation range and some simulations result in specialists matching the resource (column 7, table 1). More than half of the repeated simulations (112 of 200) with asexuals with similar parameter settings resulted in the evolution of a multispecialist [1010] (Table 1 in Holmgren & Getz 2000). This suggests that mutation rates are sufficient to create the plant-matching guilds, but recombination from sexual hybridization with generalists [1111] overrides the selection for them.

Table 1 Sensitivity of phenotype guild solutions to mutation and crossover in sexually reproducing diploid exploiters in an environment of plant resources of the value [40, 0.1, 40, 0.1]. (The solutions are divided into three categories: (i) Exploiter population (more ...)

If mutational changes are too large, both in terms of probability for point mutations and mutational range, favourable perceptron settings will degrade faster than they can evolve leading to poorly adapted extant sexual generalists [1111] (columns 3 and 4, table 1). In asexuals, 97% of the repeated simulations resulted in a guild of two specialists: [1000] and [0010] (Table 1 in Holmgren & Getz 2000). Hence, homogenization by sexual reproduction also plays a role here to prevent selection for specialists.

When mutation range is sufficiently high and mutation probability is sufficiently low, both sexuals and asexuals readily evolve guilds of two specialists (column 1, table 1). Adding crossover, the selection–recombination antagonism (Felsenstein 1981) is flipped in favour of the recombination effects, thus homogenizing the genetic structure of the population in favour of the multispecialist (column 2, table 1). The parameter range in which host-matching guilds of sexuals evolve may seem narrow and unlikely to appear in nature. However, we can expect natural selection to adjust the rate and range of mutations by trading-off adaptive and detrimental effects.

If theoretical models can point out possible mechanisms of sympatric speciation, they can initiate ideas on experiments and field studies which may produce evidence of their existence. A strong candidate example of sympatric speciation is two species of cichlids from a small isolated lake (Barluenga et al. 2006). In order to fully understand sympatric speciation through exploiter adaptation to hosts, the question of underlying mechanisms for specialization must be addressed. Existing models also show that specialization on resources in combination with intraspecific competition is required for sympatric speciation (Dieckmann & Doebeli 1999). Dieckmann & Doebeli (1999) rightly point out that specialization can be a direct consequence of physiological or morphological traits for resource utilization, such as beak size. In the models presented here specialization freely evolves (Holmgren & Getz 2000; Norrström et al. 2006). Exploiter–victim systems that include victim signal and exploiter perceptual mechanisms suggest two potential mechanisms for disruptive selection and specialization: guilds of specialists, with less inter-phenotypic competition, are better at matching the environment of hosts, and specialists are more robust to mutations due to a more simple perceptual system (with the exception of a generalist responding to all signals; Holmgren & Getz 2000; Norrström et al. 2006). Empirical testing of these hypotheses would be very welcome.

Also, there has to be a mechanism for reproductive isolation between specialists (figure 4). For host races, this is assumed to be upheld by the exploiters' mating on hosts. Experiments with Drosophila flies in the laboratory environment have shown that after resource specialization has occurred, reproductive isolation may evolve as a correlated character (Rice & Salt 1990). An intrinsic reproductive barrier, defined as being genetically based, is a prerequisite for sympatric speciation (Orr & Smith 1998). Although the act of mating with individuals exploiting the same host is an inherited trait, it still responds to an external key (the host) and may only partially reflect the genome of a potential partner. Some individuals are expected to make erroneous choices or have limited mutations altering their host preferences, while the rest of their genome is unaltered. Exploiters should avoid less fit hybrid offspring by selecting mates based on phenotypic cues (one or multiple) highly correlated with the genotype. Additionally, there is also selection for phenotypic cues signalling the genotype. Mate choice based on the genotypically correlated cues of their partners enhances reproductive isolation and is likely to create full species. These species will also be more robust to changes in their host environment (see below). This three step process: (i) specialization on hosts, (ii) partial reproductive isolation through mating on victims or host species, and (iii) (nearly) complete reproductive isolation through mating with similar genotypes, appears to be a viable hypothesis of sympatric speciation (figure 4). Since each step involves selection, speciation will be faster than if the reduced hybrid vigour were due purely to randomly accumulated mutations in lineages separated by external barriers.

Figure 4 Species are hypothesized to evolve sympatrically in two steps. First, there is disruptive selection for specialization on hosts. Two hypothetical hosts are represented by squares. Specialists in guilds are better than generalists at matching their host (more ...)

If sympatric speciation results from specialization on biological resources, the resource may coevolve with its exploiters. When exploiters are entrained in the cyclic coevolutionary process with their victims, selection for generalists and specialists may shift back and forth (Norrström et al. 2006). Recent investigations reveal new dynamic properties of the specialization process (Janz et al. 2001; Nosil 2002). They question the view of the specialization process as always going from generalization towards specialization, hence suggesting that specialization is not a dead end. Janz et al. (2001) investigated the phylogeny of the nymphali butterfly tribe Nymphalini. They concluded that there is no directed evolution towards specialization and that the changes in host range show a very dynamic pattern. Nosil (2002) used phylogenies from 15 groups of phytophagous insects to investigate the rates of evolution towards specialization and generalization. They found that the rate of the evolution towards specialization is significantly higher than the rate towards generalization. In some cases, however, the rate of generalization was higher, or equal to the rate of specialization, indicating a dynamic property of the evolution of specialization, hence supporting the view of evolution of specialization as dynamic and not dead-end.

A continuously changing environment, in which selection shifts between specialization and generalization, may promote faster rates of speciation than previously anticipated. Species having evolved during the specialization phase may prevail under the generalization phase due to reproductive isolation through genotype-mediated assortative mating, an area where future modelling should give further insight. Disruptive selection with assortative mating as a pleiotropic by-product is postulated to potentially underlie sympatric speciation events (e.g. Rundle & Nosil 2005). Specialization has theoretically been shown to be a prerequisite for disruptive selection on food niches (Dieckmann & Doebeli 1999). Here, we provide a potential mechanism for specialization on food resources in evolving guilds of exploiters, some of which may more easily lead to speciation than others. It is premature to speculate on how important or common the mechanisms outlined in this paper will be in the natural world, but we hope we have argued that it is at least worthy of further consideration.

Tomas Jonsson and two anonymous reviewers made comments that improved the manuscript. This work was funded in part by a James S. McDonnell Foundation Twenty-first Century Science Initiative Award to WMG.