As with simulated research sites, the most suitable sites for existing agricultural systems will be determined from the research goals. Examples of where existing sites may be better than simulated sites for systems research include:
- Farm-scale questions, particularly those relating to effects of farm-scale vegetation or land use, farmer decision-making, marketing strategies and farm enterprise budgets
- Studies in which cross-scale interactions may be important. This is generally the case for research involving integrated pest management or organic farming systems where natural biological control is an important line of defense
- Research targeting processes that operate at scales too large to be accommodated by research stations, such as economics and other social system processes, questions focused on highly mobile pests or beneficial insects, and landscape-level ecological processes
- Research on interactions between environmental variation (e.g., soil type, texture or topographical location) and management
- Research aimed at improving management practices that require unique expertise or equipment not available at the research station
- Studies of innovative systems located on private land and that would require time, money or new skills to recreate at experiment stations.
Site selection is usually an iterative process that alternates between information gathering and decision-making (Figure 3.5). As the team explores sites, the experimental plan may need to be adapted to make it compatible with available sites. Although study sites may be identified during proposal writing (since site selection often requires preliminary data collection), make the final site choices after funding is secured.
The most important consideration when choosing existing agricultural systems study sites is to minimize confounding variables across farms; however, these variables depend on the questions being addressed. For example, when studying how management practices affect soil properties, the farm location and the adjacent land use or vegetation may not be important, but parent soil type and prior uses would be critically important. In contrast, when studying aboveground arthropod communities across farms, the surrounding habitat, microclimate and field size all need to be considered. Not surprisingly, as the number of disciplines involved in the study increases, the list of confounding variables becomes longer, and it becomes more challenging to select sites that do not have confounding variation.
Although there is no single “correct” process for identifying farm sites, the following key steps are broadly applicable:
- Familiarize all team members with the literature from onfarm agricultural systems research before beginning the site-selection process.
- Define and prioritize the site-selection criteria: location, scale, crops, soil type, management strategy, etc.
- Conduct a preliminary survey of potential study sites to obtain basic knowledge of the common characteristics of these farms. Consult with growers with whom team members have established relationships.
- If the study requires a large number of farm sites, consider identifying more potential sites than needed, and then collect basic information after finalizing the selection criteria. Extra site surveys can be useful for providing a larger context for the study.
- Consider conducting site visits as a group. Some groups find it efficient to visit sites after they have narrowed the list down to probable study locations. Final site selections should be made after these visits.
- Consider including preliminary soil tests or other diagnostic measurements (e.g., presence or absence of pests or beneficial insects) as additional information for the final site selection.
- When finalizing selections, include extra sites to allow for the possibility that some farms may be eliminated as the study progresses (the farmer’s plans may change, sites may be lost through crop failure, etc.).
- Select farm sites where growers are enthusiastic about the research project or have a track record of hosting research on their farms. This is particularly important when the project will be manipulating the system or the farmer will be asked to make special accommodations for the research.
- Compensate growers appropriately for their participation in the research, and include a plan for sharing the research findings with all cooperating farmers.
Box 3.3 describes the extensive farm-site selection processes used for a project comparing ecological characteristics among a large number of farms.
Box 3.3 Site-Selection Process for Comparing Groups of Farms with Distinct Management Systems
The goal of this team-led study, funded by SARE, was to test hypotheses about the impacts of management on ecological and agronomic characteristics in conventional and organic production systems. The team developed interdisciplinary hypotheses and an experimental design that addressed interactions among components of the systems (Drinkwater et al., 1995). In addition, each investigator had a focused, discipline-specific hypothesis about soil processes, crop production, plant pathology or arthropod community dynamics, which supported the study’s overall goals (Workneh and van Bruggen, 1994; Letourneau and Goldstein, 2001). After agreeing on the major questions, the team began the process of site selection by listing all the criteria representing all the disciplinary perspectives. While the list highlighted differences in priorities, the team agreed to maximize overlap in geographic location, local climate and parent soil types between both types of farming systems. It also produced a list of secondary criteria consisting of attributes that at least some of the sites needed to contain. Initially, the team considered two important vegetable-producing regions as potential study areas: the Central Coast valleys of California, which produce mainly cool-season vegetables; and the inland Central Valley, which produces warm-season vegetables. The team used site visits and a questionnaire to gather information on organic and conventional vegetable producers in both regions. The coastal region had favorable characteristics for some of the project’s objectives; most notably, a potentially serious root pathogen of lettuce was present on both types of farms. However, several obstacles made the Central Coast region problematic for this interdisciplinary study. Specifically, the organic and conventional farms were segregated into different valleys and had almost no geographical overlap. In addition, the very short duration of lettuce crops (six weeks) made data collection at many field sites impossible. In contrast, in the Central Valley, organic and conventional farms overlapped in their geographical distribution, and although there was a climatic gradient, both types of farms were located along the gradient. Furthermore, all of the organic farms had conventional agricultural neighbors, and some were surrounded by large conventionally farmed fields. As a result, the team chose tomato production in the Central Valley for the project. In contrast, if the team’s plant pathologist had been conducting the research alone, she would have opted to study lettuce production in the Central Coast region. Once the team selected the region and the crop to be studied, it needed to select farms as study sites. This process required compromise among disciplines. Each team member ranked sites by priorities related to their area of expertise. In this way, the sites that were most important for each discipline were sampled by the entire team, providing a strong basis for developing integrated questions. Because there was a limited number of organic farms, the team first selected organic farm sites and then identified an appropriate mix of comparable conventional farms. The team originally planned to pair sites even though they expected to use multivariate statistics, but in the end, they were not able to arrange the sites in pairs that were acceptable to all disciplines. In the first year, in addition to farm sites that were sampled by all disciplines, individual researchers chose and sampled extra sites to strengthen their own disciplinary work. At the time, this seemed like a realistic compromise because it allowed the group to take an integrated interdisciplinary approach on a majority of sites while also providing some autonomy to more rigorously test sampling methods and selection criteria for each research component. At the end of the first year, however, the group discovered that the findings from the sites that were sampled by all disciplines were much more interesting and informative. In the second year, the group agreed on a set of 18 sites that were the best for integrated questions, and these were sampled by all disciplines.
See Drinkwater et al., 1995, for more details.
Design Considerations
Along with site selection possibilities, multiple types of systems can be studied when using existing systems. The following examples offer insight into how to match methodology with research goals.
Comparing Farm Pairs
Comparing farm pairs is the most common method used to study working farm systems. It assumes that confounding variation can be reduced by carefully matching paired farm sites. This design considers each pair as a replication, which allows common statistical analyses to be used (Lockeretz et al., 1981; Reganold, 1993). The strategy works fairly well if the study involves fewer disciplines and the geographic distribution of the farm types to be compared is similar. As the number of disciplines involved in the study increases, agreeing on how to designate farm pairs can become difficult because the number of criteria used to match the pairs also grows. For example, soil type would be the most important criteria for matching farms in a study focusing on the impact of tillage intensity on soil properties. In contrast, soil type, field size, microclimate and surrounding landscapes would all need to be matched to examine the impact of farming systems on soil processes and arthropod pests. Furthermore, if the farm types are geographically segregated or differ by landscape position or soil type, establishing farm pairs without confounding environmental variability will not be possible.
Comparing Groups of Farms
When pairing farms is not possible, another option is to design the study to compare groups of farms using multivariate statistics (Drinkwater et al., 1995; Wander and Bollero, 1999). This approach allows greater flexibility in site selection because the need to find farm pairs that meet matching criteria is eliminated. Instead, the criteria are applied to groups of farms. Farm sites are selected so that confounding variables have similar distributions in each group. For example, groups of farms are usually defined in terms of specific environmental variables (Needelman et al., 1999) or management types (Drinkwater et al., 1995) and are then compared to address the question of interest. A variation on this approach is to identify a set of farms that form a continuum (in terms of environmental or management characteristics) rather than contrasting groups (Steenwerth et al., 2002). For example, the use of a chronosequence—a set of soils, farms or ecosystems that have been under differing management regimes for a varying length of time—can provide useful information about how quickly the agricultural system responds to changes in management.
In-Depth Study of a Single Site or a Single Pair of (Usually Adjacent) Farms
Farms can serve as sites for mechanistic studies of small-scale processes such as microbially mediated processes. In this case, rather than focus on the effects of management practices on ecosystem processes, researchers concentrate on interactions within a farming system that is already well characterized by previous research (Steinheimer et al., 1998).
Larger-Scale Studies
Systems projects often address questions about processes that occur at scales larger than a field or farm. Examples include watershed comparisons (Sovell et al., 2000; Napier and Tucker, 2001) or studies that examine how land management varies across regions or through time (Auclair, 1976; Donner, 2003).
Mother-Baby Trials
This hybrid method combines the use of existing farm sites and experimental research station plots to systematically link biological performance with farmer assessment of technologies (Snapp et al., 2002). This approach is extremely powerful for developing improved management options, and it allows researchers to evaluate a wide variety of management strategies across varying farm environments and in a replicated field station design in a single experiment. It is also appealing to farmers because they can choose the options most relevant to their operation after viewing trials at the research station. A systematic framework to guide this approach was developed by Snapp et al. (2002) to evaluate soil fertility management options available to smallholder farmers in Malawi. The study, using replicated experiments at research stations, included all fertility management options and investigated ecological mechanisms and outcomes; farmers then selected a subset of these options, usually three or fewer, to test in their own fields under realistic conditions.
Case Studies
Case studies are useful when in-depth, qualitative information is needed. The social sciences rely more heavily on this approach than other disciplines, but case studies have also been proven useful in providing a holistic overview of specific farms. In fact, case studies can be excellent educational tools because stories are effective for reaching diverse audiences (Mikkelsen, 1995) and for enlightening agricultural researchers about management systems they rarely encounter. Case studies can also serve as the basis for generating hypotheses and can lead to new discoveries about how particular management systems function on working farms. The SARE case study on p. 58 provides an in-depth example of how one researcher used qualitative methods to holistically evaluate the decision-making process at three very different dairy systems in rural Wisconsin.