The Process

Following these 10 steps will help you develop a successful on-farm research project.

1. Identify your research question and objective.
2. Develop a research hypothesis.
3. Decide what you will measure and what data you will collect.
4. Develop an experimental design.
5. Choose the location and map out your field plots.
6. Implement the project.
7. Make observations and keep records throughout the season.
8. Collect research data.
9. Analyze the data.
10. Interpret the data and draw conclusions.

Each of these steps is expanded on below, providing an overview of the entire on-farm research process from initial planning to implementation to drawing final conclusions. Keep in mind that the focus here is on crop-based research, but the same process applies in livestock- or pasture-based systems.

TABLE 1: From Research Question to Research Hypothesis

STEP 1: Identify your research question and objective. Identifying your research question involves moving from the general to the specific—from ideas or hunches to a clear objective—and selecting just one yes-or-no question to answer. In developing your question, consider your own capabilities and if the information needed to answer the question is actually measurable. The question will usually ask whether a new approach is an improvement over the current one or if it will help you meet some goal or objective. Here are some sample research questions:

• Can a legume cover crop substitute for my standard commercial nitrogen fertilizer application?
• Will a new tomato variety produce a higher yield than the standard tomato variety that I usually plant?
• Can I eliminate a particular pesticide application, replace it with a more environmentally sound approach, and increase my bottom line per acre?
• Will changing my tillage practices change the amount of irrigation I need? Or, if I switch to a no-till or reduced-tillage system, will my yields be reduced?

You can think of the research question as a comparison between two or more practices. The examples above compare: a cover crop versus commercial fertilizer; the performance of one variety versus another; a pesticide versus an alternative pest control practice; and a current tillage practice versus a reduced-tillage practice. The practices compared in the research project are called treatments. To further clarify your intent, you may also want to re-write the research question as an objective. Using the legume-cover crop example above, an objective based on that question might look like this: My objective is to determine if a legume cover crop will supply enough nitrogen to meet the needs of my subsequent cash crop.

If you are having trouble articulating your research question or objective, talk to other farmers or an agricultural advisor to help clarify your thinking. Again, keep it simple: The simpler the research question, the simpler the project will be to conduct.

STEP 2: Develop a research hypothesis. Your research hypothesis stems directly from the research question or objective. A hypothesis is simply a clear statement of what you expect the outcome of your experiment to be, based on the limited evidence you have at hand. A well-written hypothesis statement can be confirmed (or denied) with actual data. In fact, the hypothesis gives an indication of what will actually be measured in the experiment. A well-developed hypothesis will help you obtain the most useful and practical information for the time and resources you invest in your research project. Possible hypothesis statements for the research questions outlined above are summarized in Table 1.

STEP 3: Decide what you will measure and what data you will collect. The next step in planning your on-farm research project is to determine the data you will be collecting. Your research hypothesis should give you a general idea, but now is the time to be specific: What will you measure and record in order to answer your question and test the validity of your hypothesis? This is also the time to decide what techniques you will use to get your data, looking at factors such as cost, practicality and feasibility.

In many crop research projects you will be collecting yield data, but depending on your project, you might also be collecting data on soil nutrient levels, crop development, plant health, plant height, leaf number, chlorophyll content, pest numbers, yield quality parameters (e.g., protein, Brix levels, fruit size, insect damage, moisture, etc.), costs or anything else you want to know about. The key determinant in deciding what to measure is whether the information will be useful in answering your research question.

Say, for example, you are looking at whether a higher planting density reduces weed competition in the field. Once you have your treatments defined (i.e., narrower row spacing and/or more plants within the row), you will need to decide what you will measure as an indicator of weed competition. Some possible options include percent weed cover at specific time intervals during the growing season, or the weight of weed biomass. You might also measure the effect of higher planting density on both weed density and final crop yield. Remember that each variable you decide to measure will come with its own time commitment in data collection and analysis, and may incur costs.

STEP 4: Develop an experimental design. It is tempting to rush through the previous steps and start planning what the experiment will look like in the field. But the task of designing your experiment should flow from the previous steps. Experimental design includes arranging treatments in the field so that error and bias are reduced, and data can be accurately analyzed using statistics. Experimental design and statistical analysis (step 9) go hand in hand: If an experiment has a poor design, you cannot have confidence in the data. For example, see the profile of farmer Steve Groff, who studied grafting to control disease in high tunnel tomatoes. In the first year, a mistake was made in the experimental design that prevented him from addressing some of his research questions, and the mistake was corrected for the second year.

There are several standard experimental design layouts used in on-farm research. Which one you choose will be based primarily on the number of treatments you are investigating. You can explore experimental design concepts and techniques in more detail in the next section, Basics of Experimental Design. If possible, plan your experiment for at least two growing seasons to increase the reliability of your results.

STEP 5: Choose the location and map out your field plots. After you have figured out your experimental design, you are ready to choose a location and design your field setup. You should be specific about plot size and layout, how the crop will be planted, which treatments are to be applied in each plot, and any other important aspects of managing the plots. Some guiding principles to help site your project:

• Select a field that has the right characteristics for what you are testing. Look at the field history and make sure there are no major problems that might prevent you from establishing the plots, or that could negate your results.
• Research plots should be accessible and easy to maintain. To facilitate management, for example, you may want to set up plots that run the length of the field and are wide enough for one or two tractor passes. It should be located close to the home farm so you can make observations regularly.
• Each treatment plot should be large enough to collect the data you need. If you can, separate your treatments with buffers to reduce cross-contamination.
• To moderate the effect of external variation, choose an area that is as uniform as possible in terms of soil characteristics, management history or slope, to name a few important types of variation.
• If there is some variation in the field that cannot be avoided, such as slope, drainage or soil type, try to set up your plots so that they are as uniform as possible with respect to field conditions. Since it is not always possible to achieve this, you can use blocking, replication and randomization to separate out the effect of field variability from the actual treatment effects. More information on these techniques is provided in the next section, Basics of Experimental Design.
• Keep in mind that land adjacent to the research plots can also have an impact on your research due to runoff, pesticide drift or by harboring pests that migrate into the research plots. This is potentially another source of external variation. To control these effects, establish a border or buffer zone around the entire research project. Ideally, a buffer should be a minimum of one tractor pass on all sides, or larger if conditions permit. Your technical advisor can help you determine what is most appropriate for your particular project.
• Last, create a detailed plot map for your chosen location based on your research design.

See Figures 3 and 4 for examples of plot maps that incorporate these principles.

STEP 6: Implement the project. Now that you are ready to implement the project, begin by establishing the research plots based on the map you created. Measure and mark your plots with clearly visible stakes or flags. In order to prevent mishaps with the project, make sure you discuss plot design, location, timeframe (one year or multiyear) and implementation with your entire farm crew, and share the detailed plot map with everyone involved.

Throughout the experiment, be careful to manage all plots exactly the same, except for the treatments (the practices you are testing or comparing.) For example, if your experiment is a comparison of two different varieties of tomatoes, plant all the plots on the same day using exactly the same planting technique, make the same number of passes with the tractor on all plots, cultivate all the plots in the same way and use the same pest control techniques in all plots. Follow this same principle when you set up your treatments. If you are comparing fertilizer treatments, for example, set the equipment for the first application rate and fertilize all the plots that are to receive that rate at the same time. Then change the setting for your second application rate and do all the plots assigned to receive that rate, and so on. The goal is to standardize as much as possible the techniques by which all field work is done. If possible, have the same group of people involved throughout the project so that there is consistency in how the plots are managed.

Most importantly, plan ahead and communicate. Before you start any field work, create a management plan and calendar for the project. Be specific about how the plots and the crop will be managed, how and when treatments are to be applied, and what data will be collected and how. Then make sure you review this plan with everyone who will be involved in the project. Good planning and communication can help ensure that the project is implemented correctly, that the work is done on time, and that you have the equipment and labor available when you need it.

STEP 7: Make observations and keep records throughout the season. Separate from your actual data collection (step 8), make observations and take notes throughout the season on influential factors such as rainfall, temperature, other weather events, seedling emergence, crop growth, soil condition, pest problems, field operations or anything else that seems relevant. Keeping a designated notebook, file or spreadsheet with this information will help you interpret your data and put your research results in context. In some cases, your observations will apply to the entire experiment: “Plants in all plots appear to be suffering from the extended dry period.” In others, you may want to record observations about specific plots or treatments: “Plants in treatment A appear taller than treatment B.” If you notice such differences between treatments, you may decide to measure those differences, even if you did not plan to do so originally.

STEP 8: Collect research data. For successful data collection:

• Be highly organized and specify your data collection techniques ahead of time.
• Prepare your data record sheets beforehand and have all your copies ready to fill out.
• If you are collecting samples, have all your bags or containers labeled accurately and organized by treatment and plot to facilitate the process.
• Remember to keep all treatments and plots separate! Do not lump data together thinking that you will be able to just take an average. Doing so will invalidate your data.
• If you are measuring yield, try to harvest from the center of the plots for your research data and, again, keep each treatment and plot separate. You will eventually harvest the whole area, but do not include buffer rows in your data.
• If you are measuring other effects (e.g., soil characteristics, weed cover, disease or insect damage, etc.), use random sampling procedures.
• Allow adequate time for sampling. For instance, expect plant sampling in 12 experimental units to take at least four hours; collecting soil samples will likely take longer.

STEP 9: Analyze data. Statistics are the most common tool used to determine if any differences observed in the treatments or comparisons are truly a result of the change in practice or merely a result of chance, due to natural variation. The statistical techniques that you will use to analyze your data depend on the research design you have used. You can learn to do your own data analysis, either by hand or with a statistical software program. In most situations, you will also want to consult with your technical advisor or Cooperative Extension personnel for guidance and assistance with your data analysis. The most common designs and statistical tests for on-farm research are discussed in more detail in the Experimental Design and Statistical Analysis sections.

STEP 10: Interpret the data and draw conclusions. Now that you have analyzed the data from your on-farm research, what do the results tell you? What can you infer from the data, and how can you apply that information to your farm? The statistical analysis you use will indicate whether or not there is a real or “significant” difference in the treatments, practices or varieties you are comparing. If there is a difference, and you feel confident about the results, you may decide to begin making changes in your farming practices.

But before you proceed, first discuss your results with your management team, other farmers or Cooperative Extension staff; it is always good to get a second opinion. Even then, you may still want to repeat the study for a second or third year to confirm the results and enhance the reliability of the data. If you are not sure of the results, or if the data seems off base, then you will need to dig deeper to determine what might explain the findings. Refer back to the observations and notes you made throughout the season (step 7). Was there some kind of environmental effect you did not anticipate? Did rainfall or temperature patterns over the course of the experiment influence the outcome? Was there a problem with how the plots were managed or in how the treatments were applied? Again, discuss your thinking with others before you decide how to proceed. Most important in this final stage of your project is to be objective and to be careful about making major changes in your management until you have accurate and reliable information.