Evaluating New Products

Nov 22, 2013

Evaluating claims of new products that could potentially improve yield and tree health is a daunting task. Every week I get calls and literature from people promoting fertilizers and techniques that "resist insects," "reduce salt levels in the soil," "increase crop quality," "release that natural fertility of your soil," and numerous other claims. There just is not enough time in the day to approach each and every one of these materials or techniques, even though some may, in fact, be promising.

So what does a grower do? You hear about a new product. It only costs $20 an acre to apply. Might as well fly it on all 50 acres. But then, how do you know it has done anything? What results do you have to compare it with? Last year's yield which was miserable? We know how variable avocado yields are, so last year's harvest may not be a good comparison.

When we conduct field trials, we assume a clear comparison is available to test the effects of the treatment. With field trials, there are usually small plots, repeated several times (at least three), and arranged in an apparent haphazard (random) fashion. The reason is threefold: 1) to account for variability in the field, 2) to prevent a systematic bias in favor of one treatment over another and 3) to see if differences in treatments are due to chance or to the superiority of the treatment.

How are observational trials different from replicated one? The big difference is that they are not replicated. Each treatment occurs only once, so we have no measure of the natural variability in the field or trees. As a result, we risk thinking we have a difference due to treatment which is actually due to field variability. Without replication there is no way to tell.

Let's examine this replication idea a little more closely. We had a frost trial where we applied copper or a water control spray to young trees in November. Copper is a noted bactericide and the idea was to control the frost-nucleating bacteria. Forty trees, randomly spaced in the orchard were sprayed with either a dilute copper spray according to instructions or water alone. We evaluated frost damage to the trees in January. The first counts showed 40% frost damage with the copper spray and 60% with the water alone. Great. Let's go out and spray the whole orchard next year with copper. However, successive counts showed 50% frost damage with the copper and only 30% from the water. In the end, there was no significant difference to trees that had been sprayed with either material.

These results show the natural variability in biological systems and demonstrate the disadvantages in looking at results from a non-replicated trial based on a single year. This becomes even more important when interpreting information from a trial site different from your own. If every grower sprayed a non-replicated treatment at their own ranch, the risk of coming to the wrong conclusion about that treatment at each location is still 50%. Just like flipping a coin. Is that worth spending money on?

As each of the variables (soil type, irrigation quality, management, etc.) increases, the risk of making a poor decision about a product or practice increases, as well. You can see that there are difficulties associated with relating information from a non-replicated trial based on a single year of data at a different location to your own situation.

How does someone go about evaluating a new practice or material at home without going through all the complications of a complicated research trial? Mary Bianchi, farm advisor in San Luis Obispo and I came up with a little checklist.

  1. Be conservative in your approach and critical observations. Resist the urge to spray the whole grove. Leave something, so that a comparison can be made. Preferably run a side-by-side comparison.
  2. Use consistent farming practices across all areas of the trial.
  3. Compare the new practice to one which is a standard for your operation.
  4. Don't bias your results by implementing the new practice where it stands to have the best effect anyway. For example, don't spray boron on the trees that always give a good yield.
  5. Run the test more than one year and in more than one location, especially if the new practice is costly.
  6. Talk to the industry and use the experience of others in different locations as a check on your own experience. A good place to swap ideas is at the California Avocado Society/University of California Cooperative Extension sponsored bimonthly meetings.

By Ben Faber
Author - Advisor

Attached Images: