EDGAR

Experimental Design Generator And Randomiser

by James K. M. Brown
Cereals Research Department, John Innes Centre, Colney, Norwich, NR4 7UH, England


Choose an experimental design


News (16/11/04): I am currently re-writing EDGAR. It will produce spreadsheets in Microsoft Excel format. My intention is to make EDGAR faster, more reliable and easier to use and to make the output more convenient. I also hope, in due course, to include incomplete block designs. Eventually, this will become EDGAR version 2. I would greatly appreciate feedback on the new programmes.

More news (2/3/05): The Fortran routines which the old EDGAR programmes use are about to be taken off our server. Please use the new programmes where available. I am rewriting the other programmes rather slowly so if you need one desperately, please let me know and I will try to speed it up.

News (13/6/05): The server which ran the old versions of the EDGAR programmes has now been switched off.

News (23/12/05): EDGAR has moved. The new address is www.edgarweb.org.uk. Please bookmark it for future reference.

News (20/08/10): I've fixed a bug in the way that the Latin Squares and Complete Randomisation with Unequal Reps programmes randomised the assignment of treatments to experimental units. Thanks to Michael Hinchcliffe for pointing it out.


Note that, in the documentation below, the specifics of how EDGAR operates are out-of-date. I'll get round to updating them sometime soon. The main thing you should know at the moment is that EDGAR produces spreadsheets in MS-Excel 2003.

What is EDGAR?
For those from outside the JIC
Principles of experimental design
What designs can EDGAR handle?
How large a design can EDGAR generate?
Are there any restrictions on the format of input to EDGAR?
How can I save output from EDGAR?
Future developments
Acknowledgments


What is EDGAR?

EDGAR is a programme which generates experimental designs and randomises the position of experimental treatments in the design. Version 1.0 of EDGAR handles a small number of relatively simple designs, but these are appropriate for many experiments of the kind done at the John Innes Centre. My purpose in writing EDGAR is to help users of the programme to design their experiments in such a way that the subsequent analysis of the data is comparatively straightforward.

EDGAR is written so that you can access it via the Internet. You will need a world-wide-web (WWW) browser capable of interpreting forms, such as version 2.0 or higher of Netscape.

This introduction is not intended to be a substitute for proper advice. It is much better to spend a little time seeking advice from a statistician about the design and analyis of your experiment beforehand than to end up with a set of data, acquired at the cost of considerable time and money, from which it is very difficult or even impossible to draw any conclusions. To quote Sir R. A. Fisher, "to call in a statistician after the experiment is done may be no more than asking him to perform a post-morten examination: he may be able to say what the experiment died of".


For those from outside the JIC

The John Innes Centre is an internationally renowned centre for research in plant sciences, with particular strengths in molecular biology, cell biology and genetics; this should explain my choice of examples in this introduction. However, the various designs which EDGAR 1.0 generates are appropriate for experiments in many other subjects, and the descriptions given below should provide much of the information you need to choose an appropriate design for your work.

Principles of experimental design

This section gives a brief overview of the principles of good experimental design. However, it is not intended to be a substitute for personal advice or for a statistics textbook. There are fuller discussions about experimental design in many books, such as Chapter 6 of Steel & Torrie, Principles & Procedures of Statistics: a Biometrical Approach.

A little terminology

Experimental treatments are whatever it is that you control in the experiment. These might be, for example, different environmental conditions or fertiliser regimes in which plants are grown, or different media used for tissue culture. Furthermore, if you are interested in how different species, or different genotypes of one species, vary in a particular character, those species or genotypes are considered to be treatments. The term factor refers to a set of different treatments. A typical experiment might investigate long and short days - two treatments within a factor of daylength - and three genotypes (i.e. treatments) within a factor of plant genotype.

A unit is the part of the experiment to which each treatment is applied. For instance, units may be pots to which different fertilisers are applied or petri dishes in which you put different media or growth rooms in which you have different environmental conditions. In an experiment involving different genotypes, you obviously can't choose the genotype of any particular plant; here, the experimental unit could be considered to be the space in the growth cabinet, glasshouse or field plot in which you decide to put a particular genotype. Other experiments may involve doing something to a set of different samples in order - extracting RNA or scoring microscope slides, for example. The experimental unit here is the slot in the time sequence in which you work on each sample.

Three principles of good experimental design are replication, randomisation and blocking.

Replication

If your experiment is intended (as many experiments are) to test whether or not a number of treatments differ in their effects on a character in which you're interested, those treatments must be applied to replicate units of the experiment. You can only show that two treatments have different mean effects on the character if you have measured several samples given the same treatment.

For instance, observing that one plant of a particular genotype is more resistant to a disease than one plant of a different genotype tells you nothing about the difference between the mean disease resistance of the two genotypes; the difference you observed could have been caused by the environment or the inoculation procedure affecting the two plants differently. To make any inference about the mean difference between the genotypes, you need to test several plants of each.

The more replicates you use, the smaller the differences you should be able to identify between the means for the various treatments. In some cases, you may wish to know if some treatments are more variable than others; you may be able to reach such a conclusion if you have used sufficient replicates of each treatment.

Randomisation

The purpose of randomising an experiment is to ensure that the layout of the experiment does not consistently favour one or other treatment in such a way that your results would be misleading. For instance, suppose that you are measuring the time of flowering of plants in a growth cabinet. If you arrange the pots so that all the plants of a variety are next to one another, and you find that one variety flowers earlier than the rest, does it mean that that variety is inherently earlier-flowering, or does it mean that the light conditions in that part of the cabinet cause plants (of whatever genotype) to flower early? You can't tell from an experiment designed in this way. Or suppose that you are studying the resistance of a number of plant varieties to a disease. The experiment is laid out so that one variety (A, say) is always next to another (B). Both of them are highly diseased: does this mean that they are both susceptible, or that one of them is in fact more resistant but has been infected by a large number of spores from its more susceptible neighbour? Again, you can't tell from this experiment.

Randomising the treatments in time or space is an insurance policy, to take account of variation that you may or may not know to exist under the conditions of your experiment. For instance, the levels of light in growth cabinets vary considerably, so randomising the layout of the plants of different genotypes is essential to make sure that no one genotype is consistently exposed to light levels which are particularly high or low. In the pathology example, randomising the layout of plants of the various varieties means that no one variety is consistently exposed to particularly high or low levels of inoculum from nearby plants.

Blocking

If you randomise the treatments among all the units in your experiment, you will almost certainly end up with clusters of units given the same treatment. For instance, this is a randomised sequence of plant genotypes (A-D) for extracting RNA to be used in a quantitative assay of transcription levels:
B B D D C C C B A D A A C A B A B D D C
Notice that the plants of genotype A have tended (purely by chance) to be clustered towards the later times. Another example: this is a randomised layout of plants of four genotypes on a glasshouse bench:
D A B D B
A A C A D
C D A B D
C C C B B
Here, all of the C genotypes have been placed (again, just by chance) at the bottom left-hand corner. Such layouts, where each treatment has an equal chance of being applied to each unit of the experiment, are known as completely randomised designs.

However, this clumping effect can be avoided to a large extent if you group the units into blocks. Typically, in each block, one unit is given each treatment, and the treatments are randomised among units within blocks. The following are similar to the two examples above, but randomised blocks have been used rather than completely randomised designs:

Block: 1         2         3         4         5
       D C B A : D B C A : D B A C : A C D B : B D C A
and
Block: 1 2 3 4 5
       D C D A C
       C B B B B
       A A C C D
       B D A D A
Blocking should be used to control systematic factors which might affect your experiment. Such factors might include, for example, light levels and temperature in glasshouses and growth cabinets or the fertility of soil in a field trial. Time could also be a block factor, since your concentration or expertise could alter as you carry out a task, such as measuring disease levels, scoring microscope slides or making quantitative extracts of nucleic acids. Blocks should be arranged so that systematic differences are expected to vary more between blocks than within them.

Confounding

Sometimes, experiments are done in such a way that the application of the various treatments is confounded with some aspect of the design. I gave an example earlier of a growth cabinet experiment in which all plants of one variety are placed in one area, all plants of a second variety in a second area, and so on. Here, any effect of genotype is confounded with the location within the growth room. Confounding like this must be avoided, because (in this example) it's impossible to separate the effects of the factor you're interested in (genotype) from one which is irrelevant (location).

What designs can EDGAR handle?

Completely randomised design

This is available only for one treatment factor. All treatments must have the same number of replicates.

Complete randomisation allows differences between the mean effects of treatments to be estimated with higher precision than other designs do, if you are sure that there is very little extraneous, systematic variation. However, it doesn't allow for the possibility that there may be some unknown extraneous factor, so if in doubt, use a blocked design. In principle, you can use different numbers of replicates of each treatment in a completely randomised design, but this is not an option in EDGAR 1.0.

Randomised complete blocks

This is available for either one or two treatment factors. All treatments in the one-factor design, and all combinations of treatments in the two-factor design, are represented once in each block.

This is much the most widely-used design for a single factor, and is the easiest way of controlling extraneous, systematic variation. Randomised complete blocks are also widely used for two treatments, but split plot designs (see below) may offer an advantage in some experiments.

If you have many treatments, it may be better to use an incomplete block design, in which the experiment is arranged in smaller blocks, each of which has some of the treatments. EDGAR 1.0 does not generate incomplete block designs.

Split plots

In a split plot design, the experiment is arranged so that one factor is applied to large units of the experiment (called main plots, from the terminology of field trials), while the second factor is applied to smaller units of the main plot, called sub-plots. Split plots are particularly useful in at least two important kinds of work at the JIC. One of these involves disease trials, in which different varieties of a plant species are grown as sub-plots (e.g. as pots in a glasshouse or rows in a field trial) while different isolates of a pathogen are applied to main plots (groups of pots or rows). The other type of work involves controlled environment experiments, in which different cabinets (main plots) have different enviroments, while pots or trays of different plant genotypes constitute sub-plots within those main plots.

EDGAR 1.0 can generate one common type of split-plot design, in which the experiment is arranged in blocks, each of which has one main plot of each treatment of the first factor, while each main plot has one sub-plot of each treatment of the second factor. Main plot treatments are randomised within blocks and sub-plot treatments are randomised within main plots. For instance, this is a design for an experiment in which four varieties of a plant (A-D) are infected with three isolates of a fungus (P-R); the experiment is in two blocks:

             Block:       1             2
         Main plot:   1   2   3     1   2   3
           Isolate:   R   P   Q     Q   R   P
Variety: Sub-plot 1
   D   C   B     C   B   B
                  2   C   D   C     A   C   D
                  3   A   B   D     B   A   A
                  4   B   A   A     D   D   C
Here, the main plots are arranged in randomised blocks, while the sub-plots are completely randomised within main plots.

Many other split-plot designs can be constructed, although EDGAR 1.0 does not provide other options. In principle, any arrangment of sub-plots - randomised blocks, a complete randomisation, Latin squares, etc - can be nested within any arrangement of main plots.

Latin squares

This design is particularly useful in the limited number of cases in which there are two known sources of systematic variation. An example of this involves growth cabinets which have shelves; two sources of variation are the height of the shelf and the position along the shelf from the door, both of which may affect the micro-climate within the growth room. Another example might concern a number of people carrying out a particular operation in which their level of skill may vary; here, the sources of variation are the person and the sequence in which the material is scored.

In a Latin square design, the units are arranged in a square (real or imaginary), and each treatment appears once in each row and each column. For instance, with four varieties (A-D):

B A C D
D C B A
A B D C
C D A B
Since, in a Latin square, one has to estimate the effects of three sources of variation - the row factor, the column factor and the treatment that you're actually interested in - this design is less efficient than the simpler block design. It is best used, therefore, when there is good reason to suppose that there are indeed two extraneous sources of systematic variation. Latin squares are rarely used when there are more than a few treatments.

How large a design can EDGAR generate?

EDGAR 1.0 can generate designs for very large experiments:
Completely randomised, one factor:
200 treatments, 200 replicates, total of 10,000 units
Randomised complete blocks, one factor:
200 treatments, total of 10,000 units
Randomised complete block, two factors:
100 treatments for each factor
Split plots (main plots as randomised blocks, randomised sub-plots):
50 main plot treatments, 200 sub-plot treatments
Latin squares:
12 treatments
Please contact me if you wish to use EDGAR to generate a design for an experiment which is even larger than this.

Are there any restrictions on the format of input to EDGAR?


How can I save output from EDGAR?

Each time you run EDGAR, a different randomisation of your design is generated. If you want to save a design generated by EDGAR in order to use it in another programme (e.g. Excel, Genstat, Minitab) you can either
i) save the output as a file (in Netscape: File menu, Save As), then edit the file (e.g. using Notepad) to remove lines that you don't need from the top and bottom of the file, or
ii) cut the table with the design from the screen (highlight the table with the mouse, type Control-C), and paste it into a text editor, such as Notepad. You can then save this file and import it into another application.

Future developments

The extent to which I will develop EDGAR depends on how much it is used, whether or not those who use it wish to have facilities that are not provided in version 1.0 and whether or not I have time. A short "wish-list" of things that I might develop if there is sufficient demand include: I would very much like to know whether or not you found EDGAR useful and what improvements would help you. I would also be interested to know about the kind of research that you are doing.

Acknowledgments

Many thanks to Pat Newby, Computing Centre, UEA, for her help with CGI programming. The writing of EDGAR was supported by the Ministry of Agriculture, Fisheries and Food.

EDGAR is written by James Brown, John Innes Centre, Norwich, England.

Comments welcome. I would particularly like to know what kind of research you do, if you found this programme useful and what changes or improvements would be helpful.

Last updated on 13th June 2005