Educ Psychol Rev (2017) 29:583–598 DOI 10.1007/s10648-015-9339-x

REVIEW ARTICLE

A Critical Review of Line Graphs in Behavior Analytic Journals

Richard M. Kubina Jr.1 & Douglas E. Kostewicz2 & Kaitlyn M. Brennan2 & Seth A. King3

Published online: 3 September 2015 # Springer Science+Business Media New York 2015

Abstract Visual displays such as graphs have played an instrumental role in psychology. One discipline relies almost exclusively on graphs in both applied and basic settings, behavior analysis. The most common graphic used in behavior analysis falls under the category of time series. The line graph represents the most frequently used display for visual analysis and subsequent interpretation and communication of experimental findings. Behavior analysis, like the rest of psychology, has opted to use non-standard line graphs. Therefore, the degree to which graphical quality occurs remains unknown. The current article surveys the essential structure and quality features of line graphs in behavioral journals. Four thousand three hundred and thirteen graphs from 11 journals served as the sample. Results of the survey indicate a high degree of deviation from standards of graph construction and proper labeling. A discussion of the problems associated with graphing errors, future directions for graphing in the field of behavior analysis, and the need for standards adopted for line graphs follows.

Keywords Line graphs . Time series . Graphical construction guidelines . Graphing standards

Behavior analysis, a subfield of psychology, owes a great debt to the visual display of data. For example, the cumulative recorder offered a standard visual display of an organism’s perfor- mance data. The distinctive visual patterns of behavior led to the discoveries such as schedules of reinforcement (Lattal 2004). As behavior analysis moved forward in time, the visual displays shifted from cumulative recorders to line graphs. Data show that cumulative records in the

* Richard M. Kubina, Jr. [email protected]

1 Special Education Program, The Pennsylvania State University, 209 CEDAR, Building University Park, State College, PA 16802-3109, USA

2 University of Pittsburgh, Pittsburgh, PA, USA 3 Tennessee Technological University, Cookeville, TN, USA

584 Educ Psychol Rev (2017) 29:583–598

Journal of the Experimental Analysis of Behavior continue to appear infrequently and in other years not at all (Kangas and Cassidy 2010).

The shift away from cumulative records to line graphs coincided with the advent of an emphasis on applied work. The oft-cited paper of Baer et al. (1968) laid the foundation for discerning the facets of behavior analysis. Three of seven characteristics of applied behavior analysis have a direct link to visual displays of data. First, Analytic refers to a convincing demonstration of an experimental effect. The preferred medium for all analysis of data occurs through graphs. Second, Effective conveys the requirement for the intervention to produce a practical and meaningful magnitude of behavior change. Line graphs allow for the determi- nation as well as the public documentation and communication of the significance of behav- ioral improvements (Spriggs and Gast 2010). And third, Generality means that the behavior persists across time, environments, and operant responses within a class. The line graph, part of the family of time series graphs, directly portrays the extent to which behavior does or does not persist.

Principles of graphic presentation for line graphs have quality standards necessary for the accurate representation of data. A number of publications have described the standards for proper construction for line graphs (American National Standards Insti- tute and American Society of Mechanical Engineers 1960, 1979; American Standards Association 1938; American Statistical Association 1915; Department of the Army 2010). For example, the publication of Time series charts: a manual of design and construction set forth agreed upon standards for line graphs (American Standards Association 1938). The committee provided guidance on many specific features ranging from scale rulings and graph dimensions to the weight of lines and use of reference symbols. Through time, many professional organizations and researchers have continued to offer principles of design and procedure for constructing high- caliber line graphs (e.g., Behavior Analysis—Cooper et al. 2007; Statistics—Cleveland 1993, 1994; General Science—Scientific Illustration Committee 1988; Technical Drawing, Drafting, and Mechanical Engineering—Giesecke et al. 2012). Table 1 lists major quality features of line graphs tailored toward use in the behavioral sciences.

An analysis of the following basic behavior analysis (Alberto and Troutman 2013; Catania 1998; Cooper et al. 2007; Malott and Shane 2014; Mayer et al. 2014; Pierce and Cheney 2013; Vargas 2013) and single case design books (Barlow et al. 2009; Gast 2010; Johnston and Pennypacker 2009; Kazdin 2011; Kennedy 2005) corresponds to the graphical standards for a line graph previously listed. In addition to quality standards line graphs have an essential structure consisting of two axes, the horizontal and vertical, representing a time unit and a quantitative value, respectively. Time units can cover minutes, hours, days, weeks, and years based on the second (National Institute of Standards and Technology 2014). The range of behavior on the vertical axis spans dimensionless quantities like percentages and ratios to dimensional quantities such as repeatability and temporal extent measured with frequency and duration, respectively (Johnston and Pennypacker 2009).

Not adhering to the essential structure may yield distorted, exaggerated, or imprecise information. The essential structure shows change over time. Figure 1 shows three line graphs with the same data. The first line graph made following the Bproportional construction ratio,^ discussed later, displays a series of data with a moderately increasing variable trend. The line graph has an extended vertical axis changing the variability from moderate to low. The trend also increases when compared to the previous graph. Stretching the horizontal axis in the third line graph depresses the trend and decreases variability.

585

T ab

le 1 Q ua lit y fe at ur es

of a lin

e gr ap h an d m ea su re m en t

E ss en tia l st ru ct ur e

Fu nc tio

n M ea su re d

V er tic al ax is la be le d w ith

qu an tit at iv e m ea su re ;

To sh ow

t he

ch an ge

in th e m ea su re

ov er

tim e

W ha t la be l do es

th e ve rt ic al an d ho ri zo nt al ax es

m ai nt ai n?

ho ri zo nt al a xi s la be le d w ith t im

e un it

(H ar ri s 19 99 )

D oe s th e fi gu re m

ai nt ai n a lin

e fo r ea ch a xi s?

Q ua lit y fe at ur e

V er tic al a xi s le ng th h as a 2 :3 r at io t o th e

To p ro pe rl y di sp la y da ta v ar ia bi lit y an d lim

it di st or tio

n Is t he r at io o f ve rt ic al t o ho ri zo nt al a xi s 5: 8 to 3 :4 ( 63 t o 75

% ho ri zo nt al ax is

(C oo pe r et al . 2

00 7;

Pa rs on so n an d B ae r 19 78 )

di ff er en ce )?

D o ax es

lin e up

in al l m ul tip

le ba se lin

e gr ap hs ?

A re a xi s le ng th s th e sa m e fo r al l fi gu re s w ith

t he s am

e un it

w ith

in e ac h ar tic le ? A re a ll fi gu re s w ith t he s am

e un it sc al ed

to t he s am

e m in im

um an d m ax im

um ?

T ic k m ar ks

po in t ou tw ar d

To pr ev en t or

m in im

iz e da ta ob fu sc at io n (C le ve la nd 1 99 4)

O n bo th

ax es ,a re

tic k m ar ks

po in tin g ou tw ar d fo r th e en tir e le ng th

of t he a xe s?

A m in im

al nu m be r of

ev en ly sp ac ed

tic k m ar ks To

de cr ea se

gr ap h cl ut te r em

ph as iz in g th e m os t pr om

in en t

O n bo th ax es ,a re tic k m ar ks

ev en ly sp ac ed

(i .e ., at eq ua li nt er va ls )?

fe at ur e, t he d at a (C le ve la nd 1 99 4)

T ic k m ar ks

ha ve

la be ls

D el in ea te s th e va lu e of

th e ax es ’ un its

(R ob bi ns 2 00 5)

O n bo th

ax es ,a re

tic k m ar ks

nu m be re d?

A re

th e sc al e co un ts

co rr ec t?

D at a po in ts c le ar ly v is ib le

E nh an ce s da ta c la ri ty ( R ob bi ns 2 00 5)

A re d at a po in ts o n th e fi gu re c le ar ly v is ib le ?

D at a pa th s cl ea rl y vi si bl e (i f us ed )

Sh ow

s th e di re ct io n of d at a cl ea rl y (C oo pe r et a l. 20 07 )

If t he f ig ur e co nt ai ns a d at a pa th , i s it vi si bl e?

C on di tio

n ch an ge

lin es

(i f us ed )

V is ua lly

se pa ra te s da ta be tw ee n co nd iti on s (C oo pe r et al . 2 00 7) If th e fi gu re

co nt ai ns

a co nd iti on

ch an ge

lin e, is it vi si bl e?

C on di tio

n la be ls

Id en tif ic at io n of e xp er im

en ta l co nd iti on s (C oo pe r et a l. 20 07 )

If t he f ig ur e co nt ai ns a c on di tio n ch an ge l in e, d oe s it ha ve l ab el s?

Fi gu re

ca pt io n

W he n co m bi ne d w ith

ot he rg

ra ph ic el em

en ts ,c on ve ys

m ea ni ng

D oe s th e fi gu re

co nt ai n a ca pt io n?

(C oo pe r et a l. 20 07 )

Educ Psychol Rev (2017) 29:583–598

586 Educ Psychol Rev (2017) 29:583–598

Fig. 1 Sample graphs containing the same scaling with variable length axes

The Behavior of Organisms of Skinner (1938), BSome Current Dimensions of Applied Behavior Analysis^ of Baer et al. (1968), and chapters of Cooper et al. (2007) on the construction and interpretation of graphical displays represent examples for the use, rationale, and creation of behavior analytic line graphs. As a result, line graphs have become the primary visual display for presenting behavioral data in fieldwork, theses, dissertations, lectures, conference presentations, and journal articles (Cooper et al. 2007; Mayer et al. 2014; Poling et al. 1995; Spriggs and Gast 2010). How well the field of behavior analysis attends to essential structure and quality features of line graphs, however, remains unknown. The current survey examines the quality of line graphs contained in behavioral journals and attempts to answer two questions. First, how well do selected visual graphics follow the essential structure of line graph construction? Namely, to what extent do selected line charts have time units on the horizontal axis and quantitative units on the vertical axis? Second, how well do selected visual graphics follow the quality features of line graphs (Table 1)?

Method

Initial selection followed criteria established in previous surveys for the identification of prominent behavioral journals (Carr and Britton 2003; Critchfield 2002; Kubina et al. 2008). Journals had to explicitly pertain to behavior analysis and have at least a 10-year publication record. The survey sampled a variety of behavior analytic foci (e.g., education, cognitive behavior modification, experimental analysis). Eleven journals met criteria.

Six journals covered technical applications, practices, and issues related to the field of behavior analysis (Behavior Modification, Behavior Therapy, Child and Family Behavior

587 Educ Psychol Rev (2017) 29:583–598

Therapy, Cognitive and Behavioral Practice, Journal of Applied Behavior Analysis, and Journal of Behavior Therapy and Experimental Psychiatry). Five additional journals discussed behavior analysis in relation to education (Education and Treatment of Children, Journal of Behavioral Education), experimental behavior analysis (Journal of the Experimental Analysis of Behavior, Learning and Behavior), and the analysis of verbal behavior (The Analysis of Verbal Behavior).

After journal identification, one random issue from every 2-year block served as the basis for selecting graphs. The process began at each journal’s inception date and concluded in 2011. The investigators examined all graphs that had a vertical and horizontal axis with data moving left to right. First, the graph must have contained a maximum of one data point per data series on the horizontal axis interval excluding scatterplot graphs (i.e., multiple data points can occur on the same horizontal interval) and bar charts. Second, a unit of time or sessions and a quantitative value must have occurred on the horizontal and vertical axis, respectively. Graphs scaled with nominal or ordinal vertical axes and/or non-time based horizontal axes did not meet inclusion criteria. Third, graphs with dually and/or logarithmically scaled vertical axes, as line graph variants, also failed to satisfy inclusion criteria. Investigators analyzed each included graph individually whether appearing alone or in the context of other graphs (i.e., multiple baselines).

Investigators scored each graph for components of line graph essential structure and the presence or absence of line graph quality features (specific questions appear on Table 1). Scorers initially determined presence or absence of axes and labels. Using rulers or straight edges, scorers then determined the ratio of the length of the vertical to the horizontal axis and the axes scaling and alignment to other graphs within the same figure (i.e., multiple baseline figures) and/or article. Scorers continued to evaluate each graph according to the remaining questions noted on Table 1 and entered all data on an accompanying Excel file. The process repeated for each graph meeting criteria.

Scorer Calibration, Reliability, and Interobserver Agreement

Scorers received instruction on all procedures. Instruction consisted of review and guided practice of scoring and entering data on the Excel spreadsheet for three graphs. At the conclusion of the instructional sessions, experimenters had scorers evaluate a random, but previously scored issue and compared results. Scorers had to meet 100 % agreement prior to independent scoring.

Two measurement assessment techniques evaluated scoring: reliability and interobserver agreement. For reliability (Johnston and Pennypacker 2009), each scorer rescored 20% of issues. A comparison occurred between the second examination and the initial score. An exact agreement approach (Kennedy 2005) determined the percent of agreement between individual cells of data in Excel sheets. The average reliability totaled 95 % with a range of 94–100 %. Interobserver agreement followed the same procedure but took place between different scorers on 20 % (28) of issues. The average interobserver agreement equaled 91 % with a range of 89–100 %.

Results

A total of 11 behavioral journals served as the basis of the graph analysis. A random sampling produced 191 issues, an average of 17 issues per journal with a range of 9–27. A potential 5989 time series graphics occurred in 1622 articles. Removing graphs that contained

588 Educ Psychol Rev (2017) 29:583–598

logarithmically scaled (114) or dually scaled axes (203), the number of graphs came to 5672. Graphs with nominal (60) or ordinal (227) scaled vertical axes (i.e., qualitatively scaled) or included a label for the horizontal axis that did not fall into a unit of time (1072) failed to meet review criteria resulting in a total of 4313 coded graphs or approximately 23 per issue.

Essential Structure

The essential structure of a simple line graph starts with two drawn axes, a vertical and horizontal. Of the 4313 graphs meeting criteria, 98 % (4206) and 97 % (4200) of graphs contained a drawn line on the vertical and horizontal axis, respectively. Two additional essential structure criteria require the labeling of the two axes with a quantitative value on the vertical axis and time along the horizontal axis. Two dot charts (Cleveland 1984b), in Figs. 2 and 3, show the breakdown that authors used for labeling axes. On both figures, dots represent categorical instances and appear from greatest to least.

Figure 2 shows that, of the eight vertical label categories, five contained 98 % of instances. Authors used a percent label 27 % (1159) of the time on the vertical axes, more often than any other classification. Count (20 % or 865), ratio (18 % or 782), no label (17 % or 753), and frequency/rate (16 % or 677) groupings round out the initial 98 %. The remaining 2 % consist of latency (40), duration (36), and interresponse time (1).

On Fig. 3, labels for the horizontal axes revealed less diffusion as compared to the vertical axis breakdown. No label (1486) and sessions/trials (1469) accounted for 69 %. A total of 459 (11 %) graphs incorporated seconds, 268 (6 %) days, and 262 (6 %) minutes along the horizontal axis. The final 9 % include other unit of time (172), multiple units on the same axis (160), and hours (37).

Quality Features

Tables 2 and 3 show a variety of quality features coded for each graph and groupings of graphs. Graphs meeting each instance receive a comparison to the total number of opportu- nities resulting in a percent occurrence. Coded graphics not meeting the quality feature (except

Fig. 2 A dot chart showing the instances of vertical axis labels

589 Educ Psychol Rev (2017) 29:583–598

Fig. 3 A dot chart showing the instances of horizontal axis labels

for data connected across condition change line) result in an error shown as a remaining percentage in the final column. Each instance of the category data connected across condition change line already constitutes an error; thus, the same percentage occurs in both columns.

Tick marks and scaling Considerable differences in quality occurred between tick marks and scaling of the vertical and horizontal axes. Table 2 shows that, across the five tick mark

Table 2 Quality features

Graphic quality feature Instances per opportunity Percent of error (%)

Vertical axis tick marks Data occur on tick marks 3481/4313=81 % 19

Full or partial tick marks on 2460/4313=57 % 43 outside of graph

Tick marks occur at equal 3768/4313=87 % 13 intervals

Numbers occur on tick marks 3493/4313=81 % 19

Scale count is correct 3702/4313=86 % 14 (e.g., 10, 20, 30, etc.)

Horizontal axis tick marks Data occur on tick marks 2866/4313=66 % 34

Full or partial tick marks on 2115/4313=49 % 51 outside of graph

Tick marks occur at equal intervals 2714/4313=63 % 37

Numbers occur on tick marks 2214/4313=51 % 49

Scale count is correct 2304/4313=53 % 47 (e.g., 10, 20, 30, etc.)

Data points clearly visible 3703/4313 =86 % 14

Data connected and data path clearly visible 4193/4313=97 % 3

Figure caption 4253/4313=99 % 1

Condition change labels present 1774/2172=82 % 18

Data connected across condition change line 158/2151=7 % 7

590 Educ Psychol Rev (2017) 29:583–598

and scaling features, half as much average error occurred on the vertical (22 %) rather than horizontal axis (44 %). On both axes, the highest percentages (43 and 51 %) of error related to fully or partially placing tick marks on the outside of the figure. Instead, graphs either had no tick marks or tick marks on the inside of the graph.

Data points and paths, condition labels, and figure captions A decrease in error appeared when coding quality features associated with data points and paths and condition and figure labels (Table 2). Only 1 and 3 % of the sampled graphs failed to contain a figure caption and had connected data paths clearly visible. Although, 18 % of the time graphs containing condition change lines failed to have a label for each condition and 7 % had data paths connected across condition change lines.

Axes and axis comparisons Table 3 highlights quality features associated with axis com- parisons. The first main comparison occurred between the vertical and horizontal axes with 15 % meeting a ratio of 5:8 to 3:4 or 63 to 75 % difference in length. For graphs on the same page, both vertical and horizontal axes aligned, when possible, 79 % of instances. Expanding the analysis to each article, graphs received coding for maintaining the same physical and scaling structure. On both axes, approximately 70 % of instances failed to maintain similar scaling for the same unit across the article and 40 % had variable axis length.

Discussion

Graphs have one fundamental purpose: to affect the interpretative behavior of the graph reader (Johnston and Pennypacker 2009). Information in graphs includes documenting performance, analyzing intervention effects, interpreting experimental and applied outcomes, and predicting the future course of behavior. Graphs generate meaning based on physical distinctions of

Table 3 Additional quality features: comparisons of individual graph axes and of axes on multiple graphs

Graphic quality feature Instances per opportunity

Percent of error (%)

Ratio of vertical to horizontal axis length: 5:8 to 3:4 (63 to 75 % difference)

637/4313=15 % 85

For multiple graphs within the same figure on the same page:

Vertical axes align

Horizontal axes align

1016/1285= 79 %

1053/1331= 79 %

21

21

For multiple graphs within the same article

Vertical axes that share the same label:

Scaled to same unit (min and max)

Drawn to the same physical length

262/838= 31 %

467/838= 56 %

69

44

Horizontal axes that share the same label:

Scaled to same unit (min and max)

Drawn to the same physical length

222/710= 31 %

436/710= 61 %

69

39

591 Educ Psychol Rev (2017) 29:583–598

shape, size, color, positioning, and symbols (Cleveland and McGill 1985; Tufte 1990). While all graphed data convey a message, BA graphical method is successful only if the decoding is effective. No matter how clever and how technologically impressive the encoding, it fails if the decoding process fails^ (Cleveland and McGill 1985, p. 828).

The science of behavior fundamentally relies on simple line graphs for decoding information (Cooper et al. 2007). Not all simple line graphs, however, have equal merit. Line graphs can vary along critical dimensions such as scaling, length of axes, and labeling. Widely divergent construction practices distort the interpretative function of graphs. The use of essential structures and quality features promotes order and guides graphical construction, subsequently enhancing visual clarity and a clear explanation of the data (Cleveland 1984a). At the time of the present review, however, no comprehensive evaluation of simple line graphs in behavior analysis has occurred. The specific research questions asked how well do selected visual graphics follow the essential structure and quality features of line graph construction.

Essential Structure

Quantity and time form a line graph’s essential structure (Few 2009; Kriglstein et al. 2014). Behavior analysis has long valued the graphic display of quantitative rather than qualitative representations of behavior (Baer et al. 1968; Ferster and Skinner 1957; Parsonson and Baer 1978; Poling et al. 1995). Quantification leads to precision provided by numbers. Numerical representations of behavior, or quantitative measurement, serve as the medium through which all analysis occurs. The data show that, out of 4313 reviewed graphs, 3560 had a quantitatively scaled vertical axis. In other words, 83 % of the reviewed graphs prominently displayed behavior as a quantity.

The quantitatively displayed behavior can change across time. Time series line graphs must have a unit of time on the horizontal axis (Robbins 2005). Examples of units of time range from seconds (e.g., Preston 1994) and minutes (e.g., Norborg et al. 1983) to hours (e.g., Ramirez 1997) and days (e.g., Gutentag and Hammer 2000). Twenty-eight percent of line graphs in behavioral journals maintained a time unit label. The remaining 72 % scaled the horizontal axis with a non-time unit (e.g., sessions, trials, no label). By labeling the horizontal axis with sessions or trials, the line graph technically no longer qualifies as a time series graphic and markedly influences visual analysis.

All graph readers use visual analysis to uncover functional relations and experimental effects (Cooper et al. 2007; Kazdin 2011). In point of fact, visual analysis of line graphs serves as the cornerstone of studies using a single case experimental design. BData are graphed for each participant during a study with trend, level, and stability of data assessed within and between conditions (Lane and Gast 2014, p 445).^ Trend refers to the slope or angle of a data series; level applies to the median score of a data set, and stability captures the degree of variability in a set of data (Gast and Spriggs 2010). Table 4 explains and Fig. 4 illustrates how session usage produces three categories of errors affecting visual analysis: labeling error, false equality, and non-representative data.

Labeling error involves assigning a designation other than a unit of time to the horizontal axis. As an example, graph 1 in Fig. 4 shows data plotted along a horizontal axis labeled with sessions and a vertical axis with quantified behaviors per session. Baseline data show a gradually increasing, moderately variable trend with a level of 25 behaviors per session. Data in intervention show the level rising to 40 behaviors per session with a slightly increasing and

592 Educ Psychol Rev (2017) 29:583–598

Table 4 Labeling error and visual analytic distortions

Session practice Problem when horizontal axis has Bsessions^ as the label and data graphed consecutively and contiguously.

Session duration reported as a time unit in text and…

Sessions graphed with respect to time and…

Session duration not reported as a time unit and…

Held consistent for the duration of the study

Not held consistent (i.e., presented as a possible range of time)

Occur consecutively in time (i.e., one session per day)

Occur consecutively in time (i.e., one session per day) but do not occur every day (i.e., sick days and/or weekends)

Do not occur consecutively in time (e.g., multiple se