Chapter 6

Formal Assessment of Reading: Individualized

Assessment

Introduction

• Development of Formal Reading Measures • Types of Scores • Administration and Scoring of Formal Tests • Determining Chronological Age • Individualized vs Group Tests • What do individualized Norm-referenced Measures of

Reading Look Like? • Test Bias • Selecting Formal, Individualized Instruments • Special Considerations for Formal, Individual Assessment

of Adult and English Language Learners

Test Development

• Table of specifications (blueprint) (Figure 5.2) • Development version • Pilot or field testing • Standardization version • Norm sample

• Representativeness of sample

Figure 6.1

Types of Scores

• Numerical Scales: Nominal, Ordinal, Interval, Ratio

• Raw Score: Actual number correct, taking into account the basal

• Derived Scores • Standard Scores: Z Scores, IQ Scores, T

Scores, Scale Scores • Grade Equivalents, Age Equivalents • Percentiles

Assessment Terms

• Grade Equivalent • Age Equivalent • Grade-Based

Norms • Age-Based Norms ØAs a Rule, Use Age-

Based Norms to Determine Standard Scores

• Standard Scores- can Add and Subtract-Used for Comparisons

• Percentiles- Provides A Rank Order 1-99

• Stanines-A Rough Gauge 1-9

Text Box 6.1

Age and Grade Equivalents

Age Equivalent Grade Equivalent 6-1 1.2 6 years, 1 month 1st grade, 2nd month

Percentile versus Percentage Misha Jesse Percentage 50 90 Mastery

Percentile 16 80

Text Box 6.2

Text Box 6.3

Contrasting Standard Scores for Word Recognition and Reading Comprehension

Mean = 100; Standard Deviation = 15

Word Reading Recognition Comprehension

Shelby 85 84 Saimah 95 70

Text Box 6.4

Comparing IQ Scores with Reading Achievement Scores (Mean = 100; Standard Deviation = 15)

Saimah Shelby Standard Score IQ 94 103 Reading Comprehension -70 -85 24 18

Administering and Scoring of Formal Tests

Test Administration • Test Manual: Administration Instructions,

Scoring and Norm Information, Reliability and Validity Data

• Protocol • Calculation of Chronological Age • General Guidelines for Test Administration • Raw Scores • Establishing Basals and Ceilings • Derived Scores: Percentiles, Grade Equivalents,

Standard Scores

Test Administration

• Determine chronological age

• Know start and stop rules

• Establish rapport with examinee

• Administer subtests according to directions

• Do NOT coach, prompt, or give feedback EXCEPT as directed in the manual

• Obtain raw scores- take into account basal and ceiling

• Obtain derived scores (standard scores, percentiles, etc.

Tips for Administering Standardized Tests • Text Box 6.5

Word Recognition

Basal Rule: 3 in a row correct. Ceiling rule: 3 in row incorrect.

1. ___ 2. ___ 3. ___

→ 4. _1_ 5. _1_ 6. _1_ 7. _1_ 8. _1_ 9. _1_ 10. _0_ 11. _0_ 12. _0_

Note: Starting point for this student indicated by the →

Vocabulary

Basal rule: All items in a set correct. Ceiling rule: All items in a set incorrect.

1. ___ 2. ___ 3. ___

4. ___ 5. ___ →6. _1_ 7. _1_ 8. _1_ 9. _1_ 10. _1_

11. _0_ 12. _0_ 13. _0_ 14. _0_ 15. _0_

Note: Starting point for this student indicated by the →

Figure 6.2. Sample Reading Subtests Showing Basal and Ceiling Rules

Raw Score Conversion Table for Reading Comprehension Test

Age: 8 years, 9 months Raw Score

Stand ard Score

Percen tile

Grade Equiva lent

Age Equivale nt

↑

15 95 37 2.1 7-6 16 96 39 2.4 7-9 17 97 42 2.7 7-11 18 98 45 2.9 8-3 19 99 47 3.2 8-6 20 100 50 3.5 8-9

↓

Figure 6.3. Sample Excerpt from a Norm Table for an Individually Administered Reading Comprehension Test for Students Age 8 years, 9 months. Examiners calculate the raw score for the test, then look up derived scores (standard scores, percentiles, grade- and age-equivalents) based on the examinee’s chronological age.

Determining Chronological Age

Calculation of Chronological Age

Subtract Date of Birth from Current Date:

2016 06 22

– 2009 07 14

Months always have 30 days and years (of course) have 12 months). Always double check your work.

Standardized Diagnostic Testing

• When to Use Diagnostic Tests • Instructional Level • Informal Instruments • Probes • Direct Measurement • Domains

Individualized vs. Group Tests

Individual versus Group?

Individual • Special education

eligibility • Determine goals and

objectives for IFSPs, IEPs

• Progress monitoring • May be more reliable

for certain students

Group • Determine mastery of

curriculum standards • Determine if teachers

are effective (value added) • Controversial but

happening! • Accountability and

general school planning

• May be less reliable for certain students

Test Interpretation • Analyze test scores • Are scores above

average, average or below average?

• Do the scores yield any instructional information?

• Compile with other data

• Maintain confidentiality

• Is there a pattern of strengths and weaknesses? • e. g. Areas of learning

disability? basic reading, reading fluency, reading comprehension?

What do Individualized, Norm- Referenced Measures of

Reading Look Like?

Figure 6.4

Figure 6.5

Figure 6.6

Figure 6.7

Figure 6.8

Figure 6.9

Methods of Assessing Comprehension

• KTEA-III • Reading Comprehension subtest

• Picture identification, Answering questions

• WIAT-III • Reading Comprehension subtest

• Picture identification, Answering questions

• WJ-IV • Passage Comprehension subtest

• Picture identification, Fill in the blank • Reading Recall subtest

• Silent reading followed by oral retelling

Comprehension

• Wechsler Individual Achievement Test (WIAT-III) • Reading Comprehension subtest • The examinee is required to read short sentences or

passages and respond to comprehension questions

Reading Comprehension Similar to WIAT-III

Sample Maze Activity Norm-referenced tests are standardized (without/on/for) a clearly defined group, termed the (test/sample/norm) group, and scaled so that each (score/instrument/variance) reflects a rank within the (evaluation/reliability/norm) group. Psychologists have developed norm-(standardized/referenced/rated) tests to assess, for example, (height/attitude/intelligence), reading, mathematics, writing, etc. Although we (aren’t/are/is) fortunate in having a choice of (poorly/haphazardly/well)-standardized and psychometrically sound tests (without/above/with) which to evaluate children, some (numbers/tests/practitioners) do not meet acceptable psychometric standards. (Passage from Sattler, 2001)

Nelson-Denny Reading Test

• Comprehension • 7 reading passages • 38 comprehension questions • Literal and interpretive question types • 5 answer choices per question • 20 minutes

Methods of Assessing Vocabulary •WJ-IV: Reading Vocabulary

•Synonyms, Antonyms, Analogies

•Nelson Denny •Multiple choice

Vocabulary

•Woodcock Johnson – IV Tests of Achievement (WJ-IV) •Reading Vocabulary subtest • The examinee is required to orally state synonyms and antonyms for printed words and orally complete written analogies

Reading Vocabulary Similar to WJ-IV

Analogies

Synonyms

Antonyms

Nelson-Denny Reading Test

• Vocabulary • 80 items • 5 answer choices each • 15 minute time limit • Example

• Which word best completes the opening statement? • A chef works with: A. bricks B. music C. clothes D. food E. statues

Methods of Assessing Fluency •KTEA-III •Decoding Fluency subtest

•Nonsense word list (timed) •Word Recognition Fluency subtest

•Word list (timed) • WIAT-III •Reading Comprehension subtest

•Reading timed passages •WJ-IV •Sentence Reading Fluency subtest

•Reading sentences and marking whether statements are true or false (3 minute time limit)

•Word Reading Fluency •Choosing two words that go together from an array of choices

Fluency

• The Test of Silent Word Reading Fluency-2 (TOSWRF-2)

• Examinees are required to identify increasingly difficult words that have no spaces between them by drawing lines between the boundaries of as many words as possible within the time limit (3 minutes)

• The Test of Silent Contextual Word Reading Fluency (TOSCRF-2) • Similar to TOSWRF-2 but examinee reads connected text

of increasingly difficult graded passages

TOSWRF-2: Example

Nelson-Denny Reading Test • Reading Rate

• Part of Comprehension Test • The examiner calls “Mark” after one minute has

elapsed, and the examinees are asked to record the number to the right of their current line of text and continue reading.

Methods of Assessing Phonics

• KTEA-III • Nonsense Word Decoding subtest

• Nonsense words

• WJ-IV • Word Attack subtest

• Nonsense words

• WIAT-III • Pseudoword Decoding subtest

• Nonsense words

Phonics • Wechsler Individual Achievement Test (WIAT-III)

• Pseudoword Decoding subtest • Requires the examinee to read aloud a list of nonsense

words designed to mimic the phonetic structure of words in the English Language.

Pseudoword Decoding Similar to WIAT-III

• heb • mib • fum • bim • pon • vun • dreeb

Methods of Assessing Phonemic Awareness

• KTEA-III • Phonological Awareness subtest • Rhyming, Sound Matching, Blending, Segmenting,

Deletion

• WJ-IV • Spelling of Sounds subtest • Rhyming, Deletion, Substitution, Reversal

• WIAT-III • Word Reading subtest • Rhyming, Sound Matching, Blending,

Phonemic Awareness

• Woodcock Johnson Tests of Achievement (WJ IV) • Spelling of Sounds subtest • Requires the examinee to use rhyming, deletion,

substitution, and reversal to manipulate sounds in words.

Spelling of Sounds Similar to WJ-IV

• Rhyming: “What rhymes with go?”

• Deletion: “Say cowboy without saying boy”

• Substitution: “Change /s/ in sun to /f/.”

• Reversal: “Listen to the sounds in the word pot – /p/ /o/ /t/. Now you say the sounds backward.”

Other Measures that Assess Phonemic Awareness

• CTOPP-2 • Durrell-2 • ERDA-2

• PAL-2 RW • TOWRE-2 • TERA-3

Methods of Assessing Sight Word Recognition

• WIAT-II • Word Reading subtest

• Letter and word identification

• KTEA-II • Letter and Word Recognition subtest

• Letter and word identification

• WJ-IV • Letter Word Identification subtest

• Letter and word identification

Sight Word Recognition

• Kaufman Tests of Educational Achievement (KTEA III) • Letter and Word Recognition subtest • The examinee is asked to point to or name various

letters, sounds, and words.

Letter Word Recognition Similar to KTEA III

Test Bias

• Scores from a test should not provide a systematic advantage to a particular group or subgroup based on irregularities in test content, administration, or interpretation

• If a test produces such an unfair advantage it is said to be biased against the group that is disadvantaged (Payne, 2003)

Evidence of Test Bias

• a) Does the test assess the same content for all the groups using it?

• (b) Are the constructs assessed by the test measured consistently for all groups using it?

• (c) Does the test predict (later achievement) as well for all the groups using it?

• (d) Does the test provide equally useful instructional information for all groups using it?

Avoiding Bias

• Expert judgment • Statistical analyses: Are the constructs assessed

the same across different groups (e.g., gender, race, SES level?)

• Remember tests that are not inherently biased may be misused (e.g., use of test in English for students with limited English skills)

Selecting Formal, Individualized Instruments

• Consider test characteristics • Consider psychometric properties • Consider needs of children, classrooms, schools,

or systems • Determine whether a group or individualized test

is needed

Special Considerations for Formal, Individual Assessment of Adult and English Language

Learners

Tips for Formal, Individualized Assessment of Adult Learners • Text Box 6.7 • Assess adult learners’ educational histories, background

experiences, and interests as well as specific reading skills. • Multiple assessments will likely be needed to provide complete

information about adult learners’ reading strengths and weaknesses.

• Ensure that the test used with adults is standardized on that population.

• Put adult learners at ease; establish rapport and a purpose before launching into formal testing.

• Using grade equivalent scores, and to a lesser extent age equivalent scores, is problematic due to ease of misinterpretation of these score, and their psychometric limitations.

Tips for Adults cont’d

• Tests should include item content appropriate for adults, rather than young students, even though the difficulty level of the content may be low in order to achieve a reasonable test floor.

• Consider entering your test data and using the instructional skills profile available at the Adult Reading Components Study website: https://lincs.ed.gov/readingprofiles/resources.ht m

Tips for Formal, Individualized Assessment of English Language Learners

• Text Box 6.8 • Users should follow standards set for test administration,

scoring and interpretation in general and for ELL and other diverse learners as developed by experts. Many of these standards are reproduced in Chapter 6; some of the most relevant are reproduced in this Text Box in abbreviated form.

• Be aware that an ELL student’s formal test scores will be compared to students in the U.S., who may not resemble the ELL in background, culture, knowledge, etc.

• Related to Number 2 above, an ELL’s scores on formal tests may be depressed because of cultural, linguistic, and knowledge differences.

Tips for ELLs cont’d • Do not administer a language-loaded test for ELL

examinees unless the language content in the test is the focus of assessment.

• Related to Number 4 above, if a student’s English language proficiency is the target of assessment English language- loaded tests (i.e., English reading tests) are appropriate

• Ensure that the test does not contain item content that may be offensive in the ELL’s home country or culture.

• Ensure that reliability and validity of the test have been established for ELLs.

• Ensure that the items are not biased against ELL learners by evaluating recommendations from expert panels.

Tips for ELLs cont’d

• Ensure that items are not biased against ELL learners by examining evidence from statistical techniques designed to determine bias.

• When possible, use an examiner who can communicate in the ELL’s native language to establish rapport and aid in administering the test.

• Be aware that an ELL’s progress in learning a second language and ability to understand test demands may be limited by negative transfer from the native language.

• Reduce importance of speeded performance on scores; some cultures do not value speed.

• Ensure that the test only measures what you intend to measure (i.e., specific types of reading skills) and not some irrelevant content.

Tips for ELLs cont’d

• If your ELL students score below the mean, ensure that if mean differences occur, the differences are not the result of bias.

• Use ample teaching items to ensure that ELL learners understand the task demands.

• Using grade equivalent scores, and to a lesser extent age equivalent scores, is problematic due to each of misinterpretation of these score, and psychometric limitations.

Assessment at a Glance