nimble ies sbir< home about legal

IES SBIR Funded Research

Expanding, Testing, and Validating the Universal Assessment

Nimble Assessment Systems, Inc. has been awarded a $850,000 contract by the Institute of Education Sciences to expand the access and accommodation features built into NimbleTools. This project allows Nimble to enhance its signing tools, expand interoperability with electronic Braille displays, and embed writing supports, such as word prediction, into the test delivery interface. This project also provides funding to implement NimbleTools for one or more operational state tests. Funding is provided through the Small Business Innovation Research Program (SBIR). June 2008 – December 2010

Abstract:

Many students who are deaf or hard-of-hearing are eligible for a signing accommodation for state and other standardized tests. The signing accommodation, however, presents several challenges for testing programs that attempt to administer tests under standardized conditions. One potential solution for many of these challenges is the use of computer-based test delivery that integrates recordings of signed presentation of test content into the test. In addition to standardizing conditions, computer-based delivery also holds potential to decrease the cost of developing recordings of signed presentation by using avatars rather than humans. However, because avatars are relatively new and are not as expressive or lifelike as humans, they may not be as affective as humans in presenting content in a clear and interpretable manner. The study presented here employed a randomized experiment to compare the effect that a computer-based provision of the signed accommodation using a recorded human versus a signing avatar had on students’ attitudes about performing a mathematics test and on their actual test performance. This study found that students generally reported that it was easy to perform a mathematics test on computer, and that both the recorded human and the signing avatar tools were easy to use and to understand. Students also reported a strong preference for performing future tests on computer, and generally preferred using the recorded human and the avatar for future tests rather than a DVD. While students also reported that they preferred the recorded human rather than the signing avatar, this preference did not affect test performance. The use of the recorded human and the avatar did not have affects on either the amount of time required to complete the test items or on students’ performance on the test items. Implications for future research are discussed in light of these findings and the shortcomings of this study.

More Information:

Providing adequate test accommodations for students with disabilities and special needs is a critical problem for large-scale testing programs and classroom teachers. NimbleTools was developed to increase the efficiency, affordability, equity, and quality of providing students with appropriate and adequate access to accommodations during testing. As indicated by several national organizations that focus on special needs and assistive technologies, and several state assessment directors, NimbleTools provides a viable, commercial solution to providing valid test accommodations.

NimbleTools is an innovative computer-based test delivery system that applies principles of Universal Design for Assessment (UDA) to flexibly meet the accessibility and accommodation needs of individual students with disabilities in grades 3–12. Flexibly meeting individual needs is accomplished by allowing teachers to activate one or more of the accessibility and accommodation tool built into NimbleTools, and then allowing individual students to use these tools as needed while performing achievement tests. This SBIR Fast Track project supports the development of code and software tools that:

  1. allow Intellikeys to interact with the system
  2. provide access to text that appears in test items for students who are deaf and communicate through American Sign Language.

In addition, the security protocols are being tested and stress testing is being conducted to assure the system can handle user loads typical of state test administrations. Finally, a set of test validity studies are being conducted to build a body of evidence used to evaluate the effect that the use of NimbleTools has on the validity of inferences about student achievement based on test scores.

Through this SBIR project, NimbleTools will include 18 accessibility and accommodation tools, will provide adequate test security, and will be able to handle user loads typical of a state testing program. Evidence will also be collected to examine the effect that the use of NimbleTools has on the validity of inferences about student achievement based on test scores. In addition, NimbleTools will be piloted for at least one state achievement test, and thus it will be demonstrated that NimbleTools is ready for use for state assessment programs.

Accomplishments to date:

Learn more about the accessibility and accommodation tools provided by NimbleTools.

Developing the Digital Work Pad

Nimble Assessment Systems, Inc. was awarded a Phase I Small Business Innovation Research Grant. This contract with the Institute of Education Sciences provided seed money to develop a tool that allows students to produce handwritten responses to open-ended items presented by a computer-based test. The work pad allows students to show their work for mathematics problems, produce graphs and drawings, label figures, and write short answer responses. An alpha version of the work pad has been piloted

©2009 Nimble Assessment Sytems, Inc.

Please send your comments to access@nimbletools.com