Research in Education Corner

Conducting Research, Standards, Raising Achievement, Assessment, Assisting Struggling Readers, Technology Integration

Get Adobe Acrobat Reader, free software for pdf files, which appear on this page.

NCLB and Scientifically Based Research

According to Laitsch (2003),  "To access much of the federal funding allocated through NCLB, states and districts will be required to adopt programs and policies that are supported by scientifically based research, and teachers will need to adapt their practice to reflect the competencies necessary to implement the new programs" (para. 2).  Educators might require professional development in what constitutes scientifically-based research and implementation of new curricula.  CT4ME's Research Corner will give you a good start on this topic, as well as how to conduct your own action research. 

North Central Regional Educational Laboratory (2004) discusses and provides examples of the six components of scientifically based research (SBR).  SRB must:

1. Use empirical methods.
2. Involve rigorous and adequate data analyses.
3. Rely on measurements or observational methods that provide reliable and valid data.
4. Use either an experimental or quasi-experimental design.
5. Allow for replicability.
6. Undergo expert scrutiny. (pp. 3-4)
    

The U.S. Department of Education set up the What Works Clearinghouse (WWC) in 2002 to provide easily searchable databases containing such scientific evidence.  During its first year, the WWC focused on seven topics: interventions for beginning reading; curriculum-based interventions for increasing K-12 math achievement; preventing high school dropout; increasing adult literacy; peer-assisted learning in elementary schools for reading, mathematics, and science gains; interventions to reduce delinquent, disorderly, and violent behavior in middle and high schools; and interventions for elementary school English language learners.  

The WWC defines educational interventions as a product, practice, policy, or program that has been shown to be effective.  In Determining 'What Works,' Therese Mageau (2004), guest editor of T.H.E. Journal and Dr. Grover 'Russ' Whitehurst, director of the Institute of Education Sciences, discussed the question of effectiveness.  The WWC gives preference to studies involving randomized trials because randomized trials are the gold standard for determining effectiveness.   Whitehurst indicated, however,  that "there are two prongs to evidence-based practice" (p. 34).  Something that may work in one location might not work in another.  Therefore, evidence-based practice includes the scientific studies of effectiveness as found on the WWC, and the integration of professional wisdom based on "locally collected performance data that indicates whether changes are occurring in the desired direction when a particular program or practice is implemented" (p. 34).

Thus, in order for educators to use both SBR and their professional wisdom, it is important for educators to read and analyze professional journals.  According to Simpson, LaCava, and Graner (2004), authors of such journal articles should enable educators to clearly identify the following:

• "the research design upon which a method is based,
• the targeted populations,
• the outcomes, and
• the resources needed by classroom personnel to replicate within classrooms the outcomes reported by researchers" (p. 74).

Back to top

Efforts to Determine Product Effectiveness

Interested readers will find WWC reports on the following math programs at http://www.whatworks.ed.gov/ along with other materials currently under review:

• Cognitive Tutor
• Connected Mathematics Project
• The Expert Mathematician
• I CAN Learn Education Systems
• Saxon Math

Mathematica Policy Research, Inc. and SRI International announced a study on February 16, 2004 of interest to mathematics educators (http://edtech.mathematica-mpr.com/Press/index.asp).  The study would use a random-assignment design to evaluate 16 computer-based reading and math products over three years to determine the effectiveness of technology in bolstering student achievement.  The study would assess the effectiveness of learning technology in teaching reading in grade 1, reading comprehension in grade 4, pre-algebra in grade 6, and algebra in grade 9.  Pre-algebra products included in the study would be Successmaker from Pearson Digital Learning, SmartMath from CompuTaught, Inc., Achieve Now from PLATO Learning, Inc. and Larson Pre-Algebra from Meridian Creative Group.  Algebra products include Cognitive Tutor from Carnegie Learning, Inc., Algebra from PLATO Learning, Inc., and Larson Algebra from Meridian Creative Group.  The study actually was conducted on 15 products.

Readers might also be interested in Scientifically Based Research: Preliminary Review of Mathematics Curricular Materials by the Idaho Department of Education (June 2003).  This report shows how the state of Idaho is determining the scientifically based research behind publishers' materials as it undergoes its five-year state mathematics textbook adoption.  Their rubric contains six criteria for judging the quality of a study:

1. Systematic and empirical
2. Rigorous data analysis
3. Reliable and valid data collection
4. Strong research design
5. Detailed results that allow for replication
6. Expert scrutiny

The Idaho committee of teachers rated curricular materials that had been submitted to them to date using a Continuum of Evidence of Effectiveness rubric with three levels: most rigorous, somewhat rigorous, and marginal.  The rubric addresses theory/research foundation, evaluation-based evidence of effectiveness, implementation, and replicability.  Round 1 findings include that Everyday Math (K-6) and Connected Mathematics (6-8) fell into the "Most Rigorous" category.  See the report for other results.

Back to top

Reading and Conducting Research

Those who are not familiar with reading and conducting research studies will appreciate a few tips and documents on this topic.  Commentary on and additional resources for conducting Action Research are also presented.

Reading Research Studies

• Identifying and Implementing Educational Practices Supported by Rigorous Evidence: A User Friendly Guide, an 18-page guide from The U.S. Department of Education, will help educators determine whether an educational intervention is supported by rigorous evidence.  It contains a three-step evaluation process, a checklist to use in the process, definitions of research terms, and what to look for in research studies.
• A Policymaker's Primer on Education Research: How to Understand It, Evaluate It, and Use It (February, 2004) is by Patricia Lauer at Mid-continent Research for Education and Learning.  The Primer is intended to help readers understand what education research says, whether it's trustworthy and what it means for policy.  Readers will also learn some of the technical statistical and scientific concepts touched upon in research reports and gain a deeper understanding of education research methodology.  Practical tools, including a flowchart for analyzing research and an understanding statistics tutorial, are included.
• D. Blewett of the College of DuPage in Illinois provides advice on How to Read a Research Study Article.  A research article generally is structured with an Abstract, Introduction, Methods, Results, Discussion, and References.  Each section has specific content, which he summarizes.  It will take more than one reading to fully understand the bulk of the research, but the general start is to read the abstract, the first paragraph or so of the introduction and the hypothesis, then skip to the discussion to find how the study turned out.   Go back to read the sections on methods focusing on how the hypotheses were tested, results, and re-read the discussion section.  Finally, read the entire report again from first page to last for greater understanding.
• Making Sense of Research for Improving Education, an issue brief (April 2003) from the Charles A. Dana Center at the University of Texas at Austin, will help practitioners clarify their understanding of what scientifically based research is and learn more about research designs used in education research: experimental, quasi-experimental, correlational, and case study.  When using research to make decisions about teaching and learning, practitioners should consider the research's relevance, generalizability to their particular circumstances, statistical soundness, and preponderance of evidence.

The following online texts are for anyone who really wants to understand the statistics involved in research: 

• HyperStat Online Statistics Textbook by D. Lane also contains links to free statistical analysis tools and instructional demos.
• Introductory Statistics: Concepts, Models, and Applications by D. W. Stockburger.  This Web Edition also includes many examples showing how to use SPSS/WIN 7.0 to do statistical procedures.
• Statistics Every Writer Should Know by Robert Niles.  Learn about mean, median, percent, per capita, standard deviation, margin of error, data analysis, and more.  Link to sites for data sources and interactive help to select the right statistical test.
• StatSoft, Inc. (2004). Electronic Statistics Textbook. Tulsa, OK: StatSoft.  According to StatSoft developers, "The Electronic Textbook begins with an overview of relevant elementary concepts and continues with a more in depth exploration of specific areas of statistics, organized by "modules," accessible by buttons, representing classes of analytic techniques. A glossary of statistical terms and a list of references for further study are included."

• Statnotes: Topics in Multivariate Analysis by G. David Garson is an online text.

Conducting Research Studies

In general, research studies involve qualitative, quantitative, or mixed methodologies.

• American Evaluation Association identifies online resources of interest to anyone doing evaluations.  Their collection includes online multi-chapter handbooks and texts on various methodologies, software links for qualitative data analysis and developing and administering surveys, and more.
• Brief Guide to Questionnaire Development by R. Frary is useful for collecting factual information and opinions in survey research.
• Electronic Resources for Research Methods by T.D. Wilson, Ph.D., Professor Emeritus, University of Sheffield, UK., is comprehensive.
• Qualitative Research Web Ring is a database of resources on all aspects of conducting qualitative research.
• Research 101 from the University of Washington addresses how to conduct research, including how to distinguish between scholarly/popular communications and primary/secondary sources, which are often problems for learners.

Back to top

Action Research

According to John Tillotson (2000), a reason why teachers lack interest in educational research is that many of the topics chosen for study seldom have direct implications for what happens in the classroom.  "One doesn't have to look far in most public schools today to find outdated teaching practices and assessment strategies, in spite of ample research findings that suggest more effective alternatives" (p. 31, par. 3).  He suggests that an expansion of action research at the K-12 level is a promising solution to the dilemma of research failing to inform practice. 

Action research, in this author's view, would satisfy the No Child Left Behind mandate for educators to employ researched-based instructional materials and methodologies in their instruction that get results.  It fits the exercise of a district's professional wisdom based on "locally collected performance data that indicates whether changes are occurring in the desired direction when a particular program or practice is implemented" (Mageau, 2004, p. 34). 

Although action research is “usually qualitative and participatory” (Dick, 1997, par. 5), Bob Dick (1998a) acknowledges that “action research is not synonymous with ‘qualitative’ either” (par. 7).  Essentially, the action research cycle involves problem formulation, data collection, data analysis, reporting results, and action planning (Tillotson, 2000).  The research study has many cycles, the theories of which are intended to guide actions, such as: "In situation S, to produce outcomes O1, O2, ..., try actions A1, A2, ..."  Further, "[i]n specifying the important features of the situation it also allows its generalizability to be tested in other similar settings" (Dick, 1998b, par. 7).  The potential to generalize results is key. 

The following resources provide more information on the process and examples from completed studies:

• Bob Dick has an extensive Action Research Resources Web site at Southern Cross University in Australia. 
• The Center for Technology in Education at Johns Hopkins University provides an overview of action research, considerations for starting action research, and explanations for a six step process.
• Madison (WI) Metropolitan School District provides a matrix for conducting action research.  Of particular value is the database of abstracts and selected papers from completed action research studies from 1990 to the present.
• Teacher Research from the Graduate School of Education at George Mason University is devoted to resources and how to's for conducting action research in classrooms.

Back to top

Finding Education Research

Searching the Web with key phrases will yield some education research.  In addition to resources below, journals are a good source for education research.  See the list of Journals at CT4ME.

Education Policy Analysis Archives: http://epaa.asu.edu/epaa/ a peer-reviewed online journal of education research.

Education Resources Information Center (ERIC): http://www.eric.ed.gov/ sponsored by the U.S. Department of Education, Institute of Education Sciences.

U.S. Department of Education: http://www.ed.gov/index.jhtml Use the search phrase "education research."

What Works Clearinghouse: http://ies.ed.gov/ncee/wwc/

Standards, Raising Achievement, Assessment, How People Learn

American Educational Research Association. (2000, July). Position statement concerning high-stakes testing in preK-12 education. http://www.aera.net/policyandprograms/?id=378  

The American Educational Research Association (AERA) is the nation's largest professional organization devoted to the scientific study of education.  AERA's position statement on high-stakes testing is based on the 1999 Standards for Educational and Psychological Testing. The Standards represent a professional consensus concerning sound and appropriate test use in education and psychology. The statement includes that high-stakes testing should meet all of the following criteria:

• Protection against high-stakes decisions based on a single test; availability of alternative assessments where appropriate
• Availability to adequate resources and opportunity to learn prior to stakeholders being held accountable for failure to meet standards
• Validation for each separate intended use
• Full disclosure of likely negative consequences of high-stakes testing programs
• Alignment between the test and the curriculum; high stakes tests should not be limited to testing curriculum that is easiest to measure
• Validity of passing scores and achievement levels
• Opportunities for meaningful remediation for examinees who fail high-stakes tests
• Appropriate attention to language differences among examinees
• Appropriate attention to students with disabilities
• Careful adherence to explicit rules for determining which students are to be tested
• Sufficient reliability for each intended use
• Ongoing evaluation of intended and unintended effects of high-stakes testing

American Federation of Teachers. (2001). Making standards matter. http://www.aft.org/pubs-reports/downloads/teachers/msm2001.pdf

This report in four sections examines state efforts at setting standards, curriculum development, assessment, incentives, and intervention for students at risk of not meeting standards.  It does not address professional development.  The AFT has made several recommendations.  For example, states should (p. 7):

• Provide examples of standards and student work at various grades and performance levels so that teachers, students, parents, and the public have a shared understanding of what is expected.
• Specify the learning continuum in the core subjects to show the progression and development of critical knowledge and skills from grade to grade.
• Identify instructional resources--reading materials, textbooks, software, and so forth--that are aligned to the standards.
• Provide performance indicators to clarify the quality of student work required for mastery of the content standards
• Develop lesson plan data banks that include exemplary lessons and student work related to instruction in the standards.

Australian Council for Educational Research has posted a number of relevant publications on brain-based research and learning: http://www.acer.edu.au/brain_research/publications.html

Barley, Z., Lauer, P. A., Arens, S. A., Apthorp, H. S., Englert, K. S., Snow, D., & Akiba, M. (2002). Helping at-risk students meet standards: A synthesis of evidence-based classroom practices. Aurora, CO: Mid-continent Research for Education and Learning. http://www.mcrel.org/topics/products/116/ 

This research synthesis provides evidence of five strategies to help low-achieving students meet standards: 

• cognitively oriented instruction
• heterogeneous grouping structures
• tutoring
• peer tutoring
• computer-assisted instruction.

Educational Leadership magazine, September 2001, is devoted to making standards work.  ASCD has made three of those articles available to help you understand more about standards:  http://www.ascd.org/portal/site/ascd/menuitem.519a0e62f418ba0cbfb3ffdb62108a0c/   

Gandal, M. & Vranek, J. (2001, September). Standards: Here today, here tomorrow. Educational Leadership, 59 (1) 6-13.

Standards must be teachable to have an impact on what goes on in the classroom.  Teachable standards contain clarity and specificity.  In other words, they contain enough detail and precision so that teachers, students, and parents know what is to be learned.  Making decisions as to what all students should learn is difficult.  Including everything that a student could learn helps no one.  This "laundry list" tends to undermine the power of standards as common expectations.  Teachers feel overwhelmed by the enormity of what needs to be taught, and hence, in-depth coverage of concepts suffers. 

According to Gandal and Vranek, those who design tests that measure state standards might consider three key principles: 

• If it's not in the standards, it shouldn't be on the test.
• When the standards are rich and rigorous, the tests must be as well.
• Tests should become more challenging in each succeeding grade.

Professional development linked to standards is key, but lacking for the majority of teachers in the United States. According to Education Week's (2001) Quality Counts survey, "fewer than half of teachers in the United States say that they have plenty of access to curriculum guides, textbooks, or other teaching materials or to specific training connected to state standards."

If standards truly define essential skills and knowledge that students should acquire, then students must be given a fighting chance to reach them, be it more time or extra targeted help on weaknesses through intervention and support programs.  Schools require a focused curriculum aligned to standards. Students should have adequate preparation time for tests and multiple opportunities to retake tests, if they do not succeed on a first try.

Gayler, K., Chudowsky, N., Kober, N., Hamilton, M., & Yeager, M. (2004, August). State high school exit exams: A maturing reform. Washington, D.C.: Center on Education Policy. Available under section: High School Exit Exams at http://www.cep-dc.org/ 

Gayler, Chudowsky, Kober, Hamilton, and Yeager present findings and recommendations about exit exams in 2004, which is the third annual report on this topic.  In six chapters, the authors address how exit exams are affecting curriculum, instruction, and students; the characteristics of exit exams and how well they align with state standards; the kinds of exit exam supports and options that states are providing for students; the kinds of changes that states are making in exit exam systems; and how well exit exams are connected to other education policies such as No Child Left Behind.  State profiles are included.

Gayler, K., Chudowsky, N., Kober, N., & Hamilton, M. (2003, August). State high school exit exams: Put to the test. Washington, D.C.: Center on Education Policy. Available under section: High School Exit Exams at http://www.cep-dc.org/ 

Gayler, Chudowsky, Kober, and Hamilton present findings and recommendations about exit exams based on a survey study of 24 states with current or planned exit exams.  This five chapter report, which is supported by review of literature on relevant studies of the prior year, addresses how exit exams are affecting curriculum, instruction, and students; the main features of state exit exams as they existed in 2003; the cost of implementing state exit exams; and the challenges that states face as they implement exit exams.  This report also includes state high school exit exam profiles (data and descriptions) for all 24 states that have or plan to have mandatory exit exams by 2008. Test types administered are either standards-based, minimum competency, or end-of-course.

Among findings:

• NCLB legislation is starting to impact the performance goals, content, and timetables of state exit exam systems.  
• Exit exams appear to encourage school districts to cover more of the content in state standards, better align curriculum with state standards, and add remedial and special courses to address needs of students at risk of failing.  
• Costs of implementing such systems are substantial, with many states expecting school districts to foot most of the bill.  Hidden costs include that for professional development of teachers and prevention services for students at risk of failing.  Such services include, for example, revamping instruction to align it with state standards, instituting early reading and math programs to prevent failure, and implementing special instructional programs to help English language learners and students with disabilities to learn the knowledge and skills being tested. 

Ginsburg-Block, M. (1999).  Standards-Based Interventions in Elementary Mathematics.  Center for Applied Research and Educational Improvement: Standards Based Education in Minnesota, 7(2).   http://education.umn.edu/CAREI/Reports/Rpractice/Fall99/elem-math.html

Ginsburg-Block discusses innovative programs in mathematics education and their implications for research and practice.  An example of a classroom sequence that incorporates problem solving and peer collaboration strategies is provided.

Kauffman, D., Johnson, S. M., Kardos, S. M., Liu, E., & Peske, H. G. (2002, March). Lost at sea: New teachers' experiences with curriculum and assessment. Teachers College, Columbia University: Teachers College Record, 104(2), 273-300. http://www.tcrecord.org/Content.asp?ContentID=10822

Research that included semi-structured interviews and brief surveys was conducted in 1999-2000 with a diverse group of 50 first and second year Massachusetts teachers working in a wide range of  public schools to uncover their experiences with curriculum and assessment.  Authors indicate the results of the study might not be generalizeable to the nation as a whole, but do have an implication for practice.  

In terms of the interaction of state standards and curricula, research revealed:

• New teachers were often expected to use the state curriculum framework as the actual curriculum.
• If curricula and teaching guides were at the school, they often did not fully align with the curriculum framework.
• The state frameworks covered too much content.  
• Teachers experienced anxiety with too much content to cover and insufficient guidance on what topics to cover in depth and what topics to just introduce.  Without support, teachers tended to focus on content addressed on the standardized test by "piecing together test-related materials and by using specific MCAS questions as the basis for instruction" (p. 290).

The authors recommended actions in three arenas: state policy, curriculum research and development, and collaboration around curriculum at the school site. Beyond developing standards and assessments, states must support their implementation.  Research is necessary to evaluate the effectiveness of existing materials, the conditions under which those materials will be effective, and to develop additional materials for use in various academic subjects.  School-based collaboration around curriculum development would go a long way towards orienting new teachers to the curriculum and helping them learn what to teach and how to teach it.

Marzano, R. (2003). What works in schools: Translating research into action. Alexandria, VA: Association for Supervision and Curriculum Development. ISBN: 0-87120-717-6. 

Marzano synthesized 35 years of research on effective schooling and identified 11 school, teacher, and student level factors that have the greatest affect on student achievement:

School Level

• Guaranteed and viable curriculum
• Challenging goals and effective feedback
• Parent and community involvement
• Safe and orderly environment
• Collegiality and professionalism

Teacher Level

• Instructional strategies
• Classroom management
• Classroom curriculum design

Student Level

• Home environment
• Learned intelligence and background knowledge
• Motivation

Marzano and his associates have created a 66 item survey based on those factors, which is located online at http://www.whatworksinschools.org/ .  For a fee, your school or district can obtain an analysis of staff perceptions on those factors, which will identify areas on which to focus for school improvement.  Marzano indicated that the intent of the survey is to learn how to improve your students' achievement on assessments and standardized tests.  Customized reports will also contain resources matched to your organization's needs.  Whether or not you take advantage of the online survey, just reading the 66 items will give teachers suggestions for improvement of their practice.

McNeil, N. M., Grandau, L., Knuth, E. J., Alibali, M. W., Stephens, A. C., Hattikudur, S., & Krill, D. E. (2006). Middle-school students' understanding of the equal sign: The books they read can't help. Cognition and Instruction, 24(3), 367–385. http://labweb.education.wisc.edu/knuth/taar/papers_rep_pub/CI_galley.pdf

"This study examined how 4 middle school textbook series (2 skills-based, 2 standards-based) present equal signs. Equal signs were often presented in standard operations-equals-answer contexts (e.g., 3 + 4 = 7) and were rarely presented in nonstandard operations on both sides contexts (e.g., 3 + 4 = 5 + 2). They were, however, presented in other nonstandard contexts (e.g., 7 = 7). Two follow-up experiments showed that students’ interpretations of the equal sign depend on the context. The other nonstandard contexts were better than the operations-equals-answer context at eliciting a relational understanding of the equal sign, but the operations on both sides context was best. Results suggest that textbooks rarely present equal signs in contexts most likely to elicit a relational interpretation—an interpretation critical to success in algebra" (p. 367).

• Skills-based: Saxon Math (Hake & Saxon, 2004) and Prentice Hall Mathematics (Charles, Branch-Boyd, Illingworth, Mills, & Reeves, 2004)

• Standards-based: Connected Mathematics (Lappan, Fey, Fitzgerald, Friel, & Phillips, 1998) and Mathematics in Context (Romberg et al., 1998).

According to McNeil et al. (2006), "all four textbook series focus on both skills and concepts to some degree; however, [they classified] for the sake of comparison and conciseness" (p. 372).

The caveat to this study is that results might only apply to the four textbook series for grades 6-8 examined.  Educators should examine their own texts for how the equal sign is used. 

National Council of Teachers of Mathematics. (2006, January). Position statement on high stakes testing. http://www.nctm.org/about/content.aspx?id=6356 

NCTM has a section on its web site for position statements.  With regard to high stakes testing, NCTM states:

"The National Council of Teachers of Mathematics recognizes the importance of measuring the learning of students and the effectiveness of instruction. Large-scale tests can and should be among several measures that are used to make significant decisions about students and instruction. However, such critical decisions about students and instruction must involve more than the results of any single test. We strongly support a balance of day-to-day classroom assessments, which help teachers improve instruction, and external tests that track progress and provide for national comparisons."

Rudner, L., & Schafer, W. (Eds.). (2002). What teachers need to know about assessment. Washington, D.C.: National Education Association. http://edres.org/nea/teachers.pdf   

This 111-page book in pdf format is available free online. You will learn fundamental concepts related to all assessments; essential classroom assessment concepts; and useful concepts and issues pertaining to district, state, and national assessments.  Teachers will learn how to construct multiple choice and performance assessments, how to construct and evaluate scoring rubrics, and read about essential skills for students.  This latter includes improving the quality of student notes, how to study for tests, and how to avoid the traps associated with standardized tests. 

Scherer, M. (2001, September). How and why standards can improve student achievement. Educational Leadership, 59(1), 14-18.

In an era of accountability that has been created by technology and the information explosion, we must be specific about what students must know and be able to do. Marge Scherer, Editor in Chief of Educational Leadership, posed several key questions on how and why standards can improve student achievement in a conversation with Robert J. Marzano, Senior Fellow at the Midcontinent Research for Education and Learning Institute (McREL, http://www.mcrel.org ). Marzano's comments fell into three categories: a manageable number of standards, a change in our record-keeping system that does not increase teacher clerical work, and the need for a repertoire of instructional strategies. Highlights follow.

Step one toward implementing standards is to cut the amount of content addressed within standards by about two-thirds. The sheer number of those standards is the biggest impediment to their implementation. Someone at the district or school level has to cut down the content to essentials related to standards. This would give teachers ample time to cover the essential knowledge in the time allotted and provide them with room to supplement that content. Schools and teachers are looking to state departments of education for guidance.

In addition to trimming standards, a monitoring system is needed that allows tracking of student progress on specific standards. Marzano suggests a change from our grading practices to standards-based grading. This would require a change in record keeping and the use of rubric scores or percentage scores on specific standards that were covered in a course. Administrators have to set up a record-keeping and monitoring system that is easy for teachers to use, if they expect teachers to implement standards. Over time, you would plot student progress on specific standards. Those patterns are more reliable and valid than a single score from a year-end test. Getting feedback on student progress as often as possible, at least once a year, is absolutely essential to the teaching and learning process.

Research at McREL identified classroom practices that generally lead to achievement gains. These include:

• identifying similarities and differences
• summarizing and note-taking
• receiving reinforcement for effort and recognition for achievement
• doing homework and practicing
• using nonlinguistic representations
• learning cooperatively
• setting objectives and receiving feedback
• generating and testing hypotheses
• using cues, questions, and advanced organizers.

Schoenfeld, A.H. (2002). Making mathematics work for all children: Issues of standards, testing, and equity. Educational Researcher, 31(1), 13-25. Available: http://www.aera.net/publications/?id=431  

Schoenfeld discussed four conditions necessary for providing high quality mathematics instruction for all students: high quality curriculum; a stable, knowledgeable, and professional teaching community; high quality assessment that is aligned with curricular goals; and stability and mechanisms for the evolution of curricula, assessment and professional development.  

It has taken time for curricula to be developed and implemented that is aligned to 1989 NCTM Principles and Standards, and more time for hard data based on large-scale implementations to be collected on their effectiveness.  In terms of the current state, Schoenfeld stated that the body of data indicates the following (p. 16):

1. "On tests of basic skills, there are no significant performance differences between students who learn from traditional or reform curricula.
2. On tests of conceptual understanding and problem solving, students who learn from reform curricula consistently outperform students who learn from traditional curricula by a wide margin.
3. There is some encouraging evidence that reform curricula can narrow the performance gap between Whites and under-represented minorities."

Schoenfeld presented data from Pittsburgh Public Schools to illustrate effectiveness of reform curricula, and noted reform experiences in Michigan and Massachusetts.

He stated that "teaching is a profession more in name than in reality", which is "a national outrage and a national pathology" (p. 16).  The reason for this stems from the need for more training, the typical expectation being that "one year of teacher training will prepare candidate teachers to take on full responsibilities of the classroom" and that once in the field "the vast majority of teachers have minimal opportunities for professional growth" (p. 16).

There are at least two large-scale assessments aligned with NCTM Principles and Standards.  These are New Standards Mathematics Reference Examination and a standards-based assessment developed by the Mathematics Assessment Resource Service (MARS).  See:

• Briars, D., & Resnick, L. (2000, August). Standards, assessments -- and what else? The essential elements of standards-based school improvement. Los Angeles, CA: Center for the Study of Evaluation: CSE Technical Report 528. Available:  http://www.cse.ucla.edu/products/summary.asp?report=528   [This report discusses the Pittsburgh Public Schools experience and the New Standards Reference Examination for Mathematics and has examples of content.]
• Ridgway, J., Crust, R., Burkhardt, H., Wilcox, S., Fisher, L., & Foster, D. (2000). MARS report on the 2000 tests. San Jose, CA: Mathematics Assessment Collaborative.

Stepanek, J. (2000). Mathematics and science classrooms: Building a community of learners. It's just good teaching. Northwest Regional Education Laboratory Math and Science Center. Available: http://www.nwrel.org/msec/just_good/10/titlepg.html  

Stepanek has several chapters, but among them are three models for collaborative learning (democratic, caring, and ecological) and group process skills and strategies to eliminate stratification in groups.

Trends in International Mathematics and Science Study (TIMSS, formerly the Third International Mathematics and Science Study). TIMSS information is located at the National Center for Educational Statistics.  For math, there is a sampling of questions (Dare to Compare) that appeared on prior TIMSS and National Assessment of Education Progress Exams for 4th-grade, 8th-grade, and 12th-grade: http://nces.ed.gov/timss/. 

Wenglinsky, H. (2002, February 13). How schools matter: The link between teacher classroom practices and student academic performance. Education Policy Analysis Archives, 10(12). Retrieved May 29, 2003, from http://epaa.asu.edu/epaa/v10n12/ 

Wenglinsky found that teacher classroom practices have a significant effect on student achievement. Additionally, high-quality professional development focusing on higher-order thinking skills and diversity issues does appear to strongly influence classroom practice. Teacher quality and classroom practice can have an effect on student achievement equal to or exceeding that of socioeconomic status (SES) of students.  

In addition, he notes aspects of teacher quality that are related to student achievement when class size and SES are taken into account. In particular, the following five variables are positively associated with achievement:

• Teacher major
• Professional development in higher-order thinking skills
• Professional development in diversity
• Use of hand-on learning in classrooms
• Focus on higher-order thinking skills

Wagner, T. (2003, November 12). Beyond testing: The 7 disciplines for strengthening instruction. Education Week, 23(11), 28, 30. Available: http://www.gse.harvard.edu/~clg/pdfs/EdWeek7Disciplines.pdf

Tony Wagner, co-director of the Change Leadership Group (CLG) at Harvard University's graduate school of education, reports on strategies used for improving teaching in districts that have dramatically raised the level of student achievement for the lowest quartile of students, including those from the most at-risk populations. He discusses seven practices, which the CLG identified, that appear  to be central to any successful instructional-improvement effort.  All of these might not be implemented at once, some must come before others, but none can be skipped.  Quoting his words:

• The district creates an understanding and a sense of urgency among teachers and in the community for the necessity of improving all students' learning, and it regularly reports on progress. Data are disaggregated and are transparent to everyone. Qualitative data (for example, from focus groups and interviews), as well as quantitative data, are used to understand students' and recent graduates' experience of school.

• There is a widely shared vision of what good teaching is, which is focused on rigorous expectations, the quality of student engagement, and effective strategies for personalizing learning for all students.

• All adult meetings are about instruction and are models of good teaching.

• There are well-defined standards and performance assessments for student work at all grade levels. Both teachers and students understand what quality work looks like, and there is consistency in standards of assessment.

• Supervision is frequent, rigorous, and entirely focused on the improvement of instruction. It is done by people who know what good instruction looks like.

• Professional development is primarily on-site, intensive, collaborative, and job-embedded, and is designed and led by educators who model the best teaching and learning practices.

• Data are used diagnostically at frequent intervals by teams of teachers, schools, and districts to assess each student's learning and to identify the most effective teaching practices. There is time built into schedules for this shared work.

Back to top

Technology Integration

Technology enhanced learning activities should be aligned with local, state, and national standards, should include well-defined assessments, should be accessible to all learners (including those with disabilities), and should contain multiple learning strategies.  Strategies for technology integration that might be used include active learning strategies, constructivist learning strategies, cooperative learning strategies, authentic learning strategies, and intentional/reflective learning strategies. Read more about Good Models of Teaching with Technology at the Knowledge Loom, a resource of the Education Alliance at Brown University.

Battista, M. T. (1998). Computer Environments that Engender Students’ Construction of Mathematical Ideas and Reasoning: A Constructivist Perspective. Paper presented at the ENC Technology and NCTM Standards 2000 Conference. Arlington VA, June 5-6, 1998. 

Carrol, T.G. (2000). If we didn't have the schools we have today, would we create the schools we have today? Contemporary Issues in Technology and Teacher Education, 1(1), 117-140.  

Carroll includes discussion of the classroom of tomorrow and six reasons on why the web has won.

Johnson, J., & Toms Barker, L. (eds.) (2002).  Assessing the impact of technology in teaching and learning: A sourcebook for evaluators.  Ann Arbor, MI: Institute for Social Research at the University of Michigan. http://rcgd.isr.umich.edu/tlt/TechSbk.pdf  

This 184 page sourcebook, available for free in PDF format online, provides an overview of measurement issues in seven areas as well as examples of measures used in current projects.  It would be of value to evaluators who are assessing the role of technology in American education.  The areas include learner outcomes in the cognitive and affective domains and in adult education, teacher outcomes related to changed pedagogy and improving technology skills, technology integration, and disseminating the lessons of technology projects.  The first chapter on learner outcomes in the cognitive domain, for example, includes the merits and difficulties of using standardized tests, tailored tests, and authentic assessments in the evaluation of educational projects.

Kerrey, B., & Isakson, J. (2000, December 19).  The power of the Internet for learning: Moving from promise to practice. Washington, DC: Web-Based Technology Commission.  http://interact.hpcnet.org/webcommission/index.htm    

This final report of the Web-based Technology Commission contains over 100 pages and was presented to the President and the Congress of the United States in December, 2000. Among the contents are discussions of access to broadband technologies; professional development and how technology can enhance teaching; correcting the paucity of research and development; online content; removing regulatory restrictions to e-learning; privacy, protection, and safety; funding for e-learning, and the call for national action. The report is available online in several formats for download.

Kleiman, G.M. (2000, April-June). Myths and Realities about Technology in K-12 Schools. Leadership & the New Technologies, 14. http://www.edtechleaders.org/documents/myths.pdf  

Glenn Kleiman of The Center for Online Professional Education discusses the realities surrounding five myths that relate to computer availability in schools, goals and best practices for computer use in classrooms, teacher implementation stages for effective use, district technology plans, and equity and the digital divide.  The message is that short term solutions will not work.  The key is not how many computers are available, but, in his words, "how we define educational visions, prepare and support teachers, design curriculum, address issues of equity, and respond to the rapidly changing world" [Online].

NCREL (North Central Regional Education Laboratory) has compiled an annotated Bibliography of Research and Resources on Technology and Engaged Learning.  There are over 170 articles listed, many of which have web access.

Shields, J., & Poftak, A. (2002). A report card on handheld computing.  Technology & Learning, 22(7), 24-36.

Jean Shields and Amy Poftak discuss the pros and cons of handheld computers in K-12.  They present a short history of these small devices for one-on-one computing in schools, their potential for learning, and explore applications for integrating handhelds into instruction.  They reference actual classrooms where handhelds are used and include a list of links to resources to learn more.

Wells, J., & Lewis, L. (2006, November).  Internet access in U.S. public schools and classrooms: 1994-2005 (NCES 2007020). U.S. Department of Education. Washington, DC: National Center for Education Statistics.  http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2007020

John Wells and Laurie Lewis presents survey findings (no survey in 2004) on school   connnectivity, student access to computers and the Internet, technologies and procedures to prevent student access to inappropriate material on the Internet, teacher professional development on how to integrate the use of the Internet into the curriculum, and use of the Internet to provide opportunities and information for teaching and learning.

Willard, N. (2002). Keeping kids safe online. Education World. http://www.educationworld.com/a_tech/tech119.shtml 

The Children's Internet Protection Act requires districts to monitor student use of the Internet and to implement technology-based measures to protect against student access to online content that may be harmful to minors. Willard notes four core components for a comprehensive plan to address online safety and Internet use by districts:

• A focus on the educational purpose with Internet use limited to activities that support education, enrichment, and career development;

• Education about safe and responsible use, which is included in the National Education Technology Standards Project;

• Supervision and monitoring that are age appropriate and based on circumstances of use;

• Discipline that not only is appropriate to circumstances of misuse, but also educates students on standards for use.

July 24, 2007 Note: Willard also notes that districts that want to have blocking in place should consider the use of the Internet Content Rating Association system.  ICRA is now part of the Family Online Safety Institute.

Back to top

Back to top

References:

Dick, B. (1998a). Rigour (1).  Occasional pieces in action research methodology, # 13.  Available: http://www.scu.edu.au/schools/gcm/ar/arm/op000.html

Dick, B. (1998b) Grounded theory (2).  Occasional pieces in action research methodology, # 17.  Available: http://www.scu.edu.au/schools/gcm/ar/arm/op017.html

Dick, B. (1997).  What is "action research"? Occasional pieces in action research methodology, # 2. Available:  http://www.scu.edu.au/schools/gcm/ar/arm/op002.html

Laitsch, D. (2003, August). Into the mix: Policy, practice, and research. ASCD InfoBrief, Issue 34. Available in Archived Issues: http://www.ascd.org/portal/site/ascd/menuitem.c30040c1b9063eeeb85516f762108a0c/

Mageau, T. (2004, January). Determining 'What Works'. T.H.E. Journal, 31(6), 32-27.

Mathematica Policy Research, Inc. (2007, April 5). New report released on congressionally mandated evaluation of 15 educational technology products. [Press release].  Available: http://edtech.mathematica-mpr.com/Press/index.asp

North Central Regional Educational Laboratory (2004). NCREL quick key 7: Understanding the No Child Left Behind Act of 2001: Scientifically based research. Naperville, IL: Learning Point Associates. Available: http://www2.learningpt.org/catalog/category.asp?SessionID=825996520&ID=13

Reeves, D. (2006). The learning leader: How to focus school improvement for better results. Alexandria, VA: Association for Supervision and Curriculum Development.

Simpson, R. L., LaCava, P. G., & Graner, P. S. (2004). The No Child Left Behind Act: Challenges and implications for educators. Intervention in School and Clinic, 40(2), 67-75.

Tillotson, J. W. (2000). Studying the game: Action research in science education. The Clearing House, 74(1), 31-34.

Viadero, D. (2007, July 12). U.S. gives What Works Clearinghouse to new contractor. Education Week, 26(43). Available: http://www.edweek.org/ew/articles/2007/07/18/43whatworks.h26.html?tmp=541546481 

Back to top 

Last revised 04/30/08

Deubel, P. (fill in year from last revised). Research corner [Online]. Retrieved [fill in date] from Computing Technology for Math Excellence at http://www.ct4me.net/Research_Corner.htm