Chemical Education Journal (CEJ), Vol. 13, No. 2/Registration No. 13-21 /Received March 12, 2010.
URL = http://chem.sci.utsunomiya-u.ac.jp/cejrnlE.html


The Influence of the Type of Question on Students' Responses during Assessment

Franz L. Wimmer*1 and Harkirat S. Dhindsa2

1. Department of Chemistry,
2. Department of Science and Mathematics Education,
Universiti Brunei Darussalam, Gadong BE1410, Brunei

E-mail: franz.wimmerubd.edu.bn

Abstract: Chemical structures and bonding are repeatedly used to explain and rationalise the physical and chemical properties of the elements and their compounds. These concepts are first introduced in secondary school along with simple models and are then further elaborated and extended upon at both the secondary and tertiary levels. However, it was noticed that many of the senior students felt uncomfortable with questions that require the taught information in a sequential different order to that in which it had been taught. The aim of this study was to evaluate how the senior chemistry students respond to questions that require simply a recall of learned information (Theoretical questions) and questions that require the application of the learned information to answer the questions (Conceptual questions). The subjects were second year inorganic chemistry students at Universiti Brunei Darussalam. The students were given two types of question: (i) a direct question relating to theory in the same order as they were taught and (ii) conceptual questions that required the application of a part of the information in the direct question. We found that despite students knowing the information as indicated by >90% correct responses to theoretical questions, their correct responses drop by about 30% when the questions required application of a part of the information reported in the direct question. This paper explains these discrepancies using recent advancements in the area of how our brain works. Implications to classroom teaching and learning will also be discussed.


1. Introduction

2. Experimental

3. Theory - Chemical Bonding

4. Results

5. Discussion

References


1. Introduction

The discrepancy between students' and scientists' understanding of natural phenomena has been closely studied since Piaget and Inhelder [1]. Osborne [2] pointed out that this discrepancy was due to the different ways in which students and scientists perceive natural phenomena. Since then a considerable amount of research has been carried out in this area. During the early 1990s, Bowen and Bunce [3] listed over 50 articles focusing on misconceptions in chemistry alone. Science educators in a number of countries have found that student misconceptions concerning fundamental laws of science are not only numerous, but diverse [4-10].

Conservation of matter, chemical change and chemical bonding are the areas that illustrate the problems students have with the fundamental laws in science. Because knowledge in chemistry is based on the understanding of the basic principles, the shortcomings of education in dealing with these notions unquestionably hinder students' ability to progress in the sciences.

Researchers [11] found that only a small number of students in secondary school and in their first year at universities could solve conceptual problems in chemistry. According to Lythcott and Sawrey [cited in 11], this inability can be explained by an insufficient knowledge foundation. There is a need to probe deeper to understand why students struggle with the basic principles of chemistry.

Although identifying student misconceptions is an important instructional step to helping them develop scientific understanding, Stavy [12] observed that student misconceptions are often quite resistant to common classroom instruction. Calix and Ayas [13] highlighted how student misconceptions about solutions outweighed their factual knowledge about the topic. Students in their study had difficulties making connections between their knowledge and everyday life experience, and, as a result, their knowledge was limited to the context of their curriculum.

One strategy for clarifying the degree and nature of student understanding is through questions that focus on conceptual versus algorithmic ways of knowing. The importance of conceptual understanding was underscored in a survey of 436 chemistry professors at 205 institutions in the United States [14]. Faculty from four areas of chemistry identified "conceptual understanding" as one of the two most important learning outcomes for students. The other was that students "develop reasoning skills" where they use fundamental concepts to solve problems. The recognition of the importance of conceptual understanding has increased the demand for assessments that embody a conceptual structure. In response, the American Chemical Society, which has designed national examinations since the 1930s to test general chemistry knowledge, now offers a conceptually oriented examination [3].

There have been studies on students' understanding of chemical bonding and most of these studies are limited to secondary students. The studies show students' poor understanding of chemical bonding and learners have little appreciation of the underlying electrostatic nature of the chemical bonding [15-18]. Moreover they show that students have more problems with metallic bonding than those other forms of bonding [15]. The learners were able to use the simple electron sea model to explain the properties of metals [16, 19], but are unable to use complex models like the electron drift model [20]. Brick and Kurtz [21] reported confusion between ionic and covalent compounds. The learners believe that ionic compounds are molecular in nature. Coll and Treagust[22, 23] explored students mental models of metallic and covalent bonding in three academic levels (Secondary, Graduates, and Postgraduate students) and reported that "despite evidencing expertise in a number of highly complex and mathematically sophisticated mental models, tertiary students including graduates (M. Sc. and Ph. D.) show a strong preference for simple realistic mental models. They struggle to use their mental models to explain the physical properties of covalently bonded substances". About the metallic bonding they reported that learners in their study hold realistic views about the bonding and structure for metallic substances and prefer the "sea of electrons" model. Few of learners were able to describe bonding in alloys. Students at all levels were able to explain conductivity, but not malleability.

The influence of question type and students' responses has been studied in various fields [24-25]; however, in sciences such as chemistry, it has attracted little attention. The questioning technique in oral or written format is widely used to evaluate students' comprehension of a scientific topic. Bloom [26] recommended six levels for learning, teaching and assessing. Bloom's levels were revised by Anderson and Krathwohl,[27] by changing nouns to verbs: (1) Knowledge to Remember, (2) Comprehension to Understand, (3) Application to Apply, (4) Analysis to Analyze, (5) Synthesis to Synthesize and (6) Evaluation to Evaluate. However, Anderson and Krathwohl [27] reversed the order of Synthesis and Evaluation in Bloom's taxonomy to Evaluate and Synthesize [26-27].

The assessment component of the above taxonomies guides the different types of question (multiple choice, short answer and essay) that can be written to assess each level. Moreover, these questions can be administered in an examination setting or as an assignment to complete. Schiffmann et al. [28] stated that in response to questions about particular events of interest that can be enumerated in advance, it is possible to perform a deeper semantic analysis focusing on the entities, relations, and sub-events of interest. On the other hand, the deeper analysis may be in error as it will not always provide complete coverage of the information relevant to the query. The challenge, therefore, is to blend a shallower, robust approach with the deeper approach in an effective way. This study therefore concentrates on the multiple choice questions administered in the examination setting as well as short answer questions given in an assignment setting. These questions concentrated on the recall questions (referred to as theoretical questions) and application questions (referred to as conceptual questions).

There is very little known about Bruneian students' misconceptions about chemical bonding especially at the tertiary level. Hence this study was planned and executed. The aim of this study was primarily to evaluate tertiary students' responses to recall questions and application questions.

2. Experimental

2.1 Subjects

The subjects were a group of year two students (n = 43) in the Department of Chemistry, Faculty of Science studying inorganic chemistry in a combined class for a range of degree programmes (B.Sc., B. Sc. Ed., B. Biotech, B. Chem. Eng.)

2.2 Data Collection Procedure

The first instrument consisted of a series of questions wherein the students were asked to make predictions, describe the structures of compounds and explain theoretical concepts. This instrument was administered under supervision near the end of the semester after the content had been taught (theory plus application to the representative elements of the periodic table). All students were provided with a periodic table during the assessment process. The second instrument was a take-home assignment. Here, the students were free to consult reference works and had ample time to answer the questions.

Students' responses to test and assignment questions were analyzed and grouped as per their understanding. Percentage data on students' correct responses and typical responses are reported in this study.

2.3 Test and Assignment Questions

Instrument one was a test with six questions: two multiple choice, two requiring short descriptions and two asking for a decision on the type of bonding. The topic of the test was structure and bonding (i.e. ionic/covalent/metallic). The questions were based on the objectives of the course with the students being given adequate time to finish the test. The content validity of the instruments was checked by three experienced members of staff. The first three questions and part of question 6 are used in Examples 1 and 2 below.

The assignment consisted of five questions; these were all of the conceptual type requiring students to apply the information in the lectures. Two of the questions are used in Examples 3 and 4 below. The example 4 theoretical question (Na2O, CdI2) is taken from the two-hour end of semester examination.

The alpha reliability, using SPSS, was 0.86 for the test and 0.79 for the assignment questions respectively.

3. Theory - Chemical Bonding

Structure and bonding are the foundation/ cornerstones of chemistry. These concepts are then used to rationalise the physical and chemical properties of both the elements and their compounds.

Models are used to rationalise bonding. This process commences in secondary school with simple models. This is followed at University with more complex models and finally at the research level state of the art models are employed. Merely learning the concepts and bonding models themselves is of limited use, students need to be able to apply them to:

The prediction/rationalisation of physical and chemical properties

Understanding the molecular and crystal structures of actual compounds.

With this proliferation of models, the questions that can be asked are:

At which level of models is the students' conceptualisation?
Do they understand the models?
Can they apply them in the correct manner?
Can they choose a suitable model?

All bonding is electrostatic in nature. For the sake of convenience and teaching however, bonding is divided into three types:

Metallic
Ionic
Covalent

as summarized in the bonding triangle (see e.g. Jolly [29])

Ionic bonding involves electrostatic interactions between cations and anions. Electrostatic forces act uniformly in all directions. There are no bonds as such and all the ions interact with each other, resulting in a three-dimersional lattice. There are no molecules in ionic bonding. Ions need to be formed from neutral atoms. The formation of ions thus implies a metal with low ionization energy reacting with a non-metal with a high electron affinity.

Metallic bonding can be rationalized with the simple "sea of electrons" model. Again, we have cations, but this time they are embedded in a mobile "sea of electrons". Both metallic and ionic bonding imply a 3-D lattice. Covalent bonding involves the sharing of one or more pairs of electrons between two atoms, usually non-metals, resulting in the first instance in the formation of discrete molecules (omitting for the moment covalent network solids). Each type of bonding has an associated range of physical and chemical properties.

4. Results

This section is divided into two sub sections: (a) test questions from instrument 1 (Examples 1 and 2) and (b) assignment questions from instrument 2 (Examples 3 and 4). Note, however, that the theoretical question in example four is taken from the end-of-semester examination as such a question in an assignment would have limited merit as these are standard compounds.

Test Questions


Example 1

Theoretical question: When the following test question is given either at the start or the end of the semester:

When looking for examples of ionic bonding, would you look for a compound formed between ___________.

(a) two metals

(b) two non-metals

(c) a metal and a non-metal

(d) all the above


the mean correct number of responses was typically 94 ± 4% (Figure 1).

This information is already taught in schools and it stays with the student. This response demonstrates that students know that ionic bonding occurs in compounds formed by the interaction of metal and non metal. Some typical answers are:

Ionic bond forms by transfer of valence electrons from a metal to a non-metal in order to achieve a noble gas configuration.

Ionic bonding is formed between a metal and a non-metal. In this bonding, electron is transfer from one atom to another.

Ionic compound is formed by transferring of electron from one element to another. And this ionic bonding occurs between a metal at the far side of the periodic table with a non-metal at the right side of the periodic table.

However, do they understand the implications or the reasons why?

Fig. 1. Typical distribution of responses for the theoretical question on ionic bonding.


Conceptual question: At room temperature and pressure, would you expect an ionic compound to be a:

(a) solid
(b) liquid
(c) gas
(d) any of the above



This question requires students to operate in the context of room temperature and pressure. From theory, an ionic compound is a solid at room temperature. However, Figure 2 shows the spread in responses. These indicate a poor understanding. Already we begin to see the appearance of misconceptions. Some of these are given below:

Fig. 2. Typical distribution of responses for the conceptual question on ionic bonding.

Ionic compound has ionic bonding. One of the properties of ionic bonding is that they have low melting and boiling point.

My answer is molten. The nearest is liquid.

This is because at room temperature and pressure, ionic compound remains as a crystal lattice.

An example of a better answer is;

Ionic bonding is very strong; therefore it has a high m.p. and b.p. to break the bonds apart. Therefore, it requires high energy to break the bonds. So, usually ionic compounds present are solid.


Example 2

Theoretical question: Describe the structure (with a sketch if necessary) of NaCl(s)


About 79% of the students stated that solid NaCl has a lattice structure made up of ions, and only 6% stated that it was molecular (i.e. contains Na-Cl molecules). This knowledge is taught in secondary school and is retained by the student. Here we glimpse the beginnings of a misconception that an ionic solid contains molecules and hence confusion with covalently bonded compounds such as H-Cl.


Conceptual question: (a) Describe and sketch the structure of LiCl(s) and MgCl2(s)


The use of these salts in the teaching process is less common compared to NaCl(s). Therefore, students should evaluate that these salts are similar to NaCl(s). The acceptable responses for the two substances ranged from 64 to 74%, suggesting a significant deviation from complete understanding. A significant portion of the class (20 - 30%) gives a molecular covalent structure, i.e.:

even thought some students acknowledge that the bonding is ionic. This misconception is particularly resistant and remains locked in the students even after one semester of active teaching of these concepts. LiCl has the rock salt lattice, like NaCl, while MgCl2 has a layer structure.

Simple extrapolation from Na to Li and Mg should enable the student to predict a lattice.

(b) Describe the structure (with a sketch if necessary) of RbBr(s)

This question elicits the distribution (%) of responses shown in Table 1.

Table 1. Distribution of responses for Example 2(b) - Solid-state structure of RbBr
Response

Percentage
Comment
Correct (rock salt)

5%
Linear (Rb-Br)

60%
(state ionic bonding 12%)
No structure

18%
(state ionic bonding*)
Ionic crystal lattice

10%
Blank

7%

*Some students take structure to mean electronic structure and draw Lewis structures instead

Only 15% of the class stated that RbBr has an ionic lattice, with only 5% volunteering that it was like NaCl, while a further 18% identified the bonding as ionic with no elaboration as to the structure (did not know?). 60% drew a linear molecule (i.e. RbBr).

Even worse, 12% of the class drew a molecule even thought they knew the bonding to be ionic. The wide range of responses suggests a lack of ability to make a small extrapolation from the known (NaCl) to a closely-related compound. The students have been taught that the alkali metals are similar in their properties and that the majority of the alkali metal halides have the same structure. Secondly and more serious, knowing that the bonding is ionic, some students still draw the (familiar) molecule and thus fail to differentiate between RbBr and say HBr.

Assignment Questions


Example 3: For the reactions:

describe the structures of the reactants and products.

Theoretical question

For the structure of elemental Ca, the following distribution of responses (%) was obtained (Table 2):

Table 2. Distribution of responses for Example 3 - Structure of Ca Metal
Response Percentage / %
No response 80 - 85
Crystal solid 7 - 12
Metallic 4 - 8
FCC 4

The actual structure is face centred cubic. Metallic is not an acceptable answer as it describes the type of bonding, not the structure. As this was an assignment, students were expected to consult reference works and on-line resources.

Conceptual question

For the structure of calcium salts, the descriptions are summarised in Table 3.

Table 3. Distribution of responses for Example 3 - Solid-state structure of Ca(OH)2 and CaCl2

Ca(OH)2(s) CaCl2(s)
Response Percentage Response Percentage
No response 40 No response 35
CaOH 4
linear 48 linear 38
bent 8 bent 12
ionic 12
crystal lattice 4

Of the class, 60% believed that Ca(OH)2 has some sort of a covalently-bonded molecular structure, mostly linear as depicted by the representations:

while 8% gave a bent structure, even though Ca has no lone pairs of electrons to induce such a bend (4% forgot the valence of Ca). There was no mention of ionic bonding or a crystal lattice. A similar result was observed for CaCl2, except now "ionic" is mentioned and only two students mentioned the key words - crystal lattice, but no details were forthcoming. One student who stated that the bonding is ionic, then went on to state that the shape is linear (the actual structures are Ca(OH)2 layer (CdI2); CaCl2 3-dimensional lattice - rutile). A frequent response in tutorials to the question: what is the structure of compound X, is:

Crystalline
Crystal structure
Ionic
Metallic

All solids (except amorphous) are crystalline.


Example 4

The theoretical question is taken from the end-of-semester examination.

Theoretical question: Describe the structures of:

Na2O (antifluorite)
CdI2

For each compound, you may need to sketch the structure, include the type of packing, the type of sites occupied and the coordination number of the anion and cation.


These are both standard compounds and need to be memorised. An example of an excellent answer is shown in the box. Na2O has the antifluorite structure and CdI2 has a layer structure.

Na2O (antifluorite)
Type of packing: cubic close packed
CN of
Na+ = 4
CN of
O2- = 8
Type of sites of
Na+ = tetrahedral
Type of sites of
O2- = f.c.c.
CdI2 (forms layer structures)
Type of packing: layer structure
Cd2+: fills 1/2 of the octahedral sites
I- : hexagonal-close-packed
Cd2+ and I are held together by strong electrostatic forces
I- layers are held by weak van der Waals forces

The number of correct responses for Na2O and CdI2 were 72% and 69% respectively. Even the incorrect responses still described some type of lattice structure, albeit an incorrect one; no student draws a molecular structure like those shown above for MgCl2 or CaCl2. The high response rate for CdI2 is surprising in that it is has a layer structure, which generally causes conceptual difficulties amongst students.


Conceptual question: Describe the solid-state structures of AlCl3, MnO2, GeF2 and PbCl2.


Students are not commonly exposed (in a teaching sense) to these compounds during lectures, thus some research is required. Even with free access to books and the internet, typically some 50% to 67% students reported the following structures:

trigonal planar linear linear linear

All of these (solid) compounds have a polymeric structure: AlCl3 layer, MnO2 3-dimensional (rutile), GeF2 chain, PbCl2 3-dimensional. Meanwhile, the students have treated all of them as molecular substances with covalent bonding and while doing so, have ignored the following concepts, all discussed during the course:

The octet rule
The nature of the bonding
The electronegativity of the elements
Layer structure for AlCl3 (as opposed to BCl3)
Aluminium, manganese and lead are metals and mostly do not form molecules
The coordination number of a metal is typically 6
MnO2 is not a molecular compound (neither its colour nor melting point are consistent with a molecular solid)
Existence of lone pairs of electrons (Ge and Pb)
Lead(II) salts have a 3-dimensional structure


Conceptual question: Describe the structures of Al2O3, SiO2 and BeO.


These are mostly drawn as:

All of these compounds form a 3-dimensional lattice with a high melting point. In the nonsensical aluminium oxide structure shown above the valence of the oxygen is not satisfied (a simple Lewis structure would reveal this) and the compound can not possibly be a monomer. Neither Si=O nor Be=O double bonds are stable, single bonds are preferred; therefore the structures must also be polymeric. In the students' rush to draw familiar and comfortable molecules, they have failed to allow for the properties of the substances and the nature of the bonding, e.g. O=Si=O and Be=O would be gases.

5. Discussion

This research shows that the quality of the answer depends on how the question is formulated. Students' responses to theoretical questions were far superior to conceptual questions. This may be because the definitions and facts are memorised (by rote) with the students demonstrating tutor-dependent passive learning instead of active learning. Rote learning is a learning technique which avoids understanding of a subject and instead focuses on memorization (Wikipedia, 2010) [30]. There may be a problem with the transfer of knowledge for long term retention with rote learning if the learning has not yet reached the mastery level and also if it is applied to information that requires understanding. Information retention requires that students remember what they have learned, whereas transfer requires students not only to remember but also to make sense of and be able to use what they have learned [31,32].The basic bonding concepts and more importantly the consequences thereof are poorly understood and therefore unable to be applied.

This research shows that despite tertiary students demonstrating that they have the theoretical knowledge, they are unable to demonstrate its application. For example, the students know that ionic bonds are formed between metals and non-metals, yet they draw ionic compounds as molecular. This finding is similar to that by Brick and Kurtz [21], who reported that students are confused between ionic and covalent compounds. The learners believe that ionic compounds are molecular in nature. A question before teachers or lecturers is: where have things gone wrong? Has this concept been taught wrongly in classrooms or the books have this presentation wrong - both of these scenarios are unlikely. One problem the authors see is that the lecture method used for teaching at the tertiary level does not allow for the correction of mistakes. Large tutorial classes and a large syllabus could also be a limitation. On the other hand, lecturers also feel that the students at university level are intelligent, so they should be able to handle such issues. However, the research shows that these misconceptions often resist change unless a conceptual change approach is applied.

The students in this study were also unable to describe the structure of Ca metal. Research shows that students use a simple model of electron clouds to describe the metallic bonding. Moreover, they have problems explaining the metallic bonding in metals. This finding is in line with that of Coll and Treagust [22,23]. They reported that "despite evidencing expertise in a number of highly complex and mathematically sophisticated mental models, tertiary students including graduates (M. Sc. and Ph. D.) show a strong preference for simple realistic mental models".

Another way to look at the problem is that students are able to answer the question when a theory type of question is asked directly. However, when a part of the information is not from the beginning of the course material, the students are unable to answer. These types of questions involve synthesis and the fusion of information from several parts of the course material and some slight extrapolation. Neurocognitive theory suggests that free recall of information depends upon the way the information is interlinked in the human memory [33-35]. These studies also report that when information is linked more linearly, the student's recall for a particular part is difficult. However if the memory is well cross-linked, then the recall of various parts of the cognitive structure about a concept is possible. It is therefore important that the memory organization of these students on chemical bonding is evaluated. Moreover, the teaching should facilitate more cross-linking of memory.

 

References

  1. Piaget, J. and Inhelder, B. (1974) The child's construction of quantities. London: Routledge and Kegan, Paul.
  2. Osborne, R. (1982) Science education: Where do we start? The Australian Science Teachers' Journal, 28, 21-30.
  3. Bowen, C. W. and Bunce, D. M. (1997) Testing for conceptual understanding in general chemistry. The Chemical Educator, 2, 1430-4171.
  4. Agung, S. and Schwartz, M. S. (2007) Students' Understanding of Conservation of Matter, Stoichiometry and Balancing Equations in Indonesia. International Journal of Science Education, 29, 1679-1702.
  5. Ayas, A. and Demirbas, A. (1997) Turkish secondary students' conceptions of introductory chemistry concepts. Journal of Chemical Education, 74, 518-521.
  6. Driver, R. Children's ideas in science. 1985, Philadelphia: Open University Press. Driver, R. and Easly, J. (1978) Pupils and paradigms: A review of literature related to concept development in adolescent science students. Studies in Science Education, 5, 61-84.
  7. Hesse, J. J. and Anderson, C. W. (1992) Students' conceptions of chemical change. Journal of Research in Science Teaching, 29, 277-299.
  8. Meerah, T. S. M. Dampak penyelidikan pembelajaran sains terhadap perubahan kurikulum. Malaysia: Universitas Kebangsaan Malaysia. 1998.
  9. Ozmen, H. and Ayas, A. (2003) Students' difficulties in understanding conservation of matter in open and closed-system chemical reactions. Chemistry Education: Research and Practice, 4, 279-290.
  10. Stavy, R. (1990) Children's conception of changes in the state of matter: From liquid (or solid) to gas. Journal of Research in Science Teaching, 27, 247-266.
  11. Boujaoude, S. and Barakat, H. (2000) Secondary school students' difficulties with stoichiometry. School Science Review, 296, 91-98.
  12. Stavy, R. (1991) Using analogy to overcome misconceptions about conservation of matter. Journal of Research in Science Teaching, 28, 305-313.
  13. Calix, M. and Ayas, A. (2005) A comparison of level of understanding of eight-grade students and science student teachers related to selected chemistry concepts. Journal of Research in Science Teaching, 42, 638-667.
  14. Slavings, R., Cochran, N. and Bowen, C. W. (1997) Results of a National Survey on College Chemistry Faculty Beliefs and Attitudes of Assessment-of-Student- Learning Practices. Chemistry Educator, 2, 1430-1471.
  15. De Posada, J. M. (1997) Conceptions of high school students concerning the internal structure of metals and their electric conduicion; structure and evolution. Science Education, 81, 445-467.
  16. Taber, K. S. (1995) An analogy for discussing progression in learning chemistry. School Science Review, 76, 91-95.
  17. Taber, K. S. (2000) Multiple frameworks? Evidence of manifold conceptions in individual cognitive structure. International Journal of Science Education, 22, 399-417.
  18. Taber, K. S. (2001) Should physics teaching be a research-based activity? International Journal of Science Education, 23, 731-753.
  19. Taber, K. S.(1998) An alternative conceptual framework from chemistry education. International Journal of Science Education, 20, 597-608.
  20. Stephens, S., McRobbie, C. J. and Lucas, K. B.(1999) Model-based reasoning in a year 10 classroom. Research in Science Education, 29, 189-208.
  21. Brick, J. P. and Kurtz, M. J. (1999) Effect of experience on retention of misconceptions about molecular structure and bonding. Journal of Chemical Education,76, 124-128.
  22. Coll, R. K. and Treagust, D. F. (2001) Learners' mental models of chemical bonding. Research in Science Education 31, 357-382.
  23. Coll, R. K. and Treagust, D. F. (2002) Exploring tertiary students' understanding of Covalent bonding. Research in Science and Technological Education, 20, 241-267.
  24. Alexandera, P. A., Hareb, V. C. and Garnerc, R. (1984) The effects of time, access, and question type on response accuracy and frequency of lookbacks in older, proficient readers, Journal of Literacy Research, 16 (2), 119-130.
  25. Hewitt, L. E. (1998) Influence of question type on response adequacy in young adults with autism. Journal of Communication Disorders, 31 (2), 135-152.
  26. Bloom, B. S. (ed.) (1956) Taxonomy of educational objectives, the classification of educational goals - Handbook I: Cognitive Domain New York: McKay.
  27. Anderson L. W. and Krathwohl, D. R. (eds) (2001) A taxonomy for learning, teaching, and assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Longman.
  28. Schiffman, B., McKeown, K., Grishman, R., Allan, J., (2007, April). Question answering using integrated information retrieval and information extraction. In: Proceedings of HLT-NAACL'07, Rochester, New York.
  29. Jolly, W. L. (1991) Modern Inorganic Chemistry, 2nd edition, Singapore: McGraw-Hill.
  30. Wikipedia, (2010) Rote learning. http://en.wikipedia.org/wiki/Rote_learning (retrieved on June 22, 2010).
  31. Haskell, R. E. (2001) Transfer of learning. San Diego: Academic Press.
  32. Mayer, R. E. (1992) Thinking, problem solving, cognition (2nd ed.). New York: Freeman.
  33. Anderson, O. R. (1991) Neurocognitive models of information processing and knowledge acquisition. Progress in Sensory Physiology, 12, 115-192.
  34. Anderson, O. R. (2009) Neurocognitive theory and constructivism in science education: a review of neurobiological, cognitive and cultural perspectives. Brunei International Journal of Science and Mathematics Education, 1 (1), 1-32.
  35. Dhindsa, H. S. and Anderson, O. R. (2004) Using conceptual-change approach to help pre-service teachers reorganize their knowledge structures for constructivist teaching. Journal of Science Teacher Education, 15 (1), 63-85.