Apertura

Vol. 8, Núm. 2 / octubre 2016 – marzo 2017 / ISSN 2007-1094

 

Working memory and reading comprehension: Measurement through reading span test and type test for high school and university levels

 

Ismael Esquivel Gámez[1]

Waltraud Martínez Olvera[2]

Rafael Córdoba Del Valle[3]

Carlos Reyes Gutiérrez[4]

 

Abstract

The significance that working memory has in the performance of various highly demanding cognitive tasks such as reading comprehension and, in turn, the impact it has on student performance has been examined in numerous studies applied to different levels of education. Therefore, standardized tests properly supported have been implemented in order to measure the degree of relationship between working memory and reading comprehension. The aim of this study is to assess such relationship by using reading span tests and cloze tests due to the fact that these instruments are not frequently used in conjunction in Hispanic American contexts. All of this takes place despite the practical development, application and analysis of tests made with questions embedded with (cloze) answers in open virtual learning platforms like Moodle. The results refer to a correlation for those over nineteen years of age, highly significant for women and significant for men, in part due to a higher proportion of undergraduates in both the instructional reading level as well as for fourth reading span level.

 

Keywords: Memory test, working memory, reading comprehension, reading test, reading span test, cloze test, higher education, high school.

 

 

INTRODUCTION

 

The working memory (WM) is a system that maintains and manipulates information required to execute complex cognitive tasks related to learning, problem resolution, and comprehension. In accordance with the fact that the reading process intervenes in both the recognition of graphic symbols (forms) as well as the integration of information on prior schemes of the reader (meanings), a constant interaction is required between the subsystems that comprise the WM in order for the reader to be able to decipher the information contained in the text (Gutiérrez-Martínez, García, Elosúa, Luque and Gárate, 2002; Just & Carpenter, 1992). Recognition, memory, elaboration of inferences, access to prior knowledge, and the ability to integrate relevant and new information, are aspects in which it has been strongly accepted that a low WM intervenes negatively in the comprehension of a text, given that a broad WM allows the use of a larger quantity of resources (Gómez-Veiga, Vila, García-Madruga, Contreras and Elosúa, 2013; García-Madruga, Elosúa, Gil, Gómez, Vila, Orjales, Contreras, Rodríguez, Melero & Duque, 2013).

 

Among the studies that reflect the measurement of the WM regarding the comprehension of texts on children in primary school are those by Baqués and Sáiz (1999); Sánchez and García (2003); Carriedo and Iglesias-Sarmiento (s.f.); Iglesias-Sarmiento, Carriedo and Rodríguez-Rodríguez, J. (2015); and Gómez-Veiga et al. (2013). Regarding secondary education there are those by Gutiérrez-Martínez, Ramos and Vila (2011), García-Madruga and Fernández (2008), Gutiérrez-Martínez et al. (2002), Miranda-Casas Fernández, Robledo and García-Castellar (2010), and at the university level, Del Río and Lopez-Higes (2006), meilán and Vieiro (2001) and Barreyro, Burin and Duarte (2009). The cited studies, in order to measure the WM, have used the prueba de amplitud lectora (PAL), this being the Hispanic version of the Reading Span Test, proposed by Daneman and Carpenter (1980).

 

The cloze method, proposed by Taylor in 1953, has been extensively studied and validated as both a legibility measurement of a text as well as to evaluate the general comprehension of the reader (Bickley, Ellington & Bickler, 1970; Quintero, 1986). A text is presented in prose, with systematic suppression every fifth word, so that it evaluates the success of the reader by adequately completing the empty spaces (López, 1983; Monsalvo, 1985).

 

The works that have implemented the PAL or its derivations, already mentioned, have used tests different to the cloze method in order to measure the performance of the readers, except for the study of O’Shanahan, Siegel, Jiménez and Mazabel (2010), who utilized a similar instrument (oral cloze), but to evaluate, among others, the syntactic conscience and its relation with the WM, in English and Spanish.

 

Our objective is to determine the degree of association between the capacity level of the WM and the reading comprehension, using both instruments, in high school and higher education students.

 

 

WORKING MEMORY AND READING COMPREHENSION

 

The WM is understood as a group of subsystems—the phonological, the visuospatial, and a checkpoint that connects the two aforementioned subsystems with the long-term memory—dependent on a central executive that stores and manipulates information, controlling and processing it active and simultaneously, although in a manner limited by the attention capacity (Baddeley, 2000). The short-term memory, divided in subsystems distinguishable from the WM, briefly stores the sensorial captured data and, thanks to a limited attention control system, it transfers, activates or recovers them to and from the long-term memory (Baddeley, 2003; Juffs and Harrington, 2011).

 

Following this approach, the WM constitutes the previous step through which the information must be stored in the long-term memory (Gómez-Veiga et al. 2013, Juffs and Harrington, 2011). The ability to replace unused information by new data relevant to the restructuring of the schemes in the memory, has contributed to calling the span of the MO as an update executive function (Carriedo and Iglesias-Sarmiento, s.f.; Iglesias-Sarmiento, Carriedo and Rodríguez-Rodríguez, 2015).

 

For its part, the approach of Ericcson and Kintch (1995), based on the reading comprehension processes identified in competent readers, incorporates the original idea of temporary storage of the WM in a long-term storage. The short-term working memory (ST-WM) stores provisional information that could end up held in the long-term working memory (LT-WM), depending on the ability of the reader to decode it, i.e., of the recovery keys that it can generate and store in the ST-WM, presumably with the implementation of strategies. In general, the ST-WM is in charge of maintaining the memory keys elaborated through a constructive exchange in the long-term memory, supported in the structure of the text and on prior knowledge. Based on said approach, the efficiency of the WM varies in light of the familiarity or not of the task, which therefore better explains the execution of competent individuals in determined tasks.

 

 

READING SPAN TEST

 

For Gutiérrez-Martínez et al. (2005), the span and functionality of the WM corresponds to the efficiency with which certain operations are carried out, given that the span of the storage capacity is given by requiring, on behalf of the processing, less resources. The aforementioned establishes the basis for this most usual type of measurement, which involves the processing and storage, in an interdependent manner, by combining both tasks in the one denominated PAL. Developed by Daneman and Carpenter (1980), it entails a measurement instrument for the storage and processing functions attributed to the WM and required in the reading process, given that the tasks imply retaining information while requiring the simultaneous processing.

 

Following the original version of Daneman and Carpenter (1980), the test has been adapted for a Hispanic population, for children and with an auditory application. In spite that it is considered that the WM measurements based on numbers or non-verbal are also related with the comprehension of the discourse (Canet-Juric, Urquijo, Richards & Burin, 2009), the studies show a greater correlation between the verbal WM measurements and the reading comprehension (Daneman and Merikle, 1996), which would be related to the model of the auditory component in the language of Baddeley (2003).

 

In the cited test, a series of phrases that increases in number per level reached so that the subject reads aloud, is presented, in a first level, three series of two phrases that increase until reaching the fifth level with series of six phrases, for a total of sixty. The phrases do not maintain semantic relation in order to avoid the implementation of strategies and there is a time limit (once the subject finishes reading each phrase) in order to avoid for repetition to affect memory. After concluding each series, the subject is asked to remember the last word read in each phrase and to write them in the most orderly form possible (Elosúa, Gutiérrez, García, Luque and Gárate, 1996). It is assumed then, that a high score in the test signifies a greater capability in the WM and reflects the good readers, who can easily store the product taken from the reading, and therefore their capacity to process information does not decrease (Osaka & Osaka, 1992).

 

 

READING COMPREHENSION AND ITS LEVELS

 

In order to identify a reading task as a comprehensive act, various authors highlight the relevance of determining reference criteria, such as behavioral patterns to observe, expected performance level, types of material on which the behaviors are proven, proportion of the reading task in which the person must show adequacy, as well as characteristics of the reader, such as their aptitude and goals (Bormuth, 1973; Irigoyen, Jiménez and Acuña, 2008). Thus, the reader, text, and activity interrelated in a specific sociocultural context are elements that outline comprehension (Snow, 2002).

 

For the classification of the task in comprehension levels, Valléz-Arándiga (2005) documents the proposals of various authors and evidences, in spite of the variation in terminology, the decoding process as an initial category, whereas the apprehension of the sense of the text would occupy a higher hierarchy. Although the influence of the decoding on the reading process is not excluded, it is important to differentiate that reading comprehension in and of itself requires the reconstruction—integration, interpretation, reflection and evaluation—of the content rather than literal memory of the words contained in the text (Muñoz-Valenzuela and Schestraete, 2008); therefore, it is differentiated from identification reading.

 

A competent reading implies the in-depth, inferential, analytic and reflective reading of a vast sample of resources, whether for entertainment or learning (Snow, 2002). For this purpose and according to Andrade-Calderón (2007), the reader requires sub-competencies that could be divided into three large groups: a) linguistic competence, that covers, in accordance to the linguistic focus, the grammatical, semantic, syntactic and lexical skills, corresponding to a literal reading; b) communicative competence, which, from a psycholinguistic focus, includes the analysis, interpretation, deduction, induction, inferential reading, and the identification of the text structure; and c) pragmatic competence, which, from the sociolinguistic and sociocultural focus, is related to the use that the reader makes of the text, the identification of the author’s intention, the analysis of the language, and the vision of the world. The latter, from an interpretative and critical level of the text.

In order to evaluate the levels of reading competence in subjects that have concluded their compulsory education, based on the premise of having obtained the necessary competences to be able to apply knowledge and skills in their adult life (Junta de Castilla and León, 2011), the Project for International Student Assessment (PISA) conceptualizes the term as comprehending, using, reflecting and feeling engaged with the written texts in order to achieve their own goals, develop knowledge and potential, and participate in society (Organization for Economic Cooperation and Development [OECD], 2009), which constitutes one of the most widespread standards. In the PISA there are two levels of reading comprehension: literal and non-literal. The first implies locating, recognizing and comparing fragments of information, whether explicitly or synonymously. The second is subdivided into two phases: interpretative, which requires the construction of meaning and the generation of inferences, analysis, synthesis and reorganization of a text, and the critique which, in addition to the previous characteristics, allows to demonstrate the capability to relate the text with prior knowledge, as well as recognize the intentions of the author (Caño and Luna, 2011).

 

CLOZE TEST

 

Cognitive and metacognitive processes intervene in the cloze method in order to carry out the reading task, parting from syntactic and semantic aspects towards the simultaneous involvement of inferences, problem resolution and autoregulation (Difabio, 2008; González, 1998; López, 1983). In order to assess the test, a point is given for each correct answer, the number of correct answers is divided by the total and the result is then multiplied by one hundred in order to obtain the achieved percentage (Condemarín and Milicic, 1988, cited in Alegre-Bravo, 2009).

 

The studies indicate that the correct answers may correspond to a precise word, or to synonymous terms. Although there are differences in that regard (Bickley, Ellington and Bickley, 1970; Rodríguez-Trujillo, 1983), the use of synonyms is justified in the fact that the test, more than a superficial structure, seeks to capture meanings from the text (Difabio, 2008). Once the score has been obtained, it is placed on a scale in order to determine the level of comprehension and anyone who achieves 75% or more is considered an independent reader, 44 to 74% as instructional, and 43% or less was placed in the frustration level (Condemarín and Milicic, 1988, as cited in Difabio, 2008). Since its appearance, the cloze method has been used in various studies in order to test the reading comprehension of the participants from both basic as well as higher education; some recent investigations in Latin American scenarios are those by Difabio (2008), De Oliveira and Dos Santos (2005), Márquez, Ibáñez and Pérez (2011), Alegre-Bravo (2009), and Calderón-Ibáñez and Quijano-Peñuela (2010).

 

 

PROCEDURE

 

Before the implementation of the tests, each participating researcher, independently, explained the objective and the extra value that the result obtained in said tests would have on their final grade, coordinating the application of the same and resolving any doubts, equipment and connectivity failures.

For the cloze test, the group was told that they had an hour to answer the exam, that they would place a single word in each space, and that they could correct their answers however many times they would like before closing and sending the finished exam for its automated evaluation. For the implementation of the PAL, each professor explained to the dynamic to the group, putting emphasis on the fact that they had to read each phrase aloud and write the last word of said phrase whenever this was requested. This was an individual exam and, at the end of it, the result would appear on the computer screen—at the same time as it was recorded in a file, the name of which was the registration number—, each one of the answered words, the time spent, and the final score.

 

 

POPULATION

 

It was comprised by six groups of educational institutions of a public character from the state of Veracruz, Mexico, and gathered 136 students (46% from high school and 54% undergraduates) with average ages of 17.6 and 21.5, respectively, and a distribution of 67% men and 33% women. The groups were comprised by students from a technological high school (1 and 2), undergraduate in Administrative Computational systems (3), Engineering in Computational Systems (4), Pedagogy (5), and Nursing (6).

 

 

INSTRUMENTS

 

Reading span test

 

The study developed the test accepted by Elosúa et al. (1996) through a programming language, ideal for the processing of texts, denominated Regina-Rexx. Some of the phrases and, therefore, the last words were slightly changed in the fact that they had no accents. For the pilot, the test was applied to the participating researchers, who provided their feedback on the initial screen, which contained the instructions. Furthermore, in order to help their students with the application process, they were given a guide with the particularities of the test.

 

Cloze type test

 

In this case, we utilized the text La muralla mágica, used with university students of recent admission (Márquez, Ibáñez and Pérez, 2011). We decided to use an electronic text in Moodle with embedded questions, which contained the cited text and accepted synonyms. The exam was pilot tested with a group of ten advance degree students and the written instructions and some errors found during the automated evaluation process, referring to missing synonyms, were corrected. It is worth noting that Moodle records, once the exam has been answered, the ease index of each question and its standard deviation. According to Blanco and Ginovart (2012), said index represents the ratio of participants that answer a question correctly, if it is very high or very low it is not useful as a measurement instrument. In general, in our case we obtained an ease index of 71.95, which falls within what is acceptable and with a standard deviation of 0.0921.

 

RESULTS

 

Of the close type test, the results were obtained directly from the Moodle platform, through the exportation of the scores on an Excel file. Of the PAL test, we automatically processed the files that recorded the individual scores in order to comprise the group, the registration, the score, and the time of execution in a single file. By gathering those from both tests, we proceeded to adding the sex and age, which were requested directly by each professor.

 

In order to determine the level reached in reading comprehension and its correlation with the WM measurements, we first proceeded to obtain the descriptive data shown in Table 1. Then, we also obtained comparative data from different groupings, followed by the group location in the corresponding reading and memorization level; finally, we determined the association factors between the scores of the two tests.

 

With the results categorized by evaluated group and by age, we verified their normal distribution and found, for both, that those of the cloze test complied with this assumption, but not those of the PAL; therefore, we applied the ANOVA and Kruskal-Wallis statistic tests, in that order.

 

Per group of students, we did not identify significant differences between the groups for the cloze test (F=1.758, p=0.126), but we did for the PAL (Chi-square=22.149, p=0.000), mainly between groups 6 and from 2 to 3, in favor of the first. Per age groups, we excluded the groups with less than two members and found, in the case of the cloze test, significant differences (F=3.240, p=0.015) between the groups of seventeen and twenty years, in favor of the former by 8.79 points; whereas for the PAL, there was no significant difference (Chi-square=1.710, p=0.789).

 

Table 1. Scores obtained for each participating group.

 

Furthermore, we made groupings for comparison in accordance with the level of studies, sex and time of application (start or end of the semester). Per level of studies, we found that the cloze results followed the normal distribution, but not those of the PAL; therefore, when implementing Student’s t and Mann-Whitney’s U statistics, respectively, we did not notice any significant different for the former (p=0.277), but we did for the latter (p=0.013), in favor of the bachelor’s (76.25 vs 59.52). For the sex, by not meeting the assumption of normality, it was obtained through the Mann-Whitney’s U, and there were no relevant differences for both tests. Regarding the time of application, for groups 1, 3 and 4 it was at the end of the course and coincided with the exam period; for the others it was at the start of the semester. By applying Student’s t test, we observed that there were no differences in the results of the PAL test, but there were in those of the cloze test (p=0.026), in favor of the initial groups (74.81 vs 71.38).

 

According with the classification by Condemarín and Milicic (1988) for the cloze test and with the PAL (WM) levels, we carried out the distribution of the participants per group, level of studies, and total (see Table 2).

 

Table 2. Distribution of the participating groups per level.

 

In the Cloze tests no participant was placed in the frustration level and there is a greater ratio of readers with independent level for high school students. On the contrary, in the PAL test there is a greater ratio in the 5 to 6 levels for university students.

By verifying the associating level between the scores of both tests, we found a low level between the results (0.303, p=.000), although it results rather significant if we compare both levels of study; for high school there is no significant correlation, whereas there is a moderate significance for university students (0.481, p=.000). For those older than nineteen, the correlation by gender is highly significant for women (0.592) and significant for men (0.387); it stands out that for groups 5 and 6 the ratio of the feminine sex is of approximately 3 to 1, whereas for group 4 it is the inverse. Furthermore, those who were placed in the instructional level of Table 2, we found a highly significant (0.325, p=0.005), though low, association factor. Table 3 shows association factors for diverse groupings.

We divided the students in three, according to the PAL scores: high (N=46, M=4.97), medium (N=49, M=3.45), and low (N=41, M=2.22); each contained 33% of the cases; we only observed a low significant correlation with the results of the cloze test and it corresponded with the medium level students (0.293, p=0.041).

 

Table 3. Correlation coefficients per diverse groupings.

 

In the verification of the impact of the time of execution of the test and the score obtained, we found that, in the case of the cloze test, in general and by educational level, there was no significant correlation, whereas for PAL, in global terms, there is a moderate correlation (0.596); for university students, a high correlation (0.647); and for high school students, a very fight correlation (0.963), all of them highly significant.

 

 

DISCUSSION OF THE RESULTS

 

Cloze test

 

For the groups to which the cloze test was applied during the exam period, the average score could have been positively influenced, because group 1 obtained the highest percentage of students in the independent level, and group 4 took the test from home due to not having a computer room available; they were given one week to complete the test. Conversely, the high school group 2 had an inverse ratio, perhaps because the professor that gave the test was not the one that gave them class.

 

Of the previous studies reviewed in Hispanic American contexts for the case of the cloze test, we did not identify any reference that would fit with the type of text, evaluation scale and studied population. For the case of the same text used with students of similar ages to those of our study, the comparison of data was difficult due to the use of a different criteria (verbatim vs acceptance of synonyms) to classify the answers given. Specifically, no reference is recorded for the Mexican population, except for the study of Sánchez (2009), who used a different comprehension placement scale. In our case, with the acceptance of synonyms, we observed that no participant was placed in the frustration scale, which is considered to be consistent with their academic degree, since the frustration level would indicate a severe lack of the prerequisites for a reading that has been corroborated as indispensable for said schooling.

 

Although the majority of high school students were placed in the independent reading level, there were no significant differences with the higher education level in comparison with other studies (Dos Santos, Boulhoça and Vendemiatto, 2009). The aforementioned can be understood if we reflect on the PISA standards, which establish that, from the conclusion of the basic level studies, the student acquires the ability to apply knowledge and skills in different environments, which would lead us to consider that for the intermediate and higher levels said competences could level out.

 

Regarding execution by gender, we noticed better results for males; however, their sample size (N=91) is twice the feminine (45); therefore, it is possible that because of this the gender and comprehension relation is opposite that of other studies (Márquez, Ibáñez and Pérez, 2001; De Oliveira and Dos Santos, 2005).

 

PAL test

 

The results given by the PAL test refer to a 3.4 mean for high school students, who were placed at an average age of 17.6, whereas for university students it was a 3.8 mean, with an average age of 21.5. According with these results, there is a certain tendency to vary the execution according to age, which is supported in the studies that allude to developmental differences in the capability, understood as efficiency, of the WM (Elosúa, Garía, Gutiérrez, Luque and Garate, 1997; Gutiérrez-Martínez et al., 2002).

 

Correlation between both tests

 

The correlation between the scores for those older than nineteen was greater for women. This difference could be due to the greater and more significant correlation factor in favor of women and to their slightly greater ratio in university (56.2 vs 43.8), whereas for high school it was much lower (6.3 vs 93.7). The aforementioned is reinforced with the fact that there is a greater ratio of university students in both the instructional reading level as well as in the level 4 reading span.

 

Such a result forces a reflection on two aspects: first, in relation with the capability measured through the PAL; i.e., the relations identified between the WM and comprehension, whether through digits or verbal, simple, or twofold tests, or rather, by their level of relation with other cognitive tasks, have led to think that the PAL provides data on the overall performance of the WM, specifically the executive performance, in charge of coordinating the exchange of resources (storage-processing) through the control point, without offering more precise data on the specific function, except until the task to be solved has loaded the processing component in said direction. Second, the least correlation found in independent reading levels corroborates the approach by Ericcson and Kintsch (1995), in which, if the WM of a reader varies according to their skill, a competent reader would not see the processing task as compromised. In this sense, the critiques of the classic PAL version are made present: the memorizing of words at the end of non-related phrases implies a more mechanical memory task rather than comprehension (Gutiérrez-Martínez et al., 2005).

 

 

CONCLUSIONS

 

Reading comprehension involves a complex interaction between the reader, the text and the task (Snow, 202) and it requires a series of grammatical, semantic, syntactic and lexical sub-abilities, in addition to the construction of meanings, inferences, analysis, synthesis, critique and reorganization of prior knowledge.

 

Although diverse studies allude to the relation between adequate reading comprehension processes and broad WM (Baquéz and Sáiz, 1999; Carriedo and Iglesias-Sarmiento, s.f.; Iglesias-Sarmient, Carriedo and Rodríguez-Rodríguez, 2015; García-Madruga and Fernández, 2008), others find no correlation whatsoever (Gutiérrez-Martínez, Ramos and Vila, 2011); or rather, variability is observed in the results in accordance with the conditions of application used (Friedman and Miyake, 2004). The results obtained among university students indicate a significant correlation between reading comprehension and WM; however, the differences found at the high school level indicate the need for a more in-depth analysis on the application protocol, as referred by Friedman and Miyake (2004), and on the type of task of involved WM, in accordance with Gutiérrez-Martínez et al. (2005).

 

Regarding the variability of the results, if the PAL test is a valid WM measurement, as noted by various studies that have proven it (Gutiérrez-Martínez et al., 2005), the use of reasoning span tests, such as that of analogies (PAL-anl), that demand the realization of simple inferences, could be extremely discriminating of the processes involved (storage and processing). Although Gutiérrez-Martínez, Ramos and Vila (2011) studied the correlation between PAL-anl and reading comprehension, both the studied population as well as the reading test utilized vary with regard to the conditions that our case proposes. The aforementioned leaves the comparison of the results between said reasoning span test and the cloze test as a pending task, if we consider that the tasks of syntactic, semantic and inferential analysis contained here can discriminate between good and bad readers; it would thus be interesting to corroborate the relation with the cognitive load implied in the PAL-anl.

 

Regarding the analysis by age, we observed correlation starting from those of nineteen years of age, in its majority referring to the group of university students, which agrees with the data by level of studies. From said age group, the correlation is even greater in the female participants. However, the prior studies do not provide differentiating data by sex; therefore, they must be considered cautiously and further investigations must be carried out in this regard.

 

Although the PAL-Cloze correlation must be taken with certain reservations, estimating the possible influence of other already cited elements, the results presented here appear to indicate that the use of the PAL-Cloze tests could be an option to identify the MO relation —regarding a more general component— with the reading comprehension, and to provide useful information for the design of differential remedial treatments.

 

REFERENCES

 

Alegre-Bravo, Alberto. (2009). Relación entre la comprensión lectora y las estrategias de aprendizaje en estudiantes de secundaria en un distrito de Lima. Persona, núm. 12, pp. 207-223. Recuperado de http://revistas.ulima.edu.pe/index.php/Persona/article/view/284  

Andrade-Calderón, Martha Cecilia. (2007). La lectura en los universitarios. Un caso específico: Universidad Colegio Mayor de Cundinamarca. Tabula Rasa, núm. 7, pp. 231-249. Recuperado de http://www.revistatabularasa.org/numero-7/andrade.pdf

Baddeley, Alan. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Sciences, vol. 4, núm. 11, pp. 417-423. doi: http://dx.doi.org/10.1016/S1364-6613(00)01538-2

Baddeley, Alan. (2003). Working memory and language: an overview. Journal of Communication Disorders, vol. 36, núm. 3, pp. 189-208.        doi: http://dx.doi.org/10.1016/S0021-9924(03)00019-4

Baqués, Josep y Sáiz, Dolores. (1999). Medidas simples y compuestas de memoria de trabajo y su relación con el aprendizaje de la lectura. Psicothema, vol. 11, núm. 4, pp. 737-745. Recuperado de http://www.psicothema.com/psicothema.asp?id=325

Barreyro, Juan Pablo; Burin, Débora y Duarte, D. Aníbal. (2009). Capacidad de la memoria de trabajo verbal: validez y fiabilidad de una tarea de amplitud de lectura. Interdisciplinaria, vol. 26, núm. 2, pp. 207-228. Recuperado de http://www.redalyc.org/articulo.oa?id=18011827003

Bickley, A., Ellington, Billie & Bickley, T. Rachel. (1970). The cloze procedure: A conspectus. Journal of Reading Behavior, vol. 2, núm. 3, pp. 232-249. Recuperado de http://jlr.sagepub.com/content/2/3/232.full.pdf

Blanco, Mónica y Ginovart, Marta. (2012). Los cuestionarios del entorno Moodle: su contribución a la evaluación virtual formativa de los alumnos de matemáticas de primer año de las titulaciones de Ingeniería. RUSC. Universities and Knowledge Society Journal, vol. 9, núm. 1, pp. 166-183. doi: http://dx.doi.org/10.7238/rusc.v9i1.1277  

Bormuth, John. (1973). Reading literacy: Its definition and assessment. Reading Research Quarterly, vol. 9, núm. 1, pp. 7-66. doi: http://dx.doi.org/10.2307/747227

Calderón-Ibáñez, Arlenys y Quijano-Peñuela, Jorge. (2010). Características de comprensión lectora en estudiantes universitarios. Estudios Socio-Jurídicos, vol. 12, núm. 1, pp. 337-364. Recuperado de http://revistas.urosario.edu.co/index.php/sociojuridicos/article/view/1195

Canet-Juric, Lorena; Urquijo, Sebastian; Richards, María Martha y Burin, Débora. (2009). Predictores cognitivos de niveles de comprensión lectora mediante análisis discriminante. International Journal of Psychological Research, vol. 2, núm. 2, pp. 99-111. Recuperado de http://132.248.9.34/hevila/Internationaljournalofpsychologicalresearch/2009/vol2/no2/2.pdf

Caño, Alfonso y Luna, Francisco. (2011). PISA: Comprensión lectora I. Marco y análisis de los ítems. España: ISEI-IVEI. Recuperado de http://www.isei-ivei.net/cast/pub/itemsliberados/lectura2011/lectura_PISA2009completo.pdf

Carriedo, Nuria e Iglesias-Sarmiento, Valentín. (s.f.). La función ejecutiva de actualización y el rendimiento en comprensión lectora en alumnos de 5° curso de educación primaria. Recuperado de https://www.researchgate.net/publication/271131208_La_funcion_ejecutiva_de_acdtualizacion_y_el_rendimiento_en_comprension_lectora_en_alumnos_de_5_curso_de_Educacion_Primaria

Condemarín, Mabel y Milicic,  Neva. (1988). Test de cloze: aplicaciones psicopedagógicas. Santiago de Chile: Andrés Bello.

Daneman, Meredyth & Carpenter, Patricia. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, vol. 19, núm. 4, pp. 450-466. doi: http://dx.doi.org/10.1016/S0022-5371(80)90312-6   

Daneman, Meredyth & Merikle, Philip M. (1996). Working memory and language comprehension: A meta-analysis. Psychonomic Bulletin & Review, vol. 3, núm. 4, pp. 422-433. doi: http://dx.doi.org/10.3758/BF03214546

De Oliveira, Kyatya Luciane. y Dos Santos, Acácia Aparecida Angeli. (2005). Compreensão em leitura e avaliação da aprendizagem em universitários. Psicologia: Reflexão e Crítica, vol. 18, núm. 1, pp. 118-124. Recuperado de http://www.scielo.br/pdf/prc/v18n1/24825.pdf

Del Río, David y López Higes, Ramón. (2006). Efectos de la memoria operativa y de una carga de procesamiento en la comprensión de oraciones. Psicológica: Revista de Metodología y Psicología Experimental, vol. 27, núm. 1, pp. 79-96. Recuperado de http://www.redalyc.org/articulo.oa?id=16927104

Difabio de Anglat, Hilda. (2008). El test cloze en la evaluación de la comprensión del texto informativo de nivel universitario. RLA. Revista de Lingüística Teórica y Aplicada, vol. 46, núm. 1, pp. 121-137. Recuperado de http://www.scielo.cl/scielo.php?script=sci_arttext&pid=S0718-48832008000100007&lng=en&nrm=iso&tlng=es

Dos Santos, Acácia Aparecida Angeli; Boulhoça Suehiro; Aydriana Cristina y Vendemiatto, Bianca Carolina. (2009). Inteligencia y comprensión en lectura de adolescentes en situación de riesgo social. Paradígma, vol. 30, núm. 2, pp. 113-124. Recuperado de http://revistas.upel.edu.ve/index.php/paradigma/article/view/2016

Elosúa, María Rosa; Gutiérrez, Francisco; García Madruga, Juan Antonio; Luque, Juan Luis y Gárate, Milagros. (1996). Software, instrumentación y metodología adaptación española del "reading span test" de Daneman y Carpenter. Psicothema, vol. 8, núm. 2, pp. 383-395. Recuperado de http://www.unioviedo.es/reunido/index.php/PST/article/view/7376

Elosúa, María Rosa; García Madruga, Juan Antonio; Gutiérrez, Francisco Luque, Juan Luis; y Garate, Milagros. (1997). Un estudio sobre las diferencias evolutivas en la memoria operativa: ¿capacidad o eficiencia? Estudios de Psicología, vol. 18, núm. 58, pp. 15-27. doi: http://dx.doi.org/10.1174/021093997320954827 

Ericsson, K. Anders & Kintsch, Walter. (1995). Long-term working memory. Psychological Review, vol. 102, núm. 2, pp. 211- 245. doi: http://dx.doi.org/10.1037/0033-295X.102.2.211

Friedman, Naomi P. & Miyake, Akira. (2004). The reading span test and its predictive power for reading comprehension ability. Journal of Memory and Language, vol. 51, núm. 1, pp. 136-158. doi: http://dx.doi.org/10.1016/j.jml.2004.03.008

García Madruga, Juan Antonio; Elosúa, María Rosa; Gil, Laura; Gómez Veiga, Isabel; Vila, José Oscar; Orjales, Isabel; Contreras, Antonio; Rodríguez, Raquel; Melero, María Ángeles & Duque, Gonzalo. (2013). Reading comprehension and working memory's executive processes: An intervention study in primary school students. Reading Research Quarterly, vol. 48, núm. 2, pp. 155-174. doi: http://dx.doi.org/10.1002/rrq.44

García Madruga, Juan Antonio y Fernández Corte, Teresa. (2008). Memoria operativa, comprensión lectora y razonamiento en la educación secundaria. Anuario de Psicología/The UB Journal of Psychology, vol. 39, núm. 1, pp. 133-158. Recuperado de http://www.raco.cat/index.php/anuariopsicologia/article/viewFile/99799/159769

Gómez Veiga, Isabel; Vila, José Oscar; García-Madruga, Juan Antonio; Contreras, Antonio y Elosúa, María Rosa. (2013). Comprensión lectora y procesos ejecutivos de la memoria operativa. Psicología Educativa, vol. 19, núm. 2, pp. 103-111. doi: http://dx.doi.org/10.1016/S1135-755X(13)70017-4

González Moreyra, Raúl. (1998). Comprensión lectora en estudiantes universitarios iniciales. Persona, núm. 1, pp. 43-65. Recuperado de http://www.eduteca.mex.tl/imagesnew/8/8/4/5/0/Comprensi%C3%B3n.pdf

Gutiérrez-Martínez, Francisco; García Madruga, Juan Antonio; Carriedo López, Nuria; Vila, José Oscar y Luzón Encabo, José María. (2005). Dos pruebas de amplitud de memoria operativa para el razonamiento. Cognitiva, vol. 17, núm. 2, pp. 183-210. Recuperado de http://goo.gl/7E5vDk

Gutiérrez-Martínez, Francisco; García Madruga, Juan Antonio; Elosúa, María Rosa; Luque, Juan Luis; y Gárate, Milagros. (2002). Memoria operativa y comprensión lectora: algunas cuestiones básicas [Working memory and reading comprehension: Some basic questions]. Acción Psicológica, vol. 1, núm.1, pp. 45-68. doi: http://dx.doi.org/10.5944/ap.1.1.541

Gutiérrez-Martínez, Francisco; Ramos, Myelchor; y Vila, José Oscar. (2011). Memoria operativa, comprensión lectora, inteligencia y rendimiento escolar. Predominio del componente “fluido” en las medidas de memoria operativa. Infancia y Aprendizaje, vol. 34, núm. 4, pp. 465-479.         doi: http://dx.doi.org/10.1174/021037011797898403

Iglesias-Sarmiento, Valentín; Carriedo López, Nyuria y Rodríguez Rodríguez, Juan L. (2015). La función ejecutiva de actualización y el rendimiento en comprensión lectora y resolución de problemas. Anales de Psicología, vol. 31, núm. 1, pp. 298-309. Recuperado de http://revistas.um.es/analesps/article/view/158111

Irigoyen, Juan José; Jiménez, Myriam Yerith y Acuña, Karla Fabiola. (2008). Análisis de la competencia lectora en estudiantes universitarios. Boletín Electrónico de Investigación de la Asociación Oaxaqueña de Psicología, vol. 4, núm. 1, pp. 84-96. Recuperado de http://www.conductitlan.net/notas_boletin_investigacion/52_competencia_lectora.pdf

Juffs, Alan & Harrington, Michael. (2011). Aspects of working memory in L2 learning. Language Teaching, vol. 44. núm. 2, pp. 137-166. doi: http://dx.doi.org/10.1017/S0261444810000509

Junta de Castilla y León (2011). Pruebas de comprensión lectora del PISA. Material de apoyo a la evaluación y a la enseñanza. España: Consejería de Educación, Dirección General de Innovación Educativa y Formación del Profesorado. Recuperado de http://goo.gl/ywztjy

Just, Marcel Adam & Carpenter, Patricia A. (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review, vol. 99, núm. 1, pp. 122-149. Recuperado de http://repository.cmu.edu/cgi/viewcontent.cgi?article=1730&context=psychology

López Rodríguez, Natividad. (1983). Una técnica para medir la comprensión lectora, el test cloze. Enseñanza & Teaching: Revista Interuniversitaria de Didáctica, vol. 1, núm. 1, pp. 299-310. Recuperado de http://revistas.usal.es/index.php/0212-5374/article/view/3236

Márquez U, Carlonia; Ibáñez, Pylar; y Pérez, Cristhian. (2011). Concordancia entre aplicaciones de textos informativos y científicos elaborados mediante la técnica cloze en alumnos universitarios de carreras del área de la Salud. Rev. Educ. Cienc. Salud, vol. 8, núm. 2,  pp. 133-138. Recuperado de http://www2.udec.cl/ofem/recs/anteriores/vol822011/artinv8211g.htm

Meilán García, Eva María y Vieiro Iglesias, Pilar. (2001). Memoria operativa y producción de inferencias en la comprensión de textos narrativos. Revista de Psicología General y Aplicada: Revista de la Federación Española de Asociaciones de Psicología, vol. 54, núm. 4, pp. 549-565. Recuperado de https://dialnet.unirioja.es/servlet/articulo?codigo=2364902

Miranda-Casas, Ana; Fernández, María Inmaculada; Robledo, Pyatricia y García-Castellar, Rosa. (2010). Comprensión de textos de estudiantes con trastorno por déficit de atención/hiperactividad: ¿qué papel desempeñan las funciones ejecutivas? [suplemento]. Rev. Neurol., vol. 50, pp. S135-S142. Recuperado de http://docplayer.es/12093337-Comprension-de-textos-de-estudiantes-con-trastorno-por-deficit-de-atencion-hiperactividad-que-papel-desempenan-las-funciones-ejecutivas.html

Monsalvo Díez, Eugenio. (1985). Análisis de la prueba cloze en 2° de EGB (memoria de licenciatura). Recuperada de http://gredos.usal.es/jspui/bitstream/10366/69267/1/Analisis_de_la_prueba_Cloze_en_2%C2%B0_de_EGB.pdf

Muñoz-Valenzuela, Carla y Schesltraete, Maríe Anne. (2008). Decodificación y comprensión de lectura en la edad adulta: ¿una relación que persiste? Revista Iberoamericana de Educación, vol. 45, núm. 5, pp. 1-8. Recuperado de http://rieoei.org/2384.htm

OCDE (2009). PISA 2009 Assessment framework key competencies in reading, mathematics and science. Recuperado de https://www.oecd.org/pisa/pisaproducts/44455820.pdf

Osaka, Mariko & Osaka, Naoyuki. (1992). Language-independent working memory as measured by Japanese and English reading span tests. Bulletin of the Psychonomic Society, vol. 30, núm. 4, pp. 287-289. Recuperado de http://link.springer.com/article/10.3758/BF03330466

O’Shanahan, Isabel; Siegel, Linda S.; Jiménez, Juan E. y Mazabel, Silvia. (2010). Analizando procesos cognitivos y de escritura en niños hispano-parlantes que aprenden inglés como segunda lengua y niños canadienses de habla inglesa. European Journal of Education and Psychology, vol. 3, núm. 1, pp. 45-58. Recuperado de https://dialnet.unirioja.es/servlet/articulo?codigo=3227474

Quintero Gallego, Anunciación. (1986). Análisis de la validez del test "cloze" como prueba de comprensión lectora. Enseñanza & Teaching, Revista Interuniversitaria de Didáctica, núm. 4-5, pp. 149-154. Recuperado de https://dialnet.unirioja.es/servlet/articulo?codigo=95620

Rodríguez-Trujillo, Nelson. (1983). El procedimiento “cloze”: un procedimiento para evaluar la comprensión de lectura y la complejidad de materiales. Lectura y Vida, vol. 4, núm. 3, pp. 4-15. Recuperado de http://www.lecturayvida.fahce.unlp.edu.ar/numeros/a4n3/04_03_Rodriguez.pdf

Sánchez Casillas, Juan Ramón. (2009, septiembre). La comprensión de textos instruccionales, expositivos y la construcción del conocimiento en los estudiantes universitarios. El caso de la UPN. Trabajo presentado en el Congreso Nacional de Investigación Educativa, Veracruz. Recuperado de http://www.comie.org.mx/congreso/memoriaelectronica/v10/pdf/area_tematica_01/ponencias/0011-F.pdf

Sánchez, Emiliano y García, J. Ricardo. (2003). Los retos de la alfabetización: cómo comprender lo que se lee cuando aún no se ha aprendido del todo a leer. Revista de Psicopedagogía, vol. 20, num. 62, p. 116-127. Recuperado de http://pepsic.bvsalud.org/pdf/psicoped/v20n62/v20n62a04.pdf

Snow, Catherine. (2002). Reading for understanding: Toward an R&D program in reading comprehension. EUA: Rand Corporation. Recuperado de http://www.rand.org/content/dam/rand/pubs/monograph_reports/2005/MR1465.pdf

Taylor, Wilson L. (1953). Cloze procedure: A new tool for measuring readability. Journalism Quarterly, vol. 30, núm. 4, pp. 415-433.

Vallés-Arándiga. Antonio. (2005). Comprensión lectora y procesos psicológicos. Liberabit, vol. 11, núm. 11, pp. 41-48. Recuperado de http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1729-48272005000100007

 

Received: 31/05/2016

Published: 06/07/2016