Abstract
Several studies have indicated that ESL students in China are weak in answering comprehension questions which are based on Higher Order Thinking Skills (HOTS). The objectives of this study were to investigate students’ overall performance in answering LOTS and HOTS ESL comprehension questions, to compare students LOTS and HOTS comprehension question answering skills according to gender and to gauge teachers’ views on how to improve students LOTS and HOTS ESL comprehension skills. The LOTS and HOTS comprehension questions in this study were based on Bloom’s taxonomy. This descriptive study used the QUAN-qual model, whereby the quantitative data was collected using a comprehension test and the qualitative data was collected through interviews via SKYPE. The sample of this study consisted of 100 ESL third year high school students from Urumqi, China and their three ESL teachers. The data collected from the comprehension test was analyzed using the independent samples t-test and paired samples t-test (SPSS Program for Windows). The qualitative data was analyzed based on emerging themes. The findings revealed that the students performed significantly better at answering LOTS questions compared with HOTS questions. The findings also indicated that female students scored significantly higher in answering LOTS and HOTS comprehension questions compared with their male counterparts. The analysis of teacher interviews indicated that Chinese students were able to answer LOTS questions but they struggled to answer HOTS questions because they had limited vocabulary, lacked critical thinking skills and failed to use the right strategies.
Keywords: ESL reading comprehension, HOTS, LOTS, Chinese students, Teachers’ views.
Received: 8 February 2019 / Revised: 13 March 2019 / Accepted: 18 April 2019/ Published: 2 July 2019
Contribution/ Originality
This study contributes in the existing literature of HOTS and LOTS ESL comprehension among students in China. This study has strong pedagogical implications as it serves as a guide to teachers on how to improve students’ HOTS comprehension. In addition, the study also has theoretical and practical implications.
1. INTRODUCTION
With the advent of economic globalization, English is increasingly considered a global language; the language of communication used by people all over the world. In 2001 and 2008, China joined the WTC and hosted the Olympic Games. Since then, English has gradually become an important language associated with the Chinese people’s daily lives (Hu, 2009). According to a China Daily article in 2010, around 400 million Chinese people are learning English today; the number is approximately one third the population of China.
English is taught as a second language in China, and it is taught in public schools from Grade 1 to university level. The four basic skills (listening, speaking, reading and writing) are being taught in ESL classes. Among the four skills, reading is the most important skill for second language learners to acquire information and knowledge (Cheng and Good, 2009). In China teachers’ pay much attention to improving students’ reading comprehension skills because it plays a large part in the National College Entrance Examination and entrance to the university.
In 2001, a new National English Curriculum Standard was proposed by the Education Ministry of China. It emphasizes the development of students’ comprehensive language abilities through language application (Hu, 2009). The new curriculum requires ESL teachers to not only train students in Lower Order Thinking skills (LOTS) but also in the higher order thinking skills (HOTS), especially in reading comprehension. Compared with LOTS questions, HOTS questions are more difficult and challenging for Chinese students because it requires students to think critically and make inferences.
In addition, in China’s ESL classrooms today, many high school teachers still use the traditional method to teach English reading comprehension. Generally most of the ESL classes are teacher-centered in instruction methods, with the focus being on learning and practicing vocabulary and grammar. Therefore, such teaching environments are not favorable when developing students’ reading comprehension skills utilizing HOTS. According to Pei, quoted in Seliger (1972) during comprehension lessons in ESL classrooms, Chinese students tend to overemphasize the importance of decoding every single word in the reading passage, by translating it into Mandarin; thereby often separating the whole content into isolated pieces and focusing their attention on the individual units. Wu and Wang (2006) indicated that Chinese students’ reliance on their first language is one of the main factors which affect their ESL reading proficiency.
Students’ schema also plays an important role in ESL reading comprehension (Rumelhart, 1980). Unfortunately, many teachers do not see the importance of enriching students’ schema when teaching ESL comprehension. As such, due to lack of prior knowledge, students fail in ESL comprehension. Wang (2016) also stressed that Chinese students lack background knowledge and cultural understanding of the subject matter in ESL reading texts, thus directly affecting their reading comprehension. Yorio (1971) stated that the second language reader’s knowledge of the foreign language is not like that of the native speaker, therefore the guessing or predicting ability is hindered due to lack of prior knowledge related to the text. Therefore, teachers should enrich students’ background knowledge or schema related to the text to facilitate them in understanding the text.
One of the main problems affecting Chinese students’ reading comprehension is teachers’ teaching strategies in ESL reading classrooms. Many researchers criticize the fact that teachers still use the grammar-translation method which is teacher- centred rather than employing ESL reading strategies which are student-centred in ESL comprehension classes (Wang and Xing, 1993; Zhang, 2004).
Several studies have indicated that girls have higher ESL reading achievement compared with boys (Dolores, 2013). In China the girls outperform the boys in reading comprehension (OECD, 2015). With regards to ESL reading comprehension among students in China, top performers were able to retrieve information that required them to locate and organize several pieces of deeply embedded information from the text, but the low performers struggled even to recognize the main ideas in the text (OECD, 2015).
Lately, numerous researchers have studied students’ attitude towards reading comprehension (Smith and Wilhelm, 2002) and students’ reading strategies in ESL reading comprehension. However, only a few researchers have investigated students’ performance in HOTS and LOTS comprehension questions. With regard to changes in the ESL curriculum in China (2001) the difficulty level of ESL reading comprehension is increasing because students are now required to answer HOTS questions.
As shown through the literature review, few researchers have specifically aimed at investigating gender differences in ESL comprehension among Chinese students. Studies conducted on Chinese students’ achievement in HOTS and LOTS comprehension is also very limited. As such the current study seeks to shed more light on this area by investigating Chinese students’ ability to apply HOTS and LOTS in ESL comprehension based on revised Bloom’s taxonomy (Anderson and Krathwohl, 2001). The study also compares male and female students’ ability to apply HOTS and LOTS in comprehension. In addition, the study also explores teachers’ views on how to improve students’ ESL comprehension.
1.1. Research Objectives
The following are the research objectives of this study:
1. To investigate whether there is a significant difference between the comprehension of LOTS and HOTS questions among students.
2. To investigate whether there is a significant difference between the male and female students in their mean score in LOTS comprehension question.
3. To investigate whether there is a significant difference between the male and female students in their mean score in HOTS comprehension question.
4. To investigate the teacher’s perceptions on how to improve students’ ESL reading comprehension.
1.2. Research Questions
Based on the research objectives the following five research questions are formulated:
1. Is there a significant difference in students’ comprehension of LOTS and HOTS questions?
2. Is there a significant difference between the male and female students in their mean scores in LOTS comprehension questions?
3. Is there a significant difference between the male and female students in their mean scores in HOTS comprehension question?
4. What are the teachers’ views on how to improve students’ reading comprehension?
2. REVIEW OF LITERATURE
Bloom’s taxonomy was developed by Benjamin Bloom and his associates in 1956. There are six categories from the lowest to the highest level questions in Bloom’s taxonomy (Bloom, 1956). At the knowledge level, students are required to remember or retrieve previous learned material. At the second level, (comprehension) students need to grasp or construct meaning from material. In relation to the third level (application) students are required to utilize the learned material or to implement material in a new and concrete situation. The forth level (analysis) requires students to break down or distinguish the parts of material into its components. Synthesis is the second highest level in Bloom’s taxonomy. This level requires students to piece parts together to form a coherent or unique new whole. The highest order in Bloom’s taxonomy is evaluation which requires students to judge, check and critique the value of the material.
Bloom’s taxonomy was widely utilized as the model for designing low level and high level questions in the field of education. Marzano (2006) stressed that many students are unable to answer higher level questions which require them to make application, analysis synthesis and evaluation. In 1984, a conference was held to discuss the solution towards students’ poor performance in higher level thinking tasks. They decided Bloom’s taxonomy should be revised (Marzano, 2006). Therefore, Bloom’s taxonomy was further revised by Anderson and Krathwohl (2001) and the new taxonomy is known as Bloom’s Revised Taxonomy Figure 2.1.
Figure-2.1. Revised Bloom’s Taxonomy.
Anderson and Krathwohl (2001).
The lowest level in Bloom’s revised taxonomy is remembering which requires students to recognize and recall the relevant knowledge from long-term memory. The key words used for questions at this level are: define, duplicate, list, memorize, repeat, and reproduce. The next level is known as understanding which requires students to construct meaning from oral, written and graphic messages. The key words used in the questions of this level are interpret, exemplify, classify, summarize, infer, compare, explain, paraphrase and discuss. The third level is known as applying which requires students to use information in new ways. The key words used in the questions are demonstrate, dramatize, interpret, solve, use, illustrate, convert, discover, discuss and prepare.
The fourth level in the revised Bloom’s taxonomy is analyzing. Questions at this level require students to distinguish between parts and study how they relate to each other and to the overall structure and purpose. The key words used at this level are compare, contrast, criticize, differentiate, discriminate, question, classify, distinguish, and experiment. The fifth level is known as evaluating which requires students to make judgements and justify decisions. The key words at this level are appraise, argue, defend, judge, select, support, evaluate, debate, measure, test, and verify. The highest level in this taxonomy is the skill of creating which requires students to put elements together to form a functional whole to create a new product or point of view. The key words used in this level are assemble, generate, construct, design, develop, formulate, rearrange, rewrite and devise.
Bloom’s revised taxonomy has been widely used in education. There are six categories of thinking skills in Bloom’s taxonomy: remembering, understanding and applying are lower order thinking skills (LOTS) while analyzing, evaluating, and creating are higher order thinking skills (HOTS). Compared with the original Bloom’s taxonomy, the revised Bloom’s taxonomy describes each category as the verb and arranges them from lower order thinking skills to higher order thinking skills. It helps educators identify the intellectual level which students are capable of working with and it also develops students’ higher order thinking skills gradually.
In the 21st century, the development and changes in economy and technology require workers to master decision-making and collaborative problem-solving skills (Nuramah et al., 2016). As such, these skills should be taught to students across the curriculum using Bloom’s taxonomy. Current study used Bloom’s revised taxonomy (Anderson and Krathwohl, 2001) to gauge students’ level of LOTS and HOTS comprehension.
2.1. Reading Comprehension
Reading comprehension is one of the important skills in ESL learning. Bernhardt (1987) stressed that reading comprehension is the process of relating to a learner’s background knowledge. Hoover and Gough (1990) assumed that if the readers could decode the words on the page, they would be able to monitor what was being read to understand what they were reading. However, this assumption was disagreed upon by many contemporary researches. Hammerberg (2004) stated that reading comprehension is more than decoding the words or saying them aloud in your head. Instead, it is a complex process which requires readers to employ the reading strategies to come up with a meaningful understanding of the input (Lin, 2010).
In line with traditional views, comprehension is a major skill consisting of four main types of comprehension sub-skills. The four sub-skills are literal comprehension, inferential comprehension, evaluative comprehension, and critical comprehension (Swaby, 1989). With reference to Bloom (1956) literal comprehension is a lower order thinking skill which is also known as LOTS in revised Bloom’s taxonomy (Anderson and Krathwohl, 2001). Comprehension questions at this level only require the reader to understand ideas and information explicitly from the text. The reader may recall facts, ideas and the sequence to know the word meanings. Inferential comprehension requires the reader to combine literal knowledge and prior knowledge to make inferences about things not explicitly stated in the text. This skill involves making predications, identifying cause and effect relationships, contrasting, explaining, and drawing conclusions (Swaby, 1989). Evaluative comprehension requires readers to generate their own opinions based on the information in the text. Readers are required to make judgments and decide whether to reject or adapt the ideas. Critical comprehension requires the reader to analyze the form, term and content in the text. Readers are required to distinguish fact from opinion, distinguish the literary forms and recognize the mood and intention of the author (Swaby, 1989). Analyzing, evaluating, and creating are HOTS level questions in revised Bloom’s taxonomy (Anderson and Krathwohl, 2001).
2.2. Theories of Reading Comprehension
There a several theories related to reading comprehension. Among them, the Bottom-up Theory, Top-Down Theory, Interactive Theory, and Schema Theory are well-known theories in reading comprehension.
The Bottom-Up Theory basically assumes that the reader decodes the letters in the words linearly in order to construct meaning in the text (Hudson, 2007). The reader reads the text from left to right, letter by letter (Gough, 1972). The bottom-up approach is a low-level reading process which has some shortcomings. For example, it focuses on a reader’s ability at decoding or recognizing words. It is time consuming and prevents the reader from understanding the meaning of the whole text.
The Top-Down Theory on the other hand is just the opposite. The bottom-up approach is activated by specific data from the text, while the top-down approach starts with the general to confirm the predications. The top-down approach is a high level reading process because the reader does not necessarily read the text word by word; instead the reader has to make predictions about the incoming information. The Top-Down Theory stresses on readers’ background knowledge. Smith (1994) stated that knowledge of relevant schemes is obviously essential if we are to read and comprehend any text. Therefore, the readers have to apply background knowledge in order to create meaning in the text. Goodman (1976) was of the view that reading is a psycholinguistic guessing game. This is because while reading, the readers use their knowledge of syntax and semantics to reduce their dependence on the print and phonics of the text.
The Interactive Theory views reading as an interactive process; interaction occurs at three levels. First is the interaction between top-down and bottom-up processing; second, the interaction between low-level skills and high-level skills and third, the interaction between reader’s background knowledge and the information in the text (An, 2013). There are some important features related to the Interactive Theory, that is, the automatic recognition of words using lower-level comprehension skills (Gough, 1972; Ehri, 1995) and the interaction of background knowledge and text (Hudson, 1982; Carrell, 1983; Anderson and Pearson, 1984). The Interactive Theory assumes that each process works parallel with the other. A reader will fail to give meaning to the text if he does not have the background knowledge (Rumelhart, 1980).
The Schema Theory thus explains that the knowledge we store is recalled from long-term memory (McVee et al., 2005). Rumelhart (1980) stated that the schema theory explains how readers use prior knowledge to comprehend and learn from the text. In the learning process, learners cannot only rely on the words in the passage (bottom-up processing) but also use their background knowledge (top-down processing) to build all the elements to enable comprehension. Brown (2001) also stressed the importance of schema in reading comprehension because with regard to reading, a text does not carry any meaning; the reader gives meaning to the text based on his prior knowledge.
2.3. Studies Related to Comprehension
Various studies have investigated gender differences in L2 reading comprehension. Bernhardt (2003) claims that second language reading is affected by a 20% first language literacy, 30% second language knowledge and the rest of the 50% includes background knowledge and gender factors. Male and female students possess different schemata, experience, motivation and knowledge while engaging in reading comprehension. That is the reason why many researchers pay much attention to the relationships of gender and background knowledge in reading comprehension (Jalilehvand and Samuel, 2014).
In recent years, many researchers are devoting time to research students’ ability to apply LOTS and HOTS in reading comprehension. Nuramah et al. (2016) conducted a research to examine Thai students’ EFL reading comprehension. The participants were 60 third-year college students in Pattani, Southern Thailand. Six reading comprehension passages were provided to the students. The students; were required to answer six questions on each passage (four LOTS questions and two HOTS questions from each passage). The results revealed that students performed significantly better at answering LOTS questions (M=76.94) compared with HOTS questions (M=51.53).
A study by Jeyamahla et al. (2010) also used Bloom’s Taxonomy to gauge students’ ability at reading comprehension. The participants were 50 diploma students. They used a reading comprehension test to collect data and the test consisted of 35 multiple-choice-type questions. The results revealed that 66% of the students were able to answer knowledge-level questions; 57.2% of the students were able to respond to comprehension-level questions; 47% of the students managed to answer application-level questions; 48.4% of the students were able to answer the analysis-level questions; 33.6% of the students were able to respond to synthesis-level questions and only 28% of the students were able to answer evaluation-level questions correctly. The data showed students faced difficulties in answering HOTS questions. The findings also indicated that it is vital to teach students effective reading strategies in the ESL classroom.
According to a OECD (2015) top performers in reading can retrieve information in the text and low performers struggle at recognizing main ideas in the text. On an average, across the OECD countries, 80% of students have achieved a baseline level of reading proficiency while only 8.3% of the students have achieved a higher level reading proficiency. A PISA (2012) stated that 14% of boys and 9% of girls did not reach the OECD average achievement in reading comprehension.
Many studies show that female students achieve better scores than male students in reading comprehension. Arellano (2013) conducted a research to examine the level of girls’ and boys’ achievements in reading comprehension. A total of 141 students in Spain, including 72 girls and 69 boys of a secondary school took part in this research. The researcher selected eight texts and used open-ended questions to test students’ comprehension ability. The results revealed the girls’ average score for answering reading comprehension questions was 2.7 while the boys’ average score was 2.34. As such, the female students’ overall ESL reading comprehension score was higher than that of the males. The researcher also stated that the female students achieved better scores than the boys in HOTS questions, such as in deducing meaning from the text and understanding text structure. The female students also performed better in lower order thinking skill questions, such as in getting specific information.
Anantasa (2016) conducted a case study at the teacher training faculty of Iain Syekh Nurjati, Cirebon to investigate gender difference in reading comprehension achievement. 12 girls and 12 boys participated in the reading comprehension test. The result showed that 47% of the girls could answer the comprehension questions while only 38% of the boys could answer the comprehension questions. Therefore, overall the girls performed better at reading comprehension than the boys.
On the other hand, there are researchers such as Yazdanpanah (2007) and Al-Shumaimeri (2005) who have stressed that text topic familiarity and gender difference do not affect students’ performance in reading comprehension. However, Brantmeier (2004) and Bügel and Buunk (1996) stressed that different types of texts and topics together with students’ familiarity with content may affect their achievement.
Bügel and Buunk (1996) conducted a research on the impact of passage topics on gender difference. The participants were 2980 high school students in Netherlands. The researchers selected 11 different English reading passages, including 5 ‘male’ passage topics and six ‘female’ passage topics. The results indicated that male students scored better on the multiple-choice type comprehension questions based on the passage of laser thermometers, volcanoes, cars and football players. On the other hand, the female students achieved a higher score on the passages based on midwives, a sad story and a housewife’s dilemma. Similarly a study by Brantmeier (2004) showed that female students obtained higher comprehension scores in multiple-choice-type questions than male students and female students were able to recall more ideas in the written recall task. In relation to Bloom’s taxonomy, recall questions (remembering) are LOTS questions; therefore, this research implies female students perform better at LOTS questions than male students.
Al-Shumaimeri (2005) examined gender difference in reading and comprehension in relation to content familiarity of gender-neutral texts. The participants were 66 male and 66 female undergraduate English language students at King Saud University in Riyadh. Two types of passages (familiar and unfamiliar) were selected. The researcher randomly divided the students into two groups and conducted a reading comprehension test. One group had a familiar passage and another group had an unfamiliar passage. Each passage had 10 multiple-choice type questions. The results showed that male students outperformed (M= 7.455) their counterparts (M= 6.390) in both reading comprehension tests. Meanwhile, the findings also indicated that there was no relationship between gender and content familiarity in L2 reading comprehension performance. Al-Shumaimeri (2005) stated that high-ability students performed better in both familiar and unfamiliar passages while low-ability students performed better in the tests on familiar passages. Therefore, content familiarity facilitated students’ comprehension.
In a study by Yazdanpanah (2007) two ‘male’ topic texts and one neutral topic text were given to 187 participants in North Cyprus. The finding revealed that males and females performed differently on different items. The males outperformed the females in reading for specific information, identifying referential information and matching titles with paragraphs. On the other hand, females scored higher on identifying main ideas, guessing the meaning from context and text coherence questions. This study showed that female students achieved better scores in the higher order thinking skill (HOTS) questions, whereas male students outperformed the female students in the lower order thinking skills (LOTS) comprehension questions.
Linguist Wilkins (1972) stressed the importance of vocabulary. He stated that “without grammar very little can be conveyed to understand a text; without vocabulary nothing can be conveyed to understand a text.” Therefore, in order to better comprehend the text, leaning how to efficiently increase vocabulary is vital. Several studies (Clarke and Nation, 1980; Prince, 1996; Thornbury, 2006; Edwards, 2009) have shown that learning words in context is an effective approach which enables students to improve their vocabulary. Prince (1996) indicated three benefits of learning vocabulary in context. First, it improves students’ reading strategies such as inferencing and anticipating. Second, it draws students’ attention to words that are used in discourse for communication purposes. Finally, context enables students to better understand how to use the words in the right way. Dole et al. (1995) and Grabe (2009) stressed that many words had multiple meanings and looking up the meaning in the dictionary results in a superficial understanding of the words. This does not help students to understand the meaning in the text. Therefore, students cannot simply select the meaning from the dictionary without referring to the context.
Dole et al. (1995) studied the effectiveness of an alternative model. Two groups were established in the study. The first group was taught vocabulary using the traditional method, and the alternative group was learning words within the context of the text. The researchers found that students in the alternative group performed better than their counterparts in the reading comprehension test. The traditional group did not connect the words with the stories they read whereas the alternative group used story information in the sentences.
Masound et al. (2014) conduct a research to examine the authentic materials on vocabulary development. 50 IELTS students were selected from Iran. Three instruments (authentic and non-authentic materials, a questionnaire and observation) methods were used in the study. The researcher found that authentic materials were effective in both learning and teaching processes. The results showed that the majority of students agreed that listening to English songs/news, watching English movies and reading authentic materials enhanced their vocabulary. Furthermore, students enjoyed themselves while using authentic materials to learn vocabulary.
Since 1970, several second language theorists have advocated teaching reading strategies in order to help students improve their reading skills (Barnett, 1988). Allen (2003) stressed that reading strategies efficiently enhance a student’s reading comprehension; without using strategies most readers face difficulties in grasping the meaning of the text.
Amanda (2011) conducted a research in the use of skimming and scanning techniques to improve students’ reading comprehension achievement. The participants were year 8 students in Ambulu Jember. A classroom action research was applied in the study. The results showed that the students made improvement in cycle 2, whereby the students increased the use of skimming and scanning skills. The researcher encouraged the students to use skimming and scanning skills. She stated that skimming helped the students to comprehend general information whereas scanning helped the students to comprehend specific information quickly. These findings were consistent with that of Fatmawati (2014) who found that using skimming and scanning strategies in the classroom minimized students’ boredom while reading.
In relation to learning vocabulary from passages, Nassaji (2004) conducted a study to examine the relationship between ESL learners’ depth of vocabulary knowledge, lexical inferencing strategy used and their success in deriving meaning from context. Participants read a passage containing 10 unknown words and tried to figure out the meaning in context. The results indicated that the students who had stronger depth of vocabulary preferred to use certain reading strategies more frequently than the weaker students. Stronger students used more lexical inferencing strategy than weaker students.
Mohammed (2015) conducted a research using the Think-Aloud strategy to improve English reading comprehension. The participants were in two groups of 23 students each in Saudi Arabia. One group designed as the Control Group was subject to traditional instruction. The other group designed as the Treatment Group used the Think-Aloud strategy. Four assessments were carried out to obtain the data. The results showed that the Treatment Group had significant improvement from a pretest mean score of 1.66 to the posttest mean score of 4.04. The mean score for the Control Group remained almost the same. McKeown and Gentilucci (2007) conducted a research to examine how the Think-Aloud strategy affects reading comprehension in middle school. 27 students were purposefully selected. They were 5 early intermediate students (L2), 11 intermediate students (L3) and 11 early advanced students (L4). Two tests (pretest and posttest) were designed for the students. The results indicated that the Think-Aloud strategy may not appropriate for L2 students as they don’t have the ability to utilize the top-down strategy. The Think-Aloud strategy benefits L3 students but is not suitable for the L4. Therefore, teachers should use the right strategy on the right level of students.
3. METHODOLOGY
The researchers utilized the descriptive design using the QUAN-qual model to collect data in this study. This study applied Purposive Sampling and selected 100 3rd year high school students from a school in Urumqi, China. They comprise 55 male students and 45 female students between the ages of 16 and 18. The researcher also interviewed three ESL teachers from the school to obtain their views on how to enhance students’ HOTS and LOTS in reading comprehension. These three teachers are currently teaching the samples at the high school and they have more than four years of experience in teaching English.
Two types of instruments were utilized in this study, namely a set of semi-structured interview questions and an ESL reading comprehension test. First, the researcher with the help of an ESL teacher conducts a reading comprehension test for students. The test paper includes five different types of passages on the topics of sports, technology, narrative, advertisement and environment. Each passage was around 350 words and contains five multiple-choice-type questions (two HOTS questions and three LOTS questions).
The researchers conducted a pilot study before the data collection, in order to ensure the reliability and validity of the research instruments. A different group of students and ESL teachers were involved in this pilot study. The pilot study consisted of 30 students, 15 male and 15 female students representing the 3rd year high school students in different schools. The purpose of the pilot study was to obtain the reliability of the instrument and to ensure students do not encounter any problems in the reading comprehension passages and to confirm the questions are suitable for their level. The pilot study also verifies the time allocated is enough for the students to complete the reading comprehension test. The researcher also carried out the pilot test for the interview questions to obtain their reliability and to ensure that the interview questions were adequate to gather the data to answer the research question four. In order to obtain the validity of the comprehension test, the researcher asked two senior ESL teachers to verify the test paper in terms of content validity and the language level of the students.
Based on the results of the reliability estimates in the pilot study, the value of KR21 (0.60) and KR20 (0.83) are quite reliable. The researcher also received feedback from the three ESL teachers to prove the reading comprehension questions were reliable and acceptable.
In relation to the procedures, since the researchers were not in China, in the first step the researchers sent emails to obtain permission from the principal of the high school to conduct the research. The ESL teachers helped the researcher to conduct the reading comprehension test within two weeks. The test was marked by an ESL teacher from a different school to avoid biasness. The answers were provided by the researchers to the teacher. Following this, the researchers conducted interviews via skype with the three ESL teachers who were teaching the samples. The interviews were recorded and transcribed for qualitative data analysis. The researcher used emerging themes to analyze the qualitative data.
4. FINDINGS AND DISCUSSION
1. Is there a significant difference in students’ comprehension of LOTS and HOTS questions?
2. Is there a significant difference between the male and female students in their mean scores in LOTS comprehension question?
3. Is there a significant difference between the male and female students in their mean scores in HOTS comprehension question?
4. What are the teachers’ views on how to improve students’ reading comprehension?
Table-4.1. Comparison of the students’ Scores in Comprehension Questions.
Mean |
SD |
Mean Difference |
t value |
df |
p value |
|
LOTS |
88.53 |
8.15 |
44.54 |
38.96 |
2 |
0 |
HOTS |
38.6 |
14.21 |
* Level of significance at p<0.05.
The findings in Table 4.1 show that the students’ mean score for LOTS ESL comprehension questions is higher (Mean=88.53; SD=8.15) than HOTS comprehension questions (Mean= 38.60; SD= 14.21). The results of Paired samples t-test indicate that students scored better for the LOTS questions than the HOTS questions (t= 38.96; df= 2; mean difference= 44.54; p= 0.000). These findings distinctly show that there is a significant difference between the LOTS and HOTS questions in students’ ESL reading comprehension performance. The findings in the current study have revealed that students performed better in the LOTS questions compared with the HOTS questions. This finding is consistent with findings by Jeyamahla et al. (2010) which indicated that students face difficulties when answering HOTS question and performed better in LOTS questions. The current findings also support the study by Nuramah et al. (2016) which concluded that students are able to answer LOTS questions well because the answers are directly stated in the passages. In addition, they stressed that teachers usually ask LOTS comprehension questions more frequently during ESL lessons compared with HOTS questions. As such students are more familiar with LOTS questions and are able to answer them well compared with the HOTS questions. According to a OECD (2015) students are able to answer LOTS questions better because the answers are in the text, but they struggle to answer HOTS questions which require thinking skills. These results further explain why students perform well in LOTS questions and face difficulty in answering HOTS questions (OECD, 2015) .
2. Is there a significant difference between the male and female students in their mean scores in LOTS comprehension questions?
Table-4.2. Comparison of the LOTS ESL comprehension questions according to gender.
Mean |
SD |
Mean Difference |
t value |
df |
p value |
|
Male |
83.03 |
7.68 |
-5.56 |
-3.59 |
2 |
0.001 |
Female |
88.59 |
7.74 |
* Level of significance at p<0.05.
The findings exhibited in Table 4.2 indicate that the female students’ mean scores in LOTS ESL comprehension questions is higher (Mean= 88.59, SD= 7.74) compared with the male students (Mean = 83.03, SD= 7.68). The results of the Independent samples t- test indicate that the female students performed significantly better than the male students’ in LOTS ESL reading comprehension (t= -3.59; df= 98; mean difference= -5.56; p= 0.001). These findings clearly show that there is a significant difference between the male and the female students in their LOTS ESL comprehension performance. As such, these findings answers Research Question 3.
These findings clearly show that the female students outperformed the male students in LOTS comprehension questions. The findings of this study are in line with the findings by Arellano (2013) and Brantmeier (2004) which also showed that female students scored better than male students in answering LOTS questions. Arellano (2013) also stressed that female students face less difficulties in getting specific information, getting general information, understanding textual structure and deducing meaning from context. However, male students can only understand the textual structure but face problems in understanding general information found in the text.
3. Is there a significant difference between the male and female students in their mean scores in HOTS comprehension questions?
Table-4.3. Comparison of the HOTS ESL comprehension questions according to gender.
Mean |
Std deviation |
Mean Difference |
t value |
df |
p value |
|
Male |
33.82 |
12.98 |
-10.63 |
-3.99 |
2 |
0 |
Female |
44.44 |
13.58 |
* Level of significance at p<0.05.
The findings in Table 4.3 show that the female students’ mean scores for HOTS ESL comprehension questions is higher (Mean= 44.44, SD= 13.58) compared with the male students (Mean= 33.82, SD= 12.98). The results of the Independent samples t- test also indicate that the female students performed significantly better than the male students in HOTS ESL reading comprehension (t= -3.99; df= 2; mean difference= -10.63; p= 0.000). These findings affirm that there is a significant difference between the male and the female students in their HOTS ESL comprehension performance.
The current findings are in line with previous findings by Arellano (2013) and Yazdanpanah (2007) which indicated that female students significantly outperformed the male students in answering HOTS comprehension questions such as in identifying main ideas, guessing the meaning in context and text coherence questions. In addition, students’ reading comprehension is also based on their familiarity of the passage. In other words, students’ schemata has a significant effect on their reading comprehension.
4. What are the teachers’ views on how to improve students’ reading comprehension?
An analysis of the data obtained from the teachers’ interviews revealed findings within the areas of interview questions. Three ESL teachers responded to the interview questions. The interviews were transcribed and then categorized for major themes during the analysis (Kendall, 2006). Table 4.4 presents the three interview questions used by the researcher during the Skype interviews.
Table-4.4. Teacher Interview Questions.
| Q1: How do your students perform in HOTS and LOTS ESL comprehension questions? |
| Q2: What are the common problems encountered by students in answering HOTS questions? Please explain why. |
| Q3: How do you help your students to answer HOTS comprehension questions? |
Q1: How do your students perform in HOTS and LOTS comprehension questions?
(Teacher A, female):
“Well… in my class, the students are not good at answering HOTS questions. The question such as summing up and summarizing, make inference and judgment, are the difficult parts for my student. If the students are not careless... I think they will perform very well in LOTS questions, because we practiced LOTS questions every day.”
(Teacher B, female):
“Based on my class observation and reading comprehension test results, my students do well in LOTS questions. However, they are poor in answering HOTS questions….mmm”
(Teacher C, male):
“I think LOTS questions are easy, my students can answer because the answers are in the text. My students find …difficult to answer HOTS questions because they have to think, make inference, judgement and so on. The answers are not in the text.”
From these extracts it can be concluded that the majority of students are poor in answering HOTS comprehension questions, especially questions which require students to summarize the main ideas, make inferences and judgment of text. On the other hand, all the respondents opined that students are able to answer the LOTS questions because answers for the LOTS questions are directly stated in the passage and teachers had already trained students how to answer LOTS questions. In addition students get more practice in answering LOTS questions compared to HOTS questions.
Interview question 2: What are the types of common problems encountered by students in answering HOTS questions? Please explain why.
(Teacher A, Female):
“My students always struggle in answering HOTS questions. I noticed that they find it difficult to make inferences and judgements. Students told me that they understand every word in the passage but still unable to answer HOTS questions. In my opinion, most of them do not refer to the information in the text while making inferences or judgments. Instead some students overemphasized their thoughts and give irrelevant answers.”
(Teacher B, Female):
“Well…I observed students in the class…. most students are confused by the long and complex sentences. They don’t know how to analyze the structure of sentence, and failed to comprehend them. I mean the students don’t understand the complex sentences. When students don’t have prior knowledge related to the text… difficult to understand. Their limited vocabulary is also a hindrance in comprehending the passage and answering HOTS questions. For example, some words in the passage may have two or three meanings, but students only know the literal meaning and failed to understand the underlying meaning.”
(Teacher C, Male):
“My students are poor in answering HOTS questions especially summarizing the main idea. In my opinion, most students don’t have critical thinking ability. They rely too much on the information in the passage and cannot think out of the box. Moreover, answers for the HOTS questions are not stated directly in the passage. For multiple types HOTS questions students tend to choose answers that appear in the passage. They will automatically choose these type of answers because it’s easy for them and they don’t have to think. But the HOTS questions require them to think out of the box.”
The findings of question 3 indicate that students face problems in answering HOTS questions because they fail to refer to the information in the passage while making their inferences and judgements. Some of the students overemphasize their own thoughts and give irrelevant answers. Students also face difficulty to understand the text if they do not have the schema related to the passage. Students’ limited vocabulary is also a hindrance in comprehending the passage and answering HOTS questions. Students also encounter problems in comprehending long and complex sentences in comprehension passages which lead to wrong interpretations. Student inability to understand the literal meanings and underlying meanings of the words is also a hindrance in answering HOTS questions. Besides that, students are also poor in their critical thinking ability which is crucial in answering HOTS questions. According to one of the respondents, students are not able to think out of the box which is crucial in answering HOTS questions.
Interview question 3: How do you help your students to answer HOTS comprehension questions?
(Teacher A, Female):
“As I mentioned before, my students struggle in answering HOTS questions such as making inferences, summary and etc. … in my class I prefer to read a part of the relevant paragraph from the passage and show students some pictures to guide them in making the inferences to answer HOTS questions. For example, after reading the relevant lines, I will questioning my students what, why, and how questions. These will motivate students to think aloud, infer or predict to answer HOTS questions …”
(Teacher B, Female)
“Um… I always conduct an activity called rearrange and expand sentence. Normally I will write few new words on the whiteboard, then I will choose three to five student to complete the task on the whiteboard. This activity well trained students to analyze and organize long complex sentences.” This activity will help them to understand the comprehension passage. If they understand the passage they will be able to make inferences to answer the HOTS questions.
(Teacher C, Male):
“Usually, I will divided students into group, I do this to encourage the students to discuss the HOTS questions…Each group will discuss the answers and present their answers to the class. Students feel more comfortable to argue and express their ideas in groups. They also have to give reasons to support their choice of answers. This will give students opportunity to listen to the different views, and think critically to answer the HOTS questions.
The findings from the interview question 5 indicate that guidance from the teacher is important and helps the students in answering HOTS questions. This motivates students to think aloud, make inferences or predictions to answer the HOTS questions. On the other hand,teacher B stressed that before answering the HOTS questions, it is important for the students to understand the comprehension passage. If they cannot understand the passage they won’t be able to think critically to answer the HOTS questions. In addition, group discussion also helps the students to exchange their views, argue in groups, think critically, find reasons to support their answers and finally answer the HOTS questions. The findings from the qualitative data (teachers’ interview) indicated that students performed better in LOTS comprehension questions compared with the HOTS questions. This finding is similar to and is supported by the quantitative data analysis which drew similar conclusions. The current findings are similar to the findings by Jeyamahla et al. (2010); Nuramah et al. (2016) which stated that students performed better in answering LOTS comprehension questions compared with HOTS comprehension questions.
The findings also indicate that students face problems in answering HOTS questions because they fail to make inferences, make wrong interpretations of complex sentences, are poor in critical thinking ability and cannot think out of the box. In order to help students in answering HOTS questions, teachers opined that they motivate students to think aloud and make inferences. Enriching students’ schema can also facilitate their reading comprehension. Teachers also encourage students to have group discussion to develop students’ critical thinking ability. Group discussion during comprehensionis the best way to scaffold students’ growing meta-cognitive awareness of how to develop main ideas in comprehension and enhance their critical thinking skills (Baker, 2002; Pressley, 2002; Fitzgerald and Graves, 2004; Snow et al., 2005). According to Mohammed (2015) and McKeown and Gentilucci (2007). The think aloud strategy allows teachers and students to work together to construct understanding of the comprehension text.
5. CONCLUSION
Analysis of the quantitative data indicated that Chinese students perform better in answering LOTS questions compared with HOTS questions. Second, the female students significantly out-performed the male students in answering LOTS and HOTS comprehension questions. Analysis of the qualitative data indicated that students’ poor command of vocabulary in English and their limited prior knowledge related to the comprehension passages are contributing factors to their poor scores in ESL comprehension. Students also give wrong interpretations of the passages because they tend to translate the English words into Mandarin. Teacher also mentioned that students need to learn how to use reading strategies to improve their ESL comprehension. In addition, students should be taught various skills such as scanning, skimming, and taught how to make inferences to improve their comprehension. Some teachers opined that group discussion encourages students to think critically and share their views on how to answer HOTS comprehension questions. The findings of this study have strong theoretical, pedagogical and practical implications. In terms of theoretical implications, this study supports Bloom’s Taxonomy which clearly shows HOTS questions are more difficult compared with the LOTS questions. For pedagogical implications, teachers need to utilize the right strategies to help students in answering HOTS comprehension questions. In terms of practical implications, the Ministry of Education in China should conduct training sessions for ESL teachers on how to improve students’ higher order thinking skills, especially in ESL comprehension. There are several limitations to the study. The first limitation is related to the sample size. The sample size is small because only 100 3rd year high school students were involved in the study. As such, the findings can only be generalized to similar samples. The second limitation is related to the types of reading comprehension passages used in the present study. The reading comprehension test consisted of five types of passages (sport, science, narrative, advertisement and environment). Therefore, the findings of this research cannot be generalized to other types of comprehension passages such as reports, history, and culture. The third limitation is related to the types of comprehension questions. The reading comprehension questions were all multiple-choice-type questions. Therefore, the findings of this research cannot be generalized to other types of questions such as open-ended questions.
| Funding: This study received no specific financial support. |
| Competing Interests: The authors declare that they have no competing interests. |
| Contributors/Acknowledgement: All authors contributed equally to the conception and design of the study. |
REFERENCES
Al-Shumaimeri, Y., 2005. Gender differences in reading comprehension performance in relation to content familiarity of gender-neutral texts. Paper Presented at the Second International Conference: Language, Culture and Literature. Minia University, Egypt.
Allen, S., 2003. An analytic comparison of three models of reading strategy instruction. International Review of Applied Linguistics, 41(4): 319-338.Available at: https://doi.org/10.1515/iral.2003.015.
Amanda, C.S., 2011. The use of skimming and scanning techniques to improve thterse reading comprehension achievement of grade 8G students of SMP Negeri 1 Ambulu Jember. Unpublished Masters Thesis, University of Jember, Indonesia.
An, S.Y., 2013. Schema theory in reading. Theory and Practice in Language Studies, 3(1): 130-134.Available at: https://doi.org/10.4304/tpls.3.1.130-134.
Anantasa, J.F., 2016. Gender differences in reading comprehension achievement. ELT Echo, 1(1): 27-41.
Anderson, L.W. and D.R. Krathwohl, 2001. A taxonomy for learning, teaching and assessing: A revision of Bloom's Taxonomy of educational objectives: Complete edition, New York : Longman. pp: 67-68
Anderson, R.C. and P.D. Pearson, 1984. A schema-theoretic view of basic processes in reading comprehension. Handbook of Reading Research, 1: 255-291.
Arellano, M.D.C., 2013. Gender differences in reading comprehension achievement in English as a foreign language in compulsory secondary education. Tejuelo: Didactics of Language and Literature. Education, 17: 67-84.
Baker, M.I., 2002. Reading resistance in middle school: What can be done? Journal of Adolescent & Adult Literacy, 45(5): 364-366.
Barnett, M.A., 1988. Reading through context: How real and perceived strategy use affects L2 comprehension. The Modern Language Journal, 72(2): 150-162.Available at: https://doi.org/10.1111/j.1540-4781.1988.tb04177.x.
Bernhardt, E., 2003. Challenges to reading research from a multilingual world. Reading Research Quarterly, 38(1): 112-117.
Bernhardt, E.B., 1987. Cognitive processes in L2: An examination of reading behaviours. Journal of Memory and Language, 26: 377-391.
Bloom, B.S., 1956. Taxonomy of educational objectives. New York: David McKay Company Inc.
Brantmeier, C., 2004. Gender, violence-oriented passage content, and reading in a second language. The Reading Matrix, 4(2): 1-19.
Brown, H.D., 2001. Teaching by principles: An interactive approach to language pedagogy. White Plains, NY: Addison Wesley Longman, Inc.
Bügel, K. and B.P. Buunk, 1996. Sex differences in foreign language text comprehension: The role of interests and prior knowledge. The Modern Language Journal, 80(1): 15-31.Available at: https://doi.org/10.2307/329055.
Carrell, P.L., 1983. Some issues in studying the role of schemata, or background knowledge, in second language comprehension. Reading in a Foreign Language, 1(2): 81-92.
Cheng, Y.-H. and R.L. Good, 2009. L1 glosses: Effects on EFL learners' reading comprehension and vocabulary retention. Reading in a Foreign Language, 21(2): 119-142.
Clarke, D.F. and I.P. Nation, 1980. Guessing the meanings of words from context: Strategy and techniques. System, 8(3): 211-220.Available at: https://doi.org/10.1016/0346-251x(80)90003-2.
Dole, J.A., C. Sloan and W. Trathen, 1995. Teaching vocabulary within the context of literature. Journal of Reading, 38(6): 452-460.
Dolores, A.M.C., 2013. Gender differences in reading comprehension achievement in English as a foreign language in compulsory second education. Tejuelo, 17: 67-48.
Edwards, P.A., 2009. Tapping the potential of parents: A strategic guide to boosting student achievement through family involvement. New York: Scholastic, Inc.
Ehri, L.C., 1995. Phases of development in learning to read words by sight. Journal of Research in Reading, 18(2): 116-125.Available at: https://doi.org/10.1111/j.1467-9817.1995.tb00077.x.
Fatmawati, Y., 2014. The impact of using skimming and scanning strategies of descriptive text towards students’ reading comprehension at grade 8 of SMPN 22 Bander Lampung. The Second International Conference on Education and Language. pp: 2303-1417.
Fitzgerald, J. and M.F. Graves, 2004. Scaffolding reading experiences for English language learners. Norwood, MA: Christopher-Gordon.
Goodman, K.S., 1976. Reading: A psycholinguistic guessing game. In H. Singer and R. B. Rudell (Eds.), Theoretical Models and Process of Reading. Newark, DE: International Reading Association. pp: 497-508.
Gough, P.B., 1972. One second of reading. In F. Kavanaugh and I. G. Mattingly: Language by Ear and by Eye: the relationship between speech and reading. Cambridge, MA: The MIT Press. pp: 331-358.
Grabe, W., 2009. Reading in a second language: Moving from theory to practice. Cambridge: Cambridge University Press. pp: 70.
Hammerberg, D.D., 2004. Comprehension instruction for socioculturally diverse classrooms: A review of what we know. Reading Teacher, 57(7): 648-661.
Hoover, W.A. and P.B. Gough, 1990. The simple view of reading. Reading and Writing, 2(2): 127-160.Available at: https://doi.org/10.1007/bf00401799.
Hu, R., 2009. English reading instruction in elementary schools in China. Reading Matrix: An International Online Journal, 9(2): 150-165.
Hudson, T., 1982. The effects of induced schemata on the 'short circuit' in L2 reading: Non decoding factors in L2 reading performance. Language Learning, 32(1): 1 31.
Hudson, T., 2007. Teaching second language reading. Oxford: Oxford University Press. pp: 35.
Jalilehvand, M. and M. Samuel, 2014. Content familiarity and gender-neutral texts in foreign language reading comprehension. Malaysian Online Journal of Educational Sciences, 2(3): 1-21.
Jeyamahla, V., M. Chittra, M. Rasaya and S. Angelina, 2010. Using bloom’s taxonomy to gauge students’ reading comprehension performance. Canadian Social Science, 6(3): 205-212.
Kendall, D., 2006. The power of communication: A special day class techer and herstudents’ perceptions of effective communication, lesson efficacy, and teacher-student relationship within a cross cultural framework. Unpublished Master’s Thesis, University of San Francisco, California.
Lin, L.-F., 2010. The impact of the retelling technique on Chinese students’ English reading comprehension. Asian EFL Journal, 12(2): 163-191.
Marzano, R.J., 2006. Preliminary report on the 2004–05 evaluation study of the ASCD program for building academic vocabulary. Alexandria, VA: Association for Supervision and Curriculum Development.
Masound, Z., F.M. Zadark and S.A. Kazemi, 2014. The effects of authentic materials on vocabulary development. International Journal of Language Learning and Applied Linguist World, 5(4): 155-160.
McKeown, R.G. and J.L. Gentilucci, 2007. Think-aloud strategy: Metacognitive development and monitoring comprehension in the middle school second-language classroom. Journal of Adolescent & Adult Literacy, 51(2): 136-147.Available at: https://doi.org/10.1598/jaal.51.2.5.
McVee, M.B., K. Dunsmore and J.R. Gavelek, 2005. Schema theory revisited. Review of Educational Research, 75(4): 531-566.Available at: https://doi.org/10.3102/00346543075004531.
Mohammed, A., 2015. Using think-aloud strategy to improve English reading comprehension for 9th grade students in Saudi Arabia. Masters Thesis, State University of NewYork at Fredonia, USA.
Nassaji, H., 2004. The relationship between depth of vocabulary knowledge and L2 learners’ lexical inferencing strategy use and success. The Canadian Modern Language Review, 61(1): 107-134.Available at: https://doi.org/10.3138/cmlr.61.1.107.
Nuramah, H., S.M. Nair and H.N. Krishnasamy, 2016. Thai students’ L2 reading comprehension level for lower order thinking skills and higher order thinking skills questions. Journal of Applied Linguistics and Language Research, 3(5): 83-91.
OECD, 2015. PISA 2015 results: Excellence and equity in education, 1. Available from https://www.oecd.org/pisa/pisa-2015-results-in-focus.pdf .
PISA, 2012. Results: Which country does best at reading, maths and science? Available from https://www.theguardian.com/news/datablog/2013/dec/03/pisa-results-country-best-reading-maths-science .
Pressley, M., 2002. Reading instruction that works: The case for balanced reading. 2nd Edn., New York: Guilford.
Prince, P., 1996. Second language vocabulary learning: The role of context versus translations as a function of proficiency. The Modern Language Journal, 80(4): 478-493.Available at: https://doi.org/10.2307/329727.
Rumelhart, D.E., 1980. Schemata: the building blocks of cognition. In: R.J. Spiro et al. (eds) Theoretical Issues in Reading Comprehension. Hillsdale, NJ: Lawrence Erlbaum.
Seliger, H.W., 1972. Improving reading speed and comprehension in English as a second language. ELT Journal, 27(1): 48-55.Available at: https://doi.org/10.1093/elt/xxvii.1.48.
Smith, F., 1994. Understanding reading: A psycholinguistic analysis of reading and learning to read. 5th Edn., Hillsdale, N.J: Lawrence Erlbaum Associates.
Smith, M.W. and J. Wilhelm, 2002. Reading don’t fix no chevys: Literacy in the lives of young men. Portsmouth, NH: Heinemann.
Snow, C.E., P. Griffin and M.S. Burns, 2005. Knowledge to support the teaching of reading: Preparing teachers for a changing world. San Francisco, CA: Jossey-Bass.
Swaby, B.E.R., 1989. Diagnosis and correction of reading difficulties. Needham Heights: Allyn and Bacon.
Thornbury, S., 2006. An A-Z of ELT. 1st Edn., New York: Macmillan. pp: 191.
Wang, J.H. and M.J. Xing, 1993. Discourse approach to advanced reading. In F. Konig,Y. Gao, B. K. Li, & D. X. Tang (Eds.), Proceedings of the 1993 International. Symposium on Language Teaching Methodology, Beijing – Hohhot. Cedar Falls, IA: The University of Northern Iowa Press. pp: 97-106.
Wang, Y.-H., 2016. Reading strategy use and comprehension performance of more successful and less successful readers: A think-aloud study. Educational Sciences: Theory and Practice, 16(5): 1789-1813.Available at: https://doi.org/10.12738/estp.2016.5.0116.
Wilkins, D.A., 1972. Linguistics in language teaching. London: Edward Arnold.
Wu, S. and T. Wang, 2006. On transfer of reading skills within the structure building framework from psycho-cognitive perspectives [J]. Foreign Language Teaching and Research, 2: 122-128.
Yazdanpanah, K., 2007. The effect of background knowledge and reading comprehension test items on male and female performance. The Reading Matrix, 7(2): 64-80.
Yorio, C.A., 1971. Some sources of reading problems for foreign-language learners 1. Language Learning, 21(1): 107-115.Available at: https://doi.org/10.1111/j.1467-1770.1971.tb00494.x.
Zhang, R., 2004. Using the principles of Exploratory Practice to guide group work in an extensive reading class in China. Language Teaching Research, 8(3): 331-345.Available at: https://doi.org/10.1191/1362168804lr142xx.
Views and opinions expressed in this article are the views and opinions of the author(s), Humanities and Social Sciences Letters shall not be responsible or answerable for any loss, damage or liability etc. caused in relation to/arising out of the use of the content. |