In this study, the effect of problem-based learning on students' problem-solving self-efficacy through the Blackboard system was investigated. The students’ perception of the implementation of problem-based learning into higher education and discussion through blackboard was also investigated. The participants of the study were 31 students in higher education, studying in the Curriculum and Instruction Department at Umm Al-Qura University. It is a quasi-experimental study, designed with a single group, pre-test, and post-test design. An open-ended questionnaire was used to investigate in-depth the students' perception. Both quantitative and qualitative data were obtained. The finding of the study reveals that there is a significant relationship between the pre-and-post-tests of the problem-based self-efficacy measurement scale. Further, the participants perceived advantages of using problem-based learning into their knowledge as well as into their learning and social skills. They reported that the Blackboard system had many benefits to complete the problem-solving activities. However, many students reported challenges in the implementation of problem-based learning as well as there were difficulties in using the blackboard system to achieve the raised problems.
Keywords: Blackboard, Higher education, Problem-based learning, Problem-solving, Self-efficacy.
Received: 17 July 2020 / Revised: 8 January 2021 / Accepted: 19 February 2021/ Published: 15 March 2021
This study is one of very few studies which have investigated both Problem-based Learning and Problem-solving self-efficacy in higher education. The study uses a new formula of implementing problem-solving strategy into the Blackboard system to find out its effect of students’ problem-solving self-efficacy.
The purpose of this study is to investigate the effect of problem-based learning (PBL) on students' problem-solving self-efficacy and to find out their perceptions of the advantages and challenges of implementing PBL into learning through the Blackboard (BB) system. Problem-solving is a cognitive process in which knowledge and different skills such as searching, analyzing, discussing, and evaluating work together to identify the problem and resolve the issue (Udeani & Adeyemo, 2011; Wang & Chiew, 2010). Since the higher education system has higher academic standard, it requires students to use different strategies to achieve their learning efficiently. In PBL, higher education students are given responsibility for their learning, so it encourages independence in learning. Thus, the implementation of PBL in higher education would help in improving students’ learning and reaching to the higher standards.
Instructors in the University are trying to find methods or models to help students improve their learning efficiency. Learners in higher education need to apply theoretical learning in practice. Previous researchers (Gross & Rutland, 2017; Speece, 2002) suggest that applying PBL would make the students realize that there are differences between theory and practice. Learning the theory alone would not be efficient to apply them in social life. Combining theories with real practice would be efficient to help students become confident and apply their learning in real life. Thus, using PBL would be the most suitable model to help students apply theoretical knowledge into real practice. As a result, the role of the instructor in PBL would change to be a facilitator, since they are not the source of information, they facilitate the learning process. In higher education, students' competence is discovered through the learning process. Thus, PBL could help to improve the students’ competence. It could be a challenge for the instructor to implement PBL since it requires the students to be familiar with the process. This is because applying PBL would change the whole system of learning and make swapping roles between the instructor and the students. Therefore, the suitable classroom organization in PBL could be in the form of roundtables and students working on solving problems in groups. This organization would help students to communicate well with other group members.
In the PBL model, students work collaboratively in order to help in identifying the problem and choose a suitable solution. They go through negotiation of meaning and discussing possible solutions. Thus, PBL also helps in improving student’s decision-making skills. Briefly, following five steps that students need to go through during the PBL process are: define the problem, discuss known facts, search-related knowledge, generate possible solutions, discuss consequences, and make the final decision. During the searching process, students are encouraged to discuss their perspectives with other members in the groups since this is very important in deepening their thinking (Fettahlıoğlu & Aydoğdu, 2020). Major and Mulvihill (2018) stated that PBL learning would help students to be active during the learning process and participate in formulating and solving the proposed problems. Further, the PBL could help in creating a dialogical environment during the learning process.
Previous researchers (Hung, 2006) suggested suitable criteria and guidelines in designing the model of the problem to be more effective and reflecting on students thinking. The first criterion is that the topic of the problems should be familiar to students. This would help them to formulate suitable concepts and identify suitable resources to go in-depth analysis of the problems. Further, the topic should be challenging and it should lead students to higher thinking skills (analyze, construct, evaluate) (Brabler, 2016; Dağyar & Demirel, 2015; Major & Mulvihill, 2018; Vandenhouten, Groessl, & Levintova, 2017). This study therefore presents real problems related to this research topic.
However, some researchers have found that PBL model could cause difficulty for some students, particularly if they lacked confidence in doing the in-depth analysis to solve problems (Nijhuis, Segers, & Gijselaers, 2005). Other researchers have stated that PBL would not be efficient if the problem is not challenging. Hence, all students, even those with low-efficacy, could go through the process of problem evaluation easily and without using higher-order thinking skills (Hsieh, Cho, Liu, & Schallert, 2008). Thus, the researcher in this study created challenging problems that could lead students to search and read the related resources before they could suggest possible solutions.
It is evident that students need high problem-solving self- efficacy to go through these processes. The student’s belief in their skills and ability is called self-efficacy, and this belief is needed to help them be active in the learning process, having more confidence in the discussion, analyzing, and constructing new information. Self- efficacy was first introduced by Albert Bandura who believed that a person’s belief in his skill helped him to manage the process of applying suitable strategies in learning. Self- efficacy is a person's confidence to complete the required task, define the goals of learning and work well to achieve them.
Self- efficacy is considered as a psychological process as well as a process of self-evaluation in learning. Self-evaluation could be reflected from student’s experiences in life. Students who have the belief that they can achieve the task using suitable strategies differ in efficacy from students who have the belief that they would not be able to achieve tasks completely. This could result in a sort of high-self efficacy which students would be able to use well to achieve the cognitive and metacognitive skills (Eastin & LaRose, 2000; Schunk. & Pajares, 2002). Similarly, self-efficacy could be related to the cognitive theory because when students need to succeed in a task, they need to have confidence in themselves (Bandura, 2006).
Self-efficacy could therefore be correlated to the PBL model to help in improving meaningful learning (Qarareh, 2016). This is because students would be able to construct their learning in the PBL model. PBL model could prepare students to deal with their life difficulties. In higher education, students could be classified into two groups: high-self-efficacy students and low self-efficacy students. The high self-efficacy students have high motivation in the learning process and achieve well in completing a challenging task. The low efficacy students may struggle in achieving a challenging task. They may also have low confidence in their ability to achieve the task.
In a more specific context, academic self-efficacy in problem-solving reflects the ability of students to achieve academic tasks. Schunk (2001) stated that there are different factors that affect students motivation and self-efficacy in learning. The positive behavior from students to learning could also positively affect their self-efficacy, and vice versa. Therefore, it is recommended to encourage students to be responsible for achieving the tasks of enhancing their academic self-efficacy (Kemp, 2011). Further, in educational environment, students usually compare their abilities with other students. From these comparisons, they form beliefs about their capabilities. This is how students’ motivation may reflect on their self-efficacy (Bandura, 1997).
The PBL model directs the instructor to create real problems related to the topic of the lesson in the learning process. The students then engage in in-depth thinking to present problems and read related resources trying to propose a suitable solution to the problems in small groups. There are many skills, which students need to use while they are thinking of a suitable solution to the problem. These skills are searching, reading, thinking, analyzing, constructing, evaluating, and discussing. Students are allowed to choose suitable skills, as well as to discuss the problems with others and share their proposed solution on Blackboard. Since the participating students in this study were from interdisciplinary subjects, they could develop different perspectives during their learning in the PBL model (Pepper, 2010) and discuss those perspectives on Blackboard (BB) system.
Blackboard system was used in this study as the main tool to help students to present, discuss, share, and evaluate problems. As I was the instructor for the course and the researcher for this study, I allowed the students to engage in the process of searching, discussing and presenting the related information to the problem. In this study, I presented how students familiarized themselves with the problem; how they suggested a suitable solution; and how they shared it with other students on the blackboard system. Prior to that, they could also participate in evaluating each propose solution from different perspectives.
Blackboard is a “software package designed to help educators create quality online courses” (Choy, Xiao, & Iliff, 2005). The Blackboard Collaborate is a virtual system that enables the academic to manage online communication and interaction. It could be considered as a curriculum-oriented teaching platform. It enables the instructors to integrate multimedia, virtual learning environment, and interactive communication. It has become popular in the current times in many universities around the world, as about 78% of the 100 top Universities around the world use it as integration software with their curriculum.
Using blackboard supports the learning process in higher education. It helps in transferring the traditional system to a more modern system. The modem setting usually helps students to learn in a flexible environment. There are many characteristics of using blackboard in learning: it enables students to interact with each other, it enables students to interact with their instructors. It also enables the instructors to open a discussion for their students, and engage them in these discussions at flexible timings. Creating flexibility in learning could increase students’ motivation to learn.
Blackboard also enables the instructor to manage the courses online. Although the blackboard is available in many universities, a previous study by Bennett and Bennett (2003) stated that only 20% of faculty members used it. Most of the faculty members stated that using the blackboard forced them to change their way of teaching, and they were not willing to do that. Further, many of the instructors claimed that using the blackboard may consume a long time and that would not help them to cover everything in their courses (Allen & Seaman, 2008; Jones & Moller, 2002; Nichols, 2011; Schoepp, 2005; Sneller, 2004; Woods, Baker, & Hopper, 2004; Zirkle, 2002). Some of the faculty members claimed that they lacked suitable training for using the blackboard.
The research that addresses how the PBL model affects students’ problem-solving self-efficacy in higher education is not widely found. Some related studies have only examined how self-efficacy, in general, could be improved (Dinther, Dochy, & Segers, 2011; Ritchie, 2015). The problem-solving process in higher education courses seems difficult for many students to achieve them and face the challenges. This is because studying theoretical knowledge is much easier for them than applying them to real circumstances. Therefore, this study attempts to examine the effect of implementing the PBL model on higher students' problem-solving self-efficacy.
Based on the purpose of the study, following research questions were constructed for this study:
2.1. Research Design
The present study is a single-group pretest-posttest quasi-experimental design study. An open-ended questionnaire was used after the study was completed in order to understand participants’ perceptions of implementing PBL into their learning through the blackboard system. The research data were collected during a period ranging from February to April 2020. The researcher having performed all procedures presented the questionnaire to experts in the field to validate it. After the validation process, a letter was distributed to all participants in order to collect their consent for participating in the research. The letter clarified all information related to the study and ensured to them that they can withdraw from the study at any time and for any reasons. Further, they would remain anonymous in this research and all information would be dealt with confidently. They were also informed that the findings of the study would be published. All the procedures of the research were approved by the University Ethical Guidelines.
The sample of this research comprised 31 female participants from 3 different classes studying a Master program in curriculum and instructions department in the Education College at Umm Al-Qura University. They belonged to different specialties, but all of them were studying under the general field of Curriculum and instruction. The age of the participants was between 25-45 years old. The experience element of them was varied, some of the participants were working, and others had never worked. For this study, they were divided into 7 small groups in with each group having between 4 to 6 students. These were self-selected groups by the participants themselves. This helped them to work in a more relaxed and comfortable atmosphere and solve problems raised.
2.1.1. The Pre-Test
Bandura’s (2006) Problem-Solving Self-Efficacy scale was used to measure students’ problem-solving self-efficacy in the Master Program. In this measurement scale, students indicated their degree of confidence from 0 to 100 in solving a percentage of academic problems. The higher values represented higher degrees of self-efficacy in the problem-solving process, while the lower values represented lower degree of self-efficacy in problem-solving.
The measurement scale for Bandura’s Problem-based Self-efficacy was distributed to the participants at the beginning of the term. The researcher had explained to the participants the method of measuring their academic self-efficacy.
2.1.2. Implementation of PBL Model
Having measured the self- efficacy in problem solving of the participants, a problem related to the topics of the course that students learned and discussed in each lecture was initiated. This problem was also referred to experts to check the validity and suitability of the content and written forms. After the validation, the problem was presented on the blackboard. The participants were already told that the presentation and discussion of each problem would be through blackboard. The reason was that students needed a suitable tool to help them share their perspectives with other students. Using blackboard would allow them to share and receive feedback from other students about their analysis of the problems.
Therefore, in this study, the researcher used the blackboard as a tool to present the problems after each lecture. The researcher presented the problems in a discussion forum, and each problem had a new discussion forum in which the title of the forum was the same as the title of the problem. Thus, participants were able to read the topic and content of the problem, and then they could read about it and discuss it in groups and formulate or construct a possible solution. The researcher respected all differences in participants’ opinions since there was no specific answer for solving the problems.
There are six stages that participants need to go through when they discuss problems. The first stage: they need to search and find the related resources; second, reading the related resources; third: they need to analysis the problems; fourth, they need to discuss the possible solutions for the proposed problem. Fifth, they need to write the possible solutions and evaluate them; sixth, they need to present and share their solutions and evaluate each other presented solutions on blackboard.
2.1.3. Post-Test
When eight weeks of implementing the PBL model were completed, and participants had been engaged in solving eight problems, the same measurement scale for problem-solving self-efficacy was distributed to them. Participants were asked to complete it again and record the percentage of the confidence of solving academic problem. Further, an open-ended questionnaire was distributed for all participants to investigate in-depth the students perception of implementing PBL into their learning and through the blackboard. In the questionnaire, four questions were identified, since they focused on students’ perceptions of the advantages and challenges of implementing PBL into their learning and working on PBL through the blackboard system.
The finding of the study was obtained from quantitative and qualitative data. It was required to compare the quantitative data and the result of the measurement scale before and after the implementation of the PBL. SPSS program was used to find out the result of the study. ANOVA test was used to find the differences between the pre and post-measurement test. A qualitative analysis was also applied to explore the participants' perception regarding the PBL model. It was a thematic analysis of the qualitative data to identify different codes and themes under different headings.
4.1. The Quantitative Analysis
4.1.1. Reliability Test
This study is based on the 10-point problem-solving self-efficacy scale by Bandura (2006) and to this effect, in order to establish the reliability of the scale, Carden, Camper, and Holtzman (2019) as well as Pallant (2013) recommend the use of the Cronbach’s alpha statistic. According to the literature on reliability analysis, the minimum threshold that is accepted is 0.70, and this is supported by Tavakol and Dennick (2011) as well as Coolican (2014). These scholars suggest that while 0.7 is regarded as the minimum, the higher the alpha statistic, the more reliable it is. This alpha statistic was calculated, and the results are presented in Table 1.
Procedure |
Cronbach's Alpha |
No of Items |
Before Intervention |
0.969 |
10 |
After Intervention |
0.967 |
10 |
The Cronbach’s alpha before the intervention was 0.969, and after the intervention it was 0.967 and because both coefficients were greater than 0.70, the researcher confirmed that the scale used was reliable.
4.1.2. Hypothesis Testing
This study adopted a single-group pretest-posttest quasi-experimental design to examine the effect of implementing the Problem-Based Learning (PBL) model towards the students’ problem-solving self-efficacy. In this regard, the main hypothesis that the study sought to test was:
H0: There is no significant difference in the problem-solving self-efficacy confidence score between the pre and post-measurement test.
H1: There is a significant difference in the problem-solving self-efficacy confidence score between the pre and post-measurement tests.
The one-way repeated measures ANOVA test was done as prescribed by Wywial (2015) and Kent (2016). According to Field (2016) this was optimal for this study as it facilitated the comparison of the pre and post measurements for problem-solving self-efficacy. The descriptive statistics are presented in Table 2.
Table-2 . Descriptive Statistics.
Stage |
Mean |
Std. Deviation |
N |
AVG Before |
71.3226 |
23.31864 |
31 |
AVG After |
81.3871 |
19.93101 |
31 |
From the average problem-solving self-efficacy results, it is evident that the Mean was 71.32 (SD = 23.32) before implementing the Problem-Based Learning (PBL) model and it increased to 81.39 (SD = 19.93) after implementing the PBL model. This result shows that there was an improvement in the average problem-solving self-efficacy mark, after the implementation of the PBL model. The multivariate test results are presented in Table 3.
| Effect | Value |
F |
Hypothesis df |
Error df |
Sig. |
Partial ETA Squared |
|
| AVG | Pillai's Trace | 0.451 |
24.636b |
1.000 |
30.000 |
0.000 |
0.451 |
| Wilks' Lambda | 0.549 |
24.636b |
1.000 |
30.000 |
0.000 |
0.451 |
|
| Hotelling's Trace | 0.821 |
24.636b |
1.000 |
30.000 |
0.000 |
0.451 |
|
| Roy's Largest Root | 0.821 |
24.636b |
1.000 |
30.000 |
0.000 |
0.451 |
|
| a. Design: Intercept Within Subjects Design: AVG |
|||||||
| b. Exact statistic | |||||||
The results reveal the influence of the PBL model where Wilks’ Λ = 0.549, F (1, 30) = 24.626, p = 0.000<0.05; η2 = 0.451. Thus, the implementation of the PBL model explained 45.1% of the variation in the problem-solving self-efficacy ratings. The test for the sphericity assumption was done using the Mauchly's test. However, the p-value was not generated as there were only two repeated measures, as shown in Table 4.
Table-4 . Mauchly's test of sphericity.
Within Subjects Effect |
Mauchly's W |
Approx. Chi-Square |
Df |
Sig. |
Epsilonb |
||
Greenhouse-Geisser |
Huynh-Feldt |
Lower-bound |
|||||
AVG |
1.000 |
0.000 |
0 |
. |
1.000 |
1.000 |
1.000 |
| a. Design: Intercept Within Subjects Design: AVG |
|||||||
From the foregoing, sphericity was not assumed, and thus, the Greenhouse-Geisser statistic was to be considered in the tests of within-subjects effects as shown in Table 5.
Table-5. Tests of within-subjects effects.
| Source | Type III Sum of Squares |
df |
Mean Square |
F |
Sig. |
Partial Eta Squared |
|
| AVG | Sphericity Assumed | 1570.065 |
1 |
1570.065 |
24.636 |
0.000 |
0.451 |
| Greenhouse-Geisser | 1570.065 |
1.000 |
1570.065 |
24.636 |
0.000 |
0.451 |
|
| Huynh-Feldt | 1570.065 |
1.000 |
1570.065 |
24.636 |
0.000 |
0.451 |
|
| Lower-bound | 1570.065 |
1.000 |
1570.065 |
24.636 |
0.000 |
0.451 |
|
| Error(AVG) | Sphericity Assumed | 1911.935 |
30 |
63.731 |
|||
| Greenhouse-Geisser | 1911.935 |
30.000 |
63.731 |
||||
| Huynh-Feldt | 1911.935 |
30.000 |
63.731 |
||||
| Lower-bound | 1911.935 |
30.000 |
63.731 |
||||
From the outcome, repeated-measures ANOVA was carried out with a Greenhouse-Geisser correction and the findings show that the problem-solving self-efficacy confidence score significantly differed before and after the implementation of the Problem-Based Learning (PBL) model (F(1,30) = 24.636, p = 0.000<0.05, η2= 0.451). From these findings, the researcher rejects the null hypothesis and concludes that there was a significant difference in the problem-solving self-efficacy between the pre and post-measurement tests. To identify where the differences were, the pairwise comparisons were made and the results are presented in Table 6.
Table-6. Pairwise Comparisons.
(I) AVG |
(J) AVG |
Mean Difference (I-J) |
Std. Error |
Sig.b |
95% Confidence Interval for Differenceb |
|
Lower Bound |
Upper Bound |
|||||
1 |
2 |
-10.065* |
2.028 |
0.000 |
-14.206 |
-5.923 |
2 |
1 |
10.065* |
2.028 |
0.000 |
5.923 |
14.206 |
| Based on estimated marginal means | ||||||
| *. The mean difference is significant at the .05 level. | ||||||
| b. Adjustment for multiple comparisons: Bonferroni. | ||||||
The difference in the problem-solving self-efficacy confidence score was statistically different before and after the implementation of the Problem-Based Learning (PBL) model (MD = 10.07; p<0.05).
4.1.3.Demographic Variables
The distribution of the demographic variables is presented in this section and this will cover the participants’ experience, specialty and age.
4.1.4. Experience
From the findings, the majority of the respondents did not have any experience and were unemployed (61.3%), while only 38.7% had experience, as shown in Table 7.
Table-7 . Distribution by Experience.
Frequency |
Percent |
Valid Percent |
Cumulative Percent |
||
Valid |
No |
19 |
61.3 |
61.3 |
61.3 |
Yes |
12 |
38.7 |
38.7 |
100.0 |
|
Total |
31 |
100.0 |
100.0 |
||
The distribution by specialty is presented in Table 8.
Table-8 . Distribution by Specialty.
Frequency |
Percent |
Valid Percent |
Cumulative Percent |
||
| Valid | Arabic | 3 |
9.7 |
9.7 |
9.7 |
| Art | 3 |
9.7 |
9.7 |
19.4 |
|
| English | 7 |
22.6 |
22.6 |
41.9 |
|
| Islamic | 2 |
6.5 |
6.5 |
48.4 |
|
| Science | 14 |
45.2 |
45.2 |
93.5 |
|
| Technology | 2 |
6.5 |
6.5 |
100.0 |
|
| Total | 31 |
100.0 |
100.0 |
||
The results show that a majority of respondents had a specialty in science (45.2%), followed by those whose specialty was English (22.6%). The third-rated specialties were Arabic and Art, whose proportions were 9.7%, while the least rated were Islamic and Technology whose proportions were 6.5%. The last demographic variable of Age is shown in Table 9.
Table-9 . Distribution by Age.
Frequency |
Percent |
Valid Percent |
Cumulative Percent |
||
| Valid | Less than 30 Years | 21 |
67.7 |
67.7 |
67.7 |
| 30 Years or More | 10 |
32.3 |
32.3 |
100.0 |
|
| Total | 31 |
100.0 |
100.0 |
||
The results show that the majority of the respondents were aged below 30 years (67.7%), while those aged above 30 years were only 32.3%.
4.1.5. Influence of Demographic Variables
To test the influence of the demographic variables on the improvement of the problem-solving self-efficacy, and on the confidence score before and after the implementation of the Problem-Based Learning (PBL) model, the interaction effect was tested by the tests of within-subjects effects. The results are presented in Table 10.
From the outcome, the interaction effect of experience does not show any significant influence on the improvement of the problem-solving self-efficacy confidence score before and after the implementation of the Problem-Based Learning model [F(1, 20) = 0.023; p = 0.881; η2= 0.001]. The same non-significant influence was witnessed in the interaction effect of the specialty [F(5, 20) = 0.151; p = 0.978; η2= 0.036], as well as the age group [F(1, 20) = 0.203; p = 0.657; η2= 0.010]. These findings do confirm that experience, specialty, and age group do not have a statistically significant influence on the improvement of the problem-solving self-efficacy confidence score before and after the implementation of the Problem-Based Learning model.
4.2. The Qualitative Findings
4.2.1. Perceived Benefits of Implementing PBL Model into Students’ Learning
All participants reported that implementing PBL into their learning was a benefit to them. The reported benefits were related to the students’ knowledge and their learning and social skills. The first exciting finding was that some students (9 out of 31) reported that PBL helped them to apply theoretical knowledge into real problems. Since as they reported, they used the learned theories and knowledge when they analyzed and solved the proposed problems. Some of the students admitted that they were getting better in identifying reasons for their problems, and that by linking the theory to the problems helped them in understanding the content better. This finding is consistent with the previous literature (Gross & Rutland, 2017; Speece, 2002) in which they found that applying PBL helped students to find suitable theory and apply them into real life. They stated that learning the theory without application and practice of how it should be used in real life would not be useful for students. Thus, the combination of learning theories and its practice would be efficient in preparing students into their life.
Table-10 . Tests of Within-Subjects Effects (incl. demographics).
| Source | Type III Sum of Squares |
df |
Mean Square |
F |
Sig. |
Partial Eta Squared |
|
| AVG | Sphericity Assumed | 738.642 |
1 |
738.642 |
8.976 |
0.007 |
0.310 |
| Greenhouse-Geisser | 738.642 |
1.000 |
738.642 |
8.976 |
0.007 |
0.310 |
|
| Huynh-Feldt | 738.642 |
1.000 |
738.642 |
8.976 |
0.007 |
0.310 |
|
| Lower-bound | 738.642 |
1.000 |
738.642 |
8.976 |
0.007 |
0.310 |
|
| AVG * Exp | Sphericity Assumed | 1.895 |
1 |
1.895 |
0.023 |
0.881 |
0.001 |
| Greenhouse-Geisser | 1.895 |
1.000 |
1.895 |
0.023 |
0.881 |
0.001 |
|
| Huynh-Feldt | 1.895 |
1.000 |
1.895 |
0.023 |
0.881 |
0.001 |
|
| Lower-bound | 1.895 |
1.000 |
1.895 |
0.023 |
0.881 |
0.001 |
|
| AVG * Specialty | Sphericity Assumed | 61.932 |
5 |
12.386 |
0.151 |
0.978 |
0.036 |
| Greenhouse-Geisser | 61.932 |
5.000 |
12.386 |
0.151 |
0.978 |
0.036 |
|
| Huynh-Feldt | 61.932 |
5.000 |
12.386 |
0.151 |
0.978 |
0.036 |
|
| Lower-bound | 61.932 |
5.000 |
12.386 |
0.151 |
0.978 |
0.036 |
|
| AVG * AgeGrp | Sphericity Assumed | 16.720 |
1 |
16.720 |
0.203 |
0.657 |
0.010 |
| Greenhouse-Geisser | 16.720 |
1.000 |
16.720 |
0.203 |
0.657 |
0.010 |
|
| Huynh-Feldt | 16.720 |
1.000 |
16.720 |
0.203 |
0.657 |
0.010 |
|
| Lower-bound | 16.720 |
1.000 |
16.720 |
0.203 |
0.657 |
0.010 |
|
| Error(AVG) | Sphericity Assumed | 1645.732 |
20 |
82.287 |
|||
| Greenhouse-Geisser | 1645.732 |
20.000 |
82.287 |
||||
| Huynh-Feldt | 1645.732 |
20.000 |
82.287 |
||||
| Lower-bound | 1645.732 |
20.000 |
82.287 |
||||
Some of the participants (11 out of 31) reported that through their work on PBL they would be able to model some solutions in their real life. This is because all the raised problems were real and the suggested solutions were applicable to problems in their life. A few other participants reported that the application of the method of PBL in the course linked them more with reality, which made them look more motivated to find suitable solutions. For instance, Participant 14 said:
The good thing about PBL is that we are trying to fix the problems and having broad insight which has made us feel like we are living with reality, and that is something we very much enjoyed, because I went through the process of recall knowledge, analyzed it, and then chose the suitable information and relate them to the solution of the problem.
The most common advantage of this model among all participants was that PBL helped them to improve their critical thinking skills. This was because the researcher had presented real challenging problems, and previous literature was also consistent with the idea that in order to improve students’ higher thinking skills, the problem should be challenging and related to students experiences (Brabler, 2016; Dağyar & Demirel, 2015; Major & Mulvihill, 2018; Vandenhouten et al., 2017). Most of the students (20 out 0f 31) stated that going through the searching and discussion process to find suitable solutions to real problems helped them in developing their higher and lower order thinking. Problem-solving skills helped them face the challenges in their education and in their social life. Moreover, engaging in problem-solving skills also reflected on the skills needed in their life, particularly higher-order skills (Özreçberoğlu & Çağanağa, 2018).
One of the positive effects of this model was that problem-solving directed students to several challenging tasks (Geitz, Joosten-ten Brinke, & Kirschner, 2016). It was felt by most participants that due to higher self-efficacy they are likely to succeed in challenging issues in comparison to those who lacked self-efficacy (Kurtuldu & Bulut, 2017). This is because PBL helped them and directed them to increase their critical reading to a different source. From reading, they could improve the higher thinking skills: analysis, construction, decision-making and evaluation. Also, some of the participants reported that PML helped in developing lower thinking skills such as attention and awareness, remembering.
Participant 11 admitted:
Using the problem-solving method expanded my way of thinking, and it made me think out of the box to reach appropriate solutions that are useful in solving the problem at hand, through the use of steps to solve problems starting finding out the reasons for the problem and changing and analyze it and then put all possible hypotheses to solve this problem and present it to colleagues, and then reach appropriate logical solutions to solve the problem
Another exciting finding was that some of the participants (9 out of 31) reported that they discovered the fact that the nature of communication affected the quality of the message. They also understood that implementing PBL helped them in raising awareness about the need for research and investigation for analysis, finding solutions with logical innovation, and evaluating the presentation. Besides, this method supported contemplative and creative thinking skills as well as strengthened the method of systems analysis for reaching the causes of the problems raised.
Participant 26 said:
The advantage is that implementing PBL helps us to improve the ability to generate new and innovative solutions. This is because working within a problem-solving team gives us a unique, positive, and enriching experience in which we are able to exchange knowledge and opinions with colleagues and look at the problem from different angles that we wouldn't have seen them if we were alone.
Some of the participants (7 out of 31) reported that working on the PBL model helped them to increase their confidence and responsibility. This is because every member of the group was free to express their perspectives without any specification or regulations. A few others (8 out of 31) reported that by working on PBL, they started to learn how to persuade other group members in their arguments and respect other members' views practically. This was possible only because they read and knew about the problem raised. This is consistent with previous literature which said that: PBL could be an efficient model in higher education to help students tackle their learning. Torp and Sage (2002) stated that PBL might consider as experience-based learning since students used their experiences in identifying the problems. Savery (2006) stated that PBL could consider as a student-centered learning that helped students to do research, implement theory as well as practice the learning theories into their real life. Therefore, students may engage in self-regulated learning to find out the required information from multi resources such as books, journals, reports, and newspapers.
Examples from Participant 18 and 22:
Working on PBL helped me to develop positive trends towards difficult situations, with great confidence that enabled me to overcome difficulty. Also, the higher confidence helped me in finding appropriate solutions and increasing my enthusiasm for the course, so that I could provide a logical solution to the problem. I was able to prove and highlight my personality in writing, given that the solutions presented are views based on a scientific basis, in fact, in my experience of this method, I felt that it penetrated information into the brain better than recalling it for testing.
I gained good planning skill to solve any problem, I could accept negative feelings more, my self-esteem and my capabilities increased through my colleagues and my ability to solve problems, social communication with female colleagues rose and became stronger.
The last interesting finding was that most of the participants (19 out of 31) believed that working on PBL helped them to strengthen their relationship with other members. Some participants reported that implementing PBL helped in developing communication skills by finding the solution together with the group members. Further, from the previous literature, it is recommended in the educational environment that students compare their abilities with their colleagues, and by doing so, they form a belief about their capabilities. This is how students motivation to learn may reflect on their self-efficacy (Bandura, 1997). Therefore, participants need to have a good time with members of the group, discuss together, analyze together and help each other to choose suitable solutions for each problem. Participants of the study explained that each individual had a point of view that may differ from one another. When members start a discussion in the group, opinions of all members must be accepted no matter how different they are. Some of the participants said that, because the members of the group were from various subjects, it helped them a lot practically in the process of discussion and the process of reflections of the proposed solutions. This is because each different member reflected differently on the solution. So it helped in developing the ideas to the best.
Therefore, many of the participants stated that working on PBL helped them a lot to improve the team working skills. Previous literature (Bandura, 1997; Chemers, Hu, & Garcia, 2001; Lodewyk & Winne, 2005) found that students would be able to control and achieve their learning if they proved that they had self-efficacy. However, low self-efficacy students result in academic failure. Further, some researchers (Nie, Lau, & Liau, 2011) stated that there are connections between the student’s self-efficacy and their emotion. The low-efficacy students are more anxious and stressed in learning and communication, while the high-efficacy students are more confident in learning.
4.2.2. The Challenges of Implementing PBL Model into Students’ Learning
Most of the participants (20 out of 31) claimed that although PBL had advantages in learning, some challenges could reduce the positive effect of its implementation. These challenges are related to the difficulty in connecting the theoretical knowledge to the problem raised. Further, some of them mentioned the differences in the perspectives between group members while others consider the PBL activity time consuming.
Another significant problem raised by some participants (10 out of 31) dealt with difficulties in relating the proper knowledge and theory to the problem raised. Participants explained that they were aware of the learned theory and familiar with the topic of the problems, but when it came to knowledge application, they encountered problems. Currently, they reported they could cope with these difficulties later on.
Participant 16 said thus:
We learned all the theories related to the problems raised, and we understand them very well. However, when it comes to the theory application, we face some difficulties since we cannot use the knowledge application, but by working on more than one problem, we get better in coping with these challenges.
Another crucial finding was that most of the participants (18 out of 31) stated it was difficult to work on PBL with groups since members were varying in their specializations. Since they are different in their perspective, they have some conflict in their opinions. As a result, each member of the group identifies a different perspective from a different angle, and hence the conflict. Some of the participants complained that some members in the group have the belief that their specialization was more important than other specialization, so they did not accept other members' views. They did not accept the point that each member added to solve problems. It suggests that in PBL activities, students need to have confidence in discussion with others (Udeani & Adeyemo, 2011; Wang & Chiew, 2010).
The last significant challenge from many of the participants (19 out of 31) claimed that working on PBL could take longer time than other activities since the process that they went through when they worked with others needed time for: searching, analyzing, discussion, presentation, and evaluation. Thus, they need to decide on a specific time to plan for completing the activity with other group members. Therefore, it would be difficult when they work with other members of the group. However, working on PBL in-group helps in shortening the time in completing activity since each member shares and helps in suggesting a possible solution.
4.2.3.Benefits of Using Blackboard in Implementing PBL Activity
All of the participants stated the benefits of using the Blackboard (BB) system. Most of the participants (19 out of 31) perceived that working on PBL through BB was an opportunity to them to take advantage of technology and develop their skills of electronic learning. The first and the most interesting finding was that some participants (13 out of 31) said that it was the first opportunity for them to work on any activity through technology, and they thought it would be impossible to be completed through technology, but when they tried working PBL on BB, they were convinced with the beneficial effect of working through it.
All the participants said that working on PBL through BB facilitated the process of learning with other students. Helping students to complete the tasks through BB helped in supporting the learning process in higher education. BB was considered more flexible than other learning systems that helped students in their learning (Allen & Seaman, 2008; Jones & Moller, 2002; Nichols, 2011; Schoepp, 2005; Sneller, 2004; Woods et al., 2004; Zirkle, 2002). The BB system helped them present their solutions to problems, discuss other group solutions and evaluate solutions. They explained that using BB in PBL helped them to share ideas through problem-solving and discussion with colleagues and other groups. It also helped them in familiarity and coming up with the best solutions, appropriate for all .
It was a good opportunity for the female colleagues to find solutions and examine the differences in the way each group solved the problem raised. The BB helped participants to see how these solutions sometimes took different directions. This resulted in expanding their perceptions . Thus, the most common advantage was the flexibility of presenting, discussing, and evaluating the solutions. Therefore, presenting the solutions on blackboard opened chances for participants to know together the groups' reflections to their presented suggestions.
Participants 2 and 6 expressed:
It is useful to know how each group deals with the problem presented , as well as to respect different points of view and benefit from discussions in which that revolve around the problem.
The ability to see the problem at the appropriate time provides an opportunity to discuss the problem with the rest of the members of the groups, to see various ideas in addition to enriching the knowledge and linguistic outcome. It also strengthens our commitment to persevere the discussion among colleagues on the Blackboard platform. It helps in submission of duties, preserving the intellectual rights of female colleagues and analyzing problems scientifically based on reliable evidence and references.
Participants explained they had a chance to widen their knowledge and read all possible solutions from other groups, which helped them to see various perspectives of different students and also see the work of various groups to solve problems. It enriched their knowledge about proposed solutions and benefited them from discussions and dialogue. Lastly, there was also the ease of displaying work on the BB by referring to it at any time and allowing students to get to the forum to discuss the problems raised. This enabled students to exchange views and discuss solutions between female colleagues and keeping discussions alive and returning to it later. During this process, everyone transparently expressed their ideas. Further, an interesting finding comes from Participant 16, in which she explained that discussion of the proposed problems through blackboard allowed them to present their point of view freely without interruption from other students.
The ability to view and discuss all opinions in a short time without the need to specify a specific time or place allowed us to exchange views between a large number without being interrupted or distracted by the debate
Some of the participants (14 out of 31) said the BB system helped them to save time in which students can share many files with other students. Participants said that using blackboard allowed them the flexibility in using suitable time in working on the PBL. This was because every member of the group had a chance to look at the origin of the problem from their perspective. Each group member would meet and discuss their opinion of the proposed problem. It was important to overcome the obstacle of time and place so that members could enter the system easily at any time and from anywhere. BB enabled them to constantly contact with the course instructor, present inquiries and questions, and provide feedback . The BB gave them the ability to attach solutions in different formats in the system, add comments, and communicate with groups. This is consistent with the previous literature which found the BB system enabling the instructors to manage the courses online and contact with students, and when required suggest ideas to the instructors to change their methods of teaching (Allen & Seaman, 2008; Jones & Moller, 2002; Nichols, 2011; Schoepp, 2005; Sneller, 2004; Woods et al., 2004; Zirkle, 2002).
4.2.4.Challenges Using Blackboard to Solve Problems
Many of the participants (19 out of 31) stated that the most common problem for using the BB was related to the inability to use the BB system due to poor network connection or complex features of the blackboard system. Further, participants also stated a few other challenges in using the blackboard and presented solutions for the problem raised.
Some of the participants (10 out of 31) stated that at the beginning of the work on the BB, there were problems related to the use of the BB system. This was because it was the first time for them to complete an activity through BB. Fortunately, they faced these difficulties only in the beginning of implementing PBL in the communication course. An example from Participant 26:
Often these are the most critical challenges that I faced while using the blackboard to present problems and use PBL activity, many of which were overcome over the time as we gained some experience.
Another important problem that needed consideration was that some of the students lived in a rural area in which they did not have a good network connection. Thus, they encountered problems like getting late in the discussion and evaluation with other group members. Moreover, depending 100 % on technology to solve problems could cause difficulties for some students. One of the most difficult challenges was the errors in the BB itself that compelled students to exit from the system and then enter more than once. Sometimes, they may also update the blackboard program if they found system slowing down or hanging. Thus, they faced difficulty in downloading content or attaching it. An example from participant 28:
I faced some challenges in using the BB system in the way of solving problems, in terms of the weakness of the Internet, the difficulty of knowing the problem and attaching solutions . I also faced a problem in commenting and exchanging discussions with the rest of the groups. I tried to send my comment, and as soon as I left the page, the comment would disappear, and I would receive blank messages from the rest of the colleagues in the discussions field. These may be individual problems either because of the poor internet or problems in the system, but this did not contradict the major benefits of using problem-solving in particular .
Further, some of the participants (7 out of 31) complained that the blackboard system did not have a notification message to inform students that there was a new solution for the proposed problem. They complained that they had to go through the presentation of the solution and decide which new solutions should be added. Participant 28 commented:
In the beginning, there was a challenge faced by me , which was the lack of a notification message if a discussion or uploading of any file on the Blackboard. I was able to solve this challenge by downloading an application called Black Board on Mobile and to activate alerts, it made it easier for me to discuss and follow the new process on the blackboard .
Some participants (11 out of 31) complained that a few group members were not flexible in agreeing about different perspectives. This result could be in line with previous studies about self- efficacy, since they reported that students with high self-efficacy would have difficulties in communicating with students in low self- efficacy (Schunk, 2001) and that this may create conflicting perceptions. Another difficulty stated by some participants was discussing different points of view in online learning. They faced problems and difficulties in communicating with other participants through the BB system. Making use of technology program to discuss with other group members could result in a lack of direct physical communication that promoted body language learning and visual communication.
In conclusion, Qarareh (2016) stated that self-efficacy would be related to the PBL in order to help students to make meaningful learning. However, in this study, participants presented challenges to make meaningful learning through applying PBL in BB system. The main reason for that could be the low motivation for students in completing PBL task, which cause conflicting in changing or not accepting other students’ perceptions. Schunk (2001) stated that there are different factors that affect student’s motivation and self-efficacy in learning. The positive behavior from students to learning could positively affect their self-efficacy, and vice versa. Thus, it is recommended to encourage students to be responsible for their discussion and respect other students perceptions for completing the PBL tasks , in which this is would reflect positively on their self-efficacy, as well as meaningful learning could be created (Kemp, 2011).
The result of the study suggests that implementing PBL through using the BB system could help in enhancing students’ problem-solving self-efficacy in higher education. BB system helped the participants in creating a meaningful learning; even though they faced some difficulties regard to negotiation of different perspectives Thus, the findings could be important to university instructors in general courses and in the courses of Teaching Methods in particular. The findings suggest working on related researches that apply different methods and strategies of implementing PBL through using the BB system in order to help students’ enhance their self-efficacy in learning as well as cope up with the challenges in higher Education.
Funding: This study received no specific financial support. |
Competing Interests: The author declares that there are no conflicts of interests regarding the publication of this paper. |
Allen, E., & Seaman, J. (2008). Staying the course: Online education in the United States. Newburyport, MA 01950. USA: The Sloan Consortium.
Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W.H. Freeman and Company
Bandura, A. (2006). Guide for constructing self-efficacy scales. Self-Efficacy Beliefs of Adolescents, 5(1), 307–337.
Bennett, J., & Bennett, L. (2003). A review of factors that influence the diffusion of innovation when structuring a faculty training program. The Internet and Higher Education, 6(1), 53-63.
Brabler, M. (2016). Interdisciplinary problem-based learning—a student-centered pedagogy to teach social sustainable development in higher education. In Teaching education for sustainable development at university level (pp. 245-257). Cham: Springer.
Carden, S. W., Camper, T. R., & Holtzman, N. S. (2019). Cronbach’s Alpha under insufficient effort responding: An analytic approach. Stats, 2(1), 1-14.
Chemers, M. M., Hu, L.-t., & Garcia, B. F. (2001). Academic self-efficacy and first year college student performance and adjustment. Journal of Educational Psychology, 93(1), 55-64.
Choy, D., Xiao, J., & Iliff, J. (2005). How can technology help improve the quality of Blackboard faculty training and encourage faculty to use Blackboard? Paper presented at the 2005 Annual Proceedings-Orlando.
Coolican, H. (2014). Research methods and statistics in psychology. New York: Psychology Press.
Dağyar, M., & Demirel, M. (2015). Effects of problem-based learning on academic achievement: A meta-analysis study. Ted Educationn and Science, 40(181), 139-179.Available at: 10.15390/EB.2015.4429.
Dinther, M. V., Dochy, F., & Segers, M. S. (2011). Factors affecting students' self-efficacy in higher education. Educational Research Review, 6(3), 95-108.Available at: https://doi.org/10.17323/jle.2020.9911.
Eastin, M. S., & LaRose, R. (2000). Internet self-efficacy and the psychology of the digital divide. Journal of Computer-Mediated Communication, 6(1), JCMC611.Available at: 10.1111/j.1083-6101.2000.tb00110.x.
Fettahlıoğlu, P., & Aydoğdu, M. (2020). Developing environmentally responsible behaviours through the implementation of argumentation-and problem-based learning models. Research in Science Education, 50(1), 987-1025.Available at: https://doi.org/10.1007/s11165-018-9720-0.
Field, A. P. (2016). Discovering statistics using SPSS. London, England: SAGE.
Geitz, G., Joosten-ten Brinke, D., & Kirschner, P. A. (2016). Changing learning behaviour: Self-efficacy and goal orientation in PBL groups in higher education. International Journal of Educational Research, 75, 146-158.Available at: https://doi.org/10.1016/j.ijer.2015.11.001.
Gross, Z., & Rutland, S. D. (2017). Experiential learning in informal educational settings. International Review of Education, 63(1), 1-8.Available at: https://doi.org/10.1007/s11159-017-9625-6.
Hsieh, P., Cho, Y., Liu, M., & Schallert, D. L. (2008). Middle school focus: Examining the interplay between middle school students achievement goals and self-efficacy in a Technology-Enhanced Learning Environment. American Secondary Education, 36(3), 33-50.
Hung, W. (2006). E 3C3R model: A conceptual framework for designing problems in PBL. Interdisciplinary Journal of Problem-based Learning, 1(1), 55–77.
Jones, A., & Moller, L. (2002). A comparison of continuing education and resident faculty attitudes towards using distance education in a higher education institution in Pennsylvania. College and University Media Review, 9(1), 11-37.
Kemp, S. D. (2011). Academic self-efficacy and middle school students: A study of advisory class teaching strategies and academic self-efficacy. Unpublished Doctoral Dissertation, La Sierra University, USA.
Kent, R. A. (2016). Analysing quantitative data variable-based and case-based approaches to non-experimental datasets. London: Sage.
Kurtuldu, M. K., & Bulut, D. (2017). Development of a self-efficacy scale toward Piano lessons. Educational Sciences: Theory and Practice, 17(3), 835-857.Available at: http://dx.doi.org/10.12738/estp.2017.3.0209 .
Lodewyk, K. R., & Winne, P. H. (2005). Relations among the structure of learning tasks, achievement, and changes in self-efficacy in secondary students. Journal of Educational Psychology, 97(1), 3-12.Available at: https://doi.org/10.1037/0022-0663.97.1.3.
Major, T., & Mulvihill, T. (2018). Problem-based learning pedagogies in teacher education: The case of Botswana. Interdisciplinary Journal of Problem-Based Learning, 12(1), 1.Available at: https://doi.org/10.7771/1541-5015.1543.
Nichols, D. L. (2011). An exploration of blackboard utilization by faculty at a Midwestern University. Doctoral Dissertation, Ohio University, USA.
Nie, Y., Lau, S., & Liau, A. K. (2011). Role of academic self-efficacy in moderating the relation between task importance and test anxiety. Learning and Individual Differences, 21(6), 736-741.Available at: https://doi.org/10.1016/j.lindif.2011.09.005.
Nijhuis, J. F. H., Segers, M. S. R., & Gijselaers, W. H. (2005). Influence of redesigning a learning environment on student perceptions and learning strategies. Learning Environ Res, 8, 67–93.Available at: https://doi.org/10.1007/s10984-005-7950-3.
Özreçberoğlu, N., & Çağanağa, Ç. K. (2018). Making it count: Strategies for improving problem-solving skills in mathematics for students and teachers’ classroom management. Eurasia Journal of Mathematics, Science and Technology Education, 14(4), 1253-1261.Available at: https://doi.org/10.29333/ejmste/82536.
Pallant, J. (2013). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (4th ed.). Crows Nest, NSW: Allen & Unwin.
Pepper, C. (2010). ‘There’sa lot of learning going on but NOT much teaching!’: Student perceptions of problem-based learning in science. Higher Education Research & Development, 29(6), 693-707.Available at: 10.1080/07294360.2010.501073.
Qarareh, A. O. (2016). The effect of using the constructivist learning model in teaching science on the achievement and scientific thinking of 8th grade students. International Education Studies, 9(7), 178-196.
Ritchie, L. (2015). Fostering self-efficacy in higher education students: Macmillan International Higher Education.
Savery, J. R. (2006). Overview of problem-based learning: Definitions and distinctions. Interdisciplinary Journal of Problem-Based Learning, 1(1).Available at: https://doi.org/10.7771/1541-5015.1002.
Schoepp, K. (2005). Barriers to technology integration in a technology-rich environment. Learning and Teaching in Higher Education: Gulf Perspectives, 2(1), 1-24.
Schunk, D. H. (2001). Social cognitive theory and self-regulated learning. In Zimmerman, B.J., & Schunk, D.H. (Eds.). Self-Regulated Learning and Academic Achievement: Theoretical perspectivist theories-an educational perspective (pp. 125-151). New Jersey: Prentice Prentice-Hall.
Schunk., D. H., & Pajares, F. (2002). The development of academic self-efficacy. In Development of Achievement Motivation (pp. 15–31): Elsevier.
Sneller, J. E. (2004). A web dream team: The seven principles and WebCT. Academic Exchange Quarterly, 8(4), 130–134.
Speece, M. (2002). Experiential learning methods in Asian cultures: A Singapore case study. Business Communication Quarterly, 65.
Tavakol, M., & Dennick, R. (2011). Making sense of cronbach's Alpha. International Journal of Medical Education, 2, 53-55.Available at: 10.5116/ijme.4dfb.8dfd.
Torp, L., & Sage, S. (2002). Problems as possibilities: Problem-based learning for K-16 education (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.
Udeani, U., & Adeyemo, S. A. (2011). The relationship among teachers problem-solving abilities, student’s learning styles and students achievement in Biology. International Journal of Educational Research and Technology, 2(1), 82-87.
Vandenhouten, C., Groessl, J., & Levintova, E. (2017). How do you use problem-based learning to improve interdisciplinary thinking? New Directions for Teaching and Learning, 2017(151), 117-133.Available at: https://doi.org/10.1002/tl.20252.
Wang, Y., & Chiew, V. (2010). On the cognitive process of human problem problem-solving. Cognitive Systems Research, 11(1), 81–92.Available at: https://doi.org/10.1016/j.cogsys.2008.08.003.
Woods, R., Baker, J. D., & Hopper, D. (2004). Hybrid structures: Faculty use and perception of web-based courseware as a supplement to face-to-face instruction. The Internet and Higher Education, 7(4), 281–297.
Wywial, J. L. (2015). Sampling designs dependent on sample parameters of auxiliary variables. New York: Springer.
Zirkle, C. (2002). Identification of distance education barriers for trade and industrial teacher education. Journal of Industrial Teacher Education, 40(1), 20–44.
Views and opinions expressed in this article are the views and opinions of the author(s), International Journal of Education and Practice shall not be responsible or answerable for any loss, damage or liability etc. caused in relation to/arising out of the use of the content. |