Index

Abstract

In a distance learning system, students tend to feel bored and their learning motivation decreases which makes an impact on their critical thinking (CT) being undeveloped. A motivating digital pedagogical infrastructure is needed to empower students' CT skills. The current study aimed at intervening Problem Based Learning (PBL) with PhET (Physics Education Technology) assistive virtual simulation in mobile application and evaluating its impact on students' CT skills. Experimental design (randomized pretest-posttest control and experimental groups) was carried out while the control group was intervened with expository teaching. The research sample consisted of 88 students at the Islamic senior high schools in the Mataram City, Indonesia. They were divided into experimental and control groups. Learning instruments and CT skills tests were prepared to support the research. The measurement of psychometric properties (validity and reliability) of the instrument was carried out to determine whether the instrument was feasible to be employed. The learning feasibility was observed by observers and analyzed qualitatively, based on the feedback from the observers. Data analysis of CT skills used the parameters of the average pretest-posttest and n-gain. Statistical analysis was carried out to prove the differences in students' CT between groups. The results of the learning feasibility assessment showed that PBL with PhET assistive virtual simulation is able to provide a motivating and interactive digital learning environment. The results of the student's CT skill analysis prove the efficacy of PBL with PhET's assistive virtual simulation in improving students' CT skills. This learning is better than expository teaching in improving CT skills.

Keywords: Critical thinking skills, Digital pedagogical infrastructure, Learning feasibility, Mobile application ,PhET assistive virtual simulation , Problem-based learning.

Received: 14 November 2022 / Revised: 6 March 2023/ Accepted: 3 April 2023/ Published: 26 May 2023

Contribution/ Originality

This study shows that a motivating digital pedagogical infrastructure is needed to empower students' CT skills, one of which is PBL utilizing PhET assistive virtual simulation in mobile application. It provides the evidence that that PBL with PhET assistive virtual simulation is able to provide a motivating digital learning environment, and has a powerful impact on improving students' CT skills.

1. INTRODUCTION

Various ways of learning are developed and implemented by educators in this century in their classes but CT skills are the most expected competence that learners should achieve. Expectations to get the achievement of these competencies assumedly cannot be reached when at the beginning of 2020 the COVID-19 pandemic hit the world and had an impact on the education sector. Although electronic learning (e-learning) is considered a savior for the continuity of learning to date (Mohammed, 2022). It's just that concerns about the achievement of CT competencies still exist. Especially, Indonesia right now has proclaimed the 'Freedom to Learn' program as a way of learning to create pools of developing students’ CT talent. As a concrete step in implementing the 'Freedom to Learn' program, students solve authentic problems by utilizing various learning resources while building their CT skills (Evendi, Pardi, Sucipto, Bayani, & Prayogi, 2022). To achieve this, teachers are expected to use a problem-based learning (PBL) model.

The 'Freedom to Learn' program is a strategic step to build students' CT to achieve national education standards in Indonesia and it is no surprise, considering the low CT skills of Indonesian students, even at the higher education level (Fitriani, Asy'ari, Zubaidah, & Mahanal, 2019). This program was promoted in early 2020, but a new problem arose when a few months later the learning process was stopped due to COVID-19 and continued with distance learning. Finally, until now the e-learning systems are being applied. For the purpose of training students to think critically by applying the PBL model, as far as our experiences in the teaching of sciences has been, so far, problems continue to emerge. First, teachers are required to develop a new framework to train students' CT by adapting ways of solving science problems through remote learning. This feels very inconvenient because previously teachers applied PBL in face-to-face learning mode. Second, teachers faced difficulties to perform a motivating and interesting environment in distance learning. Distance learning is expected to be a bridge to the digital learning system that motivates students to learn (Papadakis, Trampas, Barianos, Kalogiannakis, & Vidakis, 2020). Third, in fact, students are less motivated in science courses with e-learning systems as they experience boredom in the classroom. Their limited space and time to access online are much longer than before, as well as non-interactive learning content is the cause. Non-interactive learning content will only burden teachers and have no impact on the expected learning outcomes (Viktorova, 2020) and an unmotivated digital environment has an impact on knowledge input and less convincing learning outcomes (Schmitz, Klemke, & Specht, 2012).

To train CT skills with the PBL model, the technology devices of the e-learning system need to be prepared and implemented well. Adaptation of a number of technologies is needed and should be suited to students’ learning needs (Drolia, Sifaki, Papadakis, & Kalogiannakis, 2020). The orientation of fulfilling the needs is not on presenting a large amount of boring learning material content but on the management of attractive, motivating, and interesting learning. In essence, the learning process that utilizes digital platforms must have a positive effect on the expected learning objectives (Cheng, Yang, Chang, & Kuo, 2015). Previous studies have shown that teaching methods utilizing attractive and interactive digital technology have an impact on improving students' cognitive learning outcomes (NNSP Verawati, Handriani, & Prahani, 2022) and their CT skills (Bilad, Anwar, & Hayati, 2022). This is an entry point for teachers to be able to take advantage of interactive learning technologies, one of those is PhET (Physics Education Technology) assistive virtual simulation. The utilization opportunities can be further optimized considering that PhET can be connected to a mobile application and can be easily accessed by each user.

Referring to the justification of the aforementioned problems, the current study aimed to apply problem-based learning that utilizes PhET assistive virtual simulations in mobile application and evaluate its impact on students' CT skills. Specifically, this study sought answers to the following research questions.

  1. What is the learning feasibility of applying problem-based learning that utilizes PhET assistive virtual simulations in mobile application?
  2. What is the impact of using problem-based learning that utilizes PhET assistive virtual simulations on improving students’ critical thinking skills?

2. LITERATURE REVIEW

2.1. Critical Thinking Skills

In the contemporary education system, educational institutions have the fundamental task to equip students with CT skills (Erikson & Erikson, 2019). CT becomes the driving force to achieve better academic performance (D’Alessio, Avolio, & Charles, 2019; Ghanizadeh, 2017; Siburian, Corebima, Ibrohim, & Saptasari, 2019). Developing students' CT must be followed by intervention in the teaching process of critical thinking in modern ways in classes (Dekker, 2020). Teaching to get the achievement of CT skills is increasingly massive in several developed countries and they include CT as an attribute of graduate competence at all levels of education (Szenes, Tilakaratna, & Maton, 2015). Students who have good performance of CT skills are believed as the main factor to declare the quality of education (Gilmanshina, Smirnov, Ibatova, & Berechikidze, 2021).

The term CT is defined in the most familiar context as "reasonable and reflective thinking, that is focused on deciding what to believe or do" (Ennis, 2018). Within the scope of the cognitive or intellectual dimension, CT is a process of deep reasoning (Elder & Paul, 2012). Regarding the learning in the classroom, the ability to reason is a goal of all types of instruction (National Council of Teachers of Mathematics, 2000). CT skills are needed as a cognitive bridging in solving the problems (Evendi et al., 2022). Every expert or researcher engaged in educational psychology gives a baseline that learners have competencies in CT or vice versa, but the most intensive slice of indicators used in several studies covered; analyzing, inferring, evaluating, and making decisions (Verawati, Hikmawati, & Prayogi, 2020; Verawati, Ernita, & Prayogi, 2022; Verawati, Hikmawati, Prayogi, & Bilad, 2021; Wahyudi, Verawati, Ayub, & Prayogi, 2018; Wahyudi, Ayub, & Prayogi, 2019a; Wahyudi, Verawati, Ayub, & Prayogi, 2019b). Training students’ CT requires a unique innovative learning model intervention. It means that not all types of instruction can be used for this purpose (Prayogi, Yuanita, & Wasis, 2018). However, the uniqueness of learning activities to practice CT is ways of exploring and experimenting to solve problems, one of which is Problem-based Learning (PBL) (Suhirman, Prayogi, & Asy’ari, 2021).

2.2. Problem-Based Learning

Problem-based Learning (PBL) is a learning model that encourages students to conduct investigations or explorations based on authentic problems (Evendi et al., 2022). PBL model follows the learning phases, namely: a) student orientation to authentic problems, b) organizing students to learn, c) investigative processes to solve problems, d) presenting the results of the investigation, and e) reflecting on the results of problem-solving (Evendi et al., 2022). New knowledge products can be emerged by presenting a number of authentic problems (Hung, 2011). In addition, presenting authentic problems and solving them in investigative ways has an impact on knowledge retention and a better understanding of concepts (Li & Tsai, 2017). If it is associated with learning content, the pedagogy in PBL has a positive impact on students' reasoning performance (Wirkala & Kuhn, 2011). Specifically, in dealing with the achievement of CT, problem-solving through exploration and investigation helps students develop their CT performance (Calkins, Grannan, & Siefken, 2020).

The PBL model can be applied in flexible ways. It can be done by utilizing a digital environment. It is in line with the previous study (Evendi et al., 2022) PBL is employed through a digital environment using an e-learning platform (e-PBL), and the result is that it can develop students' CT skills in learning on all cognitive styles of learners (Evendi et al., 2022). The effectiveness of mobile blended PBL has been investigated and the results showed that it can improve students' problem-solving skills (Amin, Sudana, Setyosari, & Djatmika, 2021). The functions and principles contained in PBL do not regress when formatted in blended learning (De Jong, Krumeich, & Verstegen, 2017). Another study Srikan, Pimdee, Leekitchwatana, and Narabin (2021) developed cloud-based PBL to improve students' digital skills and creative thinking. In the current study, researchers intervened in PBL by utilizing the PhET assistive virtual simulation in mobile application and evaluated its impact on students' CT skills.

2.3. Assistive Virtual Simulation

Assistive technology was originally developed to fulfil and facilitate access to students’ learning rights of disabilities (Smith, 2021) but later developed as a learning visualization instrument along with the advancement of simulation technology, and is now known as assistive virtual simulation (AVS). Virtual simulation is categorized as a digital learning tool used to simplify the learning process (Zhang, Tan, Li, Han, & Xu, 2021) and as a digital learning bridge to understanding abstract concepts (Verawati et al., 2022). Virtual simulation is increasingly being employed because of its role in cultivating the new skills needed (Bedetti, Bertolaccini, Patrini, Schmidt, & Scarci, 2018) including to fostering students' critical thinking (Ikhsan, Sugiyarto, & Astuti, 2020).

One of the most assistive virtual simulation technologies widely used is PhET (Physics Education Technology). PhET is an online based science laboratory simulation which is designed interactively and attractively and developed by the University of Colorado. PhET is a suitable tool to be used when learning is constrained to carry out practical work in the laboratory, even when exploration or experimentation is carried out online. Another advantage is that PhET simulation can help students understand the concepts of abstract learning materials. In accordance with the current study context, researchers see it as an opportunity to improve students' CT performance by intervening in Problem Based Learning (PBL) using PhET's assisted virtual simulation in mobile application. The utilization opportunities can be further optimized considering that PhET can be connected to a mobile application and can be easily accessed by each student. In previous studies, PhET simulations could support students' deep conceptual mastery of different learning materials (Fan, Geelan, & Gillies, 2018) and support students’ engagement with the content of the teaching material (Toli & Kallery, 2021).

3. METHODOLOGY

3.1. Research Design

This study was designed in the form of an experimental study by employing the randomized pretest-posttest control group design (Fraenkel, Wallen, & Hyun, 2012). It was employed to show the precise effect of a controlled learning treatment. Referring to the design, two treatment groups (samples) were prepared, namely the experimental and control groups. The design is presented in Table 1.

Table 1. Research design.
Treatment group Pre-assessment Learning treatment Post-assessment
Experimental Pretest (O1) X1 (PBL + Assistive virtual simulation) Posttest (O2)
Control Pretest (O1) X2 (Expository teaching) Posttest (O2)

The treatment or learning intervention for the experimental group applied PBL with PhET assistive virtual simulation in mobile application, while the intervention for the control group used expository teaching. Assessment of critical thinking skills was carried out on students in each group, pretest before the learning intervention and posttest after it. Implementation of learning was carried out in four meetings (excluding pretest and posttest) on the material of ‘Energy Forms and Changes,’ and each meeting took about 2 x 45 minutes. The implementation of the pretest, learning treatment, and posttest was scheduled at the same time. This is a form of deliberate arrangement to avoid the possibility of bias.

3.2. Participants

Research participants involved students and observers, where students acted as research samples, while observers were those who made observations on the learning process during learning implementation. The research sample was students in one of the Islamic senior high schools (Madrasah Aliyah) in the city of Mataram, Indonesia. There were 88 students evenly divided into two groups (experimental, n = 44, and control, n = 44). The two observers were science teachers who had experience of teaching science for at least last three years and who were considered representative in observing the implementation of learning in class. The demographics of the sample are presented in Table 2.

Table 2. Demographics of samples.
Characteristics
Experimental group,
N = 44
Control group,
N = 44
Total
%
Total
%
Gender
Male
24
54.54
23
52.27
Female
20
45.46
21
47.73
Age (Year)
< 16
2
4.54
3
6.82
16 – 17
42
95.46
40
90.91
> 17
0
0
1
2.27

3.3. Instruments and Psychometric Properties

To support the implementation of the study, learning tools in the form of e-modules, learning scenarios, and CT skills tests were prepared. The CT skills test instrument employed eight test items to accommodate indicators of analytical, inference, evaluation and decision-making skills. Before being employed, learning instruments must meet adequate psychometric properties. The validity and reliability of the instrument are the best measures of psychometric properties as a prerequisite for continuous implementation (Souza, Alexandre, & Guirardello, 2017). The learning instruments that are prepared and validated cover e-modules, learning scenarios, and tests of CT skills. The measure of instrument validity is viewed from the aspect of content and construct validity (Akker, Bannan, Kelly, Nieveen, & Plomp, 2013). The learning instrument was validated by two validators, who were science learning experts, experienced in designing science lessons, and have more than fifteen years of experience in teaching science. The validators assessed the validity and provided feedback on the learning instrument. Finally, the validation results were analyzed descriptively, which showed that the learning instrument was valid in the content and construct domains. From the results of this validation, it the reliability of the learning instrument was determined (the level of consistency of the instrument based on the size of its validity). This refers to the percentage of agreement parameter (Emmer & Millett, 1970). Based on the results of this validity and reliability test, the learning instrument was therefore reliable and was feasible to be used as learning support.

3.4. Data Analysis

Data on students' CT skills were collected using the CT skill test instrument which was found valid. The CT test in the form of an essay (eight item questions) was employed to measure CT (pretest and posttest). The score for each test item used a graded scale, where the maximum score was +3, and the minimum was -1. This score was then calculated as the cumulative performance of each individual's CT skills, corresponding to criteria ranging from not critical to very critical. CT skills were calculated according to individual performance and CT indicators were adapted from previous studies  (Prayogi et al., 2018) as presented in Table 3.

Table 3. Criteria of CT skills.
Criteria
Range of CT scores based on indicators
Range of individual CT scores
Very critical
CTi > 2.20
CTs > 17.6
Critical
1.40 < CTi ≤ 2.20
11.2 < CTs ≤ 17.6
Quite critical
0.60 < CTi ≤ 1.40
4.8 < CTs ≤ 11.2
Low critical
– 0.20 < CTi ≤ 0.60
–1.6 < CTs ≤ 4.8
Not critical
CTi ≤ – 0.20
CTs ≤ –1.6

Each data on students' CT skills (pretest and posttest) in the experimental and control groups was analyzed descriptively and statistically. Descriptive analysis refers to the criteria seen in Table 3. The increase of the pretest-posttest score of students' CT skills (n-gain) was analyzed according to Hake's formulation (Hake, 1999) and categorized into n-gain criteria: low (<0.3), moderate (0.3 – 0.7), and high (>0.7). Statistical analysis (differential test) was employed to analyze the differences in students' CT performance in each treatment group (based on the pre-test-posttest mean score parameter) and between treatment groups (based on the post-test mean score parameter), both at the level of 0.05 significance. This difference indicated the effect of the intervention or learning treatment. The statistical difference test was preceded by a normality test. If the results of the normality test indicated that the data was normally distributed, parametric statistical analysis was used. Otherwise, non-parametric statistical analysis would be used. Statistical analysis employed SPSS software.

4. RESULTS

4.1. The Implementation of PBL Utilizing Assistive Virtual Simulation

Distance learning has successfully been conducted through an e-learning system using the PBL model utilizing assistive virtual simulations in the teaching of sciences. PhET assistive virtual simulation in mobile application is employed as an aid of PBL model learning. In this research study, the learning processes were done in four meetings. Each meeting took about 2 x 45 minutes. The flow of learning with PBL followed some phases: a) student orientation to authentic problems, b) organizing students to learn, c) investigative process to solve problems, d) presenting the results of investigations, and e) reflecting on the results of problem-solving (Evendi et al., 2022). PhET assistive virtual simulation intervention was utilized starting from the third phase, namely the investigation process to solve the problem. Figure 1 presents an example of the intervention of PhET assistive virtual simulation in the materials of ‘Energy Forms and Changes’

Figure 1. An example of PhET assistive virtual simulation intervention in materials of “energy forms and changes”.

Source:

https://phet.colorado.edu/.

Furthermore, the implementation of learning (learning feasibility) for each learning meeting was observed by two observers. It related to the implementation of the learning phase with the PBL model utilizing PhET assistive virtual simulation in mobile application. The data of observation was qualitative, where observers provided feedback on the learning process immediately after learning was carried out in each meeting. Feedback was discussed between teachers (researchers) and the observers.

4.1.1. First Meeting

4.1.2. Second Meeting

4.1.3. Third Meeting

4.1.4. Fourth Meeting

4.2. Assessment of Students’ Critical Thinking Skills

Data on the results of students' CT skills are based on CT indicators for each treatment group (experimental and control) as shown in Figure 2.

Figure 2. Data of analysis results of students’ CT skills based on indicators in each group.

The results of the analysis indicate a change in critical thinking scores for each treatment group from pretest to posttest. Based on the CT skill indicator parameters (analysis, inference, evaluation, decision making), the experimental group's CT score in the posttest exceeded the control group.

The pretest results of both groups are in the range of CT indicator scores: – 0.20 < CTi 0.60. This falls in the "less critical" category. Furthermore, the results of the posttest analysis in the experimental group, the average CT score for all indicators was "critical" (all indicator scores are in the range: 1.40 < CTi 2.20). While the control group varied, the posttest scores on the evaluation indicators had the criteria of "less critical" (in the range of scores: – 0.20 < CTi 0.60), and other indicators (analysis, inference, decision making) had the criteria of "quite critical" (in the range of scores: 0.60 < CTi 1.40). If the improvement is viewed from the n-gain in pretest-posttest scores, the n-gain of all CT indicators in the experimental group has the "moderate" criteria (n-gain score range: 0.3 - 0.7), while the n-gain in the control group has the "low" criteria (n-gain score < 0.3).

The data of the analysis of students' CT skills for each treatment group (experimental and control) are presented in Figure 3. Based on the test parameters (pre-test, post-test, and n-gain), the two treatment groups found varying scores.

Figure 3. Data analysis results of students’ CT skills in each group.

The pretest-posttest scores in the experimental group increased, from an average score of 1.45 with the "less critical" criteria (on the pretest) to 16.60 with the "critical" criteria (on the posttest). Unlike in the control group, students experienced an increase in score from 1.48 with the "less critical" criteria (on the pre-test) to 5.64 with the "quite critical" criteria (on the posttest). Changes in the n-gain scores of the experimental and control groups are far apart, each with a score of 0.67 (moderate criteria) and 0.18 (low criteria).

The statistical analysis was carried out to evaluate the details of the differences in increasing scores of each treatment group. As a reference of statistical testing are the results of data normality test. Details of the results of the data normality test are presented in Table 4.

Table 4. Results of normality test, p > 0.05.
Groups Parameter
Kolmogorov-Smirnov
Shapiro-Wilk
Data normality
Stat.
Df
Sig.
Stat.
Df
Sig.
Experimental Pre-test
0.183
44
0.001
0.940
44
0.023
 No
Post-test
0.333
44
0.000
0.791
44
0.000
 No
Control Pre-test
0.184
44
0.001
0.937
44
0.018
 No
Post-test
0.165
44
0.004
0.915
44
0.003
 No

The Kolmogorov-Smirnov and Shapiro-Wilk tests were employed to measure the normality of the data in experimental and control group. The results of the normality test based on the pre-test and post-test parameters were none of the test variables were normally distributed. The data tested were not normally distributed (p-value < 0.05). Based on these results, the different test used the non-parametric statistical analysis. The paired sample difference test employed the Wilcoxon test. For the independent sample, the Mann-Whitney U test was applied. The results of the two tests are summarized in Table 5 and Table 6.

Table 5. Results of Wilcoxon tests, p < 0.05.
Group
N
 Mean rank
Sum of ranks
Z
Sig.
Experimental Negative ranks
0
0.00
0.00
-5.803
0.000
Positive ranks
44
66.50
2926.00
Ties
0
0.00
0.00
Total
44
66.50
2926.00
Control Negative ranks
0
0.00
0.00
-5.798
0.000
Positive ranks
44
22.50
990.00
Ties
0
0.00
0.00
Total
44
22.50
990.00

Table 6. Results of Mann-Whitney tests, p < 0.05.
Group
N
Mean rank
Sum of ranks
Mann-Whitney
Z
Sig.
Experimental
44
66.50
2926.00
0.000
-8.214
0.000
Control
44
22.50
990.00
Total
88
89.00
3916.00

The results of the Wilcoxon test (Table 5) confirmed that there was a difference of CT skill scores on the pre-test-posttest for the two treatment groups (p-value < 0.05). It means that the learning treatment in the experimental group and the control group has an impact on CT skills. Furthermore, differences in CT skills between treatment groups based on posttest parameters were analyzed using the Mann-Whitney test (see Table 6). The results showed a difference of CT performance between the experimental and control groups (p-value < 0.05). These results have confirmed the significant impact of the PBL model with assistive virtual simulation on students' CT skills.

5. DISCUSSION

Developing students' CT skills has been attempted by implementing PBL pedagogy utilizing PhET assistive virtual simulation in mobile application. In the distance learning system, the proper pedagogy is a reflection of structured learning (Pozzi et al., 2020). To achieve this, this current study employed observers who were tasked with observing the progress of learning and providing feedback on the learning process. In addition, researchers reviewed the results of the CT measurement (pretest) before the learning treatment intervention in the experimental and control groups. The results of the pretest in the experimental group (PBL with a PhET assistive virtual simulation in mobile application) and the control group (expository teaching) were both in the "less critical" category (based on indicator parameters). After the learning intervention in the experimental group, the posttest results increased to "critical" (see Figure 2). Likewise with individual CT scores (see Figure 3), the average pretest-posttest in the experimental group was improved, from "less critical" (in the pretest) to "critical" (in the post-test). This is, of course, the effect of PBL with a well-implemented PhET assistive virtual simulation, and cannot be separated from observer feedback during learning.

Based on observer feedback during the learning process, the aspects that teachers need to pay attention to implement PBL using PhET virtual assistive simulation are motivating students, organizing learning, diversifying authentic problems, and encouraging interaction among students. Based on observer feedback on these aspects, improvements of the learning process were carried out in each subsequent meeting. The focus of improvements made by the teacher was: First, optimizing student motivation. Motivating students is one aspect of supporting the success of PBL pedagogy, especially when it is carried out systematically so as to encourage deep learning (Harun, Yusof, Jamaludin, & Hassan, 2012). Motivating by the teacher provides an overview of the importance of a learning process that will be passed by students, so it is clear the students’ reasons to continue to be involved in the learning process (Arends, 2012). Student motivation can be optimized at the beginning of learning and during the learning process (Fukuzawa, Boyd, & Cahn, 2017). The results of previous studies show that student motivation also supports the development of students' CT (Festiawan, 2021). The teachers’ encouragement in motivating students has an impact on the effectiveness of the PBL model (Luo, 2019).

Second, optimizing the organization of learning, the implementation of PBL is pursued in accordance with the phases of teaching, as a guarantee of achieving the expected learning objectives, the control of the learning process of PBL must be optimal. Third, diversify authentic problems, in PBL pedagogy, to support students' CT flexibility, authentic problems need to be diversified (Evendi et al., 2022). Knowledge construction is built based on presenting authentic problems, and solving them as evidence of students' depth of thinking (Kumar & Natarajan, 2007).  Learning based on authentic problem solving can develop students' CT skills (Dennis & O'Hair, 2010). It is also claimed to be the foundation for effectively training 21st-century skills (Preus, 2012). The ways teachers diversify authentic problems into additional preferences (tastes) when students learn (Monrat, Phaksunchai, & Chonchaiya, 2022). Fourth, continue to encourage interaction between students during the learning process. Interaction means that during the discussion students are expected to convey ideas and get responses from other students.

Observers’ feedback benefits students to get CT in learning with the PBL model using PhET assistive virtual simulation. This is different from expository teaching (as the control group in this study). On the posttest, they did not achieve good results in CT activities. This is in accordance with the results of previous studies that expository teaching methods are considered as traditional or conventional teaching, and are considered inadequate for the purpose of providing in-depth knowledge and training in critical thinking (Bilad et al., 2022). Meanwhile, PBL is considered feasible as a pedagogy that leads students to develop their CT through deep conceptual knowledge (Fitriani, Samsuri, Rachmadiarti, Raharjo, & Mantlana, 2022). The difference in learning results in the posttest of the experimental and control groups is evidence of the impact of PBL interventions using PhET assistive virtual simulations that are effective to improve students' CT skills (see Table 6). The strength of the PBL intervention is the use of PhET assistive virtual simulation. The results of previous studies show the power of PhET virtual simulation in building a comprehensive understanding of materials with a high level of abstraction (Verawati et al., 2022). The current study addressed the problems related to online learning utilizing PBL approach with PhET assistive virtual simulation as a new framework for teachers to train students' CT. Through these learning interventions, it was found that teachers provided a motivating and engaging environment for remote learning, as was expected in today's digital pedagogy (Papadakis et al., 2020). The results proved that PBL with PhET assistive virtual simulation had overcome student boredom in classes. It was evident from the dynamic interaction that has been built in learning. It also made an impact on developing students' CT skills. This finding refutes previous concerns that non-interactive learning content will only be burdensome and have no impact on expected learning achievement (Viktorova, 2020) and that unmotivated digital environments have an impact on knowledge input and less convincing learning outcomes (Schmitz et al., 2012). In fact, PBL with PhET assistive virtual simulation has provided an interactive learning situation and has an impact on improving students' CT skills. The results of this study are in accordance with previous findings, that Android-based PhET Simulation can improve students' critical thinking skills in science learning (Hasyim, Prastowo, & Jatmiko, 2020). In addition to critical thinking, the application of PhET simulation can also trigger students' creative thinking skills and conceptual understanding (Habibi, Jumadi, & Mundilarto, 2020).

6. CONCLUSION

The present study applied Problem Based Learning (PBL) by utilizing PhET assistive virtual simulation in mobile application. The implementation of the learning has been observed and the impact on students' CT skills has been analyzed. Several aspects need to be addressed in implementing PBL by utilizing PhET's assistive virtual simulation, including optimizing student motivation, organizing learning, diversifying authentic problems, and encouraging interactivity among students. Descriptive analysis shows that PBL by utilizing PhET assistive virtual simulation is better in developing students' CT skills when compared to expository teaching. Statistical analysis showed a significant difference in improving students' CT skills between PBL using PhET assistive virtual simulation and expository teaching. Finally, based on the findings in this study, researchers hope that PBL pedagogy utilizing PhET assistive virtual simulation can be maintained and implemented in regular classroom teaching. Similar studies can be conducted on other materials or other subjects to explore their effects not only on CT skills but also on other learning outcomes.

Funding: This study received no specific financial support.  

Competing Interests: The authors declare that they have no competing interests.

Authors’ Contributions: Both authors contributed equally to the conception and design of the study.

REFERENCES

Akker, J. V. D., Bannan, B., Kelly, A. E., Nieveen, N., & Plomp, T. (2013). Educational design research: An introduction. Enschede, Netherlands: Netherlands Institute for Curriculum Development (SLO).

Amin, A. K., Sudana, I. N. D., Setyosari, P., & Djatmika, E. T. (2021). The effective-ness of mobile blended problem based learning on mathematical problem solving. International Journal of Interactive Mobile Technologies, 15(1), 1-23. https://doi.org/10.3991/ijim.v15i01.17437

Arends, R. I. (2012). Learning to teach (9th ed.). New York: McGraw-Hill.

Bedetti, B., Bertolaccini, L., Patrini, D., Schmidt, J., & Scarci, M. (2018). Virtual simulation and learning new skills in video-assisted thoracic surgery. Video-Assisted Thoracic Surgery, 3, 35–35. https://doi.org/10.21037/vats.2018.08.03

Bilad, M. R., Anwar, K., & Hayati, S. (2022). Nurturing prospective STEM teachers’ critical thinking skill through virtual simulation-assisted remote inquiry in fourier transform courses. International Journal of Essential Competencies in Education, 1(1), 1-10. https://doi.org/10.36312/ijece.v1i1.728

Calkins, S., Grannan, S., & Siefken, J. (2020). Using peer-assisted reflection in math to foster critical thinking and communication skills. Primus, 30(4), 475-499. https://doi.org/10.1080/10511970.2019.1608608

Cheng, P.-H., Yang, Y.-T. C., Chang, S.-H. G., & Kuo, F.-R. R. (2015). 5E mobile inquiry learning approach for enhancing learning motivation and scientific inquiry ability of university students. IEEE Transactions on Education, 59(2), 147-153. https://doi.org/10.1109/te.2015.2467352

D’Alessio, F. A., Avolio, B. E., & Charles, V. (2019). Studying the impact of critical thinking on the academic performance of executive MBA students. Thinking Skills and Creativity, 31, 275-283. https://doi.org/10.1016/j.tsc.2019.02.002

De Jong, N., Krumeich, J., & Verstegen, D. M. (2017). To what extent can PBL principles be applied in blended learning: Lessons learned from health master programs. Medical Teacher, 39(2), 203-211. https://doi.org/10.1080/0142159x.2016.1248915

Dekker, T. J. (2020). Teaching critical thinking through engagement with multiplicity. Thinking Skills and Creativity, 37, 100701. https://doi.org/10.1016/j.tsc.2020.100701

Dennis, J., & O'Hair, M. J. (2010). Overcoming obstacles in using authentic instruction: A comparative case study of high school math & science teachers. American Secondary Education, 38(2), 4-22.

Drolia, M., Sifaki, E., Papadakis, S., & Kalogiannakis, M. (2020). An overview of mobile learning for refugee students: Juxtaposing refugee needs with mobile applications’ characteristics. Challenges, 11(2), 1-14.

Elder, L., & Paul, R. (2012). The Thinker’s guide to analytic thinking. Tomales, CA: Foundation for Critical Thinking.

Emmer, E. T., & Millett, G. B. (1970). Improving teaching through experimentation: A laboratory approach. Englewood Cliffs, NJ: Prentice-Hall.

Ennis, R. H. (2018). Critical thinking across the curriculum: A vision. Topoi, 37(1), 165-184. https://doi.org/10.1007/s11245-016-9401-4

Erikson, M. G., & Erikson, M. (2019). Learning outcomes and critical thinking–good intentions in conflict. Studies in Higher Education, 44(12), 2293-2303. https://doi.org/10.1080/03075079.2018.1486813

Evendi, E., Pardi, M. H. H., Sucipto, L., Bayani, F., & Prayogi, S. (2022). Assessing students’ critical thinking skills viewed from cognitive style: Study on implementation of problem-based e-learning model in mathematics courses. Eurasia Journal of Mathematics, Science and Technology Education, 18(7), em2129. https://doi.org/10.29333/ejmste/12161

Fan, X., Geelan, D., & Gillies, R. (2018). Evaluating a novel instructional sequence for conceptual change in physics using interactive simulations. Education Sciences, 8(1), 29-48. https://doi.org/10.3390/educsci8010029

Festiawan, R. (2021). The problem-based learning: How the effect on student critical thinking ability and learning motivation in COVID-19 pandemic? Journal of Sport Area, 6(2), 231–243. https://doi.org/10.25299/sportarea.2021.vol6(2).6393

Fitriani, H., Asy'ari, M., Zubaidah, S., & Mahanal, S. (2019). Exploring the prospective teachers’ critical thinking and critical analysis skills. Journal of Indonesian Science Education, 8(3), 379-390. https://doi.org/10.15294/jpii.v8i3.19434

Fitriani, H., Samsuri, T., Rachmadiarti, F., Raharjo, R., & Mantlana, C. D. (2022). Development of evaluative-process learning tools integrated with conceptual-problem-based learning models: Study of its validity and effectiveness to train critical thinking. International Journal of Essential Competencies in Education, 1(1), 27-37. https://doi.org/10.36312/ijece.v1i1.736

Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate re-search (8th ed.). New York: Mc Graw Hill.

Fukuzawa, S., Boyd, C., & Cahn, J. (2017). Student motivation in response to problem-based learning. Collected Essays on Learning and Teaching, 10, 175-188. https://doi.org/10.22329/celt.v10i0.4748

Ghanizadeh, A. (2017). The interplay between reflective thinking, critical thinking, self-monitoring, and academic achievement in higher education. Higher Education, 74(1), 101-114. https://doi.org/10.1007/s10734-016-0031-y

Gilmanshina, S., Smirnov, S., Ibatova, A., & Berechikidze, I. (2021). The assessment of critical thinking skills of gifted children before and after taking a critical thinking development course. Thinking Skills and Creativity, 39, 100780. https://doi.org/10.1016/j.tsc.2020.100780

Habibi, H., Jumadi, J., & Mundilarto, M. (2020). Phet simulation as means to trigger the creative thinking skills of physics concepts. International Journal of Emerging Technologies in Learning, 15(6), 166. https://doi.org/10.3991/ijet.v15i06.11319

Hake, R. R. (1999). Analyzing change/gain scores. Woodland Hills, CA - USA: Indiana University.

Harun, N. F., Yusof, K. M., Jamaludin, M. Z., & Hassan, S. A. H. S. (2012). Motivation in problem-based learning implementation. Procedia-Social and Behavioral Sciences, 56, 233-242. https://doi.org/10.1016/j.sbspro.2012.09.650

Hasyim, F., Prastowo, T., & Jatmiko, B. (2020). The use of android-based PhET Simula-tion as an effort to improve students’ critical thinking skills during the covid-19 pandemic. International Journal of Interactive Mobile Technologies, 14(19), 1-11. https://doi.org/10.3991/ijim.v14i19.15701

Hung, W. (2011). Theory to reality: A few issues in implementing problem-based learning. Educational Technology Research and Development, 59(4), 529-552. https://doi.org/10.1007/s11423-011-9198-1

Ikhsan, J., Sugiyarto, K. H., & Astuti, T. N. (2020). Fostering student’s critical thinking through a virtual reality laboratory. International Journal of Interactive Mobile Technologies, 14(8), 183. https://doi.org/10.3991/ijim.v14i08.13069

Kumar, M., & Natarajan, U. (2007). A problem-based learning model: Showcasing an educational paradigm shift. The Curriculum Journal, 18(1), 89-102. https://doi.org/10.1080/09585170701292216

Li, H.-C., & Tsai, T.-L. (2017). The implementation of problem-based learning in a Taiwanese primary mathematics classroom: Lessons learned from the students’ side of the story. Educational Studies, 43(3), 354-369. https://doi.org/10.1080/03055698.2016.1277138

Luo, Y.-J. (2019). The influence of problem-based learning on learning effectiveness in students’ of varying learning abilities within physical education. Innovations in Education and Teaching International, 56(1), 3-13. https://doi.org/10.1080/14703297.2017.1389288

Mohammed, D. Y. (2022). The web-based behavior of online learning: An evaluation of different countries during the COVID-19 pandemic. Advances in Mobile Learning Educational Research, 2(1), 263-267. https://doi.org/10.25082/amler.2022.01.010

Monrat, N., Phaksunchai, M., & Chonchaiya, R. (2022). Developing students’ mathe-matical critical thinking skills using open-ended questions and activities based on student learning preferences. Education Research International, 1–11. https://doi.org/10.1155/2022/3300363

National Council of Teachers of Mathematics. (2000). Principles, standards, and expec-tations—national council of teachers of mathematics. Retrieved from https://www.nctm.org/Standards-and-Positions/Principles-and-Standards/Principles,-Standards,-and-Expectations/

Papadakis, S., Trampas, A., Barianos, A., Kalogiannakis, M., & Vidakis, N. (2020). Evaluating the learning process: The “thimeledu” educational game case study. Paper presented at the Proceedings of the 12th International Conference on Computer Supported Education.

Pozzi, F., Asensio-Perez, J. I., Ceregini, A., Dagnino, F. M., Dimitriadis, Y., & Earp, J. (2020). Supporting and representing Learning Design with digital tools: In between guidance and flexibility. Technology, Pedagogy and Education, 29(1), 109-128. https://doi.org/10.1080/1475939x.2020.1714708

Prayogi, S., Yuanita, L., & Wasis. (2018). Critical inquiry based learning: A model of learning to promote critical thinking among prospective teachers of physic. Journal of Turkish Science Education, 15(1), 43-56.

Preus, B. (2012). Authentic instruction for 21st century learning: Higher order thinking in an in-clusive school. American Secondary Education, 40(3), 59-79.

Schmitz, B., Klemke, R., & Specht, M. (2012). Effects of mobile gaming patterns on learning outcomes: A literature review. International Journal of Technology En-Hanced Learning, 4(5–6), 345–358. https://doi.org/10.1504/ijtel.2012.051817

Siburian, O., Corebima, A. D., Ibrohim, I., & Saptasari, M. (2019). The correlation be-tween critical and creative thinking skills on cognitive learning results. Eurasian Journal of Educational Research, 19(81), 1-16. https://doi.org/10.14689/ejer.2019.81.6

Smith, E. M. (2021). Assistive technology research: Evidence for a complex and growing field. Assistive Technology, 33(4), 177-177. https://doi.org/10.1080/10400435.2021.1958650

Souza, A. C. D., Alexandre, N. M. C., & Guirardello, E. D. B. (2017). Psychometric properties in instruments evaluation of reliability and validity. Epidemiology and Health Services, 26(3), 649-659.

Srikan, P., Pimdee, P., Leekitchwatana, P., & Narabin, A. (2021). A problem-based learning (PBL) and teaching model using a cloud-based constructivist learning environment to enhance Thai undergraduate creative thinking and digital media skills. International Journal of Interactive Mobile Technologies, 15(22), 68-83. https://doi.org/10.3991/ijim.v15i22.24963

Suhirman, S., Prayogi, S., & Asy’ari, M. (2021). Problem-based learning with character-emphasis and naturalist intelligence: Examining students critical thinking and cu-riosity. International Journal of Instruction, 14(2), 217-232. https://doi.org/10.29333/iji.2021.14213a

Szenes, E., Tilakaratna, N., & Maton, K. (2015). The knowledge practices of critical thinking. In M. Davies & R. Barnett (Eds.), The Palgrave Handbook of Critical Thinking in Higher Education. In (pp. 573–591). New York: Palgrave Macmillan.

Toli, G., & Kallery, M. (2021). Enhancing student interest to promote learning in science: The case of the concept of energy. Education Sciences, 11(5), 220. https://doi.org/10.3390/educsci11050220

Verawati, N., Handriani, L. S., & Prahani, B. K. (2022). The experimental experience of motion kinematics in biology class using PhET virtual simulation and its impact on learning outcomes. International Journal of Essential Competencies in Education, 1(1), 11-17. https://doi.org/10.36312/ijece.v1i1.729

Verawati, N., Hikmawati, H., & Prayogi, S. (2020). The effectiveness of inquiry learning models intervened by reflective processes to promote critical thinking ability in terms of cognitive style. International Journal of Emerging Technologies in Learning, 15(16), 212-220. https://doi.org/10.3991/ijet.v15i16.14687

Verawati, N. N. S. P., Ernita, N., & Prayogi, S. (2022). Enhancing the reasoning per-formance of STEM students in modern physics courses using virtual simulation in the LMS platform. International Journal of Emerging Technologies in Learning, 17(13), 267-277. https://doi.org/10.3991/ijet.v17i13.31459

Verawati, N. N. S. P., Hikmawati, H., Prayogi, S., & Bilad, M. R. (2021). Reflective practices in inquiry learning: Its effectiveness in training pre-service teachers’ critical thinking viewed from cognitive styles. Journal of Indonesian Science Education, 10(4), 505-514. https://doi.org/10.15294/jpii.v10i4.31814

Viktorova, L. V. (2020). Educational conditions for implementation of adults’ distance learning of foreign languages. Information Technologies and Learning Tools, 75(1), 13-25. https://doi.org/10.33407/itlt.v75i1.2797

Wahyudi, P., Verawati, N. N. S., Ayub, S., & Prayogi, S. (2018). Development of in-quiry-creative-process learning model to promote critical thinking ability of phys-ics prospective teachers. Journal of Physics: Conference Series, 1108, 012005. https://doi.org/10.1088/1742-6596/1108/1/012005

Wahyudi, Verawati, N. N. S. P., Ayub, S., & Prayogi, S. (2019a). Effectiveness of in-quiry-creative-process learning model to promote critical thinking ability of pro-spective physics teachers. Journal of Physics: Conference Series, 1417, 012071. https://doi.org/10.1088/1742-6596/1417/1/012071

Wahyudi, W., Verawati, N. N. S. P., Ayub, S., & Prayogi, S. (2019b). The effect of scientific creativity in inquiry learning to promote critical thinking ability of pro-spective teachers. International Journal of Emerging Technologies in Learning, 14(14), 1-10. https://doi.org/10.3991/ijet.v14i14.9532

Wirkala, C., & Kuhn, D. (2011). Problem-based learning in K-12 education: Is it effec-tive and how does it achieve its effects? American Educational Research Journal, 48(5), 1157–1186.

Zhang, N., Tan, L., Li, F., Han, B., & Xu, Y. (2021). Development and application of digital assistive teaching system for anatomy. Virtual Reality & Intelligent Hardware, 3(4), 315-335. https://doi.org/10.1016/j.vrih.2021.08.005

Views and opinions expressed in this article are the views and opinions of the author(s), International Journal of Education and Practice shall not be responsible or answerable for any loss, damage or liability etc. caused in relation to/arising out of the use of the content.