Abstract
Virtualization technology has been around for many years, and its use is increasingly becoming common in education, in general and in computing fields, in particular. This can be attributed partly to its potential to reduce costs, boost efficiency and overcome limited resources through its virtual applications such as servers, storage devices and networks. However, the question is: "would that be at the cost of the quality of learning?" Besides, the way virtualized environments impact motivation needs further clarification, since it is a neglected area of research. This quasi-experimental study tried to address such questions through investigating the impact of virtualized networking software (OPNET Network Simulator) on students' learning and motivation. 116 undergraduate students enrolled in a computer networking class at one of the universities in Egypt participated in this study, who were then randomly assigned to an experimental group (N= 59), which used the OPNET Network simulator, and a control group (N= 57) which studied the same content via the traditional physical lab. Two instruments; a networking test, and a Motivation Scale were administered to both study groups prior and post the intervention. Results showed that students of experimental group who used OPNET Networking simulator had significantly higher grades and greater motivation levels than those of the control group.
Keywords: Virtualization, Virtualization in education, Simulation, Computing education, OPNET network simulator, Experiential learning.
Received: : 9 October 2018/ Revised: 14 November 2018 / Accepted: 21 December 2018/ Published: 18 January 2019
Contribution/ Originality
This paper's primary contribution is to find out how virtual learning environments can prove more effective than real environment on both learning and motivation. The current study came to this conclusion by employing a quasi-experimental method, while other studies relied mostly on anecdotal or tentative evidence.
1. INTRODUCTION
Virtualization has spread in all aspects of education, and specifically in computer education, where laboratory work is an essential part of any course. In computing, virtualization refers to the act of creating a virtual, rather than actual version of something, like a virtual computer hardware platform, an operating system, a storage device, or a computer network. Virtualization enables one to try out new operating systems, visit websites one does not trust, host configuration, network services installation, and operation. Virtualized environments were also found useful in in the development of Web-based interactive learning resources in engineering education (Ndahi et al., 2007) and in developing interactive digital media (Tan, 2008; Yildirim et al., 2018).
A need to study virtualization in computer education is obvious since in most institutions there are limited computer labs shared by students of different disciplines. There is also a frequent need to change local network topology, and operating system configuration on each host. This implies that we need a specific and exclusive laboratory for executing networks experiments, which is not always possible due to limited resources in most institutions. Advances in virtualization technology have made it possible to provide low-cost, yet effective tools to simulate real world experiences. One of the techniques that use virtualization technology to replicate substantial aspects of real world is simulation-based learning. According to Lateef (2010) simulation is a technique, not a technology that is used to replace real experiences with guided ones, that are often immersive in nature, and that evoke or replicate substantial aspects of real world in a fully interactive way. OPNET is a simulation-based software program that models the behavior of a network utilizing virtualization technology, and allows each learner to construct and manage his/her own virtual network, without affecting the physical structure of the computer laboratory.
Virtual based learning environments have been shown to engage learners, and facilitate classroom instruction in several domains (Przybylski et al., 2012). However, in computing education, studies reported mixed results; some studies were in favor of virtualization Zhu (2015); Xu et al. (2014); Plass et al. (2012); Gaspar et al. (2008) while other studies highlighted the drawbacks of using virtualization, namely; they did not allow collaborative learning, apprehending that it might have negative effects on students' motivation (Aliane et al., 2010) that they offered little benefit to students, and added unnecessary abstraction layer to the learning process (Pan, 2010). Given such contradictory views on the impact of virtualization on computer network learning, this two-group quasi-experimental study has attempted to investigate the impact of a virtual learning environment, namely, OPNET Networking simulator, on students' learning and motivation.
2. THEORETICAL BACKGROUND
2.1. Virtualization (Simulations) in Education
According to Jong and Joolingen (1998) a computer simulation is “a program that contains a model of a natural or artificial system or process”, which underlies both conceptual models (concepts underlying the simulation) and operational models (procedures that can be applied). The use of simulations in education has greatly increased in recent years, as they are very much appreciated by teachers and learners because it offers the sense of “real world” experience. Using simulations in classrooms helps learners to actively construct their learning and develop the mental representation of the model underlying the simulation (Plass and Schwartz, 2014). Besides, simulation based learning could also be more relevant to learners’ needs adding significantly to their experience (Owen, 2017).
Simulations also help learners manipulate and gain control over the learning environment and interact with simplified models, practice and solve problems in a safer environment (Rutten et al., 2012) where mistakes can happen, but without fatal consequences that might occur in real world situations (Tan, 2008). Using simulations also offers learners the opportunity to repeat certain interactions until they understand the conceptual model underlying them (Jong and Joolingen, 1998).
Simulation can be subsumed under Kolb’s learning model of experiential learning, which is sometimes described as “learning by doing”, or “learning by trial and error”, or “experience-based learning”. Gentry (1990) stated that simulations have been labeled as involving experiential learning; as they are one of the pedagogies with increasing experiential learning potential. In experiential learning, immediate personal experience is the central point for learning. Kolb et al. (1984) pointed out that personal experience gives “life, texture, and subjective personal meaning to abstract concepts”. Experiential learning also follows a recursive cycle of experiencing, reflecting, thinking and acting to increase students learning motivation and learning success (Otto, 2017) which is very much evident in virtualization and simulation, the subject under study. It involves observing the phenomenon and doing something meaningful with it through active participation.
Gentry (1990) defined Experiential learning as participative, interactive, and a kind of applied learning that allows contact with the environment, and exposure to processes that are highly variable and uncertain. It involves the whole-person; learning takes place on the affective and behavioral dimensions as well as on the cognitive dimension. Thus experiential learning emphasizes getting in direct touch with the topic being studied, rather than just watching or reading about it Kolb et al. (1984); Kohonen (2007).
According to Hoover and Whitehead (1975) experiential learning exists when a “participant cognitively, affectively, and behaviorally processes knowledge, skills, and/or attitudes in a learning situation characterized by a high level of active involvement". Holdings (2014) cited eight reasons for which experiential learning is favorable; First, it accelerates learning; Second, it bridges the gap between theory and practice by providing "firsthand experience" of practicing what has been taught, Third, it enhances engagement levels, Fourth, it influences both feelings and emotions as well as knowledge and skills, and thus, it ensures that there is high level of retention, it delivers Exceptional Return on Investment (RoI). Fifth, when combined with simulations, it provides a safe learning environment where mistakes can happen, but without the fatal consequences that might happen in real world situations. Sixth, it provides accurate assessment results; experiential training can be used to deliver assessments results accurately. Seventh, it also facilitates personalized Learning; learners can set their own learning pace, and learning can be available anytime and anywhere, across multiple devices. Finally, it can have a big impact on the learner's mindset.
The use of computer simulations in education has been well studied in several domains with abundant literature reporting its effectiveness in enhancing students' learning and conceptualization, especially in science education (Rieber, 1990; Carlsen and Andre, 1992; Jong and Joolingen, 1998; Jimoyiannis and Komis, 2001) and in boosting the conceptual understanding of mathematical concepts (Lane and Tang, 2000) and in project management in systems engineering (Davidovitch et al., 2007). Simulations were also found to enhance scientific thinking (McKagan et al., 2009). Students felt more positive and satisfied about the domain after using simulation and they were more likely to participate and take more initiative (Durán et al., 2007). In a meta-analysis study, Rutten et al. (2012) found that computer based simulations enhanced scientific learning outcomes, with an effect size of up to 1.54. In medical education, simulations revolutionized training in areas such as surgery and critical care in a way that ensured patient safety (Kneebone, 2003; Abrahamson et al., 2004; Hammond, 2004). Simulations were also reported to be effectively used in military education and training, (Cioppa et al., 2004; Keh et al., 2008).
2.2. Virtualization (Simulations) in Computing Education
Virtualization, in computing, refers to the act of creating a virtual, rather than actual version of something, like a virtual computer hardware platform, an operating system, a storage device, or a computer network. Virtualization enables one to try out new operating systems, visit websites one does not trust, host configuration, network services installation, and operation. Virtualized environments were also found useful in in the development of Web-based interactive learning resources in engineering education (Ndahi et al., 2007) and in developing interactive digital media (Tan, 2008).
Prior research in virtualization in computer education yielded contradictory results as regards the impact of virtualization on learning experience. For example, Xu et al. (2014) who used a cloud-based, virtual laboratory platform called V-Lab to teach students network security, reported that students using V-Lab had a higher completion rate, and spent fewer hours on assignments, and were able to participate more on experiments.
Gaspar et al. (2008) stated that using virtualization in networking education, namely, the VNet Lab made learning experience more authentic, offered instructors more control, and minimized the costs and consequences of students' mishaps.
Zhu (2015) reported that students who used the cloud-based network (Amazon EC2) preferred it as it offered them the flexibility to work anywhere and anytime, and at the same time without caring about the administrative policies of the physical university network. Villanueva and Cook (2005) reported that teaching management computer systems courses using VMware GSX Server, provided students with 24/7 virtual access to their coursework, and enabled them to access their personal virtual machine from anywhere, privately and securely.
However, Nedic et al. (2003) reported that students who used the virtual environment NetLab, indicated that conducting virtual experiments was not similar to conducting experiments in real labs.
Aliane et al. (2010) highlighted the drawbacks of remote virtual laboratories, such as unfit for collaborative learning and having negative effects on students' motivation. Besides, students who learnt solely through virtual labs failed to familiarize themselves with the lab equipment.
Pan (2010) argued that the traditional hands-on lab experience was indispensible for effective learning outcomes while teaching computer networks. He asserted that virtualization offered little benefit to students, and added unnecessary abstraction layer to the learning process.
In teaching Networking, Chamberlin et al. (2017) found that there were no significant differences in performance or confidence in materials between the experimental group, who learned through virtual technology, and the control group who used the physical lab.
2.3. Motivation toward Learning
Motivation is an essential factor for behavior change; therefore, understanding motivation is extremely important in the field of education, as it could offer a predictive as well as prescriptive view of behavior. Motivation is seen as the essential drive that stimulates and sustains learning, it is important to understand the factors that impact it. According to Lee (2000) motivation is particularly crucial to learning and performance in technology-mediated environments.
“Motivation is a very complex phenomenon with many facets" (Gardner, 2006). Brown (2000) defined motivation as “the anticipation of reward" while Keller (1983) viewed motivation as “choices people make as to what experiences or goals they will approach or avoid, and the degree of effort they exert in that respect”. Keller (2006) defined it as the “amount of effort a person is willing to exert in pursuit of a goal" while (Gardner, 1985) referred to motivation as the "combination of effort plus desire to achieve the goal of learning”.
In order to develop motivated learners, Keller (1979) stressed upon four principles. First, the learners' curiosity need to be kept aroused and sustained. Second, learners must perceive instruction to be relevant to their values, needs and goals. Third, learners also must believe that they have the ability to succeed. Fourth, learners must be satisfied with their learning experience. Later (Keller, 1983) elaborated these four principles into a holistic theory of motivation for learning.
2.4. ARCS Model
The ARCS model was grounded in expectancy-value theory (Vroom, 1964) which assumes that people are motivated to participate in an activity if it is perceived to satisfy their personal needs , and if there is a positive expectancy of success (Keller, 1987). Keller however expanded these two categories (value and expectancy) into four sub categories in his model. Value was subdivided into interest and relevance and expectancy into expectancy and outcomes (Keller, 1987). Keller’s work changed the view of motivational drive from an extrinsic focus, to an intrinsic one (Keller, 1979).
Keller (1983) is represented by what is now known as the ARCS model (Keller (1984;1987) based on the acronym resulting from key words representing the four categories (Attention, Relevance, Confidence, and Satisfaction). Keller (1983) classified motivational concepts and theories into four categories; namely, gaining learner attention, establishing the relevance of the instruction to learner goals and needs, building confidence in achieving realistic expectations, and making the instruction satisfying. The above-mentioned four principles of motivation may be briefly explained as follows:
2.4.1. First principle: Attention
People’s attention, curiosities and interests should be stimulated and sustained. Motivation to learn is enhanced when a learner's curiosity is aroused. The implication of this principle is that we must seek building curiosity, and sustain active engagement in the learning activity. Kopp (1982) illustrates the importance of using a variety of techniques such as graphics, animation, unresolved problems to gain learner attention. To sustain attention, we can apply the principle of variability, because no matter how interesting a given technique or strategy is, people will lose interest over time. Therefore, it is important to vary one's strategies and techniques.
2.4.2. Second principle: Relevance
People must believe that instruction is related to their personal goals or motives and keep them connected to the setting. Motivation to learn is promoted when learners perceive the content to be relevant and meaningfully related to their goals. The principle of Relevance is evident in learners' goals which can be extrinsic to the learning event such that it is necessary to pass a course in order to graduate or to get eligible for a job, but a stronger level of motivation to learn is intrinsic (Deci and Ryan, 1985) and it is achieved when the learner is engaged in actions that are personally interesting and freely chosen. Owing to the principle of relevance such learning activities are usually referred to as authentic learning experiences (Duffy et al., 1993).
2.4.3. Third principle: Confidence
As Keller (2010) observes that even if learners believe the content is relevant and are curious to learn it, they still might not be well motivated to learn it if they do not have the appropriate confidence, or expectancy for success. For example, they might possess previously-established fears about the topic, skill, or situation that could prevent them from learning effectively. Or, at the other extreme, they might develop the misconception of already knowing a topic and could overlook important details in the learning activities. Motivation to learn is enhanced when learners are confident that they can succeed in accomplishing required tasks. According to Weiner (1974) confidence is built by helping learners develop positive expectancies for success, by providing them with learning experiences that make them succeed and attribute their success to their efforts and abilities, not to good luck or easy task.
2.4.4. Fourth principle: Satisfaction
Satisfaction points at having a continuing desire to learn; people hence must have feelings of satisfaction with the process or results of the learning experience. Motivation to learn is boosted when learners have positive feelings about their learning experiences; in other words, they anticipate and experience satisfying outcomes to a learning situation. This means that extrinsic reinforcements, such as rewards and recognition, must be used. Providing students with opportunities to apply what they have learned, along with personal recognition, supports intrinsic feelings of satisfaction. The presence of a sense of equity, or fairness, is also important for learners to be satisfied with their learning experience (Adams, 1965).
Keller (2010) stated that being successful in achieving the first three motivational goals; attention, relevance and confidence) results in people being motivated to learn. People’s scores on attention, relevance, confidence and satisfaction constructs cumulatively result in an overall motivation score.
2.5. Virtualization (Simulation) and Motivation toward Learning
Though motivation is known to influence learner behavior and outcomes, little research has been done on how simulations could influence motivation toward learning. According to Owen (2017) the way simulations or virtual learning environments affect motivation toward learning needs further clarification, as there is little evidence that simulation can affect motivation.
Future research should take motivation and other learner attitudes into account while evaluating the effectiveness of simulation based educational interventions (Owen, 2017). This study is an attempt to fill the gap in the existing literature on the impact of (OPNET Network Simulator) on motivation toward learning. To accomplish this, the current study adopted a framework based on Keller's ARCS model that focuses on attention, relevance, confidence and satisfaction in order to motivate students. It is hypothesized that virtual based learning, which takes into account these four conditions, will enhance motivation toward learning.
3. RESEARCH QUESTIONS
This study tries to answer the following questions:
- What is the impact of virtualized networking software (OPNET Network Simulator) on students' motivation toward learning?
- What is the impact of virtualized networking software (OPNET Network Simulator) on students' learning?
4. METHODOLOGY
This study adopted the quasi experimental design, with two groups: an experimental group that used the virtualized networking software (OPNET Network simulator) as an operational tool to help them learn how to manage, monitor and secure networks, and a control group that used the physical laboratories to learn the same curriculum. A pretest in networking and a motivation survey, developed by the researcher, based on the ARCS Model, were administered to both groups at the start and the end of the experimental treatment.
4.1. Participants
Participants in this study involved 116 students enrolled in the second year in the college of computers and information technology in one of the universities in Egypt. The group was randomly assigned to an experimental group and a control group. The experimental group (N=59), used OPNET Network simulator as an operational tool to help them acquire the skills of utilizing, managing, and securing networks. The control group (N=57) studied the same curriculum through the physical laboratory. The experimental group consisted of 28 females and 31 males, while the control group consisted of 29 females and 28 males. The two groups were coached by the same instructor.
4.2. Instrumentation
4.2.1. The Motivation Survey
To assess whether or not students' motivation toward learning was impacted by using the virtual networking software (OPNET Network Simulator), the researcher developed a Motivation scale derived from Keller's Motivation Survey. The survey consisted of 16-items distributed on 4 subscales; attention, relevance, confidence, and satisfaction. The survey was administered to both experimental and control group at the beginning of the intervention and at its end. The survey used five-point Likert scale ranging from 5 (Strongly Agree) to 1 (Strongly Disagree).
4.2.1.1. The Motivation Survey Validity
The validity of the survey was assessed in two steps; first, the internal consistency of the Motivation Survey was assessed; second, Pearson correlation coefficients were calculated between the scores students got on each item and their scores on the sub-scale to which each item belonged.
Table (1) shows the correlation coefficients between scores on each item of the scale, and the score on the sub-scale to which this item belongs.
As shown in Table (1), all correlation coefficients were between (0.73- 0.94), and significant at the 0.01 level, which indicates that all items are valid.
Table-1. Correlation Coefficients between each item, and its sub-scale
Sub-Scale |
Items |
Correlation Coefficient ( r ) |
Sig. |
Attention |
1.1. The way the content is delivered helped to sustain my attention throughout the course. | 0.9 |
0.01 |
| 1.2. The materials of the course were so unappealing that it was difficult to keep my attention on. | 0.94 |
0.01 |
|
| 1.3. The way the information is organized helped to keep me focused. | 0.86 |
0.01 |
|
| 1.4. The course stimulated my curiosity. | 0.73 |
0.01 |
|
Relevance |
2.1. The material of the course satisfied my interests. | 0.83 |
0.01 |
| 2.2. For me, the information I got from this course is worthwhile. | 0.85 |
0.01 |
|
| 2.3. The content of the course was not useful for me. | 0.92 |
0.01 |
|
| 2.4. I could relate the new content I learned in this course to my previous knowledge and experiences. | 0.82 |
0.01 |
|
Confidence |
3.1. The content of the course seemed more difficult to understand than I expected. | 0.85 |
0.01 |
| 3.2. The way the information is organized made me confident that I could easily learn it. | 0.94 |
0.01 |
|
| 3.3. I was confident that I could easily pass this course. | 0.9 |
0.01 |
|
| 3.4. The feedback I got made me confident that I would successfully complete this course. | 0.94 |
0.01 |
|
Satisfaction |
4.1. Overall, I am satisfied with this course. | 0.84 |
0.01 |
| 4.2. The material I learned in this course was valuable to me. | 0.94 |
0.01 |
|
| 4.3. The way the content was delivered helped increase my knowledge. | 0.76 |
0.01 |
|
| 4.4. I would recommend this course for others. | 0.84 |
0.01 |
In order to assess the internal consistency of the Motivation Survey, Pearson correlations were calculated between the scores students got on each sub-scale and their total scores on the survey. Table (2) shows the correlations between scores on each sub-scale and total score of the survey.
Table-2. Correlations between scores on each sub-scale and total score of the Scale
Sub scale |
Correlation Coefficient ( r ) |
Sig. |
Attention |
0.949 |
0.01 |
Relevance |
0.955 |
0.01 |
Confidence |
0.958 |
0.01 |
Satisfaction |
0.955 |
0.01 |
As shown in Table (2), correlation coefficients were between (0.949 – 0.958), and were all significant at 0.01. The four subscales have high correlations with the total score of the survey, which indicates that they are all valid.
4.2.1.2. The Motivation Survey Reliability
To assess the reliability of the Motivation Survey, alpha Cronbach was calculated for the scale as a whole, and for each of the subscales. Table (3) shows Cronbach's Alpha for the Motivation Survey, and its sub-scales.
Table-3. Cronbach's Alpha for the Motivation Survey and its sub scales
Sub Scale |
Items |
Cronbach's Alpha |
Attention |
4 |
0.88 |
Relevance |
4 |
0.87 |
Confidence |
4 |
0.93 |
Satisfaction |
4 |
0.84 |
Motivation scale |
16 |
0.97 |
According to Mueller (1986) a well-constructed scale should have a reliability coefficient of 0.80 or higher. The alpha coefficient of the survey as a whole was 0.97. The subscale “Attention” had an alpha of 0.88; the subscale “Relevance” of 0.87; the sub-scale “Confidence"” of 0.93, and the subscale "Satisfaction" of 0.84, which indicates that the survey was highly reliable.
4.2.2. Networking Test
To assess students' learning, a Networking test, developed by the researcher, was administered to both study groups prior and post the intervention.
4.2.2.1. Networking Test Validity
The internal validity of the networking test was assessed by calculating Pearson correlations between the scores students got on each dimension of the test (Managing Network, Monitoring & troubleshooting, and securing networks) and their total score on the test. Table (4) shows the correlations between scores on each dimension and total score of the test.
Table-4. Correlations between scores on each dimension and total score of the Test
Correlation coefficient ( r ) |
Sig. |
|
Managing Network |
0.85 |
0.01 |
Monitoring& Troubleshooting |
0.86 |
0.01 |
Securing Network |
0.93 |
0.01 |
Total |
0.88 |
0.01 |
As shown in Table (4), correlation coefficients are (0.85, 0.86, 0.93), and are all significant at the 0.01, the three dimensions have high correlations with the total score of the test, which shows that the test is valid.
4.2.2.2. Networking Test Reliability
Alpha Coefficient was calculated for the Networking test, and was 0.89, which means that the test is reliable.
4.3. The Intervention
The intervention in this study lasted ten weeks; OPNET Network simulator was selected as it offered a convenient and cost-effective, and easy-to-use platform for simulating networks. It was assumed that using OPNET Network simulator would offer students not only first-hand interactive learning experience, but would also provide a safe environment that helps students explore situations that would have been risky in real contexts.
OPNet Simulator was used by the experimental group and physical labs by the control group, only as supplementary and not replacement for classroom meetings. The course was intended to help students acquire the skills of managing, monitoring and troubleshooting network problems, and securing networks, in an environment where students can learn by themselves, which is a critical demand of the workplace due to rapid change in technology.
4.4. Data Analysis
The Statistical Package for Social Sciences Version 22 was used to analyze data, Cronbach's alpha, Pearson bivariate correlation analysis, and the overall means and standard deviations of the sample were calculated in the preliminary analyses. Independent samples t-test and ANCOVA were calculated to assess the impact of the (Virtual Network Simulator/physical lab) on motivation and learning.
5. RESULTS
To ensure that there were no significant differences in motivation toward learning between the two groups of the study at the start of the intervention, the Motivation Survey was administered to both groups, and independent samples t test was calculated. Table (5) shows the significance of differences between the mean scores of two groups on Motivation prior the intervention.
Table-5. Significance of Differences between the mean scores of the study groups on pre motivation Survey.
Sub scale |
Study groups |
Mean |
SD |
T-test |
||
T |
DF |
Sig. |
||||
Attention |
Experimental |
8.71 |
1.64 |
1.828 |
114 |
0.07 |
Control |
8.16 |
1.62 |
||||
Relevance |
Experimental |
6.98 |
1.89 |
0.701 |
114 |
0.485 |
Control |
6.74 |
1.89 |
||||
Confidence |
Experimental |
8.07 |
2.29 |
0.479 |
114 |
0.633 |
Control |
8.26 |
2.09 |
||||
Satisfaction |
Experimental |
8.19 |
1.47 |
0.967 |
114 |
0.335 |
Control |
7.91 |
1.58 |
||||
Motivation scale |
Experimental |
31.95 |
4.58 |
1.089 |
114 |
0.279 |
Control |
31.07 |
4.09 |
||||
As shown in table (5), all the values of t for the whole scale, and for the sub-scales, were not statistically significant, which means that the two groups were equivalent in motivation. Figure (1) depicts the difference between the mean scores of the experimental and control group in the pre-administration of the Motivation scale.
Figure-1. Mean Scores of the two groups on Pre-Motivation Survey
Figure (2) depicts the difference between the mean scores of the experimental and control group on the pre-administration of the Motivation Survey sub-scales.
Figure-2. Mean Scores of the Experimental and Control group on Pre-Motivation Sub-Scales
Networking Pre-Test
To ensure that there were no significant differences in learning between the two groups of the study before the experimental treatment, a test in Networking, with three sub-divisions (Managing, Monitoring &Troubleshooting, and Securing Networks) was administered to both groups at the beginning of the experiment. Independent t test was calculated. Table (6) shows the significance of differences between the mean scores of the two groups on the Networking Pre-test.
Table-6. Significance of Differences between the mean scores of study groups on the pre-test.
Main skills |
Study groups |
Mean |
SD |
t-test |
||
T |
DF |
Sig. |
||||
Managing Network |
Experimental |
4.24 |
4.98 |
0.41 |
114 |
0.682 |
Control |
3.86 |
4.91 |
||||
Monitoring and troubleshooting Network |
Experimental |
4.41 |
5.01 |
0.4 |
114 |
0.688 |
Control |
4.04 |
4.95 |
||||
Securing Network |
Experimental |
4.24 |
4.98 |
0.03 |
114 |
0.977 |
Control |
4.21 |
4.98 |
||||
Networking Test |
Experimental |
12.88 |
9.83 |
0.43 |
114 |
0.671 |
Control |
12.11 |
9.77 |
||||
As shown in table (6), all the values of t for the Networking pre-test, and its sub-divisions, were not statistically significant, which means that the two groups were equivalent in learning at the start of the experiment. Figure (3) depicts the difference between the mean scores of the experimental and control group in the pre-test.
Figure-3. Mean scores of the experimental and control group on the pre-test.
The Impact of (Virtual Simulator/Physical lab) on Motivation
To assess the impact of the treatment (OPNet Virtual Network Simulator/ Physical laboratory) on students' motivation, Analysis of Covariance (ANCOVA) was calculated controlling students' scores on the pre administration of the Motivation Survey. Table (7) shows the results of ANCOVA.
As shown in Table (7), the impact of the independent variable (OPNET Network Simulator vs. Physical lab) on the scale as a whole, f=1137.99, and for the subscales: Attention, Relevance, Confidence, and Satisfaction, f values were: (244.84, 526.82, 735.40, 331.06 respectively), and they were all significant at the 0.001 level, which means that there were statistically significant differences between the experimental group on the post motivation scale, and on its four sub-scales.
Table (8) shows the means and standard deviations of the two groups on the post Motivation Scale
Table-7. ANCOVA Results
Sub-Scale |
Source |
Sum Squares |
DF |
Mean Square |
F |
Sig. |
Partial Eta Squared |
Attention |
Model |
32759.14 |
3 |
10919.71 |
3609.45 |
0.001 |
0.99 |
Pre - Measurement |
3.46 |
1 |
3.46 |
1.14 |
0.287 |
0.01 |
|
OPNET/Physical |
1481.46 |
2 |
740.73 |
244.84 |
0.001 |
0.81 |
|
Error |
341.86 |
113 |
3.03 |
||||
Total |
33101 |
116 |
|||||
Relevance |
Model |
29412.74 |
3 |
9804.25 |
4240.58 |
0.001 |
0.99 |
Pre - Measurement |
17 |
1 |
17 |
7.35 |
0.008 |
0.06 |
|
OPNET/Physical |
2436.01 |
2 |
1218 |
526.82 |
0.001 |
0.9 |
|
Error |
261.26 |
113 |
2.31 |
||||
Total |
29674 |
116 |
|||||
Confidence |
Model |
30997.29 |
3 |
10332.43 |
5788.45 |
0.001 |
0.99 |
M |
18.52 |
1 |
18.52 |
10.38 |
0.002 |
0.08 |
|
OPNET/Physical |
2625.41 |
2 |
1312.7 |
735.4 |
0.001 |
0.93 |
|
Error |
201.71 |
113 |
1.79 |
||||
Total |
31199 |
116 |
|||||
Satisfaction |
Model |
29009.82 |
3 |
9669.94 |
5008.33 |
0.001 |
0.99 |
Pre- Measurement |
10.6 |
1 |
10.6 |
5.49 |
0.021 |
0.05 |
|
OPNET/Physical |
1278.39 |
2 |
639.19 |
331.06 |
0.001 |
0.85 |
|
Error |
218.18 |
113 |
1.93 |
||||
Total |
29228 |
116 |
|||||
Motivation |
Model |
488172.5 |
3 |
162724.2 |
20083.91 |
0.001 |
1 |
scale |
Pre - Measurement |
30.31 |
1 |
30.31 |
3.74 |
0.056 |
0.03 |
OPNET/Physical |
18440.44 |
2 |
9220.22 |
1137.99 |
0.001 |
0.95 |
|
Error |
915.55 |
113 |
8.1 |
||||
Total |
489088 |
116 |
Table-8. Means and Standard Deviation of the two Groups on Post Motivation Survey
Sub-scale |
Experimental group (N=59) |
Control group(N=57) |
||
Mean |
SD |
Mean |
SD |
|
Attention |
19.1 |
0.69 |
14.04 |
2.38 |
Relevance |
18.49 |
0.97 |
12.72 |
2 |
Confidence |
18.8 |
0.83 |
13.33 |
1.8 |
Satisfaction |
17.93 |
0.74 |
13.26 |
1.88 |
Motivation scale |
74.32 |
1.79 |
53.35 |
3.69 |
As shown in Table (8), means of the experimental groups on the Motivation Survey, and all its sub-scales are higher than the means of the control group. Figures 4 and 5 show the difference in means between the two groups on the total score of the survey, and the scores of each sub-scale.
Figure-4. Mean scores of the experimental and control group on Post Motivation Survey.
Figure-5. Mean scores of the study groups on Post Motivation Survey sub-Scales.
The Impact of (Virtual Simulator/Physical lab) on Learning
To assess the impact of learning method, Virtual network software, compared to the physical laboratory, ANOVA was calculated on the post test scores for both groups, controlling their pre-test scores. Table (9) shows the results of ANCOVA.
Table 10 shows the results of ANCOVA on the post test scores for both groups, controlling for their pre-test scores. F value for the test as a whole was (f= 452. 46), which is significant at the 0.001 level, F values for the sub-divisions of the test; utilizing, managing, securing networks were 282.20, 185.64, 339.96 respectively, and they are all significant at the 0.001 level. This indicates that there is a statistically significant difference between the mean scores of the experimental and control group on the Networking posttest. Table (10) shows the means and standard deviation of the two groups on the post-test and its sub-divisions.
As shown in Table 10, the means of the experimental groups were higher than the control group on the total score of the post-test, and the scores of the sub-divisions. Figure (6) shows the mean scores of both experimental and control group on post-test. Figure (7) shows the mean scores of the two groups on the post-test sub-divisions.
Table-9. ANCOVA Results for post-test
Main skill |
Source |
Sum Squares |
DF |
Mean Square |
F |
Sig. |
Partial Eta Squared |
Managing Network |
Model |
21211.55 |
3 |
7070.52 |
258.7 |
0.001 |
0.87 |
Pre - test |
9.98 |
1 |
9.98 |
0.37 |
0.547 |
0 |
|
Learning method |
15425.84 |
2 |
7712.92 |
282.2 |
0.001 |
0.83 |
|
Error |
3088.45 |
113 |
27.33 |
||||
Total |
24300 |
116 |
|||||
Monitoring & |
Model |
23560.45 |
3 |
7853.48 |
219.69 |
0.001 |
0.85 |
Troubleshooting |
Pre - Observation |
24.8 |
1 |
24.8 |
0.69 |
0.407 |
0.01 |
Learning method |
13272.7 |
2 |
6636.35 |
185.64 |
0.001 |
0.77 |
|
Error |
4039.55 |
113 |
35.75 |
||||
Total |
27600 |
116 |
|||||
Securing Networks |
Model |
26931.92 |
3 |
8977.31 |
395.02 |
0.001 |
0.91 |
Pre - Observation |
12.41 |
1 |
12.41 |
0.55 |
0.461 |
0 |
|
Learning method |
15452.33 |
2 |
7726.16 |
339.96 |
0.001 |
0.86 |
|
Error |
2568.08 |
113 |
22.73 |
||||
Total |
29500 |
116 |
|||||
Total |
Model |
228224.5 |
3 |
76074.83 |
742.64 |
0.001 |
0.95 |
Pre - Observation |
2.67 |
1 |
2.67 |
0.03 |
0.872 |
0 |
|
Learning method |
92697.35 |
2 |
46348.67 |
452.46 |
0.001 |
0.89 |
|
Error |
11575.51 |
113 |
102.44 |
||||
Total |
239800 |
116 |
Table-10. Means and Standard Deviations of study groups on the post-test.
Sub-divisions |
Experimental group (N=59) |
Control group (N=57) |
||
Mean |
SD |
Mean |
SD |
|
Managing Networks |
18.14 |
3.93 |
10.53 |
7.18 |
Monitoring& Troubleshooting |
17.29 |
4.48 |
10.18 |
7.19 |
Securing Networks |
17.97 |
4.06 |
11.75 |
5.39 |
Total |
53.39 |
6.85 |
32.46 |
12.58 |
Figure-6. Mean Scores of the study groups on the Networking post-test.
Figure-7. Mean scores of the study groups on the Networking post-test Subdivisions
To assess the size of the impact of using OPNET Virtual Networking Simulator on both Motivation and learning, Eta square was calculated; Table (12) shows the values of Eta square for both motivation and learning.
Table-11. Eta Square values for Motivation and learning
Instrumentation |
Eta square |
Size effect |
Motivation scale |
0.95 |
Large |
Networking Post-test |
0.89 |
Large |
As shown in table (11), Eta square values for motivation and learning were 0.95, and 0.89 respectively, and they both show large effect size, which means that the use of the virtual simulator has a large impact on students' motivation and learning. Figures (8) and (9) show the impact of using Virtual Networking Simulator on Motivation and learning respectively.
Figure-8. The impact of Virtual Networking Simulator on Motivation.
Figure-9. Impact of Virtual Networking Simulator on Learning.
6. DISCUSSION
This study adopted a two-group quasi-experimental approach to examine the impact of a virtual simulator (OPNET Network simulator versus physical lab) on students' motivation and learning. Results of the study showed that there were statistically significant differences between the mean scores of the experimental group that used the OPNET Network Simulator, and the control group who used the physical lab, on both motivation and learning. By the end of the experimental treatment, students who used the virtual networking simulator showed significantly greater grades and motivation levels, than the control group, who used the physical lab.
The study started with the premise that the impact of the virtual networking simulator could be as good as using physical lab on students' learning and motivation. Surprisingly, students who used the virtual simulator achieved better and had higher motivation levels than students who used the physical lab. These findings are surprising because it is always assumed direct experience in learning is the optimal goal that teachers strive to achieve. Teachers usually opt for virtualized learning environments due to limited resources or safety issues, and in doing so; they always have concerns that this would compromise the quality of learning experience.
This study found statistically significant difference in learning between the experimental and control group as shown in post-test scores, with a noteworthy large effect size. Perhaps the fact that being present in the physical lab at fixed times is seen by many students as laborious and inflexible. The virtual learning environment on the other hand enabled students to study and work on their assignments anytime and anywhere. Students in the experimental group also reported that using the virtual networking simulator helped work on the same points until they understood the conceptual principles underlying them. Additionally, many students expressed their appreciation for using a virtual simulator instead of the physical lab, as it helped them to experiment when asked to deploy network problems, without the fear of messing things up.
This study also found statistically significant difference in motivation between the experimental and control group in learning as shown in post-Motivation Survey scores. Motivation is especially important as it the driving force behind learning. Motivation is currently an under-researched topic in virtual learning environments. Understanding motivation could be tremendously important in the field of education, as it could offer a predictive as well as prescriptive view of behavioral change.
At times when there are limited resources, there is a challenge to incorporate new technologies into the curriculum hoping that this would not compromise learning. This study strived to provide new insight for understanding the impact of one of the virtualized networking simulator on learning and motivation. The findings of the study showed that there is no need to sacrifice the quality of the educational experience when using virtual network simulator. Actually, the outcomes of using virtual network simulator exceeded expectations. What was started as a premise that virtual networking software could be as good as physical lab environment, however, it ended with the conclusion that virtual was even better than real!
7. DELIMITATIONS
The scope of this study was narrowed to one population of computer science students in one university in Egypt. Although a cause and effect relationship can be inferred from this study, further research would be needed to generalize results. This study should be replicated with different samples to investigate the impact of the Virtual Network Simulator on students' learning and motivation before reaching a generalization.
| Funding: This study received no specific financial support. |
| Competing Interests: The author declares that there are no conflicts of interests regarding the publication of this paper. |
REFERENCES
Abrahamson, S., J. Denson and R. Wolf, 2004. Effectiveness of a simulator in training anesthesiology residents. BMJ Quality & Safety, 13(5): 395-397. Available at: https://doi.org/10.1136/qhc.13.5.395.
Adams, J.S., 1965. Inequity in social exchange. In L. Berkowitz (Ed.), Advances in experimental social psychology. New York: Academic Press.
Aliane, N., R.V. Pastor and G.V. Mariscal, 2010. Limitations of remote laboratories in control engineering education. International Journal of Online Engineering, 6(1): 31-33. Available at: https://doi.org/10.3991/ijoe.v6i1.1131.
Brown, H., 2000. Principles of language learning and teaching. New Jersey: Prentice Hall.
Carlsen, D.D. and T. Andre, 1992. Use of a microcomputer simulation and conceptual change text to overcome student preconceptions about electric circuits. Journal of Computer-Based Instruction, 19(4): 105-109.
Chamberlin, J., J. Hussey, B. Klimkowski, W. Moody and C. Morrell, 2017. The impact of virtualized technology on undergraduate computer networking education. Proceedings of the 18th Annual Conference on Information Technology Education. ACM. pp: 109-114.
Cioppa, T.M., T.W. Lucas and S.M. Sanchez, 2004. Military applications of agent-based simulations. Simulation Conference, 2004. Proceedings of the 2004 Winter. IEEE, 1.
Davidovitch, L., A. Shtub and A. Parush, 2007. Project management simulation-based learning for systems engineering students. Proceedings of the International Conference on Systems Engineering and Modelling (ICSEM): 17-23. Herzeliya and Haifa, Israel.
Deci, E.L. and R. Ryan, 1985. Intrinsic motivation and self-determination in human behavior. New York: Plenum.
Duffy, T.M., J. Lowyck and D.H. Jonassen, 1993. Designing environments for constructivist learning. New York: Springer-Verlag.
Durán, M.J., S. Gallardo, S.L. Toral, R. Martínez-Torres and F.J. Barrero, 2007. A learning methodology using matlab/simulink for undergraduate electrical engineering courses attending to learner satisfaction outcomes. International Journal of Technology and Design Education, 17(1): 55-73. Available at: https://doi.org/10.1007/s10798-006-9007-z.
Gardner, R.C., 1985. The social psychology and second language learning: The role of attitude and motivation. London: Edward Arnold.
Gardner, R.C., 2006. The socio educational mole of second language learning Rowley, Massachusetts: Newbury House.
Gaspar, A., S. Langevin, W. Armitage, R. Sekar and T. Daniels, 2008. The role of virtualization in computing education. In ACM sIGCsE bulletin, 40(1): 131-132. Available at: https://doi.org/10.1145/1352322.1352181.
Gentry, J.W., 1990. What is experiential learning? In J. Gentry (Ed.), Guide to business gaming and experiential learning. London: Nichols/GP. pp: 9-20.
Hammond, J., 2004. Simulation in critical care and trauma education and training. Current Opinion in Critical Care, 10(5): 325-329. Available at: https://doi.org/10.1097/01.ccx.0000140950.47361.c9.
Holdings, K., 2014. 8 reasons why experiential learning is the future of learning - elearning industry. E Learning Industry. Available from https://elearningindustry.com/8-reasons-experiential-learning-future-learning [Accessed 8 Oct. 2018].
Hoover, J.D. and C. Whitehead, 1975. An experiential-cognitive methodology in the first course in management: Some preliminary results. Developments in Business Simulation and Experiential Learning: Proceedings of the Annual ABSEL Conference, 2.
Jimoyiannis, A. and V. Komis, 2001. Computer simulations in physics teaching and learning: A case study on students' understanding of trajectory motion. Computers & Education, 36(2): 183-204. Available at: https://doi.org/10.1016/s0360-1315(00)00059-2.
Jong, D. and V.W.R. Joolingen, 1998. Scientific discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68(2): 179-201. Available at: https://doi.org/10.2307/1170753.
Keh, H.-C., K.-M. Wang, S.-S. Wai, J.-y. Huang, H. Lin and J.-J. Wu, 2008. Distance-learning for advanced military education: Using Wargame simulation course as an example. International Journal of Distance Education Technologies, 6(4): 50-61. Available at: https://doi.org/10.4018/jdet.2008100104.
Keller, J.M., 1979. Motivation and instructional design: A theoretical perspective. Journal of Instructional Development, 2(4): 26-34. Available at: https://doi.org/10.1007/bf02904345.
Keller, J.M., 1983. Motivational design of instruction. In C.M. Reigeluth (Ed.), Instructional-design theories and models: An overview of their current status. Hillsdale, NJ: Lawrence Erlbaum Associates. pp: 386-434.
Keller, J.M., 1984. The use of the ARCS model of motivation in teacher training. In K.S.A.J. Trot (Ed.), Aspects of educational technology: Staff development and career updating. London: Kogan Page, 17: 140-145.
Keller, J.M., 1987. Development and use of the ARCS model of instructional design. Journal of Instructional Development, 10(3): 2-10. Available at: https://doi.org/10.1007/bf02905780.
Keller, J.M., 2006. ARCS design process. arcsmode.ipower.com. [online] arcsmode.ipower.com. Available from www.arcsmodel.com [Accessed 20 Oct. 2018].
Keller, J.M., 2010. The Arcs model of motivational design. In Motivational design for learning and performance. Boston, MA: Springer. pp: 43-74.
Kneebone, R., 2003. Simulation in surgical training: Educational issues and practical implications. Medical Education, 37(3): 267-277. Available at: https://doi.org/10.1046/j.1365-2923.2003.01440.x.
Kohonen, V., 2007. Learning to learn through reflection–an experiential learning perspective. Preparing Teachers to Use the European Language Portfolio–Arguments, Materials and Resources, Council of Europe Publishing.
Kolb, D.A., I.M. Rubin and J.M. McIntyre, 1984. Organizational psychology: Readings on human behavior in organizations. Englewood Cliffs, N.J: Prentice Hall.
Kopp, T., 1982. Designing boredom out of instruction. Performance & Instruction, 21(4): 23-32. Available at: https://doi.org/10.1002/pfi.4170210411.
Lane, D.M. and Z. Tang, 2000. Effectiveness of simulation training on transfer of statistical concepts. Journal of Educational Computing Research, 22(4): 383-396. Available at: https://doi.org/10.2190/w9gw-5m9c-uqvt-1e0r.
Lateef, F., 2010. Simulation-based learning: Just like the real thing. Journal of Emergencies, Trauma and Shock, 3(4): 348. Available at: https://doi.org/10.4103/0974-2700.70743.
Lee, C.-Y., 2000. Student motivation in the online learning environment. Journal of Educational Media & Library Sciences, 37(4): 367-375.
McKagan, S., W. Handley, K. Perkins and C. Wieman, 2009. A research-based curriculum for teaching the photoelectric effect. American Journal of Physics, 77(1): 87-94. Available at: https://doi.org/10.1119/1.2978181.
Mueller, D.J., 1986. Measuring social attitudes: A handbook for researchers and practitioners. New York: Teachers College Press.
Ndahi, H.B., S. Charturvedi, A.O. Akan and J. Pickering, 2007. Engineering education: Web-based interactive learning resources. Technology Teacher, 67(3): 9-14.
Nedic, Z., J. Machotka and A. Nafalski, 2003. Remote laboratories versus virtual and real laboratories. In Proceedings of the 2003 33rd ASEE/IEEE Frontiers in Education Conference. Boulder, CO. T3E.1-T3E.6, Nov. 2003.
Otto, D., 2017. Students’ interaction for enhancing learning motivation and learning success: Findings from integrating a simulation game into a university. INTED2017 Proceedings. pp. 1316-1324.
Owen, L.A., 2017. An exploration of motivation, relevance and realism in simulation based medical education: I don't want to look like an idiot. Doctoral Dissertation, University of Dundee.
Pan, J., 2010. Teaching computer networks in a real network: The technical perspectives. Proceedings of the 41st ACM Technical Symposium on Computer Science Education. pp: 133-137.
Plass, J.L., C. Milne, B.D. Homer, R.N. Schwartz, E.O. Hayward, T. Jordan, J. Verkuilen, F. Ng, Y. Wang and J. Barrientos, 2012. Investigating the effectiveness of computer simulations for chemistry learning. Journal of Research in Science Teaching, 49(3): 394-419. Available at: https://doi.org/10.1002/tea.21008.
Plass, J.L. and R.N. Schwartz, 2014. Multimedia learning with simulations and microworlds. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning. Cambridge: Cambridge University Press. pp: 729–761.
Przybylski, A.K., N. Weinstein, K. Murayama, M.F. Lynch and R.M. Ryan, 2012. The ideal self at play: The appeal of video games that let you be all you can be. Psychological Science, 23(1): 69-76. Available at: https://doi.org/10.1177/0956797611418676.
Rieber, L.P., 1990. Animation in computer-based instruction. Educational Technology Research and Development, 38(1): 77-86. Available at: https://doi.org/10.1007/bf02298250.
Rutten, N., W.R. Van Joolingen and J.T. Van Der Veen, 2012. The learning effects of computer simulations in science education. Computers & Education, 58(1): 136-153. Available at: https://doi.org/10.1016/j.compedu.2011.07.017.
Tan, H.S., 2008. Learning and motivational aspects of using interactive digital media (IDM). In Motivation and practice for the classroom, Eds. P.A. Towndrow, C. Koh, and H.S. Tan. Rotterdam: Sense Publishers. pp: 315-340.
Villanueva, B. and B. Cook, 2005. Providing students 24/7 virtual access and hands-on training using VMware GSX server. Proceedings of the 33rd annual ACM SIGUCCS conference on User services. ACM. pp: 421-425.
Vroom, V.H., 1964. Work and motivation. New York: Wiley.
Weiner, B., 1974. Achievement motivation and attribution theory. Morristown, NJ: General Learning Press.
Xu, L., D. Huang and W.-T. Tsai, 2014. Cloud-based virtual laboratory for network security education. IEEE Transactions on Education, 57(3): 145-150. Available at: https://doi.org/10.1109/te.2013.2282285.
Yildirim, G., M. Elban and S. Yildirim, 2018. Analysis of use of virtual reality technologies in history education: A case study. Asian Journal of Education and Training, 4(2): 62-69. Available at: https://doi.org/10.20448/journal.522.2018.42.62.69.
Zhu, W., 2015. Hands-on network programming projects in the cloud. Proceedings of the 46th ACM Technical Symposium on Computer Science Education. ACM. pp: 326-331.
Views and opinions expressed in this article are the views and opinions of the author(s), International Journal of Education and Practice shall not be responsible or answerable for any loss, damage or liability etc. caused in relation to/arising out of the use of the content. |