Hard-of-Hearing (HH) Students’ Perceptions of Multimodal EFL Learning

History: Received: 2021-01-21 Accepted: 2021-06-11 Published: 2021-06-18 Learners with hearing loss tend to learn visually. It attracted a teacher of hard-of-hearing (HH) students in a special needs school to implement multimodality that integrated all verbal and visual semiotic modes and used ICT in her English classroom. The study aims to explore the students’ perceptions of multimodal English learning. The interview of the two females and one male participant individually. In addition to the data gained from the interview, students' diaries were also analyzed based on themes that emerged from the codes and categories. The data were triangulated by checking the interviews with the diaries and confirming with the teacher and the students' parents. The findings of the study revealed both positive and negative perceptions emerged. The students perceived ICT-based multimodal learning positively to help them learn new vocabularies and understand a story quickly, increase their motivation to learn, improve their engagement, and make them able to learn autonomously. Meanwhile, they perceived the EFL learning negatively so that it took much time to learn all of the materials. Also, the students felt unconfident in performing activities. Therefore, it is recommended for HH teachers to provide more significant assistance, time, and patience to successfully support students in learning English.


Introduction
In today's globalized world, English is considered one of the most popular languages taught at schools. It also serves as an essential means of communication in the international world. Everyone has the right to learn English, including people with hearing disabilities. Due to their hearing disabilities, this special needs group has special needs and challenges different from hearing people. It is the task of education professionals to recognize and respond to their needs and rights to learn English as a foreign language. In addition to raise awareness of their particular needs, exploring their situation, and suggesting solutions to the difficulties they face, become the other duties of education professionals. Problems occur in teaching hard-of-hearing (HH) students, which calls for special consideration to deal with complexities. Students with hearing impairment categorized as visual learners whose learning needs are similar to those in mainstream education. They need special education because some learning methods and tools are less effective in enhancing their understanding (Csizér & Kontra, 2020). HH students have a natural orientation to visually proceed information that requires visual materials in learning a language, such as illustrating abstract concepts using images as a realistic visual display. Students in non-English speaking countries have a more significant challenge as English is a foreign language for them. It is the L3 for them after they learn sign language and their national language (Kontra, 2020). Chong & Daud (2015) highlight that teaching HH students requires particular learning tools and methods to enhance their understanding. For example, HH students need to use sign language as a primary form of communication. Other communication specificities are generally integrated simultaneously, such as facial expression, hand gestures, and lip-reading. Teachers commonly use gestures and writing on paper to facilitate small group work or discussion in the classroom. Some teachers incorporate technologies, such as phone applications or computer software. This integration of several modes is called multimodality. Jewitt (2008) views modalities as integrating various modes such as spatial, gestural, linguistic, memorable, and aural modes. Those different modalities generally come together in a digital environment to redesign the printed words and pictures or sound. Meanwhile, Kress (2003) defines multimodal resources as symbols that are both conventional and motivated. The use of these sources or modes in the teaching and learning process should be interrelated one to another in order to convey a particular message. For example, while explaining abstract things to students, they may create a concrete visualization of abstract things by drawing them aboard. Jewitt (2008) says that no single model can stand alone in the meaning-making process. Instead, every mode plays a discrete role in the whole. In line with that, Evans (2004) also argues that the combination of signs, print, and pictures in presenting the language can help the conceptually accurate translation between sign language, the national language, and English as a foreign language more meaningful. Sign language is a type of multimodal resource used in HH students' teaching and learning process (Evans, 2004). It functions similar to oral language in that it is used by the speakers to command, request, tell jokes, and express feelings. The point is that different modes such as verbal, written texts, images, and kinesis mode (including hand gestures, facial expression, lipmouthing, and eye contact) can work together harmoniously so students can understand the teaching materials more efficiently. Multimodal understandings of literacy require a thorough elaboration of the full multimodal ensemble used in the teaching and learning process, which becomes the communicative event in a pedagogical context. Some studies have reported the potential of technology, whether it is assistive or instructional, to enhance teaching and learning for special needs learners (Bari et al., 2013). In applying technology for the teaching and learning process, three factors, namely teacher, school, and technology, should be interrelated (Zhao et al., 2002). Among the three factors, Yusof et al. (2014) argue that the teacher holds the most prominent role in determining the success of the whole teaching and learning process. The technology integration decision includes how technology used in the classroom affects students' success in learning English. Technology implementation's success or failure depends heavily on teachers' knowledge and skills in integrating technology in meaningful ways.
Technology allows multimedia, which refers to the multiple representations of content in a digital environment. Through technology-induced interaction, there are some benefits for the students. For instance, by providing an external representation of the information into a more attractive and engaging representation, students' attention is maintained, and complex information is easier to understand. Multimodal learning environments also allow presenting instructional media in more than one mode of presentation that provides students with a broader variety of students learning styles. It is clear that multimedia activities are more inclusive and appealing to learning ways than visual, aural, and kinesthetic students. Accordingly, if utilized through thoughtfully planned activities, multimodal presentations, using subtitled video, can help students use both visual and auditory messages to help students improve their language skills (Sankey et al., 2010).
Considered a new area under SLA, studies about the English teaching and learning for HH students in the non-English speaking country are still not widely available. One line of such research includes investigating deaf and hard-of-hearing (DHH) students using the visual resources of a multimedia software package designed to support reading comprehension (Nikolaraizi et al., 2012). The participants of the study were 8 DHH students, ages 8 -12 years old. The study results revealed that the students used visual resources, but they did not strategically exploit them to support their reading comprehension. Therefore, the suggestion for teachers to create a mediated instruction in visual literacy helps students learn how to proceed with visual resources in a way that aids their learning.
Another study was conducted by Kontra et al. (2015). They interviewed 31 D/HH learners with ages ranging from 14 -19 years old about their beliefs about learning a foreign language. The participants desire to go work abroad so they can use the foreign language they learned. They realized that they faced some obstacles in learning the foreign language, but they claimed that studying harder and asking for help could help them deal with the difficulties. Some of the participants also acknowledged that they had limited knowledge of their national language. That what made learning another spoken language was a real challenge. Entwistle & Walker (2002) emphasize the significance of exploring students' perceptions of learning because their learning is more affected by their perceptions than by the teaching method. Therefore, by understanding HH students' life experience or their perspectives on learning EFL, educators could help them succeed in learning to adjust the learning approach and learning materials suitable for particular learners. That is why more studies that can guide teachers of HH students, policymakers, and curriculum designers in non-English speaking countries are highly needed. Regarding the lack of research focusing on the students' views about their English learning, particularly the one delivered in multimodal approach and incorporated technology, the present study explores the students' perceptions of the teacher's multimodal English learning. The research questions are as follows: What are the HH students' perceptions about learning English with Multimodality? Understanding these perceptions provides new insights into how the English teaching and learning process of HH students should be carried out regarding the students' needs as determined from their perceptions.

Method
The study aims to provide a detailed, thick description of the participants' perceptions of their English learning. Therefore, it belongs to a case study. Case studies, according to (Yin, 2017), seek to answer what and how questions. Three students were involved in the study. Participants' names were made pseudonym in order to keep the research confidential. They are Bella, Jasmine, and Ryan. Among the three students, Ryan is the only student who has a hearing disability since he was born. This 15 years old boy is not deaf. Instead, he has profound hearing difficulty. Although he still has hearing residue, he depends on sign language because of the little verbal exposure. Bella and Jasmine are 13 years old female students with profound hearing disabilities. These girls lost their hearing when they were a child. Bella lost her hearing when she was three years old because of medical treatments he had been through when she was sick. Meanwhile, Jasmine lost her hearing when she was five years old due to medication's effect.
Firstly, it was necessary to ask for the school principal's corresponding permissions, the English teacher, and the participants, who were the three HH students of grade eight, through a written consent form. The data were collected in two ways. The first was from students' diaries. After each meeting, the participants wrote a diary to write about what they learned, what they liked, what they did not like, and how they felt when they had the English class. The second data were collected from in-depth interviews. Since the students were hard-of-hearing, written interviews were carried out. I wrote the questions on a piece of paper, and they wrote down their answers. Due to the lack of English capability, the students used Indonesian to write their diaries and do the interviews. The interviewer made transcripts. These transcripts were qualitative content analysis based on themes derived from the codes and categories derived (Hsieh & Shannon, 2005). The data were triangulated by checking interviews with the diaries. I also confirmed their answers by checking with the teacher and the students' parents.

Findings
The analysis of students' reflective diaries falls into two big categories: positive and negative perceptions. In each category, some sub-themes emerged, as summarized below.

Helping students understand the materials
All three students argued that the learning process using the ICT-based multimodal approach helped them understand the materials easily. It is seen from the reflective diary written by Bella.

When new words were presented, I could tell the meaning of the words by looking at the pictures (Bella, 2019)
The sources of multimodality were printed words and pictures. As the words were new for the student, of course, he had no idea about what they meant. Luckily, the teacher provided a visual of the written words to tell the meaning of the words. The data above shows that lip-reading plays an important role in helping students with spoken words, particularly speech production and pronunciation. By looking at his teacher's clear lips movement, the students could produce the same sounds as the teacher did. In addition, reading the lip and looking at the written words allowed the students to pronounce the word correctly.
I could understand the story better when there were animated pictures. (Ryan, 2019) The pictures were presented in PPT slides. I could retell the story while the teacher was playing the slides using my own words in Bahasa Indonesia (Jasmine, 2019) On the word-level, a story presented in Power Point Presentation also allowed the student to comprehend the story.
Here is a sample of the materials they learned in the classroom. The PPT materials contain pictures, sounds, and subtitles. The pictures were taken from their surroundings so the students were familiar with the context. In delivering the materials, the teacher ran the slides with the sounds on. Unfortunately, due to the students' difficulty in hearing, the teacher was mouthing the words.

Increasing motivation
The multimodality practices involving the use of power-point presentations could motivate the students to learn more. It is seen from the interview transcripts below.

I want to learn English using PPT again (Bella, 2019)
Learning English felt so fun. I want to do that next time. (Jasmine, 2019) The more the teacher shows other slides, I want to read more. (Ryan, 2019) The data above show that the students were eager to attend the next class. It means that English classrooms were enjoyable. The fun learning activity motivated them to learn more. Besides, the curiosity about what was in the next slide also increased their motivation to see what was next.

Making students more engaged with the activities
The teacher involved visual and kinesthetic modes as parts of the multimodal learning where the students were asked to draw on the board. One student said that drawing was his favorite activity while the others also enjoyed it as well. Their responses below indicate that they found the drawing activity fun and engaging.
My favorite activity was drawing on the board. We played a game. The teacher gave me some words. She wrote the words in a piece of paper and gave the paper to me. She asked me to visualize the words by drawing on the board, and my friends had to guess the words in English by looking at my drawing. (Bella, 2019) Yes. I like drawing. It is so fun. I requested the teacher to draw more. (Ryan, 2019) The data above shows the multimodal approach; in this case, the kinesthetic mode let the students engage in the activities by doing physical activity. The physical activity here was drawing something on the word. But, of course, the drawing activity involved other modes: written words and the picture as the kinesthetic mode's outcome.

Sometimes, the teacher asked us to write something on her laptop or play the slides. (Jasmine, 2019)
As the multimodal materials were presented using ICT, in this case, PPT slides ran on a laptop; the student was given a chance by the teacher to operate the laptop. It means that the use of technology offered an opportunity to make students more involved in the activities.

Enhancing autonomous learning
As there was limited time in the classroom to learn all of the materials presented on the power-point slides, the teacher sent the materials to students' parents via social media chat so they could assist their children to learn on their own. The statement from one of the students below indicates that she could learn the materials autonomously with the help of her parents and teacher.

The teacher sent the PPT materials to our parents so I could study at home using the materials. Of course, there are many things that I didn't understand from the materials like the meaning of some vocabularies and sentences. My parents sometimes helped me, but I tried to find the answers on my own by browsing the internet or asking my teacher through social media chat. (Bella, 2019)
Because the materials were in the form of PPT files, they could be sent quickly to the students' parents. This allowed the student to learn the materials on her own. She had some options to solve any difficulties she faced in learning the materials. It could be seen that she could decide what to do when understanding the materials and doing the activities. She could either choose to ask her parents to help her or choose to solve her problems. The point is that she had the chance to control her own learning, which means being an autonomous learner.

Negative Perceptions
Despite the positive perceptions as presented above, there were also some negative perceptions of implementing the multimodality practices regarding the time and students' confidence.

Taking much time
All of the three students agreed that it took much time to learn the materials and finish the activities. It is seen from their statements below.
The story was too long. We did not have time to learn all of the new vocabularies encountered in the story. I wish it had been more time. (Bella, 2019) The teacher asked us to continue studying the material at home because the time was limited. (Jasmine, 2019) The materials were too much. When using PPT, my teacher spends time preparing everything, such as laptops, projectors, and speakers. When everything is ready, there is not much time left. (Ryan, 2019) It can be implied from the students' statements above that the teacher had poor classroom management, particularly poor time management. She did not adjust the material and did not prepare everything in advance. As a result, the students found out that the materials were too much to be studied in one meeting. As a result, it led to unfinished learning that the teacher needed the students to study the rest of the home materials.

Decreasing confidence
Although, the activities were engaging and motivating, sometimes the students found the activities a little bit difficult, complicated, and unfamiliar. Their responses below show that the activities decreased their confidence. The data above indicates that different students haddifferent anxiety. For example, Jasmine felt that doing specific tasks using the teacher's laptop was difficult because she was not familiar with the device. Meanwhile, Bella thought drawing was not an exciting activity although her teacher encouraged her to do it.

Sometimes my friends laugh at me when I make a mistake in pronouncing words. I felt hesitant. (Ryan, 2019)
One student, Ryan, was aware of their weakness, which was his pronunciation ability. It made him feel unconfident in pronouncing words in front of his teacher and his classmates. Things got worse as his friends laughed at his incorrect pronunciation. It made him feel hesitant to complete the activity.

Discussions
The first finding above reveals that the students felt they could understand the materials, particularly identifying new vocabularies and comprehending stories through visuals. The finding is in line with a study conducted by Bao (2017). He did

Nur Arifah Drajati, Bunga Ikasari, Rizka Junhita: Hard-of-Hearing (HH) Students'
Perceptions of Multimodal EFL Learning -47 -an experimental study involving two parallel classes. Multimodality is applied in the experimental class, while ordinary multimedia teaching is applied in the other class.
The study reveals that multimodality in the experimental class was more effective than in the control class. The students also found that clear lip-mouthing is beneficial to help them identify the sounds of words. Nasim et al. (2019) support this finding. In their study, HH learners were more dependent on reproducing sounds by reading lips and could not score well without lip reading. Thus, improper lip movements and the absence of visuals might result in HH learners' failure to reproduce speech correctly. However, English lip-mouthing was still a challenge for the three students. The next thing the students perceived positively about multimodal teaching and learning is that it increased their motivation to learn. This finding is supported by Bao (2017), who found out that multimodal teaching can help activate the classroom atmosphere and inspire students' motivation to read after class. The learning environment can be enhanced because combining different modes allows for flexibility in the interaction between teacher and students and creativity in delivering the material. A multimodal approach increases students' motivation in reading. It is a way to enhance motivation through a multimodal writing project. Students designed texts involving visuals, verbal, written, and other meanings (Darrington & Dousay, 2014). When these modes wrapped in technological tools, the multimodal environment increased students' motivation more as Yaman et al., (2016) found that integrating mobile application in the learning environment increased the motivation and interest of the hard-of-hearing students.
The next is about the autonomous learning that students can experience in a multimodal learning environment. The study's findings show that one of the HH students earned the motivation to learn the materials independently. She made efforts to solve the difficulties she faced by browsing the internet. What she did shows that she could learn autonomously. Autonomous learning is partially determined by these two factors; motivation and self-esteem (Joshi, 2011). Self-esteem refers to the learner's self-evaluation regarding the target learning, while the second term, motivation, refers to the efforts and passion for achieving the learning goals plus favorable attitudes towards learning.
In addition to the positive perceptions, negative perceptions emerge. The first negative perception deals with the fact that the multimodal environment, particularly incorporating technology, takes more time than traditional classroom. It leads to unfinished learning. It becomes a typical problem in the learning process of HH students. Vaughn & Wanzek (2014) say that depending on students' capacities for attention, instruction should give in longer stretches. According to Luckner et al. (2001), hard-of-hearing people cannot simultaneously use the visual and auditory channels. Instead, they have to use the visual first and shift to the auditory one. That is the reason why education becomes more time-consuming than for hearing people. If the materials are presented visually, HH students need more time to use and activate an external memory system like writing, typing, or signing.
Furthermore, due to their delay in speech production, they need to do lots of repetition and imitation. According to (Ghazi-Saidi & Ansaldo, 2017), these two approaches are very important in language acquisition. Smith & Basham (2014) also admitted that integrating online resources and using varied multimedia technologies requires more preparation time. Therefore the shortage of time indeed becomes one of the challenges in implementing the multimodality approach successfully.
The second negative perception deals with a lack of confidence in performing some activities. The findings revealed that 1 out of 3 students was anxious to complete difficult tasks due to unfamiliarity with the technology used. It does not seem to be a severe problem, but it needs special attention. Kotrlik & Redmann (2009) state that technology anxiety can result in physical and psychological symptoms. This anxiety can occur because of social, psychological, ability, and technical matters. Teachers need to find out what causes technology anxiety that happens to their students. In this study, one student was reluctant to use the technology because he was unfamiliar with the tool. However, the two other students had no problem with it. It means that the class was diverse. Some students might be affected by behavioral or psychological issues that may become more crucial if they are not designed to help exceptional learners' difficulties (Khan & Asif, 2017). Still, about anxiety, another student was afraid to read aloud and pronounce words loudly because his friends laughed at him. Giving praise to students is also essential to increase their motivation.
In general, the study results agree with the previous research and support existing theories about how multimodality benefits students with hearing impairment, particularly the benefits regarding better understanding of the materials and higher motivation. Meanwhile, the finding on how multimodal learning materials could help the HH students learn autonomously is a new finding as there has not been any previous study discussing autonomous learning in special education context. The other things that need be highlighted from this study are the difficulties and anxiety experienced by the students that led to the negative perceptions. As perception is a crucial determiner of students' behaviors towards certain environments, in this case, multimodal EFL learning, understanding the negative perceptions could be a gamechanger in the EFL learning of HH students.

Conclusion
The HH students found the practices of multimodality involving the use of different resources combining some modes such as visual, auditory, written texts, and kinesthetic are beneficial so that the multimodal teaching environment could help them understand the materials easily. In other words, it enhances their learning process. However, the students also thought that the teacher's materials were sometimes too complicated and too much. It was time-consuming to learn all of the materials. Teachers must find specific ways to deliver materials by doing some repetitions and using sign languages. Due to the different learning characteristics between hard-of-hearing and hearing students, HH students need unique treatments from their environments, especially from the teacher. Therefore, teachers need to provide a learning environment that can support their students' learning. There are many ways to do so. One of them is to provide materials that suit the students' needs. Of course, selecting suitable materials is not easy. Adaptation is required. In order to do that, teachers need to adjust. Teachers may need to cooperate with other teachers, the principal, and other stakeholders since the adjustment deal with the materials and the facilities to support the delivery of the material. Teachers have to provide more suitable materials; they have to devote more time and patience. The findings show that hard-of-hearing students need more time to learn than hearing ones. Based on the curriculum, the English subject taught more than once a week. Due to the insufficient time, teachers can assign students to learn on their own outside the school hours.
In general, this study provides a thorough exploration of how HH students perceived their EFL learning. Hence, it can give some insights for future studies in multimodality practice in the context. However, as an exploratory case study, this research entails some limitations. First, as the findings were unique in its context, it may not be applied to other context, though some extents of replications are possible. Second, the participants of the study were limited to the students only, though triangulations were done by checking the data with the teacher and students' parents. Thus, it might be worthwhile to gain more data from the teachers, parents, school principal, and other parties.