Readability of Articles in Journal of Religious and Socio-cultural Studies

s. 127–133. Tenopir, C., Mays, R., & Wu, L. (2011). Journal article growth and reading patterns. New Review of Information Networking, 16(1), 4–22. https://doi.org/10.1080/13614576.2011.566796 Vajjala, S., & Luči ́c, I. L. (2018). OneStopEnglish corpus: A new corpus for automatic readability assessment and text simplification. Xia, M., Kochmar, E., & Briscoe, T. (2016). Text Readability Assessment for Second Language Learners. The 11th Workshop on Innovative Use of NLP for Building Educational Applications, 12–22. Yulianto. (2019). An Analysis on Readability Level of English Reading Texts for Eighth Grade Students. J-SHMIC Journal of English for Academic, 6, No 1, 82–83. Yuwono, D. (2015). Translation Quality of Translated Abstracts from Indonesian to English in The Journal of STAIN Ponorogo 2015. Jurnal Bahasa Dan Sastra, 2(2), 18– 19.

of an authors' writing contributes to their research's impact (McCannon, 2019). There are national, international indexed systems. The international index system has a higher research impact than the national one. Thus, researchers and universities regard publishing in a journal indexed internationally as a valid criterion in evaluating their research output. The Ministry of Research and Technology of the Republic of Indonesia recommends Scopus and Web of Science as international journal index systems. Many universities get the academicians to publish their article in a journal indexed by Scopus or Web of Science as one of their promotion requirements.
Both Scopus and Web of Science are selective in controlling the quality, the relevance of their contents, and production costs (Aksnes & Sivertsen, 2019). One of the specific evaluation criteria under the category of content is the readability of the article (Sedghi & Heshajin, 2018). It is the effort to make sure the articles readable by an international audience. Both the writers and editorial boards need to evaluate whether it is easy to be understood or not. Readability influences readers' understanding of the text (Yulianto, 2019). There is also an important relationship between a paper's readability and its citations (McCannon, 2019). If the readers understand the study well, they can cite it correctly (Dolnicar & Chapple, 2015) because all disciplines agree that citation is essential in the grant process (Tenopir et al., 2011).
Scholars have produced and developed more than 50 readability formulas. Several researchers also have conducted a study concerning the readability test. They applied it in different texts. Many researchers applied it in education by examining the readability of textbooks such as Salihah, Sahruddin, & Degeng (2020). They analyze the readability level of English textbooks to make sure the students will understand the text. Rosita (2018) and Savitri (2018) also applied readability in the translation field to assess translation quality. In translation, readability is referred to whether both source text and the translated text can be read and understood by its readers or not (Yuwono, 2015). Daraz et al. (2018), MacLean, Basch, Ethan, & Garcia (2019), Kher, Johnson, & Griffith (2017) applied readability to online health information or material published on websites. They concluded that it is important to tailor better online patient education materials appropriate for the general population. Xia, Kochmar, & Briscoe (2016), Cha, Gwon, & Kung (2017), Vajjala & Luči´c (2018), and Graefe, Haim, Haarmann, & Brosius (2018) applied readability using technology through computational linguistics. Specifically, in the academic article, Severance & Cohen (2015) applied readability on medical research journal abstracts from 1960 to 2010. Dolnicar & Chapple (2015) analyzed the readability of articles in tourism journals.
Based on those previous studies, the readability formula has not been applied yet in the journal of religious and socio-cultural studies. It is also stressed by google scholar search using readability of journal article keyword. At the first 100 results, most of the text tested on readability formula are medical research, financial report, and disclosure. Thus, it is important to do this research.
One of religious and socio-cultural studies journal is WAWASAN: Jurnal Ilmiah Agama dan Sosial Budaya. It is the media to disseminate the research on religious and socio-cultural studies all over the world. The journal needs to prepare it indexed in Scopus. Consequently, Wawasan journal needs to use the international language and make sure the readability of its articles. It has published articles in English in three issues. This study aims to seek the readability score of each article's text in Wawasan in three issues and that appropriateness with the targeted readers. The writers use only five formulas to test the articles. Those are the Flesch-Kincaid Grade Level (FKGL), the SMOG Index, the Coleman-Liau Index (CLI), the Automated Readability Index (ARI), and Linsear Write Formula (LWF) via link https://readabilityformulas.com. This research is very important as the evaluation either for the article authors or the journal's editorial board to publish appropriate articles with the targeted readers. Testing the readability of an article is an effort to publish a good article based on its content to get more citations and to fulfill one of the requirements to be indexed by Scopus.

Method
The researchers used mix method, quantitative to analyze the readability score of the article, and the qualitative method to describe the phenomenon related to the data. The data were collected from articles in Wawasan journal. The researchers collected the data in three issues to be analyzed because the journal has just published entire articles in English in those three issues. Those are Volume 4, No 1 (2019), Volume 4, No 2 (2019), and Volume 5, No 1 (2020). In this study, the researchers analyzed only abstract sections. It's based on three reasons. First, academic articles are difficult to analyze because full stops do not always indicate the end of a sentence. It needs manual pre-processing. Second, the abstract represents the most important information of articles. Third, the authors write consistently in style across all article sections (Dolnicar & Chapple, 2015).
After collecting the data, the researchers analyzed them by applying a freeware taken from https://readabilityformulas.com. to count the readability score of those data. Although there are seven formulas available on that online freeware, the researchers just tested the articles based on five formulas as stated before because those five formulas have the same evaluation related to the score, the higher readability scores, the more difficult the text is. Then, the researchers provided the table for those readability test results. In the next step, the researchers compare the readability score with the targeted readers of the research article. There are four keys point to determine target readers or audience (Elsevier, 2021). Those are: 1) Interest: Who will want to read your research? Who will be interested in your key findings?; 2) Level of education: Is your document suited for people with an advanced level of education or relatively low?; 3) Background: Do you want to impact other scientists in the same scientific field as you or professionals in other areas of activity; and 4) Expertise: Do you want to engage beginners or experts?
Due to the keys point above to determine target readers, the researchers determine the target readers of articles journal based on the level of education, namely college, college graduate, and college graduate and above. It is based on the fact that the collegian must cite articles of the journal with 80% as references for their final assignment. So do the graduate students.

Findings and Discussion
The readability deals with the degree to which a given class of people find certain reading matter compelling and comprehensible (McLaughlin in Dubay, 2004:3). Readability is what makes some texts easier to read than others do (Dubay, 2004:3). According to Richard et al. cited in Savitri, readability is how easily written materials can be read and understood (Savitri, 2018). Over time, numerous analysts have been involved in checking the readability of text in various fields. Then, they classify their text into a scale of readability levels. In the end, they conclude whether the text is appropriate for the targeted readers. Readability is related to text difficulty that deals with the easiness of text to read and comprehend. It is influenced by many factors, ranging from the length of a word, sentence, and vocabulary (Richards & Schmidt, 2002).
According to Gray and Leary, four factors affect readability. Those are content, style, design, and structure (Dubay, 2004:18). The content consists of propositions, organization, and coherence. A good text should have a good proposition, organization, and coherence to get the reader easier to understand the text. The style consists of semantic content (vocabulary) and syntactic structure (sentence length). The better vocabulary mastery and the shorter sentence, the better readers understand the text. The design consists of typography, format, and illustrations. Good typography, format, and illustrations will influence readers' interest and mood in reading the text. The more interesting a text design, the more spirit readers, get.
Meanwhile, the structure consists of chapters, heading, and also navigation. The writers' accuracy in determining the chapter, headings, and navigation will influence readers' interest in reading the text. The long chapter, inappropriate heading, and bad navigation will make the readers stop reading the text.
Hence, the readability formulas have been designed to provide a quantifiable evaluation of how easily the readers get the content of the text. The written text should be fitting to the target readers so that they get the message appropriately. In this case, the researchers use only five readability formulas provided in freeware https://readabilityformulas.com. Those are FKGL, the SMOG Index, CLI, ARI, and LWF.
The first is the Flesch-Kincaid Grade Level (FKGL) formula. Rudolf Flesch as co-author and John P. Kincaid as author improves it upon FRE Formula. Thus, it is also called the Flesch-Kincaid Grade Level Readability Test. Hamsik (1948) stated that the FKGL formula is used to measure non-native and native reading texts (Salihah et al., 2020). It aims to assess whether the text is written in literature suitable for the target reader. It employs word lengths and sentence lengths. A result is a number that corresponds to a school grade level, which suggests that a higher score corresponds to the next level of education that is wished to recognize the text. There are four stages: 1) Counting the average number of words used per sentence; 2) Counting the average number of syllables per word; 3) Multiplying the average number of words by 0.39 and add it to the average number of syllables per word multiplied by 11.8; and 4) Subtracting 15.59 from the result.
The second is the Coleman-Liau Index. Meri Coleman and T. L. Liau designed the Coleman-Liau Index. The index relies on characters instead of syllables per word. They believed that computerized assessments understand characters more easily and accurately than counting syllables and sentence length. Thus, it is to automatically calculate samples of hard-copy text by computer-average sentence length (M. Coleman and T. L. Liau, 1975).

Ai Yeni Yuliyanti, Busro: Readability of Articles in Journal of Religious and Socio-cultural Studies
-17 - The third is the SMOG index. Harry McLaughlin created the SMOG Readability Formula in 1969. It estimates certain text appropriateness to the grade level of a person's education. It was an improvement over other readability formulas. Healthcare materials widely used this formula (Biddinika et al., 2016). It counted the number of polysyllabic words in three ten-sentence samples near the beginning, middle, and end of a piece of text (McLaughlin, 1969). If the sentences are less than 30 sentences, the formula corrects this. There are five steps: 1) Taking the entire text to be assessed; 2) Counting 10 sentences in a row near the beginning, 10 in the middle, and 10 in the end for a total of 30 sentences; 3) Counting every word with three or more syllables in each group of sentences, even if the same word appears more than once; 4) Calculating the square root of the number arrived at in Step 3 and round it off to the nearest 10; and 5) Add 3 to the figure arrived at in Step 4 to know the SMOG Grade.
The fourth is the Automated Readability Index (ARI). Its output is a number that approximates the grade level needed to comprehend the text. It is derived from ratios representing word difficulty by counting the number of letters per word and sentence. The index relies on a factor of characters per word instead of the usual syllables per word. The reason is that computer programs count the number of characters more readily and accurately than syllables. Generally, ARI determines readability only based on a portion of the factors involved in selecting appropriate written material. The author's background, interests, motivation, writing style, and skill are more important than this formula's scope. In other words, those cannot be counted mathematically.
The last is Linsear Write Formula. It is based on sentence length and the number of words with three or more syllables. The sentence length and 3 syllable words or more will influence the results of the Linsear Write formula. In conclusion, the researchers can use shorter sentences and less complex words (1 or 2 syllable words) for the lower grade level. Here are the steps: 1) Finding a 100-word sample from your writing; 2) Counting the easy words (two syllables or less) and place a number "1" over each word; 3) Counting the hard words (three syllables or more) and place a number "3" over each word as pronounced by the dictionary; 4) Multiplying the number of easy words times "1"; 5) Multiplying the number of hard words times "3"; 6) Adding the two previous numbers together; 7) Dividing that total by the number of sentences; 8) If your answer is >20, divide by "2," and that is your answer; and 9) If your answer is <20 or equal to 20, subtract "2," and then divide by "2." That is your answer. Based on FKGL, the easiest is text 4.1.1 in which the score is 12.2. In contrast, there are 2 highest score texts namely text 4.1.1 and 4.1.2 with 16.2. Based on CLI, the easiest is text 4.1.4 in which the score is 9. In contrast, there are 3 highest score texts namely text 4.1.1, 4.1.3, and 4.1.8 with 15. Tested on SMOG, ARI, and LWF, the most difficult text is text 4.1.2 in which the score for SMOG is 14.7, for ARI is 17.1 and for LWF is 20.3. The easiest text-based on SMOG is text 4.1.4, with a score of 11.6. Based on ARI, the easiest text is 4.1.3 with a score of 5.9. Meanwhile, the easiest text-based on LWF is text 4.1.8 with a score of 13.1.   Based on FKGL, CLI and ARI, the easiest is text 4.2.1, in which score for FKGL is 10.9, for CLI is 10 while for ARI is 7.1. The most difficult based on FKGL, CLI and ARI is text 4.2.8 with the score for FKGL is 19, for CLI is 18, while for ARI is 18.6. Tested on SMOG, the easiest is text 4.1.1 with a score 9.9, and the most difficult is text 4.2.3 with a score 16.8. Meanwhile, the easiest text based on LWF is text 4.2.1 and the most difficult is text 4.1.3 with the score 21.5.

Volume 5, no 1 (2020)
There are 9 articles published in Volume 5, no 1 (2020). The writers are Khoir (2020); Arif (2019); Abdillah and Kamaruddin (2020); Saihu (2020); Kholifah (2020); Huriani and Annibras (2020); Dandirwalu and Rehy (2020); Sunarto, Aliyudin, Nurdin, Laksana, Muslimah and Azis (2020); Fuad (2020). Those articles are about religious and socio-cultural studies that appropriate with the journal scope. The readability of those eight articles are drawn in table 3.  Table 3 shows that text 5.1.7 is the easiest text and text 5.1.8 is the most difficult text-based on FKGL, SMOG, ARI, and LWF. Tested on FKGL, the easiest text score is 11.5 and the most difficult text score is 20.2. Based on SMOG, the easiest text score is 10.3 and the most difficult text score is 17.5. Based on ARI, the easiest text score is 11.5 and the most difficult text score is 20.5. Meanwhile, based on LWF, the easiest text score is 11.3 and the most difficult text score is 12.3. In contrast with the other formula, based on CLI the easiest text is text 5.1.1, and there are 2 texts with the highest score namely text 5.1.2 and 5.1.5.

The Appropriateness Readability Score with the Grade Level
The result of the readability score based on five formulas: FKGL, CLI, SMOG, ARI and LWF deals with the suitable grade level of targeted readers. Thus, the researchers will explain the appropriateness of each issue below. The primary step is to ensure whether all articles of each issue are suitable for three grade levels: college, college graduate and above. Table 4 shows the appropriateness of 8 articles in Volume 4, no 1 (2019) with the targeted readers: college, college graduate, or college graduate and above. Tested on FKGL, there are 4 articles for college, 3 articles suitable for college and graduate and an article suitable for 12 th -grade level. Based on CLI, there are 6 articles suitable for college, 1 article suitable for 11 th grade, and 1 suitable for 9 th -grade level. Tested on SMOG, there are 5 articles suitable for college and 3 articles suitable for 12 th . Based on ARI, there are 4 articles suitable for college graduates, 3 articles suitable for college and an article suitable for 12 th -grade level.
Meanwhile, tested on LWF, there are 6 articles suitable for college graduates and above and 2 articles suitable for college. Generally, most of the articles are suitable for college, college graduate and college graduate and above. Tested on five formulas, almost all articles in three issues (83%) are suitable for the targeted readers.   Table 5 shows the appropriateness of 9 articles in Volume 4, no 2 (2019) with the targeted readers: college, college graduate, or college graduate and above. Tested on FKGL, there are 4 articles suitable for college graduates and above, 3 articles suitable for college, and 2 articles for 11th-grade level. Based on CLI, there are 4 articles suitable for college, 1 article suitable for college graduate, 2 article for 12th grade and 1 article suitable for 10th grade level. Tested on SMOG, there are 4 articles for college and 2 articles suitable for college graduate, 1 article suitable for 12th grade and 2 articles suitable for 10th grade. Based on ARI, there are 4 articles suitable for college graduates, 3 articles suitable for college, 1 article suitable for 10th grade and 1 article suitable for 6/7th grade level.
Meanwhile, tested on LWF, there are 6 articles suitable for college graduates and above, 1 article suitable for college, 1 article suitable for 11th grade, and 1 suitable for 8th grade. Generally, most of the articles are suitable for college, college graduate and college graduate, and above. Based on the tests through five formulas, it is found that only 29 % of the articles which is not fit to the targeted readers.  Table 6 shows the appropriateness of 9 articles in Volume 5, no 1 (2020) with the targeted readers: college, college graduate or college graduate and above. Tested on FKGL, there are 5 articles, 3 articles suitable for college graduates and above and an article suitable for 12 th -grade level. Based on CLI, there are 6 articles suitable for college, 2 articles suitable for college graduate and 1 article suitable for 11 th -grade level. Tested on SMOG, there are 5 articles suitable for college, 1 article suitable for college graduate and 1 article suitable for 10 th grade. Based on ARI, there are 4 articles for college graduates, 2 articles suitable for college, and 3 suitable for 12 th grade.
Meanwhile, tested on LWF, there are 5 articles suitable for college graduates and above, 1 article suitable for college, 2 articles suitable for 12 th grade, and 1 suitable 11 th grade. Generally, most of the articles are suitable for college, college graduate, and college graduate, and above. Based on the tests through five formulas, it is found that only 24 % of the articles which is not fitted to the targeted readers.
Based on the website accessed at 2nd March 2021, the top ten countries as visitors of Wawasan journal are Indonesia (79%), United States (14,2%), Malaysia (2,5%), Russian Federation (0,7%), India (0,45%), Singapore (0,41%), Hong Kong (0,32%), Saudi Arabia (0,30%), Netherlands (0,25%) and United Kingdom (0,22%). It shows that most visitors to the Wawasan journal are Indonesian but foreign people also visit this website. Using English is an effort to Wawasan journal accessed widely by international audiences. The readability test also becomes an effort to get the articles cited more. Nevertheless, it is not the only factor that influences article citation. Because there is lower readability are cited more (Gazni, 2011;Stremersch et al., 2007cited in Dolnicar & Chapple, 2015. In other fields such as translation health information, medical research, linguistics, and education, the better the readability score is, the more understood the text is (Savitri, 2018;Rosita, 2018;Daraz et al., 2018;MacLean et al., 2019;Kher et al., 2017;Xia et al., 2016;Vajjala & Luči´c, 2018;Graefe et al., 2018;Cha et al., 2017;Severance & Cohen, 2015;Dolnicar & Chapple, 2015). It is in line with this study that focuses on journals of religious and socio-cultural studies. The difference is that they use different readability formulas due to their consideration. The translation field is more identic because they use Nababan's scale that consists of readability, accuracy and clarity.
Based on the readability score, most of the readability of articles in journals of religious and socio-cultural studies is appropriate with the targeted readers. They are college, college graduate, and college graduate and above. However, inappropriate articles were found in the collected data of those three issues. Specifically, this study contributes to the Wawasan journal editorial board that their article's readability is appropriate with the targeted readers even few are inappropriate. Thus, they need to improve the readability score in all their next edition. Generally, this study recommends all the journal articles to test the proposed article on readability formula before publishing the articles in their journal. Making sure the readability of sociocultural studies is vital to explore and disseminate how the social and culture of people are, not only limited to specific regions or countries but also around the world and what the trend of religious and socio-cultural studies are.

Conclusion
Both the authors of an article or the journal's editorial board need to evaluate their articles' readability. It is beneficial to make sure whether the readers of articles easily understand and comprehend the content of those articles. Tested on five formulas, almost all collected data (83%) are appropriate for college, college graduate and college graduate, and above. It means 83% of whole articles are suitable for targeted readers. Meanwhile, there are many articles with targeted readers, namely the lower level or school grade level. Based on the result, the readability score of the three issues is relatively stable. It is at the college level based on FKGL, CLI, SMOG, and ARI formulas and at college graduate and above based on LWF. The limitation is that this study only analyzes the abstract of each article using FKGL, CLI, SMOG, ARI, and LWF. It also does not consider English of the targeted readers whether mother language, second language, or foreign language. Despite its limitation, the researchers suggest that the editorial board of the article needs to check the readability score of the proposed article, not only its plagiarism and grammar.