Validity of Electronic Device-Based Application for Visual Acuity Examination: A Systematic Review

Recent years, advances in the internet and communication technology have enabled the proliferation of digital medical devices with innovations in the form of health applications, including for visual acuity examination. However, the validity of these applications remains unclear. The limited mobility and health service during the COVID-19 pandemic underscores the urgent need to conduct research that validates these electronic device-based applications. Thus, this study aims to critically analyze whether the electronic device-based application is able to provide a valid and high-quality visual acuity examination. A systematic review was conducted through studies search on PubMed, MEDLINE, Springer, and Cochrane Library using specific keywords. After the studies were selected through inclusion and exclusion criteria, extraction was carried out. Publications from 2011 to the end of 2021 were reviewed, yielding in 1409 studies, of which 19 were included. The results showed a lower systematic bias for distance visual acuity testing with electronic device-based applications compared to standard reference tests with a mean difference of -0.08 to 0.10 logMAR. The validity of the near visual acuity examination with the application shows better results than the distance examination which is marked by smaller 95% limits of agreement range. The results of the analysis of Bland-Altman plots in all the studies reviewed showed that the accuracy of the examination results tended to increase in patients who had better visual acuity. In practice, the use of electronic device-based applications for visual acuity examination can increase the work effectiveness of medical personnel and the proliferation of digital medical devices. It can also be one of the breakthroughs in the field of remote medical services and support the implementation of telemedicine policies. INDEX TERMS Application, electronic device, visual acuity, systematic review

systems allowed early detection of true clinical changes in visual acuity in each patient [4].
Visual acuity is a measure of the eye's ability to clearly distinguish the shape and detail of objects at a certain distance [5]. Visual acuity examination is done by comparing a person's visual acuity with the standard normal person which usually begins with an examination using an optotype. Optotypes are marks of different sizes that are placed systematically on a visual acuity chart. The optotype is usually a number, letter, or symbol as an instrument to test visual acuity [6]. Conventionally, symbols have been printed on cards or graphics that are mounted on walls and presented to patients for examination [5].
Human visual acuity can change due to many eye problems, therefore an examination of visual acuity needs to be carried out to help in detecting various eye disorders. Eye disorder such as visual impairment is a health problem that has profound effects on quality of life, educational attainment, and economic productivity [7,8].
World Health Organization (WHO) estimates that 2.2 billion people in the world have near or distance vision problems. The most common causes of visual impairment worldwide are uncorrected refractive errors (48.99%), followed by cataracts (25.81%) and age-related macular degeneration (4.1%). Meanwhile, the most common causes of blindness were cataracts (34.47%), followed by uncorrected refractive errors (20.26%), and glaucoma (8.30%). More than 75% of visual impairments are actually preventable [9]. Nationally, the results of the 2014-2016 Rapid Assessment of Avoidable Blindness (RABB) survey in 15 provinces showed that the blindness rate in Indonesia reached 900,000 people. The main cause of blindness and visual impairment in the population aged over 50 years in Indonesia is untreated cataracts with a proportion of 77.7% [10]. This shows that there are still many cases that are not corrected or even undetected. This fact shows the importance of increasing the affordability of visual acuity examinations to assist in the early detection of visual impairment.
During the COVID-19 pandemic, social distancing, quarantine, and restrictions on face-to-face interactions were enforced to prevent and break the spread of the SARS-CoV-2 virus. This poses a challenge in providing eye care to patients, as eye examinations require the examiner to be in close contact with the patient. In fact, the first case of COVID-19 was reported by an ophthalmologist at the Wuhan Central Hospital, who also died from the new virus [11,12].
Various types of visual acuity testing applications are available and can be downloaded easily on the internet. However, the validity of these applications remains unclear. Meanwhile, the limited mobility and health service during the COVID-19 pandemic underscore the urgent need to conduct research that validates these electronic device-based applications. Thus, this study aims to critically analyze whether the electronic device-based application is able to provide a valid and high-quality visual acuity examination.

A. SEARCH STRATEGY
This systematic review is conducted based on Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) guidelines. We performed a comprehensive search of PubMed, MEDLINE, Springer, and Cochrane Library databases up to January 2011 and updated later to the end of 2021 using keywords as follows: "application", "electronic device", and "visual acuity". Boolean operators (AND, OR, NOT) and truncation (*) were applied to broaden and narrow the search results. We also used the Medical Subject Headings (MeSH) terms in the search strategy. However, the search language was limited to English and Bahasa Indonesia.

B. ELIGIBILITY CRITERIA
Inclusion criteria were set to filter the results as follows: (1) diagnostic test, observational study, or clinical trial, and (2) investigating the validity of visual acuity examination performed by electronic device-based application. It is worth mentioning that different study designs were incorporated into this review including those with one or more index tests and with any reference method that investigated visual acuity test in the general population. Conversely, the exclusion criteria defined included: (1) irrelevant topics, (2) not having index test as comparison, (3) unknown and/or inappropriate study types and settings, (4) incompatible language, and (5) irretrievable full-text articles.

C. DATA EXTRACTION AND RISK OF BIAS
The following data from articles were extracted, including author and year of publication, study design and location, sample size, index test, reference test, and outcome measures such as mean difference, sensitivity, specificity, and any other reported outcome. The quality of included studies was assessed using the Joanna Briggs Institute (JBI) checklist with >50% cut-off. Risk of bias assessment was conducted by the reviewers collaboratively and discrepancies were resolved by consensus between reviewers.

A. STUDY SELECTION
A total of 1409 studies were initially identified. After removing 324 duplicates, 1085 results were screened based on title and abstract, out of which 132 full texts were identified to be examined ( Figure-1). Finally, 113 studies were excluded due to not meeting the inclusion criteria. In total, 19 articles were included in this review. The quality assessment of all studies using the JBI checklist showed a low risk of bias.

IV. DISCUSSION
The results showed 18 of 19 studies stated that visual acuity examination with electronic device-based applications gave valid results. The overall identification results also show lower mean difference between digital applications compared to standard reference tests in assessing distance visual acuity. This indicates a low systematic bias. The mean difference ranged from -0.08 to 0.10 logMAR. The majority of the 95% limits of agreement range on the results of the distance visual acuity examination is quite wide, which indicates the variability.
The study by Satgunam et al. (2021) stated that the Smart Optometry application was not comparable to the reduced Snellen chart, but was declared valid because it only differed by 2 logMAR lines, which means it is still clinically acceptable. This difference is not a problem when digital applications are used for screening to detect visual impairments associated with decreased visual acuity, even though age and refractive errors affect measurements in Smart Optometry applications [13].
Several studies have attempted to link the use of electronic device-based applications to clinical practice. A study stated that the repeatability of using the Eye Chart application needs to be investigated before being integrated into clinical practice even though the study results are reported to be valid [16]. Correspondingly, the study by Perera et al. (2015) has valid results but still needs further research for clinical use [28]. Different things were reported in a previous study by Gounder et al. (2014) who stated that the Eye Snellen application can be used to measure visual acuity in clinical settings reliably on all measures of visual acuity [30]. More specifically, in one application, Eye Chart Pro, it was reported to be reliable for testing if Snellen's visual acuity was better than 20/200 or 0.1 in decimal [31]. The results of the analysis of Bland-Altman plots in all the studies reviewed did show that the accuracy of the examination results tended to increase in patients who had better visual acuity.
In contrast to the results of other studies, one study stated that the Eye Hand Book application was invalid. The application provides an overestimated close-range visual acuity result compared to a conventional near card with an average of 0.11 logMAR, except for the standard measurement result of 20/20. This means that patients tend to perform better on examinations, so eye disorders may go undetected. Overestimated results can result in delays in treatment. This study suspects that the main factor that plays a role in the discrepancy in the results between the application and the standard reference test is the contrast ratio. The contrast ratio of the clean printed Snellen chart or ETDRS is below 33:1, while the iPhone 5 as a digital device used in this study has a contrast ratio of 1151:1 [27]. Other studies have also reported that measurements of visual acuity in subjects can be overestimated with increasing contrast and lighting levels [32]. However, the validity of the near vision test in this systematic review overall shows better results than the remote visual acuity assessment which is characterized by a smaller 95% limit of agreement range. In daily use in clinics, examination time is critical for efficiency. This is the reason why the Snellen chart is an optotype for routine examination in clinical practice. The results of the application quality analysis on the operability component show that the Peek Acuity application that uses the ETDRS Tumbling E chart optotype has an average inspection time of 5 seconds faster than the conventional ETDRS Tumbling E chart [24]. Correspondingly, the REST application also recorded an average examination time of 2.8 seconds faster than the standard reference test [26]. These results support digital applications for routine use. Nevertheless, previous research reported there is a slight delay between the time. During this delay, the subject may begin to read the first line of the text. The result would be an underestimate of the reading time, and consequently, an overestimate of reading speed. This possibility is supported by a recent study that compared stopwatch versus automated timing in a computer-based reading test [32].
In addition, visual acuity examination by application is also affected by the basis of the electronic device. Recent studies have reported few differences in test time between paper and screens 33,34 . In a different study, it is still a debate whether reading is better on paper or LCD [35]. Furthermore, previous research found that visual acuity examinations on iPads are particularly susceptible to glare. Utilizing an antiglare coating can be the solution [36]. However, some of the information that accompanies the valid statement decisions in 18 studies shows that the feasibility of digital applications is currently still limited to early detection and has the potential to be used as an initial examination in remote medical services.
Visual acuity examination with digital applications based on electronic devices has a lot of importance for the development of remote services. This makes various studies state recommendations for the use of digital applications even though there are slight differences between the results of the examination and the application compared to standard references. Conventional examinations in hospitals require the patient to physically come to the clinic. Difficulties that may be faced by patients are living far away, for example, people living in rural areas, elderly patients, and patients who are unable to move [37]. Remote inspection can also reduce costs and speed up early detection [38]. Moreover, a smart mobile application to monitor visual function in diabetic retinopathy and age-related macular degeneration patients already existed and is being investigated [39]. With the increasing flow of digitization, the portability aspect seen from instability supports the quality of digital visual acuity check applications.
The availability and increasing use of electronic devices, especially smartphones and tablets, further emphasizes the potential for digital applications to identify the most common causes of visual impairment in Indonesia and the world, including uncorrected refractive errors. Studies report that half of the visually impaired population actually has a decrease in visual acuity that can be prevented or corrected with glasses or contact lenses [40].
In practice, the use of electronic device-based applications for visual acuity examination can increase the work effectiveness of medical personnel and increase the proliferation of digital medical devices. The results of this systematic review can also be one of the breakthroughs in the field of remote medical services and support the implementation of telemedicine policies.
As in other studies, this review also has several limitations. Due to the novelty of the topic discussed, there are limited study resources. As a result, the study design and validity parameters of the included studies varied. However, all being considered large study using the electronic devicebased application as an index and conventional visual acuity examination as the comparison.

V. CONCLUSION
In conclusion, the use of electronic device-based applications provides valid results for early detection in visual acuity examinations. This systematic review also found that electronic device-based application visual acuity examination showed better results in near-range visual acuity assessments than the distance visual acuity assessments. We also found that the results of Bland-Altman plots analysis observed in all included studies showed that the accuracy of the examination results tended to increase in patients who had better visual acuity.
Further research on the repeatability of visual acuity examination with electronic device-based applications is required to support the validity conclusion. In addition, it is necessary to conduct research that examines the potential for remote medical services.