INTRODUCTION
The lacrimal drainage system (LDS) is a mucous membrane-lined conduit, the function of which depends on a complex interplay of anatomy and physiology. Appropriate drainage of tears depends on several factors, including normal pump mechanisms and anatomic status of the drainage passages[1-2]. Epiphora is one of the common complaints of patients in ophthalmology clinic[3]. Patients presenting with epiphora may have a partial (stenosis) or complete obstruction of LDS[4-6]. LDS assessment and tests are necessary steps towards the best possible management[3,6-8]. Tests are for the assessment of LDS anatomy (diagnostic probing and irrigation test) and physiology [fluorescein dye disappearance test(FDDT) and lacrimal scintigraphy (LS)][3,5]. Previous studies have shown that the results of irrigation test are not always consistent with LS and different results could be found in more than half of the symptomatic patients[9-10].
FDDT is a simple and useful screening test in assessment of the patients with nasolacrimal duct obstruction even though it cannot differentiate an anatomical from a physiological obstruction[3,7]. While diagnostic probing and irrigation test is the gold standard for diagnosis of anatomical obstructions[2-3,7,11-12], LS is mostly requested for the patients with patent LDS on irrigation test in which stenosis and/or lacrimal pump impairment are assessed[13]. However,previous studies have found considerable variation in the technique, normative data, and interpretation of LS[13-15]. It is not unusual to encounter conflicting results of LS with patients’ complaints[13,15]. Therefore, its exact value on making an accurate decision is unclear[14]. LS images do mostly come along with a nuclear medicine specialist’s report which, to our experience, has a significant impact on clinicians’ (especially comprehensive ophthalmologists) management plan. On the other hand, lacrimal surgeons are mostly relying on their own interpretation of the images without considering the attached report. Therefore, subsequent management plan might be different.
Currently it is not clear how similar the interpretation of LS is between the nuclear medicine specialist and lacrimal surgeon(inter-observer agreement) and if ophthalmologists and other involved clinicians should rely on the attached report or consult to a lacrimal surgeon before making any decision regarding the presence and location of obstruction in the LDS. It is also not clear if the same observer’s report will be the same in different times (intra-observer agreement).
Therefore, this study was aiming to assess the agreement[16]between a lacrimal surgeon (Kashkouli MB) and nuclear medicine specialist (Hedayati R) on LS report regarding both the type (normal, partial obstruction, complete obstruction)and anatomical location of the obstruction (presac, preduct,intraduct) in general and in the subgroup of patients with normal versus abnormal lacrimal irrigation test, to evaluate intra-observer repeatability[16] for the lacrimal surgeon’s reports at different times, and to assess the reliability[16] of LS considering the lacrimal irrigation test results.
SUBJECTS AND METHODS
Ethical Approval The study was approved by the Ethics Committee of Iran University Eye Research Center(21471/124/01/92) and informed consent was obtained from all the subjects. This study adhered to the tenets of the Declaration of Helsinki.
Initially, 100 LS and irrigation test data were extracted from our previous prospective study[17] on 50 patients (＞6 years old)with unilateral anophthalmic socket which was carried on at a university hospital (Rassoul Akram Hospital) between 1st of March 2013 and 1st of February 2014. Included were subjects with more than 6 years of age (to cooperate with the tests)who attended the oculofacial plastic clinic for regular checkup.Excluded were the patients with previous ocular or periocular surgery, history of systemic diseases affecting the eye or ocular surface, long term use of any eye drop, and active infection or inflammation of the eye or ocular adnexa on the normal eye and LDS and or eyelid problem on the anophthalmic socket side.Patients were referred to the nuclear medicine department,had LS test, and handed over the images and reports (nuclear medicine specialist reports) to the lacrimal clinic secretary.A masked examiner (Kashkouli MB) performed a diagnostic probing and irrigation test[18]. All the LS images were then interpreted and reported by the same masked observer(Kashkouli MB) at the end of the previous study[17] in February 2014. The second report was made by the same lacrimal surgeon (Kashkouli MB) more than 3y after the first report(September 2017) for assessment of intra-observer variation.The second report of lacrimal surgeon was not presented in previous article. To avoid duplication of data, only the results of reliability and agreement analyses will be presented here.
Patients sat upright in front of a gamma camera equipped with a low-energy, high-resolution collimator (E. Cam,Siemens, Erlangen, Germany) with dual head fixed in an adjustable stand. The distance of the cornea to the surface of the collimator was less than 20 mm. The tracer study was started after instillation of approximately 3.7 MBq (0.1 mCi)of Tc-99m pertechnetate in a saline solution per eye with an eyedropper. Patients were instructed to continue blinking normally and if necessary to blot, not smear tears away with a tissue to prevent potential of contamination. Dynamic images were then obtained (10s/frame for first 1-2min, followed by 1min/frame for remaining time) for 30min. Images were acquired without zoom, in a 256×256 matrix. Flushing both eyes with saline after the study completion were done to help clear remaining radioactivity[6].
The same definitions and interpretation protocol were used by two observers. Visual classification was used to report both type (normal, partial and complete obstruction) and anatomical location (presac, preduct, intraduct) of LDS obstruction by two observers who were masked to each other’s report. Visualization of the lacrimal sac within 1-3min after radioactive administration with drainage of activity from the nasolacrimal duct into the nasal cavity within 5min was considered as “normal” (Figure 1). Delayed radiotracer washout from the lacrimal sac in the presence of some drainage of activity after the first 5min of the study was considered“partial obstruction” or “stenosis” (Figure 1); and the absence of notable radiotracer washout up to the end of the study was regarded as “complete obstruction” (Figure 1)[6,9]. The anatomical location of obstruction was defined as being either presac, preduct, or intraduct. Presac was a hold up of tracer at the inner canthus with failure or delay to reach the lacrimal sac in first 3min. Preduct was early filling of the lacrimal sac but no or delayed visibility of tracer beyond the sac on first static image at 5min and “intraduct” was visibility of tracer in the upper part of the nasolacrimal duct at 5min but no further drainage over the next 15min[15,19] (Figure 1).
Primary outcome measure was the inter-observer agreement between lacrimal surgeon and nuclear medicine specialist for interpreting LS in general and when considering the results of irrigation test. Secondary outcome measures were LS versus irrigation test agreement and intra-observer agreement for interpretation of LS by the lacrimal surgeon. Each LS was considered a case in this study.
Figure 1 Right intraduct stenosis with left pre-duct obstruction (A), right pre-duct stenosis with normal left side (B), and right pre-sac obstruction with normal left side (C) on LS.
Table 1 Comparing 100 LS reports by lacrimal surgeon versus nuclear medicine specialist
LS: Lacrimal scintigraphy.
Anatomical location of obstruction Lacrimal surgeon Nuclear medicine specialist Normal Presac Preduct Intraduct Normal Presac Preduct Intraduct Normal 53 - - - 60 - - -Partial obstruction - 1 9 9 - 1 2 -Complete obstruction - 3 24 1 - 2 33 2 Total 53% 4% 33% 10% 60% 3% 35% 2%Type of obstruction
All the data were analyzed with MedCalc Statistical Software version 13.3.3 (MedCalc Software bvba, Ostend, Belgium;http://www.medcalc.org; 2014). Weighted Kappa coefficient with 95% confidence interval (CI) was used to examine interand intra-observer agreements. Linear Cicchetti-Allison weights was used in order to measure the weighted Kappa and bootstrapped 95%CI was also used.
When interpreting Kappa values, strength of agreement was described based on Landis and Koch’s classification[20] which includes slight (≤0.20), fair (0.21-0.40), moderate (0.41-0.60),substantial (0.61-0.80), and almost perfect (0.81-1.00) agreement.A P value of less than 0.05 was considered significant.
RESULTS
There were 100 LS reports from 50 subjects[17].
Agreement Between Lacrimal Surgeon and Nuclear Medicine Specialist Lacrimal surgeon reported a significantly(P＜0.001) higher frequency of LDS obstruction (47% vs 40%)than nuclear medicine specialist. On the other hand, complete obstruction was reported more (37/40; 92.9%) by nuclear medicine specialist than lacrimal surgeon (28/47, 59.5%;P=0.01; Table 1).
Total proportion of agreement on the type of obstruction was 69% (69/100) comprising 46% normal, 1% partial stenosis,and 22% complete obstruction (Table 2). A significant and moderate agreement was demonstrated between lacrimal surgeon and nuclear medicine specialist for the type of obstruction (Kappa=0.55; 95%CI: 0.41-0.7; P＜0.001).
Total proportion of agreement on the location of obstruction was 69% (69/100) including 46% normal and 23% preduct location (Table 3). A significantly moderate agreement was demonstrated between lacrimal surgeon and nuclear medicine specialist for anatomical location of obstruction (Kappa=0.48;95%CI: 0.32-0.62; P＜0.001).
Inter-observer Agreement Considering the Irrigation Test Results In subgroup of the cases with normal irrigation(n=65), agreement values were substantial and moderate for the type (Kappa=0.61; 95%CI: 0.45-0.78; P＜0.001) and anatomical location (Kappa=0.56; 95%CI: 0.38-0.75; P＜0.001)of obstruction, respectively. Whereas, in cases with abnormal irrigation (n=35), moderate and fair[19] agreement values were found between lacrimal surgeon and nuclear medicine specialist for both the type (Kappa=0.41;95%CI: 0.16-0.68;P=0.002) and anatomical location (Kappa=0.31; 95%CI:0.05-0.58; P=0.01) of obstruction, respectively.
Intra-observer Repeatability on Reporting Lacrimal Scintigraphy Analysis of lacrimal surgeon’s first and second LS reports showed a strong intra-observer repeatability values for both the type (Kappa=0.66; 95%CI: 0.52-0.8, P＜0.001) and anatomical location (Kappa=0.69; 95%CI: 0.57-0.8, P＜0.001)of obstruction.
Lacrimal Scintigraphy Report Versus Irrigation Test Reliability Reliability proportion of lacrimal surgeon reports on the type of obstruction (considering the irrigation test) was 51% (51/100) including 39% normal, 8% partial obstruction,and 4% complete obstruction (Table 4) which was significant but low (Kappa=0.19; 95%CI: 0.04-0.33; P=0.002).
Table 2 Agreement of lacrimal surgeon and nuclear medicine on the type of obstruction in lacrimal scintigraphy
Complete obstruction Lacrimal surgeon Normal 46 1 6 Partial obstruction 9 1 9 Complete obstruction 5 1 22 Type of obstruction Nuclear medicine specialist Normal Partial obstruction
Table 3 Agreement of lacrimal surgeon and nuclear medicine on the anatomical location of obstruction in lacrimal scintigraphy
?
However, reliability proportion of lacrimal surgeon reports on the location of obstruction (considering the irrigation test)was 42% (42/100) including 39% normal and 3% presac obstruction (Table 5) which was not significant (Kappa=0.08;95% CI: -0.03-0.2; P=0.09).
On the other hand, irrigation test results and LS reports by nuclear medicine specialist did not show a significant agreement for both the type (Kappa=0.1; 95%CI: -0.02-0.2;P=0.05) and anatomical location (Kappa=0.07; 95%CI: -0.05-0.2;P=0.13) of obstruction.
DISCUSSION
LS is a simple and low-radiation dose method for assessment of LDS which gives a morphological documentation of tear flow[6,21-22]. Interpretation of LS could be visual (qualitative)or quantitative[6,23]. Visual classifications has been the most common type of classification used in previous studies[24]and clinical practice. LS usually demonstrate the sac and duct obstructions better than canalicular obstructions[24].Wearne et al[19] proposed that many cases with presac retention may have a primarily canalicular problem,whereas those with preduct or intraduct delay might have an abnormally functioning nasolacrimal duct. However, there is not an agreement on normative data[15,24]. More than 40%of asymptomatic individuals might have abnormalities and variability in tracer times of LS[14]. Sagili et al[15] believed that interpretation of LS is more an art than pure science.
One study (examining 242 eyes) found poor correlation between LS and both patients’ symptoms and clinical examination[14]. Therefore, LS interpretation should beperformed alongside the clinical findings. On the other hand,Fard-Esfahani et al[13] found a moderate agreement between LS and clinical findings (Kappa=0.52) in the first visit, which increased to almost perfect agreement (Kappa=0.86) after one year follow up, implying that abnormal LS results in first visit may show subtle abnormalities in the LDS which could present clinically later on. Poor agreement values between irrigation results and LS reports in this study is in accordance with previous findings[9-10]. Such observations are the reasons why clinicians (including ophthalmologists) are mostly relying on irrigation test[25-26].
Table 4 Reliability of lacrimal surgeon’s report (lacrimal scintigraphy)on the type of lacrimal obstruction considering the irrigation test
Complete obstruction Irrigation test Normal 39 11 15 Partial obstruction 14 8 9 Complete obstruction 0 0 4 Type of obstruction Lacrimal Surgeon Normal Partial obstruction
Table 5 Reliability of lacrimal surgeon’s report (lacrimal scintigraphy) on the anatomical location of lacrimal obstruction considering the irrigation test
Type of obstruction Irrigation test Normal Presac Preduct Intraduct Lacrimal surgeon Normal 39 13 0 1 Presac 1 3 0 0 Preduct 20 10 0 3 Intraduct 5 5 0 0
In the field of medical imaging, literature is mostly focused on the accuracy of the imaging systems rather than accuracy of interpretation of the images or agreement between the reports by different specialties[27-28]. For example, a recently published review in diagnostic radiology showed that agreement studies are infrequently and sub-optimally reported in the radiology literature and are limited[27]. Since different specialties have different experiences and background knowledge[28-29],consensus image interpretation by different specialties could be considered as a solution. Despite its overall positive clinical implication, reaching to a consensus is not easy[27].
Clinicians (including ophthalmologists) who are involved with the management of epiphora sometimes request LS test,even though surveys have shown that they are mostly relying on irrigation test[25-26]. LS is a simple and low-radiation dose method for assessment of LDS which gives a morphological documentation of tear flow[6,21-22]. The poor agreement values observed between irrigation results and LS reports in this study is in accordance with previous findings[9-10] which may justify the results of previous surveys in this regard[25-26].
Results of this study did not show a strong inter-observer agreement between lacrimal surgeon and nuclear medicine specialists especially in cases with abnormal irrigation test and for the location of obstruction. This observation may imply that LS of the patients with abnormal irrigation should be rereported by a lacrimal surgeon before final decision making.In the field of medical imaging, literature is mostly focused on the accuracy of the imaging systems rather than accuracy of interpretation of the images or agreement between the reports by different specialties[27-28]. For example, a recently published review in diagnostic radiology showed that agreement studies are infrequently and sub-optimally reported in the radiology literature and are limited[27]. Since different specialties have different experiences and background knowledge[28-29],consensus image interpretation by different specialties could be considered as a solution. Despite its overall positive clinical implication, reaching to a consensus is not easy[27].We suggest constituting consensus groups among nuclear medicine specialists and lacrimal surgeons in order to create a common language between them (including guidelines) for interpretation of LS. Similar guidelines have been published for interpretation of orbital imaging like CT scan and MRI[30].There are important questions to be addressed in such a consensus group, like “How often did the two specialists agree about reporting and how often did they disagree in findings?”,“Was their disagreement systematic or a random one?”, “How much the observed disagreement was influenced by experience or other factors?”[28]. Despite its overall positive clinical implication, reaching to a consensus is not easy[27].
On limitation of this study was not assessing the intraobserver repeatability for nuclear medicine specialist who was not available for such an aim. This assessment could show consistency of interpretation results in comparison to the lacrimal surgeon’s report. Furthermore, a larger sample size would increase the statistical power. We are aware of that the clinical implication of LS in the anophthalmic socket is basically different from the seeing eye which was not the aim of this study.
In conclusion, this is the first study on inter-observer agreement and intra-observer repeatability of LS in which a moderate agreement was found between lacrimal surgeon and nuclear medicine specialist. It was substantially decreased in cases with abnormal irrigation test and for the location of obstruction. LS showed no to slight reliability based on irrigation test. Strong level of intra-observer repeatability was found for the lacrimal surgeon.
Based on the results, reports of LS should not be a basis for management of LDS obstruction. A consensus group of nuclear medicine specialists and lacrimal surgeons are recommended for delivering a uniform guideline for interpretation of LS.
ACKNOWLEDGEMENTS
Authors’ contributions: Kashkouli MB: Design of study,manuscript preparation; Abolfathzadeh N: Execution;Abdolalizadeh P: Data collection; Karimi N: Execution;Hedayati R: Execution; Jafari S: Manuscript preparation;Alemzadeh A: Data collection.
Conflicts of Interest: Kashkouli MB, None; Abolfathzadeh N, None; Abdolalizadeh P, None; Karimi N, None; Hedayati R, None; Jafari S, None; Alemzadeh A, None.
1 Linberg JV, McCormick SA. Primary acquired nasolacrimal duct obstruction. A clinicopathologic report and biopsy technique.Ophthalmology 1986;93(8):1055-1063.
2 Kashkouli MB, Sadeghipour A, Kaghazkanani R, Bayat A, Pakdel F, Aghai GH. Pathogenesis of primary acquired nasolacrimal duct obstruction. Orbit 2010;29(1):11-15.
3 Kashkouli MB, Mirzajani H, Jamshidian-Tehrani M, Pakdel F, Nojomi M, Aghaei GH. Reliability of fluorescein dye disappearance test in assessment of adults with nasolacrimal duct obstruction. Ophthalmic Plast Reconstr Surg 2013;29(3):167-169.
4 Kashkouli MB, Kempster RC, Galloway GD, Beigi B. Monocanalicular versus bicanalicular silicone intubation for nasolacrimal duct stenosis in adults. Ophthalmic Plast Reconstr Surg 2005;21(2):142-147.
5 Francisco FC, Carvalho AC, Francisco VF, Francisco MC, Neto GT.Evaluation of 1000 lacrimal ducts by dacryocystography. Br J Ophthalmol 2007;91(1):43-46.
6 Al-ghamdi AA, El-saban KH. Clinical values of the lacrimal scintigraphic parameters in diagnosis of epiphora: comparison to dacryocystography.Med J Cairo Univ 2011;79(3):13-22.
7 Kashkouli MB, Mirzajani H, Jamshidian-Tehrani M, Shahrzad S, Sanjari MS. Fluorescein dye disappearance test: a reliable test in assessment of success after dacryocystorhinostomy procedure. Ophthalmic Plast Reconstr Surg 2015;31(4):296-299.
8 Amin M, Moseley IF, Rose GE. The value of intubation dacryocystography after dacryocystorhinostomy. Br J Radiol 2000;73(870):604-607.
9 Jabbour J, Van der Wall H, Katelaris L, Leslie J, Mackey D, Ghabrial R. Quantitative lacrimal scintigraphy in the assessment of epiphora. Clin Nucl Med 2008;33(8):535-541.
10 Jager PL, Mansour K, Vrakkink-de Zoete H, Poot L, Hooijmans JM,Bruin KJ, Blanksma LJ. Clinical value of dacryoscintigraphy using a simplified analysis. Graefes Arch Clin Exp Ophthalmol 2005;243(11):1134-1140.
11 Kashkouli MB, Pakdel F, Kiavash V. Assessment and management of proximal and incomplete symptomatic obstruction of the lacrimal drainage system. Middle East Afr J Ophthalmol 2012;19(1):60-69.
12 Shams PN, Chen PG, Wormald PJ, Sloan B, Wilcsek G, McNab A, Selva D. Management of functional epiphora in patients with an anatomically patent dacryocystorhinostomy. JAMA Ophthalmol 2014;132(9):1127-1132.
13 Fard-Esfahani A, Gholamrezanezhad A, Mirpour S, Tari AS, Saghari M, Beiki D, Sichani BF, Eftekhari M. Assessment of the accuracy of lacrimal scintigraphy based on a prospective analysis of patients’symptomatology. Orbit 2008;27(4):237-241.
14 Vonica OA, Obi E, Sipkova Z, Soare C, Pearson AR. The value of lacrimal scintillography in the assessment of patients with epiphora. Eye(Lond) 2017;31(7):1020-1026.
15 Sagili S, Selva D, Malhotra R. Lacrimal scintigraphy: “interpretation more art than science”. Orbit 2012;31(2):77-85.
16 Kottner J, Audigé L, Brorson S, Donner A, Gajewski BJ, Hróbjartsson A, Roberts C, Shoukri M, Streiner DL. Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. J Clin Epidemiol 2011;64(1):96-106.
17 Kashkouli MB, Zolfaghari R, Es’haghi A, Amirsardari A, Abtahi MB, Karimi N, Alemzadeh A, Aghamirsalim M. Tear film, lacrimal drainage system, and eyelid findings in subjects with anophthalmic socket discharge. Am J Ophthalmol 2016;165:33-38.
18 Kashkouli MB, Rezaee R, Nilforoushan N, Salimi S, Foroutan A,Naseripour M. Topical antiglaucoma medications and lacrimal drainage system obstruction. Ophthalmic Plastic & Reconstructive Surgery 2008;24(3):172-175.
19 Wearne MJ, Pitts J, Frank J, Rose GE. Comparison of dacryocystography and lacrimal scintigraphy in the diagnosis of functional nasolacrimal duct obstruction. Br J Ophthalmol 1999;83(9):1032-1035.
20 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159-174.
21 Reddy SC, Zakaria A, Bhavaraju VM. Evaluation of lacrimal drainage system by radionuclide dacryoscintigraphy in patients with epiphora. Iran J Nucl Med 2016;24(3):99-106.
22 Rossomondo RM, Carlton WH, Trueblood JH, Thomas RP. A new method of evaluating lacrimal drainage. Arch Ophthalmol 1972;88(5):523-525.
23 Peter NM, Pearson AR. External dacryocystorhinostomy for the treatment of epiphora in patients with patent but non-functioning lacrimal systems. Br J Ophthalmol 2010;94(2):233-235.
24 Dutton JJ, White JJ. Diagnosis, imaging and clinical evaluation of the lacrimal drainage system. In: Cohen AJ, Mercandetti M, Brazzo BG,editors. The lacrimal system. Diagnosis, management, and surgery. New York: Springer; 2006:74-99.
25 Cuthbertson FM, Webber S. Assessment of functional nasolacrimal duct obstruction: a survey of ophthalmologists in the southwest. Eye(Lond) 2004;18(1):20-23.
26 Nagi KS, Meyer DR. Utilization patterns for diagnostic imaging in the evaluation of epiphora due to lacrimal obstruction: a national survey.Ophthalmic Plast Reconstr Surg 2010;26(3):168-171.
27 Farzin B, Gentric JC, Pham M, Tremblay-Paquet S, Brosseau L, Roy C, Jamali S, Chagnon M, Darsaut TE, Guilbert F, Naggara O, Raymond J. Agreement studies in radiology research. Diagn Interv Imaging 2017;98(3):227-233.
28 Bankier AA, Levine D, Halpern EF, Kressel HY. Consensus interpretation in imaging research: is there a better way? Radiology 2010;257(1):14-17.
29 Hillman BJ. ACRIN: lessons learned in conducting multi-center trials of imaging and cancer. Cancer Imaging 2005;5(Spec No A):S97-101.
30 Ben Simon GJ, Annunziata CC, Fink J, Villablanca P, McCann JD,Goldberg RA. Rethinking orbital imaging establishing guidelines for interpreting orbital imaging studies and evaluating their predictive value in patients with orbital tumors. Ophthalmology 2005;112(12):2196-2207.