Journal Name: Journal of Biomedical Research and Reviews
Article Type: Rapid Communication
Received date: 30-April-2020
Accepted date: 28-May-2020
Published date: 04-June-2020
Citation: Angel de Soto J (2020) Medical Basis for Increased Susceptibility of COVID-19 among the Navajo and other Indigenous Tribes: A Survey. J Biomed Res Rev Vol: 3, Issu: 1.(37-41).
Copyright: © 2020 Angel de Soto J. This is an openaccess article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Abstract
Introduction:The SARs-CoV-2 virus was initially reported in Dec 2019 as the causative agent of a pneumonia breakout in Wuhan China. This virus rapidly spread from China to Europe and the East Coast of the United States eventually reaching the South West United States and indigenous tribes in mid -March. Since, then the indigenous tribes have been devasted by the virus which the Governor of New Mexico has likened as an existential threat.
Methodology: An initial PubMed search was performed utilizing specific Boolean operators and words. Three thousand one hundred and nineteen papers were elicited. Articles that had similar information were eliminated with more recent paper having preference. This left 472 papers. Each paper was then evaluated by three content experts with the most relevant and scientifically sound 50 kept for this survey.
Results: The literature suggests that the Navajo’s and other indigenous peoples may have elevated levels of ACE2 receptors in their lungs and other tissues allowing greater susceptibility to the COVID-19 virus. Increased levels of diabetes and protein nutrition are directly related to increased morbidity and mortality in this group while obesity, COPD, and heart disease are not. The increased morbidity and mortality is exasperated by an inability to test or treat for SARs-CoV-2
Conclusion: The infectivity rate of Navajo’s on the reservation is the highest in the Nation at 5.25 times higher than the national average and with a death rate at 3.35%. Co-morbidities account for some of the increased morbidity and mortality while lack of access to adequate health care unnecessarily magnifies the poor outcome. The threat to indigenous tribes in the Southwest due to COVID-19 is dire.
Keywords
COVID-19, Indigenous Tribes, Navajo, Medical Presentation, Medical Resources, Comorbidity.
Abstract
Introduction: The SARs-CoV-2 virus was initially reported in Dec 2019 as the causative agent of a pneumonia breakout in Wuhan China. This virus rapidly spread from China to Europe and the East Coast of the United States eventually reaching the South West United States and indigenous tribes in mid -March. Since, then the indigenous tribes have been devasted by the virus which the Governor of New Mexico has likened as an existential threat.
Methodology: An initial PubMed search was performed utilizing specific Boolean operators and words. Three thousand one hundred and nineteen papers were elicited. Articles that had similar information were eliminated with more recent paper having preference. This left 472 papers. Each paper was then evaluated by three content experts with the most relevant and scientifically sound 50 kept for this survey.
Results: The literature suggests that the Navajo’s and other indigenous peoples may have elevated levels of ACE2 receptors in their lungs and other tissues allowing greater susceptibility to the COVID-19 virus. Increased levels of diabetes and protein nutrition are directly related to increased morbidity and mortality in this group while obesity, COPD, and heart disease are not. The increased morbidity and mortality is exasperated by an inability to test or treat for SARs-CoV-2
Conclusion: The infectivity rate of Navajo’s on the reservation is the highest in the Nation at 5.25 times higher than the national average and with a death rate at 3.35%. Co-morbidities account for some of the increased morbidity and mortality while lack of access to adequate health care unnecessarily magnifies the poor outcome. The threat to indigenous tribes in the Southwest due to COVID-19 is dire.
Keywords
COVID-19, Indigenous Tribes, Navajo, Medical Presentation, Medical Resources, Comorbidity.
Introduction
‘In late March, New Mexico Gov. Michelle Lujan Grisham stated that the SARS-CoV-2 virus could ‘wipe out’ native 21 tribes [1]’. In Dec 2019, a pneumonia appeared of unknown etiology appeared in the Huanan Seafood Wholesale Market adjacent to the Wuhan Institute of Virology [2]. The virus rapidly spread throughout China and to Europe which then spread to the east coast of the United States. The new virus was found to be a corona virus with a close genetic sequence to the SARS virus [3]. The corona virus is normally an innocuous cold virus that spreads during the winter months [4]. Yet this virus has turned out to be much more readily passed than either the SARs virus that cause an epidemic in 2003 and the related MERs virus responsible for the epidemic that began in 2012.
The SARS-CoV-2 virus has a S protein that acts as a key to enter the cells of the lungs through an ACE2 receptor that acts as the lock [5]. Once infected the lungs often show infection of all lobes appearing as ground glass on X-ray [6]. Additional, evidence indicate that the virus may infect the nervous system and cause liver damage which can both exasperate the pneumonia [7]. Transmission of the virus is enhanced dramatically by the fact that those showing no symptoms can pass it and that the virus can survive on doorknobs, tables, and packages for up to three days [8,9]. The most important clinical symptoms initially reported are fever 88.7%, cough 67.8% and difficulty breathing 18.7% [10]. Children may present with the additional symptom of diarrhea [11]. Cases of sudden death have also been reported and may be due to increased blood clotting that increases stroke and heart attack [12]. On the Navajo Nation, the SARS-CoV-2 virus has spread rapidly and has been particularly deadly. Aggravating dire situation, the residents are in and by extension the neighboring tribes in the South West with 38% of the population in poverty, 19% suffer in extreme poverty, 32% without electricity, 31% do not have indoor plumbing, and 38% lack running water [13]. This is a national disgrace.
Here we look at the medical bases for the devastation the COVID-19 has exacted on the Navajo Nation and use this as a surrogate to the related indigenous tribes in the Southwest.
Methodology
A PubMed search was performed using the following Boolean operators and words: Wuhan Virus AND epidemiology (0), SARs-CoV-19 AND epidemiology (3), Navajo Nation AND COVID-19 (0), Indigenous Tribes AND COVID-19 (4), COVID-19 AND Pathophysiology (271) , SARS CoV-2 AND Pathophysiology (200), COVID-19 AND Clinical Presentation (978), ACE-2 AND COVID-19(48), Navajo AND Morbidities (366), Indigenous AND Physical Morbidities (1,149). Three thousand one hundred and nineteen papers were elicited. Articles that had similar information were eliminated with more recent papers having preference. This left 472 papers. Each paper was then evaluated by three content experts with the most relevant and scientifically sound 50 kept for this survey and 40 eventually used. Agreement on content by all experts was the threshold for including the information in this review.
Epidemiology
Infection and hospitalization by the SARS-CoV-2 virus seem to be linearly increased as one ages until about 65 years
Table 1: Comparative Morbidity and Mortality Rates in Southwest.
Location | Population | COVID Positive Cases | Cases per 1000 | Death Rate of those Infected |
---|---|---|---|---|
Navajo Reservation | 175,000 | 4,689 | 26.8 | 3.35% |
Benalillo County, NM (Albuquerque) | 678,701 | 1,362 | 2.00 | 4.99% |
Denver, CO | 600,000 | 5,269 | 8.78 | 5.71% |
Maricopa County AZ (Phoenix) | 3,800,000 | 8,448 | 0.91 | 4.54% |
El Paso, Tx | 649,133 | 2,393 | 1.24 | 2.51% |
United States | 329,423,917 | 1,681,418 | 5.10 | 5.88% |
* COVID Data Taken from: Coronavirus COVID-19 Global Cases by the Center for Systems Science and Engineering, Johns Hopkins University (Accessed May 27, 2020).
of age. At the age of 65, the increase becomes exponential. These observations correlate the size of the thymus gland which sits on top of the heart (Hakim, 2020). The thymus is essential in the body’s immunity to infectious disease. In the United States the average infection rate is 2.3 per 1000 individual while among the Navajo on the reservation the rate is 9.81 per 1000 individuals which is 22 times higher than the national average! See table 1. The Navajo reservation is one of the most rural areas in the country and one would expect the infection rate to be much lower than other areas of the country as density of any population increases transmission rate. Yet, the infection rate on the Navajo Nation is much higher than the local population clusters of Denver, Albuquerque, El Paso and Phoenix. Even more frightening is the death rate of 3.4% of those infected with SARs-CoV-19.
Among the factors identified that may account for increased mortality and morbidity are obesity, hypertension, diabetes, COPD/Asthma, and cardiovascular disease [10]. In a large meta-analysis it was found that hypertension was a comorbidity in 17.7%, diabetes in 8.6%, cardiovascular disease 5%, and respiratory disease 2.0% [14]. Native Americans may have some comorbidities to COVID-19 higher than the general population. Yet, many of these potential comorbidities are interrelated with each other confounding analysis.
Greater Susceptibility of Indigenous Indians
Genetic factors
The native American tribes began their migration to the New World about 15,000-25,000 years ago from Siberia and are genetically Asian overall and related to tribes in the Altaic region [15]. Analysis of the ACE2 receptor have shown no difference in the binding points of this receptor to the SARS-CoV-2 virus between ethnic groups [16]. There does seem however, to be differences in expression level with some evidence suggesting that Asian have a higher level of ACE-2 expression in their lungs than other groups [17]. Additionally, men have shown higher expressed levels of ACE-2. Though whether increased expression of ACE-2 is a genetic or epigenetic phenomenon this would lead to greater infectivity rates among the indigenous population. Whether this would increase mortality rates is unknown.
Obesity
Native Americans are slightly less likely to be overweight than Caucasians 33.1 to 33.9%.but much more likely to be obese than Caucasians 48.1 to 30% (USHHS). In a paper, by Dietz et al. Obesity was claimed to be a comorbidity for pneumonia, but no evidence was shown [18]. The argument seems to rely on a Western prejudice towards those that are overweight. Indeed, the BMI in the United States is based on New England upper middle- class body phenotypes and stereotypes of what is normal. It has actually been shown that those who are quote normal on the lower end by the current BMI may actually die sooner than those who are overweight [19]. The evidence suggests that for both pneumonia bacterial and viral pneumonia obesity is not associated with increased mortality [20,21]. Evidence suggests that obesity increases survival rate from pneumonia [22]. Again, in the large metaanalysis by Yang obesity was not found to be a comorbidity. The prevalence of obesity, prejudice against the obese, and obesity often being a comorbidity with hypertension and diabetes have falsely given some the impression that obesity is a risk factor for COVID-19. The obese tend to shed virus for a longer time however and might have to be isolated for a longer period of time [23].
Diabetes
Diabetes may be a risk factor for COVID-19. Twenty three percent of American Indians suffer from diabetes compared to eight percent of the Caucasian population a nearly threefold difference [24]. Diabetics when infected with any disease are known to decompensate leading to increasing levels of medication. This is further aggravated by corona viruses which further damages the insulin producing cells of the Pancreas [25]. Furthermore, like most with chronic disease diabetics are generally immunosuppressed and much more susceptible to infectious disease especially pneumonia [26]. The evidence suggests that the clinical course for those with COVID-19 is much worse than those without it [6]. The role of diabetes seems to be more damaging in causing a poor outcome with COVID-19 than raising the risk of initial infection [27].
Heart Disease/Hypertension
Heart disease and hypertension have both been associated with COVID-19 presentation. Among the Native American Indians their rate of heart disease is 8.6% compared to 5.8% of the population. However, their death rate from heart disease is about 10% lower than the population even though the rates of having the disease are higher. [28]. Heart disease is a significant factor in surviving any type of pneumonia as it suppresses the immune system as most chronic diseases do. Additionally, the increased work the heart must do to overcome increase lung pressure in pneumonia and to help provide adequate oxygen to a person with a failing lung can prove insufficient or cause the heart to fail. It is aggravated by evidence that the SARS-CoV-2 virus can directly damage the blood vessels feeding the heart directly perhaps causing sudden death [29].
Hypertension has also been associated with COVID-19 not surprisingly as it is closely associated with heart function and disease. American Indians are only slightly more likely to have hypertension than the population as a whole. Thus, though heart disease and hypertension both appear to be substantial comorbidities in COVID-19 they do not seem to be at additional risk to the indigenous Indians than the rest of the population as a whole.
COPD
It would be expected that COPD a disease of the lungs would be a special risk factor for COVID-19 morbidity and mortality. However, the association has been relatively weak or non- existent. The percentage of those with COPD and COVID -19 are reported to be between 1 and 2% [10,14]. This is much lower than what is found in the general population which is at 8-9% perhaps one can argue that COPD present little additional risk to COVID-19 [30]. Additionally, the rates of COPD among native American populations are only slightly higher than the population as a whole arguing that COPD is not a reason that native Americans are more susceptible to COVID-19 [31].
Poor Nutrition
The ability of the immune system to fight off disease is directly related to the nutritional status of the individual. Poor nutrition especially lack of protein severely inhibits the immune system and infection is generally the cause of death in those who are undergoing starvation and not starvation itself. Undernutrition is especially harmful to that part of the immune system called the adaptive immune system which fights off new viral and bacterial diseases introduced into a population [32]. Poor nutrition is not only related to progression or seriousness of a disease but increases the risk of becoming infected to begin with especially with pneumonia [33]. In the last half of the 20th century, a more sedentary lifestyle, coupled with poorer nutritional habits, has now replaced starvation as the major nutritional concern among the Navajo. Changes in mortality among the Navajo in recent years reflect these trends [34] Low availability and affordability of healthier foods in food stores on the Navajo Nation (NN) may also be a communitylevel risk factor for the high prevalence of obesity among the Navajo people [35], and although there is obesity, this does not imply adequate nutrition. Dietary intake of fruits and vegetables is low among the Navajo, and intake of fats is high [36] There is severe protein malnutrition that is aggravated by poverty that is especially found in the elderly who already are extremely susceptible to COVID-19 [37]. The evidence suggests that severe viral pneumonia is especially correlated with poor protein nutrition [32].
Limited Medical Resources
Availability of a health system that is efficient and effective in both the prevention, diagnosis and treatment of diseases and infections during public emergencies like this COVID-19 pandemic or in routine is a challenging task for all within tribal communities in general and Navajo communities in particular. This is the same challenge that many developing nations are now facing worldwide. With the emergence of SARS-COV-2 pandemic, where on one side healthcare infrastructure in developed nations have collapsed, the resource limited communities are hampered.
With a population of more than 250,000 inclusive of the Navajo who access the Navajo Health care system on personal protective equipment and of course trained work force. For example there are currently just 81 available ventilators across the IHS system nationwide, the agency said, emphasizing that “the core competency of IHS is primary care” and that regularly relies on a network of non- IHS facilities to provide specialized or intensive care [38- 41]. It is also reported that there are only 170 hospital beds, 13 ICU beds, 52 isolation rooms, and 28 ventilators on the entire Navajo Reservation - a reservation that encompasses 22,000 square miles. Another example particularly with reference to current pandemic is shortage/ unavailability of testing kits; current healthcare facilities remain unable to conduct tests of their own due to a lack of the necessary setup. Most of the healthcare facilities on reservation are providing general diagnosis at primary level. For all critical testing, the samples are sent off reservation to state laboratories, similarly all the specimen collected for SARSCOV- 2 diagnosis are being send to laboratories outside the reservation for evaluation, even the sample collection kits are not adequately available. These all are the factors that are adding to make this pandemic unbearable for Navajo communities.
Preventative Strategies
COVID-19 can spread by an infected person or an asymptomatic carrier, coughing and/or sneezing, as a result producing infected respiratory droplets. This is considered as person-to-person spread, most often by inhalation of infected droplets carrying virus. It also spreads by touching surfaces where infected droplets land, followed by touching your nose, and/or face, eyes, and mouth. Similarly, as with other respiratory infections like the common cold or flu, preventive measures are key to reduce the spread of these COVID-19 cases in our communities. Washing hands often with soap and water; covering mouth and nose with flexed elbow or tissue paper when coughing or sneezing (considering dispose of used tissue immediately), cleaning frequently touched surfaces and objects with appropriate disinfectant for example 5% household bleach is the simplest thing to use. It is highly recommended to stay home, avoid unnecessary travels and keep distance of at least 6 to 10 feet with other individuals in public places and gatherings.
Federal Aid in the form of increased medical personnel, hospital beds, testing facilities and infrastructure is needed to adequately assess and treat indigenous people in the Southwest. Increased economic development is needed also to ensure the ability of the indigenous peoples to afford protein rich food, have access to exercise facilities to ameliorate diabetes. Indeed, the ability to have running water and electricity are necessary in maintaining adequate hygiene and health.
Summary/Conclusion
The infectivity rate of Navajo’s on the reservation is 5.25 times higher than the national average and is the highest in the nation with a death rate near 3.35% Several potential causes of the increased infectivity, poor outcomes and death rates found among the Navajo have been evaluated. Increased Obesity, COPD among the Navajo do not seem to factor’s in the additional risk -factors for COVID-19 among the Navajo. While hypertension and heart disease do considerably increase the poor outcome of all groups to COVID-19 the Navajo are not at additional risk than the population as a whole. This survey suggests that the increased infectivity rates among the Navajo may in part be due to increased genetic/epigenetic elevation of ACE2 in this population. Increased, poor protein nutrition and diabetes seem to significantly raise the risk of poor outcome among the Navajo. This is exasperated by poverty, lack of adequate health care infrastructure, availability, technology and trained medical personnel. The response to the COVID-19 disaster among the Navajo and other southwest indigenous tribes has been a national disgrace.
Theisin L (2020) New Mexico Governor Warns Coronavirus Could ‘wipe out’ Native Tribes. New York Daily News. [ Ref ]
Li Q, Guan X., and Wu P, (2020) Early Transmission Dynamics in Wuhan, China, of Novel Coronavirus-infected Pneumonia. NEJM 382:1199-1207. [ Ref ]
Lu R, Zhao X, Li J (2020) Genomic Characterization and Epidemiology of 2019 Novel Coronavirus: Implications for Virus Origins and Receptor Binding. Lancet 395: 565-574. [ Ref ]
Carroll KC, Butel JS, Morse S (2015) Medical Microbiology 27 E. McGraw- Hill Education. [ Ref ]
Hoffman M, Kleine-Weber H, Schroeder S (2020) SARs- CoV-2 Cell Entry Depends on ACE-2 and TMPRSS2 and is Blocked by a Clinically Proven Protease Inhibitor. Cell 181: 271-280.e8. [ Ref ]
Zhou F, Yu T, Du R (2020) Clinical Course and Risk Factors for Mortality of Adult Inpatients with COVID-19 in Wuhan, China: A Retrospective Cohort Study. Lancet 395:1054-1062. [ Ref ]
Li YC, Bai WZ, Hashikawa T (2020) The Neuroinvasive Potential of SARSCoV2 May Play a Role in the Respiratory Failure of COVID-19 Patients. J Med Virol 2020: 1-4. [ Ref ]
Van Doremalen N, Bushmaker T, Morris DH (2020) Aerosol and Surface Stability of SARS-CoV-2 as Compared with SAR-CoV-1. NEJM 382:1564- 1567. [ Ref ]
- Rothe C, Schunk M, Bretzel G (2020) Transmission of 2019-nCoV Infection from an Asymptomatic Contact in Germany. NEJM 382:970-971.
Guan W., Ni Z, Liang W, et al. (2020). Clinical Characteristics of Coronavirus Disease 2019 in China. NEJM DOI: 10.1056/NEJMoa2002032. [ Ref ]
Lu X, Zhang L, Du H (2020) SARS-CoV-2 Infection in Children. NEJM 382:1663-1665. [ Ref ]
Zhang Y, Xiao M, Zhang S (2020) Coagulopathy and Antiphospholipid Antibodies in Patients with COVID-19. NEJM 382:e38. [ Ref ]
Degennaro V (2019) The Systemic Poverty in Navajo Nation is a National Travesty, KevinMD. [ Ref ]
Yang J, Zheng Ya, Gou Xi (2020) Prevalence of comorbidities in the novel Wuhan coronavirus (COVID-19) infection: A systematic review and meta-analysis. IJID 94:91-95. [ Ref ]
Pinotti T, Bergström A, Geppert M (2019) Y Chromosome Sequences Reveal a Short Beringian Standstill, Rapid Expansion, and early Population structure of Native American Founders. Curr Bio 29: 149- 157.e3. [ Ref ]
Cao Y, Li L, Feng Z (2020) Comparative Genetic Analysis of the Novel Coronavirus (2019-nCoV/SARS-CoV-2) Receptor ACE2 in Different Populations. Cell Discovery 6:11. [ Ref ]
Zhao D, Yao F, Wang L (2020) A comparative Study on the Clinical Features of COVID-19 Pneumonia to other Pneumonias. Clin Infect Dis. [ Ref ]
Dietz W, Redstone SM, and Santos-Burgoa (2020) Obesity and its Implications for COVID-19 Mortality. Obesity. [ Ref ]
Padwell R, Leslie WD, Lix LM and Majumdar SR (2016) Relationship Among Body Fat Percentage, Body Mass Index, and All-Cause Mortality: A Cohort Study. Ann Intern Med 164: 532-541. [ Ref ]
Braun ES, Crawford FW, Desai MM (2015) Obesity not associated with severity among hospitalized adults with seasonal influenza virus infection. Infection 43: 569-575. [ Ref ]
Kahlon S, Eurich DT, Padwal RS (2012) Obesity and Outcomes in Patient Hospitalized with Pneumonia. Clinical Microbiology and Infection 19: 709-716. [ Ref ]
Nie W, Zhang Y, Jee SH (2014) Obesity Survival Paradox in Pneumonia: A Meta -Analysis. BMC Medicine. [ Ref ]
Luzi L and Radealli (2020) Influenza and Obesity: its Odd Relationship and the Lessons of COVID-19 Pandemic. Acta Diabetologica. [ Ref ]
USHHS (2016) Diabetes and American Indians/Alaska Natives. Office of Minority Health. [ Ref ]
Yang JK, Lin SS, Ji XJ (2010) Binding of SARS Coronavirus to its Receptor Damages Islets and Causes Acute Diabetes. Acta Diabetol 47: 193-199. [ Ref ]
Jensen AV, Jepsen DF, Egelund GB (2017) Undiagnosed Diabetes Mellitus in Community-Acquired Pneumonia: A Prospective Cohort Study. Clin Infect Dis 65: 2091-2098. [ Ref ]
Fadini GP, Morieri ML, Longato E, Avogaro A (2020) Prevalence and Impact of Diabetes Among People Infected with SARs-CoV-2. Endocrinol Invest 43: 867-869. [ Ref ]
- USHHS (2020) Heart Disease and American Indians/Alaska Natives. Office of Minority Health.
Chen L, Li X, Chen M, Feng Y, Xiong C (2020) The ACE2 expression in human heart indicates new potential mechanism of heart injury among patients infected with SARS-CoV-2. Cardvasc Res 116: 1097-1100. [ Ref ]
Ntritsos G, Franek J, Belbasis L (2018) Gender-specific estimates of COPD prevalence: a systematic review and meta-analysis. Int J Chron Obstruct Pulmon Dis. 13:1507-1514. [ Ref ]
USHHS (2018) Obesity and American Indians/Alaska Natives. Office of Minority Health. [ Ref ]
Rajamanickam A, Munisankar S, Dolla CK, Babu S (2019) Undernutrition is Associated with Perturbations in T-Cell-, BCell- , Monocyte and Dendritic Cell- Subsets in Latent Mycobacterium Tuberculosis Infection. PLOS One 14: e0225611. [ Ref ]
Jonnalagadda S, Rodriquez O, Estrella B (2017) Etiology of Severe Pneumonia in Ecuadorian Children. PLOS One 12: e0171687. [ Ref ]
Byers Tim (1997) The Navajo Health and Nutrition Survey: Research That Can Make a Difference. The Journal of Nutrition 127: 2075S-2077S. [ Ref ]
Gayathari K (2015) Healthful Nutrition of Foods in Navajo Nation Stores: Availability and Pricing. Am J Health Promot 30: 501-510. [ Ref ]
Ballew C, White L, Strauss K, Benson L, Mendlein J, et al. (1997) Intake of nutrients and food sources of nutrients among the Navajo: findings from the Navajo Health and Nutrition Survey. J Nutr 127: 2085S-2093S. [ Ref ]
Williams R, Boyce T (1989) Protein Malnutrition in Elderly Navajo Patients. Journal of the American Geriatric Society 37: 397-406. [ Ref ]
https://www.politico.com/news/2020/03/28/native-americanscoronavirus- 152579 [ Ref ]
Hakim ST, Boyd FT, de Soto JA, (2020) The Pathophysiology of Virulence of the COVID-19 Virus. preprints. [ Ref ]
Zhao Y, Zhao Z, Wang Y (2020) Single- cell RNA expression profiling of the ACE2, the putative receptor of Wuhan 2019-nCov. Medrxiv. [ Ref ]
Zou Q, Zheng S, Wang X (2020) Influenza A- Associated Severe Pneumonia in Hospitalized Patients: Risk Factors and NA1 Treatments. IJID 92: 208-213. [ Ref ]