| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||||||||||
1 Huntsman Cancer Institute and 2 Human Development and Family Studies, University of Utah, Salt Lake City.
3 Centro Regional Universitario Bariloche, Universidad Nacional del Comahue, Argentina.
4 Department of Oncological Sciences, University of Utah, Salt Lake City.
Address correspondence to Ken R. Smith, PhD, Human Development and Family Studies, 225 South 1400 East, University of Utah, Salt Lake City, Utah 84112. E-mail: ken.smith{at}fcs.utah.edu
|
A |
|---|
|
|
|---|
It is clear that genetic factors affect adult survival probabilities, although the relative importance of environment and genes is still not fully determined. The opportunities for researchers to conduct studies that address this question have grown due to the increasing availability of survival data for families and multigenerational pedigrees. Data from large pedigrees contain valuable information with which to assess the role of genetics and environment in understanding variability in longevity. Accessible and flexible statistical tools needed to analyze these complex pedigrees still need to be fully developed and evaluated.
Historically, longevity data have been studied by calculating correlations and analysis of variance between uncensored life spans of siblings as well as between parents and offspring. When the data contain censored observations, the analysis of variance approach is not appropriate and instead survival methods need to be considered, such as the widely used Cox proportional hazards model. One of the basic assumptions in the Cox model is independence of survival times of individuals given the observed values of covariates. However, in studies involving multiple individuals from the same family, the independence assumption is not plausible unless all important familial factors were measured and controlled for in the model. It is now standard practice to adjust for these family-based sources of statistical dependencies in survival models through the introduction of a frailty component. In this usage, frailty refers to a susceptibility to death that is not captured by observed covariates. Typically, frailty includes factors that affect an individual's survival chances such as genes and unmeasured attributes of the environment, all of which may or may not be shared to some degree with others in a family or pedigree. The most common frailty specification is the "shared frailty" extension of the proportional hazards regression model. The way in which shared frailty has been used in these models has largely been to collapse all factors that are not measurable into a single random effect that is shared by individuals within the group (e.g., family). Guo (25) and Guo and Rodriguez (26) used survival techniques that incorporate a shared frailty component to study genetic and environmental influences on longevity among sets of siblings across numerous families. They interpreted the frailty component as the sum of unobserved shared factors that were likely to affect longevity, including genetic effects shared among siblings, shared genegene interactions, common influence of parental competence, and other shared household effects that were not captured by observed covariates in their study. Hougaard and colleagues (27) used several versions of the frailty model for bivariate survival to fit data for Danish monozygotic and dizygotic twins. In a shared frailty model, the frailties are unobserved random variables assumed to be independent and to follow a probability distribution, the shape of which is described by a few parameters. These models provide insight into the effects of familial risk on mortality, but have not been designed to incorporate complex genetic relationships found in family data of varying size and structure. In other words, the shared frailty model pools all effects, shared genetic and shared environmental, into a single random effect without being informed by the genetic relationships that link any two relatives.
In order to describe more complicated dependencies, Yashin and Iachine (28) and Yashin and colleagues (29) introduced a model of bivariate survival that allows for the incorporation of correlations between individual frailties. These models are known as "correlated frailty" models, which have been used with twin data to assess genetic and environmental factors influencing mortality. Ripatti and Palmgren (30) and Therneau and colleagues (31) extended correlated frailty models to survival data on n individuals, rather than on two individuals (as is the case in twin studies). Unlike the shared frailty model, in which frailties are assumed to be independent, in the correlated frailty model proposed by Ripatti and Palmgren (30) and Therneau and colleagues (31), one individual's frailty is associated (but not necessarily perfectly) with the frailty of another individual who is from the same family of origin (related genetically) or family of procreation (related by marriage). In this case, frailties are assumed to be random variables drawn from a multivariate normal distribution with an arbitrary covariance structure.
Pankratz and colleagues (32) recently applied the correlated frailty model to investigate the aggregation of breast cancer within families. They considered a correlated frailty model with one random effect per individual, with the random effects correlated to reflect the degree of genetic relationship between individuals. They interpreted the variance of this model as a polygenic (involving two or more genes) variance component and concluded that there is significant heritability of age-at-onset of breast cancer. This model assumed that family risk depends only on genetic relationships. Pankratz and colleagues (32) also explored the possibility of a shared family environment role in breast cancer by fitting a model with both polygenic and shared family random effects. They provided evidence for polygenic variance components and suggestive evidence for shared family environmental variance components.
The purpose of this article is to report how the correlated frailty model constitutes a valuable data analysis tool to incorporate both unobserved genetic and environmental influences in human longevity that allows for the inclusion of observable covariates. The utility of a correlated frailty model that incorporates both genetic and environmental sources of frailty is described here and applied to data from an ongoing family-based study of longevity called the Fertility, Longevity, and Aging (FLAG) study. We also examine, via simulations, how ignoring the dependencies among observations affects the estimation of the regression parameters and their standard errors. The problem of model misspecification related to ignoring the true dependencies in the data has been studied by Wei and colleagues (33) in the context of recurrent event data, and by Guo (25) in the case of shared frailty models. Our results point to the importance of using correlated frailty models to analyze complex pedigree data that allow for both genetic and environmental contributions to frailty.
| METHODS |
|---|
|
|
|---|
|
|
0 is an unspecified baseline hazard function, b = (b1, ..., bq) is a vector of random effects that represent family-specific frailties, assumed independent, Xi is a vector of measured covariates, ß is a vector of unknown regression coefficients, and Zi is a vector with Zij = 1 if individual i is a member of group j, and 0 otherwise. The distribution of bj is specified a priori. Distributions of bj commonly considered are the log gamma and the normal distribution for their computational advantages. The shared frailty model is attractive because it explicitly acknowledges the potential role of unobserved factors affecting an individual's risk of mortality. It does, however, assume that unobservable characteristics are perfectly shared with others in the family and that unobserved factors that are not shared are not considered.
Correlated Frailty Model
The correlated frailty model differs from the shared frailty model in that it allows for individual-level frailties that can be correlated across individuals within the family. We describe the correlated frailty model that incorporates unmeasured random effects into the analysis of censored survival data applied to a family-based sample of size n. Let Xi and Zi be vectors of known covariates, and b = (b1, ..., bq) a vector of frailties. Given b, the event times are assumed independent with the conditional hazard function for individual i following the proportional hazards specification
|
|
0 is an unspecified baseline hazard function and ß is a vector of unknown regression coefficients. It is assumed that the distribution of the vector of frailties belongs to a known family of distributions with mean 0 and covariance matrix D(
), with
denoting a vector of unknown parameters.
We assume that b follows a multivariate normal distribution with mean 0 and covariance matrix D(
). This assumption makes it computationally convenient to impose the desired covariance structure for the frailty distribution. Ripatti and Palmgren (30) give an approximate likelihood for this model and propose a computational approach for calculating the approximate maximum likelihood estimates of ß and
. Therneau and colleagues (31) implemented the approximate maximum likelihood estimation approach as R software, in the Kinship package (this software can be downloaded from http://cran.r-project.org/). This model includes as a special case the shared frailty model with normal and log normal distributions.
An important property of the correlated frailty model is that it allows for the simultaneous incorporation of unobserved genetic and environmental frailty parameters in the analysis of life-span data. As a result, genetic and environmental components of frailty can be jointly estimated from the data.
We assume that the conditional hazard of death for individuals i,
i(t | b), follows the model given in Equation 2. To evaluate the separate influences of unobserved genetic and environmental contributions to life span, we decompose the covariance matrix D(
) into two components:
|
|
and
represent the shared environmental influences and the shared polygenic effect, respectively, and
= (
f,
p). The matrix
f is a fixed matrix that incorporates the degree of shared environment among individuals, whereas
p captures the shared polygenic factors between genetically related family members.
In our analyses, the elements of
p represent the degree of genetic resemblance that two individuals would be expected to have by chance. The
p matrix is a function of K = (Kij), a matrix of kinship coefficients. The kinship matrix has elements Kij, the kinship coefficients, that measure the probability that a gene selected randomly from individual i and another selected from individual j will be identical by descent (ibd) at a given locus. Thus, if we assume no consanguinity in previous generations, the kinship coefficient is 0.25 for first-degree relatives (parent/child and sibling pairs), 0.125 for second-degree relatives (aunts/uncles paired with nieces/nephews), and 0.0625 for third-degree relatives (first cousins), etc. Kinship coefficients represent half the expected proportion of the genome that is shared by pairs of relatives. To generate
p, the kinship coefficient for the diagonal element is first set to 0.5 and
p = 2K. Hence,
p has values of 1 on the diagonal, 0.5 for parent/child and sibling pairs, 0.25 for grandparent/grandchild, uncle/niece, and so forth.
The matrix
f = (aij) with aij = 1 if individuals i and j belong to the same family and 0 otherwise. By "family," we mean the group of all individuals that are related genetically or by marriage. Clearly, the definition of family in this respect can be made more flexible for a given application. Note that, in the case where two individuals are from different families, the elements of their
p (shared polygenic effects) and
f (shared environmental effects) matrices will both be 0. Our choice of
f in this analysis restricts all individuals in the family to have the same family-specific effect. Other correlation structures that incorporate alternative environmental relationships between individuals in the data set could be accommodated with this model.
Data: Exceptional Longevity in Families in the FLAG Study
Cox proportional hazards models with correlated frailty are applied to data that were generated as part of the ongoing FLAG project. The larger study selects individuals from long-lived (LL) pedigrees who have survived to extreme ages to identify genetic and environmental factors that have allowed them to experience exceptional survival and to determine whether they have fewer disabilities and lower rates of age-related diseases over the course of their lives. The overall objective of the FLAG study is to measure aging-related demographic, epidemiologic, social, cognitive, physiological, and molecular traits in these exceptional individuals to identify a "delayed aging phenotype" and to isolate genetic markers associated with slower rates of aging.
The Utah Population Database (UPDB) is used to identify the families that participate in the study. The UPDB contains over 8 million records, including the genealogies of the founders of Utah and their descendants. In the 1970s, approximately 170,000 Utah nuclear families were identified on "Family Group Sheets" from the archives at the Utah Family History Library, each with at least one member having had a vital event (birth, marriage, death) on the Mormon Pioneer Trail or in Utah. These families have been linked across generations; in some instances, the records span seven generations. The UPDB is an active genealogy; new families and their members are continually being added as the UPDB is linked to other sources of data, including birth and death certificates. Additional information on these individuals comes from sources such as driver's license and the Utah Cancer Registry. The UPDB now holds data from migrants to Utah and their descendants that include more than 1.8 million individuals born from the early 1800s to the mid-1900s and that are linked into multigeneration pedigrees.
We used familial excess longevity (FEL) as a genealogically based method for identifying LL pedigrees. FEL is a summary measure of excess longevity (EL) among all blood relations for a given individual (34). Calculating FEL first requires an estimate of the difference between an individual's attained age and the age to which that individual was expected to live according to an accelerated failure time model. This model uses three covariates that are available in the UPDB and that are associated with longevity: gender, birth year, and affiliation with the Church of Jesus Christ of Latter-day Saints (or LDS Church or Mormons). The variable describing whether a person in the UPDB is a member of the LDS Church is included because it is known that active members of the LDS Church have significantly longer lives (35).
The expected age at death is:
|
|
, where y is the observed age at death or the age at the time last confirmed alive, in years. The FEL for an individual is calculated as the weighted average of individual EL of all blood relatives aged 65 years old or older. For a given individual, "blood relatives" refers to all persons aged 65 years old or older in the database with whom the individual has a genetic relationship, regardless of their birth year and degree of genetic relationship. The number of relatives used to generate any one individual's FEL ranged from 62 to 105,893 with a median of 1987. The weights are the kinship coefficients. The FEL for individual i is
|
|
In this study, we tested whether there is evidence that genetic and environmental factors are important determinants of age at death based on 146 kindreds of founders in the UPDB with very high FEL (the LL families) and 179 matched control kindreds. Based on the entire UPDB, LL families were identified as the top families (identified by their founder) (listed in terms of increasing p value) with a statistically significant (p <.01) excess of descendants surviving past the 97th percentile of EL or were members of the top founders most often selected by a Monte Carlo selection procedure. For the Monte Carlo procedure, variation was introduced by sampling from sibships in the data with replacement. During each iteration of the Monte Carlo procedure, the most likely LL founders were selected based on the EL of the sampled sibships. A key feature of this procedure was that each sibship was allowed to contribute to at most one of the selected founders in each iteration. Both methods were used in an effort to identify LL families that did not rely on a single selection method. The matched kindreds had founders with normative FEL values with comparable numbers of descendants as those among the exceptional FEL founders. Analyses were conducted excluding the founders themselves to eliminate the bias associated by including the exceptional founders (as they are, by definition, LL). Although all founders examined here generally have many descendants spanning up to 10 subsequent generations in the UPDB, only their children and grandchildren were used in the analysis. This was done to allow for a wider range of relationships than is done with twin studies (siblings, parents and offspring, uncle/aunt and niece/nephews, and first cousins) but also to restrict our assessment of environmental effects to circumstances where relatives are likely to share common environmental exposures. This restriction also allowed for less right-censoring because these children and grandchildren were born earlier in history and hence are more likely to be older (and, quite often, deceased). Our study focused on survival past age 50; therefore, only individuals 50 years old or older from the selected families are incorporated in the analysis. We required that individuals had to have survived to age 50 to focus our analysis on the age range during which mortality begins to rise but also to maximize our sample size as much as possible. In all, 7273 individuals 50 years old or older were used as the basis for the analysis, comprising approximately 23,300 sib pairs, 5600 parent/offspring pairs, 22,900 uncle/auntniece/nephew pairs, and 53,000 first-cousin pairs.
| RESULTS |
|---|
|
|
|---|
|
The first model is a proportional hazards model that ignores the potential statistical dependencies between individuals in the same family (relations attributable to common genes or environment) and corresponds to the standard proportional hazards model. This means that we treat all individuals as being statistically independent after controlling for the covariates. The second model assumes that all members of a family share the same unmeasured risk and represents the shared frailty model. We interpret frailty as the combined shared environmental and genetic influences that are not measured by FEL and the LL dummy variable. The third model, representing the correlated frailty model, also allows for familial risk, but this risk depends only on genetic relationships and does not explicitly consider any shared environmental effects. In the decomposition of the covariance matrix given in Equation 3, this model corresponds to
= 0. The final and most general model incorporates both shared and correlated frailties. This model provides separate estimates for the role that genetic and shared environments play in the mortality hazard rate.
In Table 2, we present the hazard rate ratios (exp(ß)) and their 95% confidence intervals (CI). The four models use the same data and have the same covariates; the difference between them is the manner in which frailty is modeled. All the parameter estimates except
and
represent relative risks. With respect to the observed covariates, all show, in a qualitative sense, the same effects although some differences exist. These models show that male mortality hazard rates are 23%27% higher than those of females and that as birth year increases by 1 year individuals enjoy a reduction in their hazard rate by 1%. Individuals affiliated with the LDS Church have hazard rates that are not significantly different than those of persons not affiliated with the LDS Church. The null effect of LDS affiliation is unexpected but is partially explained by the fact that FEL is included in the model and it controls, indirectly, for the effects of LDS status. For this cohort, men benefit much more from their affiliation with the LDS Church than do women. The results shown here pool men and women in the same sample leading to a weaker association with mortality. An interaction of gender and LDS affiliation was not estimated for reasons of parsimony. As expected, both FEL and the indicator variable for being in an LL kindred indicate that individuals with a family history of exceptional longevity have significantly lower rates of mortality. Individuals sampled from LL kindreds have a mortality risk that is 14%19% lower than that found in control kindreds; for each year of increase in FEL there is a 5%6% decrease in the mortality hazard rate.
|
f ) from the shared frailty model is 0.189 (95% CI, 0.1500.229). This estimate indicates that typical values of the family-specific risk of mortality are approximately 21% larger or smaller [exp(0.189) = 1.208] than the overall mortality risk, ceteris paribus, suggesting an important shared familial influence in life span that is unaccounted for by the included covariates. This model assigns the same familial relative risk to all members of a family without taking explicit account of shared genetic effects between individuals in the same kindred. However, it is not possible in this model to rule out some common shared alleles among family members.
The correlated frailty (polygenic) model incorporates an individual-specific random effect that is allowed to be correlated with the frailties of other relatives, as indicated by the kinship matrix K. The SD of the frailty distribution for the polygenic frailty model (
p) is 0.338 (95% CI, 0.2580.418). This result indicates that the individual relative risk of mortality is likely to be as much as (exp(0.338)) 40% larger or smaller than the average risk. This provisional result is consistent with the idea that life span has a sizable heritable component.
The shared frailty model conflates genetic with shared environmental factors, and the correlated frailty model assumes that familial risk depends only on genetic variants that are shared to varying degrees among blood relatives. The model with both shared family and correlated random effects represents a more general approach that incorporates genetic and nongenetic traits separately. The SD of the correlated frailty distribution (
p) in this model is 0.143 (95% CI, 0.0000.282), and the SD of the shared frailty distribution (
f) is 0.186 (95% CI, 0.1470.230). Given the model and this sample of kindreds, both shared familial and polygenic factors are found to play a role in explaining the variation in longevity although the size of
f is larger. As two of the fixed covariates, FEL and LL, are likely correlated with unobserved genetic factors and environmental factors, it is difficult to interpret the difference in the magnitude of the estimates of
f and
p. In an attempt to determine whether the difference in the size of the estimates of
f and
p is an indication that shared environmental factors play a larger role than genetic traits on the mortality hazard rate, we refit the model with the same decomposition of the covariance matrix but with only birth year, gender, and affiliation with the LDS Church as fixed covariates. In this model, the estimated SD of the correlated frailty is 0.265, whereas the SD for shared frailty is now 0.294. In comparing the two sets of results, we make two observations. First, the small difference in the size of the two new SD values suggests that shared environmental factors play a similar albeit slightly larger role than genetic variants shared among sets of relatives. Here, the small difference between the two SD values (i.e., a relatively large SD for the polygenic correlated frailty effect in relation to what the literature suggests) may reflect the stronger role played by alleles associated with longevity arising from our selection of families characterized by EL. The second observation is that the exclusion of both the LL and FEL variables resulted in similar increases in the SD for both the shared and correlated frailty distributions. This finding supports the idea that the LL and FEL variables capture factors that have both environmental and genetic origins.
Effect of Ignoring Frailties on the Estimates of a Cox Proportional Hazards Model
Analysts frequently possess data based on individuals clustered into families and kindreds, and yet the analysis is conducted under the assumption of independence of survival times given the observed values of covariates. Using simulated data from correlated frailty models, we examined whether ignoring the associations in the data biases the relative risk estimates and, if so, to what degree. Additionally, we investigated whether the standard errors of the parameter estimates are inefficient under the assumption of independence.
We performed 1000 computer simulations of 100 families that represent the composition and structure of families in UPDB. The data comprise two generations within a family with four basic types of relationships: parents and offspring, siblings, aunts/uncles with nieces/nephews, and first cousin dyads. Ages at death were simulated using a proportional hazards model. The baseline survival function was assumed to follow a Gompertz distribution, that is,
|
|
= 0.1 and
= 0.007. The value of
represents the proportion of individuals in our sample that survived to age 50 years and died before reaching age 51 years (36,37). The value
= 0.1 is consistent with an adult force of mortality that doubles every 7 years exp(0.1 x 7) = 2. The mortality hazard rate increased by a factor of exp(ß) for a unit increase in a covariate X. The variable X followed a normal distribution with mean 2.0 and SD 1.9. Three different values for the regression parameter ß were considered, ß = 2.0, ß = 0.7, and ß = 0.1. These values represent hazard ratios of 7.4, 2, and 1.1, respectively. We also assumed that the risk of dying is influenced by an additive polygenic effect. Hence, the hazard rate is specified as:
|
|
=
p22K, with K the kinship matrix. This corresponds to
f 2 = 0 in Equation 3. We further assumed that the fixed covariates Xi, i = 1, ..., n, are independent given b. Two common but different situations are evaluated: first, the covariates Xi, i = 1, ..., n, are generated to be independent of the polygenic frailty b; second, Xi are specified so that they are correlated with the polygenic frailty b, with covariance between Xi, and bj equal to 0.2 for i = j, and 0 otherwise. The value 0.2 was chosen arbitrarily to represent a moderate degree of association between the known covariate and the unknown factor.
The effect on the parameter estimates of ignoring the genetic relationship in the data was assessed using two different models: a proportional hazards model and a shared frailty model with variance component
. The correlated frailty model is also estimated. In what follows,
,
, and
, will refer to the maximum likelihood estimators of ß,
, and
, and SD(
) is the estimator of the standard deviation of
.
Table 3 shows results for the situation in which the simulated data sets were generated according to a model with Xi independent of the frailty b. Table 3 gives the empirical expected value of
, SD(
),
, and
; that is, the average of the maximum likelihood estimates from each of the 1000 simulated data sets. Within Table 3, the empirical SD values are given in parentheses.
|
is smaller in the proportional hazards and the shared frailty models than in the correlated frailty model. This result is consistent with the fact that related observations contribute less information in relation to a model that assumes independence; hence the standard errors of the fixed parameters are underestimated. Moreover, Ripatti and Palmgren (30) observed that the estimated SD for
, in the correlated frailty model, is underestimated because the variation in the estimation of the variance component is ignored (i.e., the estimator of the SD of
is calculated as if
were known). The correlated frailty model assumes that the fixed covariates are independent of the frailty distribution; however, in many applied settings this will not be true. Table 4 includes the results of simulations in which the fixed covariate is correlated with frailty. The results are similar to those obtained when the fixed covariate and frailty are assumed to be independent. The new result is that, in some cases, the parameter ß has a positive bias.
|
. | DISCUSSION |
|---|
|
|
|---|
The present study supports the long-established observation that longevity is subject to substantial familial aggregation. It also demonstrates that the family-specific risk of mortality, controlling for observed covariates, is typically 21% larger or smaller than the overall mortality risk. This level of variability can be attributed to environmental factors, and would likely decline as additional significant covariates were added to the model. We also show that the relative risk of mortality at the individual level that is attributable to polygenic factors is likely to be as much as 40% larger or smaller than the average risk. When we fit a model that allows for both the contributions of polygenic and shared familial traits simultaneously, we find that the variance component due to genetic factors is comparable to the one due to shared environment.
The application of the shared frailty and the correlated polygenic frailty models reported here relied on assumptions about the manner in which the environment and genetic factors are shared among relatives. Specifically, we assumed that all individuals in a family share the same environment throughout their lives and that those environmental factors that are not shared or that change with time are nonexistent. This is a strong assumption, but alternative correlation structures that allow for more complex environmental conditions are possible with the methods described here. The method permits the analyst to modify the aij values to reflect assumed or measured levels of exposure-sharing between, for example, coresident relatives as opposed to kin who live in disparate geographic areas.
Our simulation analyses illustrate the consequences of assuming independence of survival times between individuals when family-based data are analyzed. Models that ignore the environmental or genetic association among related individuals result in downwardly biased relative risk estimates, a pattern that has been reported by others (38,39). We show that, in the presence of correlated frailty based on a general polygenic model, the correlated frailty specification nearly eliminates the bias in regression parameters. These simulations also indicate that the standard errors for the regression parameter estimates are also underestimated when the dependence among individuals is ignored. Our simulations show, for example, that none of the 95% CI values for the estimated regression parameters included the true value. Overall, downwardly biased regression coefficients coupled with narrow (and incorrect) CI values will lead to poor inferences about the significance of the covariates.
For analysts with survival data comprising statistically independent individuals, estimating the effects of the covariates on the hazard rate does not require any modifications to the methodology. If, however, the sample is based on a set of relatives (i.e., all individuals are connected to other individuals because they share a common ancestor), then it is important to consider modeling unobserved environmental and genetic factors within a general analytic approach.
|
A |
|---|
|
|
|---|
We thank the Pedigree and Population Resource (funded by the Huntsman Cancer Foundation) for providing the data and valuable computing support. We also thank Dr. Terry Therneau at the Mayo Clinic College of Medicine for sharing his ideas and software used in this analysis. In addition, we acknowledge the contributions of Alison Fraser and Diana Lane Reed in providing database management and research assistance for this study.
|
F |
|---|
|
|
|---|
Received December 7, 2005
Accepted June 13, 2006
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
B. Hagberg and G. Samuelsson Survival After 100 Years of Age: A Multivariate Model of Exceptional Survival in Swedish Centenarians J. Gerontol. A Biol. Sci. Med. Sci., November 1, 2008; 63(11): 1219 - 1226. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|