A Self-administered Version of the Functioning Assessment Short Test for Use in Population-based Studies: A Pilot Study

3.1. Development of the Self-administered Version of the Fast

The original version of the FAST is an ordinally scaled, interviewer-administered questionnaire containing 24 items out of six domains of functioning, i.e. autonomy, occupational functioning, cognitive functioning, financial issues, interpersonal relationships, and leisure time [14]. The English version (as provided in [14], see supplementary file 1) was translated into German by two native speakers following a standardized forward-backward translation protocol. To adjust the scale for the self-administered setting and the new target population (i.e. general population instead of psychiatric patients), some slight modifications of content were made to the original scale: a) Question 12 (regarding the ability to solve a problem adequately) was erased due to the following reason: In a self-administered questionnaire, cognitive functioning and the ability to solve problems relate to various areas of life. Therefore, this item is hard to operationalize and linguistically unspecific. Consequently, we expected a high variance due to inaccurate responses and chose to erase the item; b) question 21 (regarding a satisfactory sexual relationship) was modified to regard the satisfactory emotional relationship with the partner, since the hypersexual component of bipolar disorder is less relevant in a population-based sample. Besides, we considered the question regarding a satisfactory sexual relationship with the partner as too offensive in a study assessing the general population and in the setting of a self-administered questionnaire.

A text instructing the participants on how to fill in the questionnaire was added (see supplementary file 2). The option ‘Not applicable for me’ was added to the original ordinal scale, which contains the selection options ‘No difficulty’, ‘Slight difficulty’, ’Moderate difficulty’ and ‘Severe difficulty’. A proxy version of the questionnaire was developed, allowing cohabitating partners to evaluate the subjects’ functional performance (proxy assessment; see supplementary file 3). This version additionally contained the selection option ‘Cannot be judged by me’.

To obtain an overall rating of everyday functioning, we added two more questions to the questionnaires: first, an ordinally scaled question asking for ‘General difficulties while managing daily activities’ with the above-mentioned selection options. Second, a Visual Analogue Scale (VAS) that had to be answered at the end of the questionnaire, reaching from 0 (i.e., problems with all mentioned IADL tasks) to 100 (i.e., no problems with the mentioned IADL tasks). Finally, the original time frame of 15 days preceding the examination was expanded to cover a total of 28 days, since we assumed substantial intra-individual variation of everyday functioning over time and expected a better regression to the mean using a larger time frame.

3.2. Assessment of Reliability

The reliability of the self-administered version of the FAST was examined by measuring the agreement of self-assessment with proxy assessment. For this purpose, all study subjects a) had to rate themselves, and b) had to be rated by their partners. The cohabitation of subjects and partners was obligatory to ensure that all proxies had close insights into their partner’s abilities to perform IADL. Cronbach’s alpha was calculated to assess the questionnaire’s internal consistency.

3.3. Assessment of Validity

The questionnaire’s construct validity was assessed by calculating correlations of the FAST score with well-known validated scores of depression (PHQ-9), anxiety (GAD-7), and health-related quality of life (SF-12). Furthermore, predictors of higher FAST scores (indicating more difficulties in IADL) were identified by conducting linear regression analyses using the above-mentioned scales besides potentially disabling comorbidities (depression, arthrosis, chronic back pain, heart failure, stroke) obtained from the subjects’ medical record.

3.4. Subject Recruitment

The study sample was derived from two ongoing population-based studies in Germany: the STAAB study in Würzburg aims “to determine the prevalence of heart failure stages A-B in a representative sample of the general population and to prospectively investigate the progression from asymptomatic cardiac dysfunction into symptomatic heart failure” [25]. In the BiDirect study in Münster, ”the bidirectional relationship between depression and (subclinical) atherosclerosis” is explored [26]. Both studies recruited subjects via the local residents registration offices with a wide range of age at enrolment (Würzburg: ≥30 to <80 years, Münster: ≥35 to <66 years). From each study, we recruited 30 study participants as well as their cohabitating partners.

3.5. Data Acquisition

All subjects from Würzburg were visited in their homes by a member of the study staff, which ensured that the participants and their partners did not communicate while filling out the FAST questionnaires. In Münster, all subjects were invited to the study centre of the BiDirect study to answer the questionnaire under similar conditions, i.e. under supervision. Whenever the participants and their partners could not visit, they were sent separate prepaid envelopes containing the questionnaires, which had to be resent separately to the study centre after completion. In both studies, the variables administered to assess construct validity were derived from the last preceding visit of the participants at the study site.

3.6. Statistical Analyses

By adding up the 23 ordinally scaled items with four valid categories (‘No difficulty’=0 points, ‘Slight difficulty’=1 point, ‘Moderate difficulty’=2 points, ‘Severe difficulty’=3 points), the FAST score comprising a range from 0 to 69 was calculated. According to a priori defined criteria, pairs of raters answering more than four items with ‘Not applicable for me’ or ‘Cannot be judged by me’, respectively, were excluded from the calculation of the sum score. If the participant was retired or not working, seven ‘missing items’ were allowed. In all other cases, the valid items (‘no/slight/moderate/severe difficulty’) were summed up, their means were calculated, and then multiplied by 23 to obtain the sum score. Hence, the invalid items answered ‘Not applicable for me’ or ‘Cannot be judged by me’ were replaced by the mean of all valid items of the participant. Agreement of self- and proxy assessment of the FAST score was assessed by calculating intraclass correlation coefficients (ICCs). Considering the not normally distributed data, a non-parametric, rank-based approach was chosen [27, 28]. To assess a possible influence of the mode of inquiry (under supervision vs. unsupervised - see ‘data acquisition’), sensitivity analyses with the calculation of separate ICCs for both groups were carried out. Agreement at single item-level was measured via weighted kappa with radical weights, following the method of Brennan and Prediger [29, 30]. The agreement of the VAS was assessed via the above-mentioned non-parametric ICC. To assess construct validity, spearman-correlations with the SF-12, PHQ-9, and GAD-7 scores were calculated. Additionally, regression analyses with these scales and variables from the subjects’ medical record (potentially disabling comorbidities) as independent variables and the FAST sum score (self-assessment) as the dependent variable were conducted. SF-12, PHQ-9, GAD-7 and age (in decades) were entered as continuous variables, whereas comorbid diagnoses were entered as dichotomous variables. The FAST sum scores were transformed with [ln (1+ sum score FAST)] to make them applicable for linear regression. After univariable regression, a stepwise backward multivariable regression was performed. Analogous correlations and regressions were computed to identify predictors for the deviation between self- and proxy assessment regarding the FAST sum score.

A power calculation was conducted prior to recruitment. It was assumed to detect an ICC of 0.8 at a power of 90% with a sample size of 30 participants, which was the aimed sample size in Würzburg and Münster, respectively.

The correlation and regression analyses were conducted with SPSS, version 25. The non-parametric ICC was calculated with MS Excel, version 15.3, and the weighted kappa coefficients were calculated with R, version 3.4.2.

4.1. Recruitment and Basic Demographic Characteristics

Recruitment and data acquisition took place between December 2015 and November 2016 in Münster, and between March and June 2016 in Würzburg. A total population of 60 individuals was examined (30 from Würzburg, and 30 from Münster). Four subjects of the Münster cohort were excluded, since they were erroneously not recruited from the general population, but were part of the BiDirect study cohort of patients with depression, which had been recruited from psychiatric departments. Two pairs of raters were excluded from FAST sum score calculation because too many items had been answered outside the predefined boundaries. Thus, 54 subjects remained for analysis of the sum score, while 56 subjects were suitable for analysis of agreement at single-item level. Of the former, 44 (81.5%) answered the questionnaire under supervision, while 10 (18.5%) were inquired by mail, i.e. unsupervised.

Since the subjects from Würzburg and Münster were derived from two comparable population-based cohort studies and had similar median values of the FAST sum score, a pooled analysis was conducted. Table 1 illustrates the basic demographic characteristic for the two individual samples as well as for the pooled sample. Furthermore, it depicts information on the participants’ working status and the number of items answered inside the valid categories. The pooled study population included subjects from a wide age range (33-71 years) and had nearly balanced proportions of sex. The general level of difficulties with IADL was quite low, and most participants had long-term relationships with the partners that performed proxy assessment. While the employed participants answered a median of 22 items (quartiles 21, 22) inside the valid categories, the retired participants answered only a median of 19 items (18, 19), respectively.

4.2. Reliability

With a Cronbach’s alpha of 0.76, the self-administered version of the FAST showed acceptable internal consistency. The agreement of self-assessment with a proxy assessment regarding the sum score was moderate: ICC 0.50 (95%-confidence interval (CI) 0.46-0.54). Fig. (1) illustrates the distribution of sum scores and the deviation from perfect agreement for each pair of raters. Since 18.5% of the participants answered the questionnaire unsupervised, it seems possible that these subjects and their partners did not answer the questionnaires independently. However, after excluding the unsupervised pairs of raters, the agreement for the FAST sum score differed only slightly: ICC 0.53 (95% CI 0.49-0.58). The excluded pairs of raters were analysed separately; the results showed poor agreement: ICC 0.38 (95% CI 0.11-0.64).

With a mean weighted kappa coefficient of 0.63 (95% CI 0.57-0.69), the agreement on single-item level was moderate, too. Table 2 shows weighted kappa coefficients for all ordinally scaled items. Weight matrices are depicted in Table 3.

Regarding the VAS, the agreement was poor: ICC 0.32 (95% CI 0.28-0.35). Fig. (2) shows the distribution of VAS scores and the deviation from the perfect agreement.

4.3. Validity

In both studies of Würzburg and Münster, well-adopted and validated scales of depression, anxiety, and health-related quality of life were obtained. However, the information was not sampled simultaneously with the FAST questionnaire, but at the subjects’ last regular visit at the study centres (i.e., baseline visit in Würzburg, second follow-up visit in Münster). While most of the participants from Würzburg were assessed with the FAST shortly after their baseline visit, the second follow-up visit of the subjects from Muenster took place more than two years before their FAST assessment. Therefore, construct validity was only analysed for the subgroup of participants from Würzburg. Three of these subjects were excluded, because their baseline visit preceded the FAST assessment by more than one year. For the remaining 27 participants, the median interval between STAAB baseline visit and FAST assessment was 6 days (IQR 5-8 days). As detailed in Table 4, significant correlations between the participants’ FAST score and the PQH-9 and GAD-7 scores were found. Thus, higher difficulties in performing IADL were found in the presence of increased symptoms of depression and anxiety. Further, a significant correlation between the participants’ FAST sum score and the mental sub-scale of the SF-12 was found. Participants with a lower level of mental health-related quality of life thus reported higher difficulties in IADL. In univariable linear regression analyses, a diagnosis of heart failure, myocardial infarction, stroke, diabetes mellitus, arthrosis and depression at any time in the participant's medical record had no significant influence on the FAST score of the participants. In contrast, chronic back pain, PHQ-9, GAD-7, and the mental sub-scale of the SF-12 significantly influenced the FAST sum score in univariable models. However, only chronic back pain and the SF-12 mental sub-scale remained significant in multivariable analysis (R² of the final model: 0.53, corrected R²: 0.48). For point estimators and p-values see Table 5 (results for clearly non-significant comorbidities, i.e. heart failure, myocardial infarction, diabetes mellitus, stroke, and arthrosis, are not shown).

4.4. Predictors of Deviation between Self and Proxy-assessment

Regarding the absolute value of deviation of self-assessment and proxy assessment in the FAST sum score, there was only a significant positive correlation with the PHQ-9 score: higher deviation (without regarding its direction) prevailed in participants that reported more severe symptoms of depression. Table 6 contains the exact values and further correlations. In univariable linear regression analyses, the PHQ-9 (p=0.08) as well as the GAD-7 (p=0.07) turned out to be tendentially significant predictors, while the SF-12 physical sub-scale had a significant influence on the absolute value of deviation: point estimate 0.79 (95% CI 0.64-0.97); p=0.03). A higher deviation was thus present in participants with lower physical health-related quality of life. Details are shown in the lower part of Table 6. No significant correlations were found when the direction of deviation was considered.

5.1. Integration of Results into Previously Published Articles

The newly-developed, self-administered version of the FAST showed acceptable internal consistency, good construct validity, and a moderate to poor reliability in two population-based studies.

The median (quartiles) score of the FAST was 6.33 (2.27, 12.48); this value is comparable to the healthy control groups in the validation studies of Rosa in Spain (mean 6.07, SD: 4.72) [14], Barbato in Italy (mean 7.90, SD 11.44) [19], Zortéa in Brazil (median 5.0) [23], and Rotger in Spain (mean 6.01, SD 4.67) [21].

Due to the self-administered application of the scale, reliability could not be assessed via interrater reliability. Instead, we chose to assess reliability by assessing the agreement of self-assessment with proxy assessment by partners. We chose this approach for two reasons: First, there is no comparable German IADL scale validated for the general population that could be used as a reference standard. Second, this is the first German version of the FAST. Since we aimed to use the scale in population-based studies, we did not develop an interviewer-administered German version of the FAST that could have been used to measure reliability by testing agreement of self-assessment with interviewer-assessment.

So far, no studies investigating the agreement of self-assessment with a proxy assessment of the FAST have been published. Therefore, our results cannot be compared directly. Ostbye et al. reported a comparable, only moderate level of agreement (weighted kappa 0.55; 95% CI 0.48-0.62) in an elderly, non-demented cohort when comparing ADL and IADL self-assessment with proxy assessment by caregivers [31]. However, in this study, the examined population was older, and the group of caregivers consisted not only of partners but also of offspring and others, which limits the comparability to our population.

The moderate agreement regarding the sum score of the FAST should not inevitably be interpreted as a sign of poor quality of the instrument itself. Multiple factors might have influenced how the participants and their partners judged the participants’ functional status. For instance, higher deviations were found when the participants reported symptoms of depression and anxiety and a lower physical health-related quality of life. These or other undocumented factors may also prevail in the partners and could have influenced their judgement of the participants’ functional status, which may explain part of the deviation. Zanetti et al. compared caregiver’s proxy assessment with direct performance-based assessment of functional impairment in patients with very mild or mild dementia and reported that the agreement was influenced by the caregivers’ burden [32]. This indicates that internal factors of proxies performing IADL assessment can affect their rating. However, since no further variables of the partners were obtained, it is not possible to confirm this hypothesis in our data.

Since there may be substantial bias of both over- and underestimation in the participants’ as well as their partners’ judgements, we would recommend using methods beyond the agreement of self-assessment and proxy assessment in further validation studies.

Considering the agreement on single-item level, it was notable that the concordance for tasks that could be directly observed was higher than for items that comprised more abstract or subjective tasks. As an example, the agreement about difficulties while ‘taking care of oneself (physical aspects, hygiene)’ was much higher than for difficulties while ‘concentrating on a book or film’. With a mean weighted kappa coefficient of 0.63 (95%-CI: 0.57 - 0.69) and a Cronbach’s alpha of 0.76, we judge the reliability on single item level as satisfactory and thus see no need to eliminate items. However, it should be noted that the Cronbach’s alpha of our German version of the FAST is considerably lower compared to the previously published FAST studies, which reached values from 0.87-0.96 [14-23].

The VAS used in the self-administered version of the FAST reflected poor agreement of self-assessment with proxy assessment. This is on the one hand, due to the many outliers; on the other hand, this is due to the high occurrence of shared ranks (see the superscript numbers in Fig. 2) that influence the rank-based approach more than if a parametric method had been used. Since the VAS is highly negatively correlated with the FAST sum score (ρ=-0.78, p≤0.001), we consider it as a dispensable element carrying no additional information and propose to omit it when applying the questionnaire in clinical studies.

The significant correlations of the FAST sum score with PHQ-9, GAD-7, and the mental sub-scale of the SF-12 are consistent with numerous previous findings: in elderly patients, there is evidence for an association between symptoms of depression or anxiety and IADL impairment [7-9, 33]. Additionally, functional impairment has been reported in adult subjects with sub-threshold symptoms of depression and anxiety disorder [10, 34, 35]. Furthermore, poor mental health-related quality of life was associated with functional-impairment in two geriatric studies [36, 37]. The fact that those associations were reproduced in our study points towards a high construct validity of the German version of the FAST. Whilst the psychiatric scales significantly influenced the FAST score, most somatic comorbidities (as heart failure, stroke or arthrosis) did not influence the FAST score. We hypothesize this to be due to 1) the low prevalence of these diseases in the study sample and 2) the fact that participants were asked for a diagnosis at any point in medical history (regardless of actual disabilities as a consequence).

5.2. Limitations and Strengths

Since the ICC of 0.5 observed in our study was considerably lower than the anticipated ICC of 0.8 that had been used for sample size calculation, the study can be considered underpowered. Furthermore, a power calculation was performed only for the primary analysis, i.e. the detection of the ICC. As the further statistical analyses were only explorative secondary analysis, we did not adjust the p-value for multiple testing. Hence, all further results need to be interpreted with caution. We did not apply a qualitative approach (e.g. focus groups) to ensure that all questions were understood adequately by all participants before beginning the validation process of the scale. This might have added to the rather low agreement between self- and proxy-assessment. We did not assess test-retest reliability since we assumed substantial intra-individual variance of the FAST score over time and concluded this would bias the results. However, we may have been able to assess test-retest reliability by choosing a rather short time gap between the two points of inquiry. Furthermore, we did not assess concurrent validity of the FAST with already validated IADL scales since there is no comparable scale in German language containing all relevant IADL dimensions that have been validated in samples of the general population and could be used as a reference standard.

Still, the correlations and regression analyses with the above mentioned validated scales illustrate the high construct validity of the German version of the FAST. Yet, it has to be noted that the FAST score and the other variables utilized to evaluate construct validity were not gathered simultaneously. However, the time interval between the two points of inquiry was not longer than one week for most of the participants, leading to a substantial overlap of the timeframes of the observation periods covered per scale. Additionally, it is reasonable to assume that health-related quality of life and symptoms of depression and anxiety do not change substantially within only a few days. We did not perform a factorial analysis due to the small sample size. However, given that only very few items were modified, and given that the original factorial structure has been reproduced in other validation studies of the instrument [18-20], it can be supposed that the factorial structure of the German version is similar to the original one. As 18.5% of the pairs of raters conducted the FAST assessment unsupervised, some of them may have answered the questionnaires not independently. However, the raters that were inquired unsupervised had only poor agreement when being analysed separately. Hence, it seems unlikely that they agreed on results. Although this study examined participants of a broad age range, younger (30-40 years) and elderly (>70 years) subjects are underrepresented in our sample. The results are therefore restricted to middle-aged to early late-aged adults. The generalizability of this study is further limited, since only participants with a cohabitating partner were recruited.

The comparability of our results regarding the self-administered version of the FAST with the previously published FAST studies is limited due to the following reasons: a) while the previous studies were carried out in samples of psychiatric patients, we tested the FAST in a sample derived from the general population, and b) we performed some slight modifications of content to adjust the scale for the new target population and self-administered assessment as detailed in the methods section.

This is the first study evaluating a self-administered version of the FAST. The scale’s coherence with well-adopted, validated scales of depression and anxiety can be considered a new and valuable finding since it illustrates the FAST’s applicability in measuring the impact of even slight manifestations of these symptoms on everyday functioning. A methodological strength of this study is the fact that a non-parametric ICC approach and Spearman correlations were applied, thereby accounting for the distributional structure of the data, instead of simply using conventional parametric methods.