Evaluation of an integrated housing and recovery model for people with severe and persistent mental illnesses: the Doorway program*

Additional keywords: health funding and financing, health services research, mental health, models of care.

It is of concern that the recent Australian Study of High Impact Psychoses (SHIP) found that 5.2% and 12.8% of individuals with a psychotic illness had been homeless in the previous month and previous year respectively, the latter with a mean of 155 homeless days.^2,3 Homeless people frequently do not have the financial resources or appropriate references to gain private housing tenure. Public housing waiting lists are often long and often availability is not in locations preferred by applicants. This further alienates them from social contacts and engenders a sense of isolation and disengagement from society. The importance of housing tenure in contributing to the recovery of individuals with a severe and persistent mental illness (SPMI) is clear.⁴

Doorway is a 3-year pilot integrated housing and recovery support program delivered by the MI Fellowship.¹ It is designed to enhance the capacity of individuals with an SPMI requiring services and who are homeless, or at risk of homelessness, to lead independent, healthy and meaningful lives in housing and communities of their choice. Homelessness in contemporary Australia should be understood to consist of several (identified) arrangements for shelter below a minimum community standard. This is a small rented flat with a separate bathroom and kitchen and an element of security of tenure.⁵ The Doorway program integrates interventions to improve the client’s housing situation with efforts to improve social inclusion and support recovery.

The Doorway model builds upon and adapts the Housing First model that was pioneered in the US in the early 1990s.⁶ These models are built on the assumption that stable housing plays a critical role in the recovery of people with SPMI. Several large randomised control trials based on the Housing First model have now been conducted in Canada as part of the 5-cities At Home/Chez Soi Trial and the studies indicate consistent improvements with residential stability, social inclusion and reduced contact with the justice system.^7,8 Effects on levels of quality of life and substance abuse show more mixed results.^9–11 The research literature overall makes clear that the Housing First model has demonstrated real and substantial social benefits to participants.

A key difference between the Doorway model and other iterations of the original Housing First model is that participants source and choose properties through the open rental market, with appropriate rental subsidy and brokerage support. This integrated care model is highly innovative, differing from widely favoured arrangements internationally that involve congregate and scattered-site housing owned or managed by the support program. The Doorway model also differs from other programs that have operated in Australia that have involved securing housing, but only through facilitating contact with other housing-specific agencies, such as the Housing and Accommodation Support Initiative (HASI) in New South Wales, the Journey to Social Inclusion (J2SI) in Victoria and Project 300 in Queensland.^12–14

The aims of the present study were to determine the effects of the Doorway program, an innovative and little-studied care model where participants source and choose properties through the open rental market, on participants’ well being (health, access to housing, employment and social inclusion), as well as to perform an appraisal of the costs of participants’ health services and housing use (in the presence and absence of Doorway).

As stated, the Doorway model supports participants to choose, access and sustain their own private rental accommodation by subsidising their rental payments where required. In addition, Doorway’s housing and recovery workers (H&RWs) support participants to, for example, develop tenancy skills and build natural support networks. The Doorway H&RWs are graduate-level staff, intentionally including some people with a lived experience of mental illness and/or homelessness. Integrated support teams are created for each Doorway participant. To do this, Doorway H&RWs are embedded in the public sector Acute Mental Health Services (AMHSs) within the relevant hospital catchment areas (see below) and provide housing and recovery inputs to care. AMHS staff also form part of these integrated support teams, providing clinical care, including case management. Other community-based health services may also be involved for specialised purposes. ACMHs, and specifically the case manager, exercise governance for these different program inputs into an individual participant’s care.

The Doorway pilot was implemented with three clinical partners across inner city and suburban Melbourne as well as country Victoria between July 2011 and June 2014. These clinical partners were Austin and Repatriation Medical Centre as part of Austin Health (covering the cities of Banyule and Nillumbik), St Vincent’s Hospital as part of St Vincent’s Health (covering the City of Yarra) and Latrobe Regional Hospital as part of Gippsland Health (covering Baw Baw Shire and Latrobe City Council). The demographic characteristics, levels of social and economic disadvantage, rental affordability and mental health characteristics of participants differ across each region, sometimes being better, sometimes worse than those in Victoria overall.¹ The program was funded by the Department of Health and Human Services of Victoria for a 3-year period (30 June 2011–30 June 2014).

The present study used a pre-post design and encompassed both quantitative and qualitative dimensions. The evaluation period concluded in November 2013, 7 months before the end of the 3-year funded period at the request of the funding body.

Inclusion criteria for admission to the program were: (1) severe mental illness requiring services from an adult mental health service; (2) homelessness or at risk of imminent homelessness; (3) currently case-managed by an adult mental health service; and (4) receiving a Disability Support Pension (DSP) or Newstart Allowance.

Operational inclusion criteria were: willing to accept support; willing to give consent for members of the integrated team to share information with each other; living in doorway catchment areas; and needing to consent to data collection, sign a lease and contribute to rental payments.

Baseline measurements were performed at entry to the program and then at 6-monthly intervals across the evaluation period.

The Behaviour and Symptom Identification Scale 32 (BASIS-32) is a consumer-oriented, self-report measure of behavioural symptoms and distress; it has five subscales and 32 items, each rated on five-point scales.^15,16 The Health of the Nation Outcome Scale (HoNOS) is an interviewer-administered measurement tool designed to assess general health and social functioning of mentally ill people.¹⁷ The HoNOS has 12 subscales, bundled under four headings (Behavioural problems, Symptomatic problems, Impairment, Social problems). Each item is rated on a five-point scale.

Outcomes Star (Homelessness) (http://www.outcomesstarsystem.org.uk, accessed 2 November 2015) has 10 domains measuring various aspects of the homelessness experience, namely internal motivation, social networks, managing money, offending, mental health, physical health, living skills, meaningful use of time, managing tenancy and substance abuse.

Doorway staff collected both quantitative and qualitative data relevant to housing, employment and social inclusion. For housing, these data included, for example, the proportion of a participant’s housing preferences met, the incidence and amount of rental arrears, and breach of duty notices. For employment, variables collected included engagement in paid and unpaid employment, steps taken to find work and seeing an employment consultant, such as an Australian Government-funded disability employment service, accessing education and vocational training opportunities and receiving qualifications. For social inclusion, data collected included the composition of participants’ natural support networks, contacts with and attendance at court and contact with police.

De-identified data from three Victorian Government Department of Health (DoH) datasets were provided for participants in the program. These datasets were: (1) the Client Management Interface/Operational Data Store (CMI/ODS), which provides data on Victorian public mental health service usage;¹⁸ (2) the Victorian Admitted Episodes Dataset (VAED) for data on Victorian hospital use; and (3) the Victorian Emergency Minimum Dataset (VEMD), which provides data on emergency department (ED) use. Data were not available for ambulance call-outs, use of drug and alcohol services or general practitioner (GP) consultations.

Costs of mental health and general health services were derived from these client contact data in conjunction with unit cost data derived from the Victorian Government DoH, as well as other published sources (see Table 1) (Victorian Government Department of Health, pers. comm.).^18–21 Doorway program costs were derived from Doorway program funding documents (see Table 1), including Doorway client support costs, rental subsidy support and program management and operational costs.^22,23 These clinical and housing components were valued at A$10 136 and A$7937 per participant per annum respectively. Together with program management and operation costs of A$1228, these totalled A$19 300 per participant per annum.

Table 1.  Cost data and sources for Doorway and other programs and services
ED, emergency department; H&RW, housing and recovery worker; HBOS, home-based outreach support; PDRSS, psychiatric disability rehabilitation and support services

Housing costs for homeless people in Australia (other than Doorway) were obtained from published data (see Table 1).^24,25 Full cost data are available elsewhere.¹

Social and demographic characteristics included sex, age (mean), country of birth, Aboriginal or Torres Strait Islander status, identified carer (at point of referral), parent (single or couple), receiving DSP payments at January 2013 and mean fortnightly income at January 2013.

Analyses of HoNOS, BASIS-32 and Outcomes Star Homelessness measures compared scores for individuals at their entry to and exit from the program (relevant quarter year ± 3 months if scores unavailable). If multiple scores were in the same quarter, mean scores were used. Null values assumed to correspond with a rating of 0.

The time of exit used varied slightly for individual measures (details are provided in table footnotes). Utilisation and cost data was also compared at program entry and end. Following normality checking and removal of outliers, statistical analysis of continuous variables was conducted using paired-sample t-tests, where paired continuous data existed. Categorical data were assessed with Chi-squared tests, generally with unpaired data. All analyses were performed using IBM SPSS version 22.0. Some data were available during but not before the program and are marked ‘Post only’. Cost data based on official estimates of average costs (multiplied by the number of contacts of use) was not suited to statistical analysis, which was not then performed. Unless indicated otherwise, data are given as the mean ± s.d. or as frequency (percentage).

Effect sizes using Cohen’s d-test were also estimated from results of pre-post change in outcome measures with statistically significant results to indicate whether changes were, in relative terms, ‘small’, ‘medium’ or ‘big’. Statistical tests were not performed on some data, such as cost data that were based on best estimates rather than directly measured values.

Ethics approval was granted for the project by Austin Health’s Ethics Committee, with matching approval from ethics committees at both St Vincent’s and Latrobe Hospitals. On entry to the program, participants provided signed approval for access to their clinical and other relevant data for program operation purposes. These data, following de-identification and which were also not re-identifiable, was judged by the human research ethics committee not to require further consent when used for research purposes. Victorian Data Linkages (VDL), established by the Victorian and Australian governments as part of the National Collaborative Research Infrastructure Strategy (NCRIS) to develop new data linkage capacity in Victoria, provided these de-identified data to the study team in encrypted form. No participants withdrew from the program as a result of the requirement to share information for program operation purposes.

During the evaluation period, 77 people went through the intake process and, of these, 59 took up residence in private rental properties under the program, with 50 still in residence at the end of the evaluation period (Fig. 1). Analyses are based on available data for 55 of the 59 participants (Austin, n = 20; St Vincent’s, n = 12; Latrobe, n = 23). As recorded by Doorway program staff, nine of these people left the program early. Reasons for early exit included social-improved outcomes (reconnecting with family), financial sustainability (living with someone else etc.), health reached recovery sustainability and increased support needs. Reasons for leaving early were judged to be evenly balanced between positive steps to recovery and ongoing recovery challenges.

Fig. 1.  Flow of participants into and through Doorway program (n = 55).

Selected social and demographic characteristics of the participants are given in Table 2. These show that participants were most commonly Australian-born, single middle-aged males receiving a DSP. Schizophrenia (49%), followed by depression (25%), was by far the most common primary mental health diagnosis. More than one-third of participants had multiple mental health diagnoses. Based on their Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) scores, (http://www.who.int/substance_abuse/activities/assist/en/, accessed 15 September 2016) the most common problems were tobacco products (60% moderate risk; 20% high risk) and alcoholic beverages (60% moderate risk; 7% high risk). The use of cannabis and amphetamine-type stimulants was also common.

Table 2.  Sociodemographic indicators for participants (n = 47), excluding participants who left the program before March 2013
DSP, disability support pension

Prior to entering the program, based on the Chamberlain definition of homelessness (which refers to the following arrangements for shelter below a minimum community standard), 17% of participants were ‘primary homeless’ (i.e. without conventional accommodation), 50% were ‘secondary homeless’ (i.e. moving between various forms of temporary shelter), 21% were ‘tertiary homeless’ (i.e. living in single rooms in private boarding houses and 10% were ‘marginally housed’.⁵ This last category refers to living in housing situations close to minimum standard such as caravan parks.

Twenty-eight per cent of participants were on the public housing waiting list. The most common primary cause for their homelessness was their mental illness (50%), followed by relationship breakdowns (15%). Three participants had previously been long-term residents of a state-funded Continuing Care Unit (a community-based treatment facility).

Total mean BASIS-32 scores decreased significantly (improved) across the evaluation period from 1.3 ± 0.8 BASIS points before Doorway to 0.8 ± 0.6 points after Doorway (Cohen’s d = 0.7; 38% improvement in baseline levels; P = 0.04; n = 23). Scores decreased (i.e. improved) for each of the five subscales, with three subscales showing statistically significantly improvements, namely Relation to self/others (Cohen’s d = 0.6, P < 0.001), Depression/anxiety (Cohen’s d = 0.5, P = 0.01) and Daily living/role function (Cohen’s d = 0.5, P = 0.02; Fig. 2).

Fig. 2.  Total and different subscale scores on the Behaviour and Symptom Identification Scale 32 (BASIS-32) before (baseline) after (post-housing) participation in the Doorway program (n = 23). Data are the mean ± s.d.

The mean total HoNOS score decreased (improved) from 10.0 ± 4.9 before housing to 8.8 ± 5.1 after housing (of a maximum score of 48; n = 35), but the change was not significant. Mean scores also improved across three of the four HoNOS domains, although none reached statistical significance.

Over the period of engagement with the program, there was one incident relating to a possible overdose, one relating to self-harm and six relating to medical concerns, physical assault and antisocial behaviour.

There were statistically significant improvements for four of the Homelessness Star’s 10 domain scores, namely Motivation and taking responsibility (16.4% improvement from baseline; Cohen’s d = 0.43, P = 0.00), Managing money (23% improvement from baseline; Cohen’s d = 0.32, P = 0.01), Emotional and mental health (14.3% improvement from baseline; Cohen’s d = 0.36, P = 0.01) and Meaningful use of time (21.4% improvement from baseline; Cohen’s d = 1.29, P = 0.01). There were improvements in all six of the other domains, but these did not reach statistical significance (Fig. 3).

Fig. 3.  Subscale scores on the Outcomes Star (Homelessness) system at the time of intake into and after completion (November 2013) of the Doorway program (n = 29). An increase in a score represents improvement.

The period of time that participants were housed in rental accommodation ranged from 3 to 21 months (Post only).

The majority of participants’ housing preferences (proximity to family, health services and community resources) were met in the rental accommodation that they eventually occupied. By the end of the evaluation period, 31 (56%) participants had received 12-month lease extensions by their Property Managers, with three more on a month-to-month basis (all Post only).

The mean rental gap paid by Doorway to participants was A$194 ± 159 per fortnight at the end of the evaluation period, down from $376 ± 174 per fortnight at the beginning of the period (P < 0.001).

With regard to adverse events, 11 (20%) of Doorway participants had fallen into rental arrears on at least one occasion. There had been 10 breach of duty notices issued, numerous verbal warnings and some complaints from neighbours, three instances of property damage and six instances of lease breaks. Only one participant had any utilities disconnected during the evaluation period (all Post only).

There were modest improvements in outcomes for the proportion of participants engaged in paid and unpaid employment, taking steps to find work, seeing an employment consultant, accessing education and vocational training opportunities and receiving qualifications for their vocational training. However, these improvements were not statistically significant.

The composition of participants’ natural support networks evolved over time. The increase in contacts with ‘Others’, such as neighbours, work colleagues and local shop and café owners, rose from 14% to 59% and was significant (χ₁² = 7.72, P < 0.01). There was also a large increase in contacts with ‘Friends’, from 45% to 68%, but this was not significant (χ₁² = 1.42, P = 0.23).

Eleven (20%) Doorway participants had had 19 reported contacts with the court system related to criminal or civil matters, with the majority resulting in positive outcomes, such as intervention orders not being placed or lifted. Eleven (20%) participants had 21 reported contacts with the police with charges laid in only one case (all Post only).

Admission to bed-based mental health services (clinical and community) decreased substantially (from 1.2 ± 2.1 to 0.5 ± 1.4 admissions per participant per annum; t₅₀ = 3.01, P < 0.1). There was also a substantial decrease in total average bed days of bed-based mental health services from 20.0 ± 34.5 to 7.4 ± 26.5 days per year per participant (n = 51; t  = 2.81, P < 0.01). Cost savings based on the reduction in bed-day usage were estimated at A$7355 per participant per annum (Table 3).

Table 3.  Savings associated with health services usage per participant per annum

Contact with ambulatory mental health professionals decreased from a mean of 39.4 to 33.5 h per year per participant. The greatest reductions were in services provided by Mobile Support and Treatment Teams (MSTT) and Continuing Care Teams (CCT). There was an estimated saving of A$1882 per participant per annum.

Doorway participants’ decreased their use of specialised mental health supports (e.g. case managers (Fisher P < 0.01) and psychiatrists (Fisher P < 0.001)) over time.

Sixteen (33%) of the 48 participants enrolled in the Doorway program at the end of the evaluation period had been formally discharged from their adult mental health services to GP or similar care. Six of the nine participants who had been subject to Community Treatment Orders (CTO) at program intake had had these lifted (all Post only).

The total number of ED presentations decreased from 1.94 to 1.42 per participant annually (t₅₀ = 1.75, P = 0.09). ED presentations leading to admission decreased from 0.50 to 0.21 per participant annually and ED presentations not leading to admission decreased from 1.48 to 1.23 per participant annually. Using available costs estimates, savings associated with ED presentations were A$349 per participant per annum (Table 3).

The total number of general hospital admissions decreased (from 0.45 to 0.12 annually per participant; t₅₀ = 3.12, P < 0.01). The largest reduction in use was in the area of General Medicine, decreasing from 0.24 to 0.05 annually per participant. Estimated savings in hospital admissions per participant annually were A$1447 (Table 3).

Based on the above estimates of health service utilisation costs, savings relating to all health service usage (excluding Doorway direct client care services) were estimated at A$11 033 per participant per annum. When Doorway direct client care services, estimated at A$7937, were included, cost savings were reduced to A$3096 per participant per annum.

Annual housing costs per Doorway participant can be compared against the costs of different types of social housing (Table 4). Where the cost of capital (to government) is available and included, the total housing cost per Doorway participant per annum of A$10 136 (as part of the total Doorway funding of A$19 300 per participant per annum) was lower than the annual costs of all types of social housing. Approximately one-third of Doorway participants resided in some form of social housing before joining the program.

Table 4.  Potential net savings to government per housing type per participant per annum (2010–11)

Total cost savings also take into account Doorway managerial and operational costs (A$1228 per participant annually). These are shown per participant per annum in comparison with various forms of social housing in Table 4. The data in Table 4 indicates that Doorway produced overall net savings, the magnitude of which depended on the form of housing type used as a comparator. These savings were estimated to be at least A$1149 (for community housing). This is an underestimate, because government investment, in capital costs for community housing, were not available and were excluded. Cost savings were as high as A$19 837 when crisis accommodation housing was used as the comparator.

The program reported here was effective in securing subsidised rental housing for 59 people from this vulnerable group. It had additional benefits in terms of significantly improved client outcomes (as evidenced by total mean BASIS-32, including 3 of the 5 domains scores as well as four of the 10 domain scores of the Outcome Star Homelessness Scale). Results for BASIS-32 were clinically significant, all with moderate size effects. Results for the Outcome Star Homelessness Scale were also clinically significant ranging from small–medium (three domains) to large (one domain). Other improvements occurred in the areas of housing, employment and training, social inclusion and antisocial behaviour; some improvements were statistically significant, others were not. There was also a reduction in the use and cost of mental and general health services, as well as in the costs of housing. However, at the time of evaluation, mean levels of rental subsidies had yet to decrease substantially. Adverse events associated with tenancy, as well as antisocial behaviour, continued among a minority of participants.

These findings relating to improvement in behavioural function and distress of participants make an important contribution to the Housing First literature. This is because the previously unstudied Doorway model, where participants source and choose properties through the open rental market, with appropriate rental subsidy and brokerage support, is shown in the present study to have beneficial effects.

The cost savings in both health service usage and housing type should be compared with other studies. A well-designed randomised control trial reported that offering housing and case management to homeless adults resulted in fewer hospital days and ED visits compared with usual care, although this involved a homeless population with chronic medical illnesses, not chronic mental illnesses.²⁶ Savings (to government) in relation to homeless people with chronic mental illness are relatively few and, if anything, demonstrate additional costs to government.^11,27–29 It is unclear to what extent the cost findings in the present study relate to the Doorway model, rather than the usual Housing First model.

It should be noted that these cost savings are to government, not participants. This is in contrast with the benefits of the Doorway program, which are considered above and should not be double counted. Further, these cost savings to government are potential, not actual. For benefit to people with SPMI and at risk of homelessness to occur, these newly available funds must be invested in relevant programs and replace economically inefficient programs where they currently exist.

The present study has some limitations, including the predominantly pre-post study design without a control group. Although it could be argued that it may have been possible to identify and possibly recruit a community-based population of homeless people with severe mental illnesses to form such a control group, this would have been very difficult. These include ethical difficulties if the study design involved withholding a program with benefits that are now established. They would also include controlling for the effects of potentially large numbers of confounding variables in two relatively small populations, either through randomisation, simple or propensity score matching. Therefore, it was necessary to accept an uncontrolled study design and to entertain the possibility that changes in outcome variables occurred for reasons other than entry to the Doorway program.

Other limitations of the study include a relatively small sample size of the Doorway participant population, making Type 2 errors possible. There were some missing paired data (pre-post) for some individual participants due principally to data gaps for the participants in the CMI/ODS. Some of these gaps may be related, directly or indirectly to early participant withdrawals from the program. More generally, however, numbers of participants withdrawing early from the program were related equally to ‘positive steps to recovery’ as to ‘ongoing recovery challenges’.

The Doorway model secured housing for this vulnerable group with additional benefits in client outcomes alongside reduced use and cost of health services. The model is eminently transferable to other countries. These findings, if confirmed in larger studies, should have widespread applicability internationally.

None declared.