http://www.researchforum.org/project_general_324.html

Arizona WORKs Pilot Program Evaluation

General Information

View a brief abstract of this project.

View a complete, printer-friendly profile of this project.

Evaluator(s) Abt Associates, Inc.
Investigator(s) Robert Kornfeld (Abt Associates, Inc.)
Sponsor(s) Arizona Department of Economic Security
Funder(s) Arizona Department of Economic Security
Consultant(s) Not applicable
Subcontractor(s) Not applicable
 
Domain Income Security/TANF
Status Completed with continuing analysis
Duration Oct 1999 - Jan 2002
Type Research and/or Program Evaluation
Goal To evaluate Arizona Works
Program/Policy Description Beginning on April 1, 1999, the TANF program in most of Eastern Maricopa County (the county that includes Phoenix) was administered by MAXIMUS, Inc., a private company. Arizona Works and Wisconsin's W-2 program are unusual in that a private company is responsible for intake and benefit determination as well as case management and return-to-work services.

This study compares Arizona Works with EMPOWER Redesign, the publicly administered TANF program in the comparison area, which is the rest of Maricopa County. Arizona Works and EMPOWER Redesign also differ because only Arizona Works receives performance incentive payments for attaining specific levels of employment rates and other outcomes, and because these two programs have some different rules regarding time limits and other issues.

Notes No notes reported.
 
Last Updated 06/18/03
Type of Summary Reviewed
External Reviewer(s) Robert Kornfeld (Abt Associates, Inc.)
Contact(s) Robert Kornfeld (Robert_Kornfeld@abtassoc.com)
Abt Associates, Inc.
4800 Montgomery Lane
(T) (301) - 913-0593
(F) (301) 718-3108
Publications Department Not Reported
Abt Publications (not reported)
Abt Associates, Inc.
4808 Montgomery Lane
(T) not reported
(F) not reported
Submitter(s) Robert Kornfeld (Robert_Kornfeld@abtassoc.com)
Abt Associates, Inc.
4800 Montgomery Lane
(T) (301) - 913-0593
(F) (301) 718-3108

Populations Studied

Target Population Single parent families
Low-income households
Subgroups Analyzed Caseworkers/managers/administrators
Sample Size and Unit About 12,000 adult TANF recipients subject to Arizona Works and about 31,000 adult TANF recipients subject to EMPOWER Redesign in the rest of Maricopa County.

Sites Studied

Arizona

Program Components, Policies, and Activities Evaluated

Employment activities

  • Employment Activities - misc.

Financial incentives

  • Financial Incentives - misc.

Administration/Implementation

  • Administration/Implementation - misc.
Variation in program components across sites? No
Notes on program components N/A

Outcomes Assessed

Service utilization

  • Service utilization - misc.

Program implementation

  • Program Implementation - misc.

Income security

  • Income security - misc.

Financial costs and benefits/cost-effectiveness

  • Financial costs and benefits/cost-effectiveness - misc.

Employment

  • Employment - misc.

Benefit termination

  • Benefit Termination-misc.

Adult outcomes

  • Adult outcomes - misc.

Types of Studies

Type Implementation/Process Study
Aim To describe the development, planning, organization, staffing, community partnerships, service delivery, and implementation experience of Arizona Works and EMPOWER Redesign.
 
Type Cost-Benefit Study
Aim To assess whether the Arizona Works Pilot Program made the welfare families and the Arizona state government and taxpayers “better off” or “worse off”
 
Type Descriptive/Analytical Study
Aim To assess client satisfaction among adults in families served by Arizona Works and EMPOWER Redesign.
 
Type Impact Study (Quasi-experiment with non-equivalent control groups)
Aim To consider the possible effects of privatization, the performance incentives, and the differences in the program rules, and to take into account the pre-existing differences in the characteristics of the caseloads and neighborhoods served by Arizona Works and EMPOWER Redesign.
 

Data Sources

Source Administrative data
Title State welfare and wage records systems
Sample Characteristics/Data Collection Not reported
Sites Arizona
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 
Source Field Research
Title Visits to local and central offices
Sample Characteristics/Data Collection Arizona Works and EMPOWER Redesign offices
Sites Arizona
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 
Source Interview
Title Interviews
Sample Characteristics/Data Collection Local and central office staff from Arizona Works and EMPOWER Redesign
Sites Arizona
Response Rate/Attrition Notes N/A
Additional Execution Notes Executed via telephone and in person
 
Source Program descriptions and documents
Title Examination of relevant program documents
Sample Characteristics/Data Collection Arizona Works and EMPOWER Redesign offices
Sites Arizona
Response Rate/Attrition Notes N/A
Additional Execution Notes No notes reported.
 
Source Survey
Title Client Survey
Sample Characteristics/Data Collection Adults in families served by Arizona Works and EMPOWER Redesign
Sites Arizona
Response Rate/Attrition Notes Telephone only, about 50 percent
Additional Execution Notes Executed during 3rd year of study
 

Findings Available

Interim Implementation Findings
Final Implementation Findings
Interim Impact Findings
Final Impact Findings
Final Cost-benefit Findings

Findings

02/02/02: Arizona Works Pilot Program Evaluation: Second Impact Report
Interim Impact Findings:

Arizona Works and EMPOWER Redesign have been about equally successful in encouraging TANF recipients to leave TANF for paid employment and attain self-sufficiency. Some of the calculated performance measures indicate that Arizona Works placed a relatively larger proportion of adults in full-time work. These performance measures do not take into account any pre-existing differences in caseload and neighborhood characteristics of those served by Arizona Works and EMPOWER Redesign in the comparison area, nor do the performance measures take into account employment and earnings obtained after families leave welfare. Once these factors are taken into account, the two programs have a roughly equal effect on employment and average earnings.

 
02/02/02: Arizona Works Pilot Program Evaluation: Second Process Study Report
Interim Implementation Findings:

"Our conclusion is that MAXIMUS has put in place a program that is capable of meeting the requirements of its contract. The Process Study has shown that the Pilot Program is not a clear-cut test of privatization because MAXIMUS has never been truly independent of DES. MAXIMUS remains dependent on the state’s automated systems. MAXIMUS hired such a large number of former DES employees that its operations must have been influenced by previous training received by these employees at DES. The denial of the waivers to operate food stamps and Medicaid meant that MAXIMUS and DES needed to integrate the operations of two organizations in one location. The perceptions of DES staff suggest more subtle ways in which DES and MAXIMUS staff share day to day work burdens: each organization may benefit from the hours spent by staff from the other organization to register applications, process files, enter household information into the state’s automated systems, manage the lobby, and handle numerous other day-to-day tasks."

"Arizona Works and EMPOWER Redesign are nevertheless different in an institutional sense. Many key functions of Arizona Works are run by MAXIMUS, a for-profit company. As long as the findings are interpreted carefully, the Pilot Program still has the potential to provide an interesting test of the effects of a major effort to privatize TANF and many related services. The Process Study has attempted to find ways in which privatization has produced differences in the organization of local offices and in the procedures for delivering public services...The two organizations have staff playing many similar roles, and they provide many similar programs and many similar support services."

 
02/01/03: Arizona WORKs Pilot Program Evaluation: A Three Year Assessment
Final Implementation Findings
  • During the first three years of the program, MAXIMUS, clearly met the requirements of its contract to administer Arizona Works.
  • EMPOWER Redesign [the public program] also featured a well-designed organization and staffing plan.
  • The two work-focused programs had many similar procedures, used identical application forms, and offered a wide array of often similar support services, job search, and readiness programs.
  • The privatization of TANF intake and benefit determination faces numerous difficulties because the federal government requires that public sector staff administer intake and benefit determination for food stamps and medical assistance.
  • Although Arizona Works and EMPOWER Redesign were different in that many TANF-related services were privatized under Arizona Works, the pilot was less than a clear-cut test of privatization. Arizona Works relied on resources and services from the public sector. Similarly, the public program was not entirely public and subcontracted several services.

Final Impact Findings
  • Relative to EMPOWER Redesign, Arizona Works did not increase the percentage of adults who were employed and did not increase the average earnings of adults.
  • Arizona Works increased the proportion of adults who participated in unpaid community service or work experience, GED or ESL programs, job search assistance, and job readiness and training activities.
  • The program had minor effects on TANF receipt.
  • Through the first three years of the Arizona Works Pilot Program, Arizona Works and EMPOWER Redesign were about equally successful in encouraging TANF recipients to leave TANF for paid employment and attain self-sufficiency.
  • Arizona Works increased many measures of employment that were based on data reported by local welfare office staff.
  • The welfare office data provide an incomplete picture of employment, however, because they do not record jobs of some welfare recipients and because they do not record jobs received after persons left welfare.
  • The discrepancies in estimated impacts on employment and earnings obtained from the two data sources (UI wage records and welfare office data) exist mainly because the proportion of adults with employment recorded only employers, and not by welfare office staff, was larger among those subject to EMPOWER Redesign than among those subject to Arizona Works.
  • The Arizona Works program rules also gave recipients a relatively greater incentive to conceal earnings under EMPOWER Redesign.

Final Cost-Benefit Findings
  • The pilot program’s total costs — including contract payments and all categories of costs —probably exceeded the projected costs that would have been incurred under the state program.
  • It is ultimately difficult to assess whether TANF-related programs were relatively more expensive to administer under Arizona Works or EMPOWER Redesign because it is difficult to measure the separate costs of administering TANF-related programs, food stamps and Medicaid under EMPOWER Redesign.
  • It should be recognized, however, that the total cost of Arizona Works also included the cost of professional staff to oversee the pilot, the cost of the independent evaluation, and the substantial increase in average monthly TANF benefits caused by the program rules of Arizona Works.
  • The sum of all of these costs of Arizona Works most likely exceeded the costs that would have been incurred without the pilot.
 

Recommendations

Arizona Works Pilot Program Evaluation: Second Impact Report (02/02/02)
None reported.
 

Existing Publications

02/02/02 Arizona Works Pilot Program Evaluation: Second Impact Report Abt
02/02/02 Arizona Works Pilot Program Evaluation: Second Process Study Report Abt
02/01/03 Arizona WORKs Pilot Program Evaluation: A Three Year Assessment Abt
02/01/03 Arizona WORKs Pilot Program Evaluation: The Arizona Works Pilot Program: A Three Year Assessment: Executive Summary Abt